Edition 5
Abstract
Mono-spaced Bold
To see the contents of the filemy_next_bestselling_novel
in your current working directory, enter thecat my_next_bestselling_novel
command at the shell prompt and press Enter to execute the command.
Press Enter to execute the command.Press Ctrl+Alt+F2 to switch to a virtual terminal.
mono-spaced bold
. For example:
File-related classes includefilesystem
for file systems,file
for files, anddir
for directories. Each class has its own associated set of permissions.
Choose Mouse Preferences. In the Buttons tab, click the Left-handed mouse check box and click to switch the primary mouse button from the left to the right (making the mouse suitable for use in the left hand).→ → from the main menu bar to launchTo insert a special character into a gedit file, choose → → from the main menu bar. Next, choose → from the Character Map menu bar, type the name of the character in the Search field and click . The character you sought will be highlighted in the Character Table. Double-click this highlighted character to place it in the Text to copy field and then click the button. Now switch back to your document and choose → from the gedit menu bar.
Mono-spaced Bold Italic
or Proportional Bold Italic
To connect to a remote machine using ssh, typessh
at a shell prompt. If the remote machine isusername
@domain.name
example.com
and your username on that machine is john, typessh john@example.com
.Themount -o remount
command remounts the named file system. For example, to remount thefile-system
/home
file system, the command ismount -o remount /home
.To see the version of a currently installed package, use therpm -q
command. It will return a result as follows:package
.
package-version-release
Publican is a DocBook publishing system.
mono-spaced roman
and presented thus:
books Desktop documentation drafts mss photos stuff svn books_tests Desktop1 downloads images notes scripts svgs
mono-spaced roman
but add syntax highlighting as follows:
package org.jboss.book.jca.ex1; import javax.naming.InitialContext; public class ExClient { public static void main(String args[]) throws Exception { InitialContext iniCtx = new InitialContext(); Object ref = iniCtx.lookup("EchoBean"); EchoHome home = (EchoHome) ref; Echo echo = home.create(); System.out.println("Created Echo"); System.out.println("Echo.echo('Hello') = " + echo.echo("Hello")); } }
Note
Important
Warning
Install Media
section and download the install DVD that matches your hardware.
Network Configuration
section you’ll want to:
Important
system settings
and select network
. From there you can select what device to configure.
Software Selection
section (try saying that 10 times quickly), choose Minimal Install
so that we see everything that gets installed. Don’t enable updates yet, we’ll do that (and install any extra software we need) later.
Important
/
(aka. root
) partition which cannot be dynamically reduced in size (dynamic increases are fine by-the-way).
Installation Destination
section where you are be given an opportunity to reduce the size of the root
partition (after chosing which hard drive you wish to install to).
Date & Time
section. [3]
root
and the password you created earlier.
Note
# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 52:54:00:d7:d6:08 brd ff:ff:ff:ff:ff:ff inet 192.168.122.101/24 brd 192.168.122.255 scope global eth0 inet6 fe80::5054:ff:fed7:d608/64 scope link valid_lft forever preferred_lft forever
Note
# manually edit /etc/sysconfig/network-scripts/ifcfg-${device} # nmcli dev disconnect ${device} # nmcli con reload ${device} # nmcli con up ${device}
NetworkManager
aware that a change was made on the config file.
[root@pcmk-1 ~]# ip route default via 192.168.122.1 dev eth0 192.168.122.0/24 dev eth0 proto kernel scope link src 192.168.122.101
default via
, then you may need to add a line such as
GATEWAY=192.168.122.1
# ping -c 1 192.168.122.1 PING 192.168.122.1 (192.168.122.1) 56(84) bytes of data. 64 bytes from 192.168.122.1: icmp_req=1 ttl=64 time=0.249 ms --- 192.168.122.1 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.249/0.249/0.249/0.000 ms
# ping -c 1 www.google.com PING www.l.google.com (173.194.72.106) 56(84) bytes of data. 64 bytes from tf-in-f106.1e100.net (173.194.72.106): icmp_req=1 ttl=41 time=167 ms --- www.l.google.com ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 167.618/167.618/167.618/0.000 ms
beekhof@f16 ~ # ping -c 1 192.168.122.101 PING 192.168.122.101 (192.168.122.101) 56(84) bytes of data. 64 bytes from 192.168.122.101: icmp_req=1 ttl=64 time=1.01 ms --- 192.168.122.101 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 1.012/1.012/1.012/0.000 ms
beekhof@f16 ~ # ssh -l root 192.168.122.11 root@192.168.122.11's password: Last login: Fri Mar 30 19:41:19 2012 from 192.168.122.1 [root@pcmk-1 ~]#
Warning
Important
TODO: Create an Appendix that deals with (at least) re-enabling the firewall.
# setenforce 0 # sed -i.bak "s/SELINUX=enforcing/SELINUX=permissive/g" /etc/selinux/config # systemctl disable firewalld.service # systemctl stop firewalld.service
# setenforce 0 # sed -i.bak "s/SELINUX=enforcing/SELINUX=permissive/g" /etc/selinux/config # systemctl disable iptables.service # rm '/etc/systemd/system/basic.target.wants/iptables.service' # systemctl stop iptables.service
# uname -n pcmk-1.clusterlabs.org # dnsdomainname clusterlabs.org
hostnamectl
tool to strip off the domain name.
# hostnamectl set-hostname $(uname -n | sed s/\\..*//)'
# uname -n pcmk-1 # dnsdomainname clusterlabs.org
# ping -c 3 192.168.122.102 PING 192.168.122.102 (192.168.122.102) 56(84) bytes of data. 64 bytes from 192.168.122.102: icmp_seq=1 ttl=64 time=0.343 ms 64 bytes from 192.168.122.102: icmp_seq=2 ttl=64 time=0.402 ms 64 bytes from 192.168.122.102: icmp_seq=3 ttl=64 time=0.558 ms --- 192.168.122.102 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.343/0.434/0.558/0.092 ms
# grep pcmk /etc/hosts 192.168.122.101 pcmk-1.clusterlabs.org pcmk-1 192.168.122.102 pcmk-2.clusterlabs.org pcmk-2
# ping -c 3 pcmk-2 PING pcmk-2.clusterlabs.org (192.168.122.101) 56(84) bytes of data. 64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=1 ttl=64 time=0.164 ms 64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=2 ttl=64 time=0.475 ms 64 bytes from pcmk-1.clusterlabs.org (192.168.122.101): icmp_seq=3 ttl=64 time=0.186 ms --- pcmk-2.clusterlabs.org ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.164/0.275/0.475/0.141 ms
Warning
# ssh-keygen -t dsa -f ~/.ssh/id_dsa -N "" Generating public/private dsa key pair. Your identification has been saved in /root/.ssh/id_dsa. Your public key has been saved in /root/.ssh/id_dsa.pub. The key fingerprint is: 91:09:5c:82:5a:6a:50:08:4e:b2:0c:62:de:cc:74:44 root@pcmk-1.clusterlabs.org The key's randomart image is: +--[ DSA 1024]----+ |==.ooEo.. | |X O + .o o | | * A + | | + . | | . S | | | | | | | | | +-----------------+ # cp .ssh/id_dsa.pub .ssh/authorized_keys
# scp -r .ssh pcmk-2: The authenticity of host 'pcmk-2 (192.168.122.102)' can't be established. RSA key fingerprint is b1:2b:55:93:f1:d9:52:2b:0f:f2:8a:4e:ae:c6:7c:9a. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'pcmk-2,192.168.122.102' (RSA) to the list of known hosts.root@pcmk-2's password: id_dsa.pub 100% 616 0.6KB/s 00:00 id_dsa 100% 672 0.7KB/s 00:00 known_hosts 100% 400 0.4KB/s 00:00 authorized_keys 100% 616 0.6KB/s 00:00 # ssh pcmk-2 -- uname -n pcmk-2 #
[ALL] # yum install -y pacemaker pcs
[ALL] # yum install -y pcs
# systemctl start pcsd.service # systemctl enable pcsd.service
pcs
to talk to itself on other nodes in the cluster. This is necessary in order to perform tasks such as syncing the corosync config, or starting/stopping the cluster on remote nodes
pcs
can be used locally without setting up these user accounts, this tutorial will make use of these remote access commands, so we will set a password for the hacluster user. Its probably best if password is consistent across all the nodes.
# passwd hacluster
password:
--stdin
option for passwd
:
# ssh pcmk-2 -- 'echo redhat1 | passwd --stdin hacluster'
# pcs cluster auth pcmk-1 pcmk-2 Username: hacluster Password: pcmk-1: Authorized pcmk-2: Authorized # pcs cluster setup mycluster pcmk-1 pcmk-2 pcmk-1: Succeeded pcmk-2: Succeeded
Important
pcs
and crmsh
. This edition of Clusters from Scratch is based on pcs
. Start by taking some time to familiarize yourself with what it can do.
Note
Important
pcs
has the ability to manage all aspects of the cluster (both corosync and pacemaker), it requires a specific cluster stack to be in use, (corosync 2.0 with votequorum + Pacemaker version >= 1.1.8).
# pcs
Control and configure pacemaker and corosync. Options: -h, --help Display usage and exit -f file Perform actions on file instead of active CIB --debug Print all network traffic and external commands run --version Print pcs version information Commands: cluster Configure cluster options and nodes resource Manage cluster resources stonith Configure fence devices constraint Set resource constraints property Set pacemaker properties status View cluster status config Print full cluster configuration
# pcs status help
Usage: pcs status [commands]... View current cluster and resource status Commands: [status] View all information about the cluster and resources resources View current status of cluster resources groups View currently configured groups and their resources cluster View current cluster status corosync View current membership information as seen by corosync nodes [corosync|both|config] View current status of nodes from pacemaker. If 'corosync' is specified, print nodes currently configured in corosync, if 'both' is specified, print nodes from both corosync & pacemaker. If 'config' is specified, print nodes from corosync & pacemaker configuration. pcsd <node> ... Show the current status of pcsd on the specified nodes xml View xml version of status (output from crm_mon -r -1 -X)
# pacemakerd --features
Note
# pcs cluster start --all pcmk-1: Starting Cluster... pcmk-2: Starting Cluster...
# pcs cluster start
Starting Cluster...
# systemctl start corosync.service # systemctl start pacemaker.service
corosync-cfgtool
.
# corosync-cfgtool -s Printing ring status. Local node ID 1 RING ID 0 id = 192.168.122.101 status = ring 0 active with no faults
id
and no faults
for the status.
# corosync-cmapctl | grep members runtime.totem.pg.mrp.srp.members.1.ip (str) = r(0) ip(192.168.122.101) runtime.totem.pg.mrp.srp.members.1.join_count (u32) = 1 runtime.totem.pg.mrp.srp.members.1.status (str) = joined runtime.totem.pg.mrp.srp.members.2.ip (str) = r(0) ip(192.168.122.102) runtime.totem.pg.mrp.srp.members.2.join_count (u32) = 1 runtime.totem.pg.mrp.srp.members.2.status (str) = joined # pcs status corosync Membership information -------------------------- Nodeid Votes Name 1 1 pcmk-1 (local) 2 1 pcmk-2
# ps axf PID TTY STAT TIME COMMAND 2 ? S 0:00 [kthreadd] ...lots of processes... 28019 ? Ssl 0:03 /usr/sbin/corosync 28047 ? Ss 0:00 /usr/sbin/pacemakerd -f 28048 ? Ss 0:00 \_ /usr/libexec/pacemaker/cib 28049 ? Ss 0:00 \_ /usr/libexec/pacemaker/stonithd 28050 ? Ss 0:00 \_ /usr/lib64/heartbeat/lrmd 28051 ? Ss 0:00 \_ /usr/libexec/pacemaker/attrd 28052 ? Ss 0:00 \_ /usr/libexec/pacemaker/pengine 28053 ? Ss 0:00 \_ /usr/libexec/pacemaker/crmd
# pcs status Last updated: Fri Sep 14 09:52:25 2012 Last change: Fri Sep 14 09:51:55 2012 via crmd on pcmk-2 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 0 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources:
# grep -i error /var/log/messages
# journalctl | grep -i error
# pcs status Last updated: Fri Sep 14 10:12:01 2012 Last change: Fri Sep 14 09:51:55 2012 via crmd on pcmk-2 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 0 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources:
pcs cluster cib
command.
Example 5.1. The last XML you’ll see in this document
# pcs cluster cib
<cib epoch="4" num_updates="19" admin_epoch="0" validate-with="pacemaker-1.2" crm_feature_set="3.0.6" update-origin="pcmk-1" update-client="crmd" cib-last-written="Wed Aug 1 16:08:52 2012" have-quorum="1" dc-uuid="1"> <configuration> <crm_config> <cluster_property_set id="cib-bootstrap-options"> <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0"/> <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/> </cluster_property_set> </crm_config> <nodes> <node id="1" uname="pcmk-1" type="normal"/> <node id="2" uname="pcmk-2" type="normal"/> </nodes> <resources/> <constraints/> </configuration> <status> <node_state id="2" uname="pcmk-2" ha="active" in_ccm="true" crmd="online" join="member" expected="member" crm-debug-origin="do_state_transition" shutdown="0"> <lrm id="2"> <lrm_resources/> </lrm> <transient_attributes id="2"> <instance_attributes id="status-2"> <nvpair id="status-2-probe_complete" name="probe_complete" value="true"/> </instance_attributes> </transient_attributes> </node_state> <node_state id="1" uname="pcmk-1" ha="active" in_ccm="true" crmd="online" join="member" expected="member" crm-debug-origin="do_state_transition" shutdown="0"> <lrm id="1"> <lrm_resources/> </lrm> <transient_attributes id="1"> <instance_attributes id="status-1"> <nvpair id="status-1-probe_complete" name="probe_complete" value="true"/> </instance_attributes> </transient_attributes> </node_state> </status> </cib>
# crm_verify -L -V
error: unpack_resources: Resource start-up disabled since no STONITH resources have been defined
error: unpack_resources: Either configure some or disable STONITH with the stonith-enabled option
error: unpack_resources: NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
-V may provide more details
enabled
. However it also knows when no STONITH configuration has been supplied and reports this as a problem (since the cluster would not be able to make progress if a situation requiring node fencing arose).
# pcs property set stonith-enabled=false # crm_verify -L
Warning
Important
# pcs resource create ClusterIP ocf:heartbeat:IPaddr2 \ ip=192.168.0.120 cidr_netmask=32 op monitor interval=30s
# pcs resource standards
ocf
lsb
service
systemd
stonith
# pcs resource providers
heartbeat
linbit
pacemaker
redhat
# pcs resource agents ocf:heartbeat
AoEtarget
AudibleAlarm
CTDB
ClusterMon
Delay
Dummy
.
. (skipping lots of resources to save space)
.
IPaddr2
.
.
.
symlink
syslog-ng
tomcat
vmware
# pcs status Last updated: Fri Sep 14 10:17:00 2012 Last change: Fri Sep 14 10:15:48 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 1 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
# pcs status Last updated: Fri Sep 14 10:17:00 2012 Last change: Fri Sep 14 10:15:48 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 1 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1
#pcs cluster stop pcmk-1
Stopping Cluster...
# pcs status Last updated: Fri Sep 14 10:31:01 2012 Last change: Fri Sep 14 10:15:48 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition WITHOUT quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 1 Resources configured. Online: [ pcmk-2 ] OFFLINE: [ pcmk-1 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Stopped
pcmk-1
is now offline. However we can also see that ClusterIP
isn’t running anywhere!
total_nodes < 2 * active_nodes
# pcs property set no-quorum-policy=ignore # pcs property dc-version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 cluster-infrastructure: corosync stonith-enabled: false no-quorum-policy: ignore
# pcs status Last updated: Fri Sep 14 10:38:11 2012 Last change: Fri Sep 14 10:37:53 2012 via cibadmin on pcmk-2 Stack: corosync Current DC: pcmk-2 (2) - partition WITHOUT quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 1 Resources configured. Online: [ pcmk-2 ] OFFLINE: [ pcmk-1 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
pcmk-1
and check the cluster’s status. Note, if you get an authentication error with the pcs cluster start pcmk-1 command, you must authenticate on the node using the pcs cluster auth pcmk pcmk-1 pcmk-2 command discussed earlier.
# pcs cluster start pcmk-1 Starting Cluster... # pcs status Last updated: Fri Sep 14 10:42:56 2012 Last change: Fri Sep 14 10:37:53 2012 via cibadmin on pcmk-2 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 1 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2
Note
pcmk-1
). Usually this is no longer the case.
# pcs resource rsc defaults resource-stickiness=100 # pcs resource rsc defaults resource-stickiness: 100
# yum install -y httpd wget
Loaded plugins: langpacks, presto, refresh-packagekit fedora/metalink | 2.6 kB 00:00 updates/metalink | 3.2 kB 00:00 updates-testing/metalink | 41 kB 00:00 Resolving Dependencies --> Running transaction check ---> Package httpd.x86_64 0:2.2.22-3.fc17 will be installed --> Processing Dependency: httpd-tools = 2.2.22-3.fc17 for package: httpd-2.2.22-3.fc17.x86_64 --> Processing Dependency: apr-util-ldap for package: httpd-2.2.22-3.fc17.x86_64 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-2.2.22-3.fc17.x86_64 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-2.2.22-3.fc17.x86_64 --> Running transaction check ---> Package apr.x86_64 0:1.4.6-1.fc17 will be installed ---> Package apr-util.x86_64 0:1.4.1-2.fc17 will be installed ---> Package apr-util-ldap.x86_64 0:1.4.1-2.fc17 will be installed ---> Package httpd-tools.x86_64 0:2.2.22-3.fc17 will be installed --> Finished Dependency Resolution Dependencies Resolved ===================================================================================== Package Arch Version Repository Size ===================================================================================== Installing: httpd x86_64 2.2.22-3.fc17 updates-testing 823 k wget x86_64 1.13.4-2.fc17 fedora 495 k Installing for dependencies: apr x86_64 1.4.6-1.fc17 fedora 99 k apr-util x86_64 1.4.1-2.fc17 fedora 78 k apr-util-ldap x86_64 1.4.1-2.fc17 fedora 17 k httpd-tools x86_64 2.2.22-3.fc17 updates-testing 74 k Transaction Summary ===================================================================================== Install 1 Package (+4 Dependent packages) Total download size: 1.1 M Installed size: 3.5 M Downloading Packages: (1/6): apr-1.4.6-1.fc17.x86_64.rpm | 99 kB 00:00 (2/6): apr-util-1.4.1-2.fc17.x86_64.rpm | 78 kB 00:00 (3/6): apr-util-ldap-1.4.1-2.fc17.x86_64.rpm | 17 kB 00:00 (4/6): httpd-2.2.22-3.fc17.x86_64.rpm | 823 kB 00:01 (5/6): httpd-tools-2.2.22-3.fc17.x86_64.rpm | 74 kB 00:00 (6/6): wget-1.13.4-2.fc17.x86_64.rpm | 495 kB 00:01 ------------------------------------------------------------------------------------- Total 238 kB/s | 1.1 MB 00:04 Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction Installing : apr-1.4.6-1.fc17.x86_64 1/6 Installing : apr-util-1.4.1-2.fc17.x86_64 2/6 Installing : apr-util-ldap-1.4.1-2.fc17.x86_64 3/6 Installing : httpd-tools-2.2.22-3.fc17.x86_64 4/6 Installing : httpd-2.2.22-3.fc17.x86_64 5/6 Installing : wget-1.13.4-2.fc17.x86_64 6/6 Verifying : apr-util-ldap-1.4.1-2.fc17.x86_64 1/6 Verifying : httpd-tools-2.2.22-3.fc17.x86_64 2/6 Verifying : apr-util-1.4.1-2.fc17.x86_64 3/6 Verifying : apr-1.4.6-1.fc17.x86_64 4/6 Verifying : httpd-2.2.22-3.fc17.x86_64 5/6 Verifying : wget-1.13.4-2.fc17.x86_64 6/6 Installed: httpd.x86_64 0:2.2.22-3.fc17 wget.x86_64 0:1.13.4-2.fc17 Dependency Installed: apr.x86_64 0:1.4.6-1.fc17 apr-util.x86_64 0:1.4.1-2.fc17 apr-util-ldap.x86_64 0:1.4.1-2.fc17 httpd-tools.x86_64 0:2.2.22-3.fc17 Complete!
# cat <<-END >/var/www/html/index.html <html> <body>My Test Site - pcmk-1</body> </html> END
[root@pcmk-2 ~]# cat <<-END >/var/www/html/index.html <html> <body>My Test Site - pcmk-2</body> </html> END
<Location /server-status> SetHandler server-status Order deny,allow Deny from all Allow from 127.0.0.1 </Location>
pcs resource create WebSite ocf:heartbeat:apache \ configfile=/etc/httpd/conf/httpd.conf \ statusurl="http://localhost/server-status" op monitor interval=1min
# pcs resource op defaults timeout=240s # pcs resource op defaults timeout: 240s
# pcs status Last updated: Fri Sep 14 10:51:27 2012 Last change: Fri Sep 14 10:50:46 2012 via crm_attribute on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 2 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-1
Note
pcs status
output, you see the WebSite resource has failed to start, then you’ve likely not enabled the status URL correctly. You can check if this is the problem by running:
wget http://127.0.0.1/server-status
Connection refused
in the output, then this is indeed the problem. Check to ensure that Allow from 127.0.0.1
is present for the <Location /server-status>
block.
Note
Important
WebSite
needs to be placed on the same machine as ClusterIP
, this implies that we must know the location of ClusterIP
before choosing a location for WebSite
.
# pcs constraint colocation add WebSite ClusterIP INFINITY # pcs constraint Location Constraints: Ordering Constraints: Colocation Constraints: WebSite with ClusterIP # pcs status Last updated: Fri Sep 14 11:00:44 2012 Last change: Fri Sep 14 11:00:25 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 2 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2
# pcs constraint order ClusterIP then WebSite Adding ClusterIP WebSite (kind: Mandatory) (Options: first-action=start then-action=start) # pcs constraint Location Constraints: Ordering Constraints: start ClusterIP then start WebSite Colocation Constraints: WebSite with ClusterIP
# pcs constraint location WebSite prefers pcmk-1=50 # pcs constraint Location Constraints: Resource: WebSite Enabled on: pcmk-1 (score:50) Ordering Constraints: start ClusterIP then start WebSite Colocation Constraints: WebSite with ClusterIP # pcs status Last updated: Fri Sep 14 11:06:37 2012 Last change: Fri Sep 14 11:06:26 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 2 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2
# crm_simulate -sL Current cluster status: Online: [ pcmk-1 pcmk-2 ] ClusterIP (ocf:heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf:heartbeat:apache): Started pcmk-2 Allocation scores: native_color: ClusterIP allocation score on pcmk-1: 50 native_color: ClusterIP allocation score on pcmk-2: 200 native_color: WebSite allocation score on pcmk-1: -INFINITY native_color: WebSite allocation score on pcmk-2: 100 Transition Summary:
# pcs constraint location WebSite prefers pcmk-1=INFINITY # pcs constraint all Location Constraints: Resource: WebSite Enabled on: pcmk-1 (score:INFINITY) (id:location-WebSite-pcmk-1-INFINITY) Ordering Constraints: start ClusterIP then start WebSite (Mandatory) (id:order-ClusterIP-WebSite-mandatory) Colocation Constraints: WebSite with ClusterIP (INFINITY) (id:colocation-WebSite-ClusterIP-INFINITY) # pcs status Last updated: Fri Sep 14 11:16:26 2012 Last change: Fri Sep 14 11:16:18 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 2 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1
# pcs constraint all Location Constraints: Resource: WebSite Enabled on: pcmk-1 (score:INFINITY) (id:location-WebSite-pcmk-1-INFINITY) Ordering Constraints: start ClusterIP then start WebSite (Mandatory) (id:order-ClusterIP-WebSite-mandatory) Colocation Constraints: WebSite with ClusterIP (INFINITY) (id:colocation-WebSite-ClusterIP-INFINITY) # pcs constraint rm location-WebSite-pcmk-1-INFINITY # pcs constraint Location Constraints: Ordering Constraints: start ClusterIP then start WebSite Colocation Constraints: WebSite with ClusterIP
# pcs status Last updated: Fri Sep 14 11:57:12 2012 Last change: Fri Sep 14 11:57:03 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 2 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1
# yum install -y drbd-pacemaker drbd-udev
Loaded plugins: langpacks, presto, refresh-packagekit Resolving Dependencies --> Running transaction check ---> Package drbd-pacemaker.x86_64 0:8.3.11-5.fc17 will be installed --> Processing Dependency: drbd-utils = 8.3.11-5.fc17 for package: drbd-pacemaker-8.3.11-5.fc17.x86_64 ---> Package drbd-udev.x86_64 0:8.3.11-5.fc17 will be installed --> Running transaction check ---> Package drbd-utils.x86_64 0:8.3.11-5.fc17 will be installed --> Finished Dependency Resolution Dependencies Resolved ====================================================================================== Package Arch Version Repository Size ====================================================================================== Installing: drbd-pacemaker x86_64 8.3.11-5.fc17 updates-testing 22 k drbd-udev x86_64 8.3.11-5.fc17 updates-testing 6.4 k Installing for dependencies: drbd-utils x86_64 8.3.11-5.fc17 updates-testing 183 k Transaction Summary ====================================================================================== Install 2 Packages (+1 Dependent package) Total download size: 212 k Installed size: 473 k Downloading Packages: (1/3): drbd-pacemaker-8.3.11-5.fc17.x86_64.rpm | 22 kB 00:00 (2/3): drbd-udev-8.3.11-5.fc17.x86_64.rpm | 6.4 kB 00:00 (3/3): drbd-utils-8.3.11-5.fc17.x86_64.rpm | 183 kB 00:00 -------------------------------------------------------------------------------------- Total 293 kB/s | 212 kB 00:00 Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction Installing : drbd-utils-8.3.11-5.fc17.x86_64 1/3 Installing : drbd-pacemaker-8.3.11-5.fc17.x86_64 2/3 Installing : drbd-udev-8.3.11-5.fc17.x86_64 3/3 Verifying : drbd-pacemaker-8.3.11-5.fc17.x86_64 1/3 Verifying : drbd-udev-8.3.11-5.fc17.x86_64 2/3 Verifying : drbd-utils-8.3.11-5.fc17.x86_64 3/3 Installed: drbd-pacemaker.x86_64 0:8.3.11-5.fc17 drbd-udev.x86_64 0:8.3.11-5.fc17 Dependency Installed: drbd-utils.x86_64 0:8.3.11-5.fc17 Complete!
# vgdisplay | grep -e Name -e Free VG Name vg_pcmk1 Free PE / Size 31 / 992.00 MiB # lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert lv_root vg_pcmk1 -wi-ao-- 8.56g lv_swap vg_pcmk1 -wi-ao-- 960.00m # lvcreate -n drbd-demo -L 1G vg_pcmk1 Logical volume "drbd-demo" created # lvs LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert drbd-demo vg_pcmk1 -wi-a--- 1.00G lv_root vg_pcmk1 -wi-ao-- 8.56g lv_swap vg_pcmk1 -wi-ao-- 960.00m
# ssh pcmk-2 -- lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert lv_root vg_pcmk1 -wi-ao-- 8.56g lv_swap vg_pcmk1 -wi-ao-- 960.00m # ssh pcmk-2 -- lvcreate -n drbd-demo -L 1G vg_pcmk1 Logical volume "drbd-demo" created # ssh pcmk-2 -- lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert drbd-demo vg_pcmk1 -wi-a--- 1.00G lv_root vg_pcmk1 -wi-ao-- 8.56g lv_swap vg_pcmk1 -wi-ao-- 960.00m
Warning
global { usage-count yes; } common { protocol C; } resource wwwdata { meta-disk internal; device /dev/drbd1; syncer { verify-alg sha1; } net { allow-two-primaries; } on pcmk-1 { disk /dev/vg_pcmk1/drbd-demo; address 192.168.122.101:7789; } on pcmk-2 { disk /dev/vg_pcmk1/drbd-demo; address 192.168.122.102:7789; } }
Note
# drbdadm create-md wwwdata
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
success
# modprobe drbd # drbdadm up wwwdata # cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 0D2B62DEDB020A425130935 1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:1015740
# ssh pcmk-2 -- drbdadm --force create-md wwwdata Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. success # ssh pcmk-2 -- modprobe drbd WARNING: Deprecated config file /etc/modprobe.conf, all config files belong into /etc/modprobe.d/. # ssh pcmk-2 -- drbdadm up wwwdata # ssh pcmk-2 -- cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 0D2B62DEDB020A425130935 1: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:1015740
# drbdadm -- --overwrite-data-of-peer primary wwwdata # cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 0D2B62DEDB020A425130935 1: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r----- ns:8064 nr:0 dw:0 dr:8728 al:0 bm:0 lo:0 pe:1 ua:0 ap:0 ep:1 wo:f oos:1007804 [>....................] sync'ed: 0.9% (1007804/1015740)K finish: 0:12:35 speed: 1,320 (1,320) K/sec
# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 0D2B62DEDB020A425130935 1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:1015740 nr:0 dw:0 dr:1016404 al:0 bm:62 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
# mkfs.ext4 /dev/drbd1 mke2fs 1.42 (29-Nov-2011) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 63488 inodes, 253935 blocks 12696 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=260046848 8 block groups 32768 blocks per group, 32768 fragments per group 7936 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376 Allocating group tables: done Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done
# mount /dev/drbd1 /mnt/ # cat <<-END >/mnt/index.html <html> <body>My Test Site - drbd</body> </html> END # umount /dev/drbd1
# pcs cluster cib drbd_cfg
# pcs -f drbd_cfg resource create WebData ocf:linbit:drbd \ drbd_resource=wwwdata op monitor interval=60s # pcs -f drbd_cfg resource master WebDataClone WebData \ master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 \ notify=true
# pcs -f drbd_cfg resource show ClusterIP (ocf::heartbeat:IPaddr2) Started WebSite (ocf::heartbeat:apache) Started Master/Slave Set: WebDataClone [WebData] Stopped: [ WebData:0 WebData:1 ]
# pcs cluster push cib drbd_cfg CIB updated # pcs status Last updated: Fri Sep 14 12:19:49 2012 Last change: Fri Sep 14 12:19:13 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-2 (2) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 4 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1 Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ]
Note
# pcs cluster cib fs_cfg # pcs -f fs_cfg resource create WebFS Filesystem \ device="/dev/drbd/by-res/wwwdata" directory="/var/www/html" \ fstype="ext4"
# pcs -f fs_cfg constraint colocation add WebFS WebDataClone INFINITY with-rsc-role=Master # pcs -f fs_cfg constraint order promote WebDataClone then start WebFS Adding WebDataClone WebFS (kind: Mandatory) (Options: first-action=promote then-action=start)
# pcs -f fs_cfg constraint colocation add WebSite WebFS INFINITY # pcs -f fs_cfg constraint order WebFS then WebSite
# pcs -f fs_cfg constraint Location Constraints: Ordering Constraints: start ClusterIP then start WebSite WebFS then WebSite promote WebDataClone then start WebFS Colocation Constraints: WebSite with ClusterIP WebFS with WebDataClone (with-rsc-role:Master) WebSite with WebFS # pcs -f fs_cfg resource show ClusterIP (ocf::heartbeat:IPaddr2) Started WebSite (ocf::heartbeat:apache) Started Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] WebFS (ocf::heartbeat:Filesystem) Stopped
# pcs cluster push cib fs_cfg CIB updated # pcs status Last updated: Fri Aug 10 12:47:01 2012 Last change: Fri Aug 10 12:46:55 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 5 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-1 WebSite (ocf::heartbeat:apache): Started pcmk-1 Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-1 ] Slaves: [ pcmk-2 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-1
# pcs cluster standby pcmk-1 # pcs status Last updated: Fri Sep 14 12:41:12 2012 Last change: Fri Sep 14 12:41:08 2012 via crm_attribute on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 5 Resources configured. Node pcmk-1 (1): standby Online: [ pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 ] Stopped: [ WebData:1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
# pcs cluster unstandby pcmk-1 # pcs status Last updated: Fri Sep 14 12:43:02 2012 Last change: Fri Sep 14 12:42:57 2012 via crm_attribute on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 5 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf::heartbeat:apache): Started pcmk-2 Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 ] Slaves: [ pcmk-1 ] WebFS (ocf::heartbeat:Filesystem): Started pcmk-2
# yum install -y gfs2-utils dlm kernel-modules-extra
Loaded plugins: langpacks, presto, refresh-packagekit Resolving Dependencies --> Running transaction check ---> Package dlm.x86_64 0:3.99.4-1.fc17 will be installed ---> Package gfs2-utils.x86_64 0:3.1.4-3.fc17 will be installed ---> Package kernel-modules-extra.x86_64 0:3.4.4-3.fc17 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: dlm x86_64 3.99.4-1.fc17 updates 83 k gfs2-utils x86_64 3.1.4-3.fc17 fedora 214 k kernel-modules-extra x86_64 3.4.4-3.fc17 updates 1.7 M Transaction Summary ================================================================================ Install 3 Packages Total download size: 1.9 M Installed size: 7.7 M Downloading Packages: (1/3): dlm-3.99.4-1.fc17.x86_64.rpm | 83 kB 00:00 (2/3): gfs2-utils-3.1.4-3.fc17.x86_64.rpm | 214 kB 00:00 (3/3): kernel-modules-extra-3.4.4-3.fc17.x86_64.rpm | 1.7 MB 00:01 -------------------------------------------------------------------------------- Total 615 kB/s | 1.9 MB 00:03 Running Transaction Check Running Transaction Test Transaction Test Succeeded Running Transaction Installing : kernel-modules-extra-3.4.4-3.fc17.x86_64 1/3 Installing : gfs2-utils-3.1.4-3.fc17.x86_64 2/3 Installing : dlm-3.99.4-1.fc17.x86_64 3/3 Verifying : dlm-3.99.4-1.fc17.x86_64 1/3 Verifying : gfs2-utils-3.1.4-3.fc17.x86_64 2/3 Verifying : kernel-modules-extra-3.4.4-3.fc17.x86_64 3/3 Installed: dlm.x86_64 0:3.99.4-1.fc17 gfs2-utils.x86_64 0:3.1.4-3.fc17 kernel-modules-extra.x86_64 0:3.4.4-3.fc17 Complete!
# pcs resource disable WebFS # pcs resource ClusterIP (ocf::heartbeat:IPaddr2) Started WebSite (ocf::heartbeat:apache) Stopped Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 ] Slaves: [ pcmk-1 ] WebFS (ocf::heartbeat:Filesystem) Stopped
Note
Warning
clustername:fsname
. For the fsname
, we need to use the same value as specified in corosync.conf for cluster_name
. If you setup corosync with the same cluster name we used in this tutorial, cluster name will be mycluster. If you are unsure what your cluster name is, open up /etc/corosync/corosync.conf, or execute the command pcs cluster corosync pcmk-1 to view the corosync config. The cluster name will be in the totem
block.
Important
/dev/drbd1: Read-only file system
# ssh pcmk-2 -- mkfs.gfs2 -p lock_dlm -j 2 -t mycluster:web /dev/drbd1 This will destroy any data on /dev/drbd1. It appears to contain: Linux rev 1.0 ext4 filesystem data, UUID=dc45fff3-c47a-4db2-96f7-a8049a323fe4 (extents) (large files) (huge files) Are you sure you want to proceed? [y/n]y Device: /dev/drbd1 Blocksize: 4096 Device Size 0.97 GB (253935 blocks) Filesystem Size: 0.97 GB (253932 blocks) Journals: 2 Resource Groups: 4 Locking Protocol: "lock_dlm" Lock Table: "mycluster" UUID: ed293a02-9eee-3fa3-ed1c-435ef1fd0116
# pcs cluster cib dlm_cfg # pcs -f dlm_cfg resource create dlm ocf:pacemaker:controld op monitor interval=60s # pcs -f dlm_cfg resource clone dlm clone-max=2 clone-node-max=1 # pcs -f dlm_cfg resource show ClusterIP (ocf::heartbeat:IPaddr2) Started WebSite (ocf::heartbeat:apache) Stopped Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 ] Slaves: [ pcmk-1 ] WebFS (ocf::heartbeat:Filesystem) Stopped Clone Set: dlm-clone [dlm] Stopped: [ dlm:0 dlm:1 ] # pcs cluster push cib dlm_cfg CIB updated # pcs status Last updated: Fri Sep 14 12:54:50 2012 Last change: Fri Sep 14 12:54:43 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 7 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: ClusterIP (ocf::heartbeat:IPaddr2): Started pcmk-2 WebSite (ocf::heartbeat:apache): Stopped Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 ] Slaves: [ pcmk-1 ] WebFS (ocf::heartbeat:Filesystem): Stopped Clone Set: dlm-clone [dlm] Started: [ pcmk-1 pcmk-2 ]
# mount /dev/drbd1 /mnt/ # cat <<-END >/mnt/index.html <html> <body>My Test Site - GFS2</body> </html> END # umount /dev/drbd1 # drbdadm verify wwwdata
# pcs resource show WebFS
Resource: WebFS
device: /dev/drbd/by-res/wwwdata
directory: /var/www/html
fstype: ext4
target-role: Stopped
# pcs resource update WebFS fstype=gfs2 # pcs resource show WebFS Resource: WebFS device: /dev/drbd/by-res/wwwdata directory: /var/www/html fstype: gfs2 target-role: Stopped CIB updated
# pcs cluster cib active_cfg
# pcs -f active_cfg resource clone ClusterIP \ globally-unique=true clone-max=2 clone-node-max=2
# pcs -f active_cfg constraint
Location Constraints:
Ordering Constraints:
start ClusterIP-clone then start WebSite
WebFS then WebSite
promote WebDataClone then start WebFS
Colocation Constraints:
WebSite with ClusterIP-clone
WebFS with WebDataClone (with-rsc-role:Master)
WebSite with WebFS
# pcs -f active_cfg resource update ClusterIP clusterip_hash=sourceip
# pcs -f active_cfg resource clone WebFS # pcs -f active_cfg resource clone WebSite # pcs -f active_cfg constraint Location Constraints: Ordering Constraints: start ClusterIP-clone then start WebSite-clone WebFS-clone then WebSite-clone promote WebDataClone then start WebFS-clone Colocation Constraints: WebSite-clone with ClusterIP-clone WebFS-clone with WebDataClone (with-rsc-role:Master) WebSite-clone with WebFS-clone
# pcs -f active_cfg resource update WebDataClone master-max=2
# pcs cluster push cib active_cfg # pcs resource enable WebFS
# pcs resource Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 pcmk-1 ] Clone Set: dlm-clone [dlm] Started: [ pcmk-2 pcmk-1 ] Clone Set: ClusterIP-clone [ClusterIP] (unique) ClusterIP:0 (ocf::heartbeat:IPaddr2) Started ClusterIP:1 (ocf::heartbeat:IPaddr2) Started Clone Set: WebFS-clone [WebFS] Started: [ pcmk-1 pcmk-2 ] Clone Set: WebSite-clone [WebSite] Started: [ pcmk-1 pcmk-2 ]
pcs stonith list
pcs stonith describe <agent name>
pcs cluster cib stonith_cfg
pcs -f stonith_cfg stonith create <stonith_id> <stonith device type> [stonith device options]
pcs -f stonith_cfg property set stonith-enabled=true
pcmk_host_map
parameter. See man stonithd
for details.
pcmk_host_list
and/or pcmk_host_check
parameters. See man stonithd
for details.
pcmk_host_argument
parameter. See man stonithd
for details.
pcs cluster push cib stonith_cfg
stonith_admin --reboot nodename
. Although you might want to stop the cluster on that machine first.
# pcs stonith describe fence_ipmilan Stonith options for: fence_ipmilan auth: IPMI Lan Auth type (md5, password, or none) ipaddr: IPMI Lan IP to talk to passwd: Password (if required) to control power on IPMI device passwd_script: Script to retrieve password (if required) lanplus: Use Lanplus login: Username/Login (if required) to control power on IPMI device action: Operation to perform. Valid operations: on, off, reboot, status, list, diag, monitor or metadata timeout: Timeout (sec) for IPMI operation cipher: Ciphersuite to use (same as ipmitool -C parameter) method: Method to fence (onoff or cycle) power_wait: Wait X seconds after on/off operation delay: Wait X seconds before fencing is started privlvl: Privilege level on IPMI device verbose: Verbose mode
# pcs cluster cib stonith_cfg # pcs -f stonith_cfg stonith create impi-fencing fence_ipmilan \ pcmk_host_list="pcmk-1 pcmk-2" ipaddr=10.0.0.1 login=testuser \ passwd=acd123 op monitor interval=60s
# pcs -f stonith_cfg stonith
impi-fencing (stonith:fence_ipmilan) Stopped
# pcs -f stonith_cfg property set stonith-enabled=true # pcs -f stonith_cfg property dc-version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 cluster-infrastructure: corosync no-quorum-policy: ignore stonith-enabled: true
# pcs cluster push cib stonith_cfg
# pcs resource Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 pcmk-1 ] Clone Set: dlm-clone [dlm] Started: [ pcmk-2 pcmk-1 ] Clone Set: ClusterIP-clone [ClusterIP] (unique) ClusterIP:0 (ocf::heartbeat:IPaddr2) Started ClusterIP:1 (ocf::heartbeat:IPaddr2) Started Clone Set: WebFS-clone [WebFS] Started: [ pcmk-1 pcmk-2 ] Clone Set: WebSite-clone [WebSite] Started: [ pcmk-1 pcmk-2 ] # pcs resource rsc defaults resource-stickiness: 100 # pcs resource op defaults timeout: 240s # pcs stonith impi-fencing (stonith:fence_ipmilan) Started # pcs property dc-version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 cluster-infrastructure: corosync no-quorum-policy: ignore stonith-enabled: true # pcs constraint Location Constraints: Ordering Constraints: ClusterIP-clone then WebSite-clone WebDataClone then WebSite-clone WebFS-clone then WebSite-clone Colocation Constraints: WebSite-clone with ClusterIP-clone WebFS-clone with WebDataClone (with-rsc-role:Master) WebSite-clone with WebFS-clone # # pcs status Last updated: Fri Sep 14 13:45:34 2012 Last change: Fri Sep 14 13:43:13 2012 via cibadmin on pcmk-1 Stack: corosync Current DC: pcmk-1 (1) - partition with quorum Version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 2 Nodes configured, unknown expected votes 11 Resources configured. Online: [ pcmk-1 pcmk-2 ] Full list of resources: Master/Slave Set: WebDataClone [WebData] Masters: [ pcmk-2 pcmk-1 ] Clone Set: dlm-clone [dlm] Started: [ pcmk-1 pcmk-2 ] Clone Set: ClusterIP-clone [ClusterIP] (unique) ClusterIP:0 (ocf::heartbeat:IPaddr2): Started pcmk-1 ClusterIP:1 (ocf::heartbeat:IPaddr2): Started pcmk-2 Clone Set: WebFS-clone [WebFS] Started: [ pcmk-1 pcmk-2 ] Clone Set: WebSite-clone [WebSite] Started: [ pcmk-1 pcmk-2 ] impi-fencing (stonith:fence_ipmilan): Started
<cib admin_epoch="0" cib-last-written="Fri Sep 14 13:43:13 2012" crm_feature_set="3.0.6" dc-uuid="1" epoch="47" have-quorum="1" num_updates="50" update-client="cibadmin" update-origin="pcmk-1" validate-with="pacemaker-1.2"> <configuration> <crm_config> <cluster_property_set id="cib-bootstrap-options"> <nvpair id="cib-bootstrap-options-dc-version" name="dc-version" value="1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0"/> <nvpair id="cib-bootstrap-options-cluster-infrastructure" name="cluster-infrastructure" value="corosync"/> <nvpair id="cib-bootstrap-options-no-quorum-policy" name="no-quorum-policy" value="ignore"/> <nvpair id="cib-bootstrap-options-stonith-enabled" name="stonith-enabled" value="true"/> </cluster_property_set> </crm_config> <nodes> <node id="1" type="normal" uname="pcmk-1"/> <node id="2" type="normal" uname="pcmk-2"/> </nodes> <resources> <master id="WebDataClone"> <primitive class="ocf" id="WebData" provider="linbit" type="drbd"> <instance_attributes id="WebData-instance_attributes"> <nvpair id="WebData-instance_attributes-drbd_resource" name="drbd_resource" value="wwwdata"/> </instance_attributes> <operations> <op id="WebData-interval-60s" interval="60s" name="monitor"/> </operations> </primitive> <meta_attributes id="WebDataClone-meta_attributes"> <nvpair id="WebDataClone-meta_attributes-master-node-max" name="master-node-max" value="1"/> <nvpair id="WebDataClone-meta_attributes-clone-max" name="clone-max" value="2"/> <nvpair id="WebDataClone-meta_attributes-clone-node-max" name="clone-node-max" value="1"/> <nvpair id="WebDataClone-meta_attributes-notify" name="notify" value="true"/> <nvpair id="WebDataClone-meta_attributes-master-max" name="master-max" value="2"/> </meta_attributes> </master> <clone id="dlm-clone"> <primitive class="ocf" id="dlm" provider="pacemaker" type="controld"> <instance_attributes id="dlm-instance_attributes"/> <operations> <op id="dlm-interval-60s" interval="60s" name="monitor"/> </operations> </primitive> <meta_attributes id="dlm-clone-meta"> <nvpair id="dlm-clone-max" name="clone-max" value="2"/> <nvpair id="dlm-clone-node-max" name="clone-node-max" value="1"/> </meta_attributes> </clone> <clone id="ClusterIP-clone"> <primitive class="ocf" id="ClusterIP" provider="heartbeat" type="IPaddr2"> <instance_attributes id="ClusterIP-instance_attributes"> <nvpair id="ClusterIP-instance_attributes-ip" name="ip" value="192.168.0.120"/> <nvpair id="ClusterIP-instance_attributes-cidr_netmask" name="cidr_netmask" value="32"/> <nvpair id="ClusterIP-instance_attributes-clusterip_hash" name="clusterip_hash" value="sourceip"/> </instance_attributes> <operations> <op id="ClusterIP-interval-30s" interval="30s" name="monitor"/> </operations> </primitive> <meta_attributes id="ClusterIP-clone-meta"> <nvpair id="ClusterIP-globally-unique" name="globally-unique" value="true"/> <nvpair id="ClusterIP-clone-max" name="clone-max" value="2"/> <nvpair id="ClusterIP-clone-node-max" name="clone-node-max" value="2"/> </meta_attributes> </clone> <clone id="WebFS-clone"> <primitive class="ocf" id="WebFS" provider="heartbeat" type="Filesystem"> <instance_attributes id="WebFS-instance_attributes"> <nvpair id="WebFS-instance_attributes-device" name="device" value="/dev/drbd/by-res/wwwdata"/> <nvpair id="WebFS-instance_attributes-directory" name="directory" value="/var/www/html"/> <nvpair id="WebFS-instance_attributes-fstype" name="fstype" value="gfs2"/> </instance_attributes> <meta_attributes id="WebFS-meta_attributes"/> </primitive> <meta_attributes id="WebFS-clone-meta"/> </clone> <clone id="WebSite-clone"> <primitive class="ocf" id="WebSite" provider="heartbeat" type="apache"> <instance_attributes id="WebSite-instance_attributes"> <nvpair id="WebSite-instance_attributes-configfile" name="configfile" value="/etc/httpd/conf/httpd.conf"/> <nvpair id="WebSite-instance_attributes-statusurl" name="statusurl" value="http://localhost/server-status"/> </instance_attributes> <operations> <op id="WebSite-interval-1min" interval="1min" name="monitor"/> </operations> </primitive> <meta_attributes id="WebSite-clone-meta"/> </clone> <primitive class="stonith" id="impi-fencing" type="fence_ipmilan"> <instance_attributes id="impi-fencing-instance_attributes"> <nvpair id="impi-fencing-instance_attributes-pcmk_host_list" name="pcmk_host_list" value="pcmk-1 pcmk-2"/> <nvpair id="impi-fencing-instance_attributes-ipaddr" name="ipaddr" value="10.0.0.1"/> <nvpair id="impi-fencing-instance_attributes-login" name="login" value="testuser"/> <nvpair id="impi-fencing-instance_attributes-passwd" name="passwd" value="acd123"/> </instance_attributes> <operations> <op id="impi-fencing-interval-60s" interval="60s" name="monitor"/> </operations> </primitive> </resources> <constraints> <rsc_colocation id="colocation-WebSite-ClusterIP-INFINITY" rsc="WebSite-clone" score="INFINITY" with-rsc="ClusterIP-clone"/> <rsc_order first="ClusterIP-clone" first-action="start" id="order-ClusterIP-WebSite-mandatory" then="WebSite-clone" then-action="start"/> <rsc_colocation id="colocation-WebFS-WebDataClone-INFINITY" rsc="WebFS-clone" score="INFINITY" with-rsc="WebDataClone" with-rsc-role="Master"/> <rsc_colocation id="colocation-WebSite-WebFS-INFINITY" rsc="WebSite-clone" score="INFINITY" with-rsc="WebFS-clone"/> <rsc_order first="WebFS-clone" id="order-WebFS-WebSite-mandatory" then="WebSite-clone"/> <rsc_order first="WebDataClone" first-action="promote" id="order-WebDataClone-WebFS-mandatory" then="WebFS-clone" then-action="start"/> </constraints> <rsc_defaults> <meta_attributes id="rsc_defaults-options"> <nvpair id="rsc_defaults-options-resource-stickiness" name="resource-stickiness" value="100"/> </meta_attributes> </rsc_defaults> <op_defaults> <meta_attributes id="op_defaults-options"> <nvpair id="op_defaults-options-timeout" name="timeout" value="240s"/> </meta_attributes> </op_defaults> </configuration> </cib>
Pacemaker Nodes: Online: [ pcmk-1 pcmk-2 ]
# pcs property dc-version: 1.1.8-1.el7-60a19ed12fdb4d5c6a6b6767f52e5391e447fec0 cluster-infrastructure: corosync no-quorum-policy: ignore stonith-enabled: true
# pcs resource rsc defaults resource-stickiness: 100
# pcs stonith show impi-fencing (stonith:fence_ipmilan) Started # pcs stonith show impi-fencing Resource: impi-fencing pcmk_host_list: pcmk-1 pcmk-2 ipaddr: 10.0.0.1 login: testuser passwd: acd123
# pcs resource show ClusterIP-clone Resource: ClusterIP-clone ip: 192.168.0.120 cidr_netmask: 32 clusterip_hash: sourceip globally-unique: true clone-max: 2 clone-node-max: 2 op monitor interval=30s
Note
# pcs resource show WebDataClone Resource: WebDataClone drbd_resource: wwwdata master-node-max: 1 clone-max: 2 clone-node-max: 1 notify: true master-max: 2 op monitor interval=60s # pcs constraint ref WebDataClone Resource: WebDataClone colocation-WebFS-WebDataClone-INFINITY order-WebDataClone-WebFS-mandatory
# pcs resource show WebFS-clone Resource: WebFS-clone device: /dev/drbd/by-res/wwwdata directory: /var/www/html fstype: gfs2 # pcs constraint ref WebFS-clone Resource: WebFS-clone colocation-WebFS-WebDataClone-INFINITY colocation-WebSite-WebFS-INFINITY order-WebFS-WebSite-mandatory order-WebDataClone-WebFS-mandatory
# pcs resource show WebSite-clone Resource: WebSite-clone configfile: /etc/httpd/conf/httpd.conf statusurl: http://localhost/server-status master-max: 2 op monitor interval=1min # pcs constraint ref WebSite-clone Resource: WebSite-clone colocation-WebSite-ClusterIP-INFINITY colocation-WebSite-WebFS-INFINITY order-ClusterIP-WebSite-mandatory order-WebFS-WebSite-mandatory
# Please read the corosync.conf.5 manual page totem { version: 2 secauth: off cluster_name: mycluster transport: udpu } nodelist { node { ring0_addr: pcmk-1 nodeid: 1 } node { ring0_addr: pcmk-2 nodeid: 2 } } quorum { provider: corosync_votequorum } logging { to_syslog: yes }
Revision History | |||
---|---|---|---|
Revision 1-0 | Mon May 17 2010 | ||
| |||
Revision 2-0 | Wed Sep 22 2010 | ||
| |||
Revision 3-0 | Wed Feb 9 2011 | ||
| |||
Revision 4-0 | Wed Oct 5 2011 | ||
| |||
Revision 5-0 | Fri Feb 10 2012 | ||
| |||
Revision 6-0 | Tues July 3 2012 | ||
| |||
Revision 7-0 | Fri Sept 14 2012 | ||
|