CentOS是RedHat的社區(qū)編譯版，所以也就采用了RedHat ES 中的集群套件：RHCS。在RHCS架構(gòu)中，F(xiàn)ence設(shè)備十分重要，關(guān)系到故障恢復(fù)和數(shù)據(jù)一致性。在這里，我們使用服務(wù)器上主板提供的IPMI設(shè)備來(lái)實(shí)現(xiàn)Fence的功能。
IPMI是一種遠(yuǎn)程查詢和控制服務(wù)器基本狀態(tài)的協(xié)議，目前的主流服務(wù)器產(chǎn)品基本都提供了這種協(xié)議的實(shí)現(xiàn)。在Linux中可以使用OpenIPMI軟件包來(lái)設(shè)置和使用IPMI。
啟動(dòng)IPMI服務(wù):
[zarra@node1 ~]$ sudo service ipmi start
Starting ipmi drivers: [確定]
設(shè)置IPMI接口網(wǎng)絡(luò)地址、掩碼、默認(rèn)網(wǎng)關(guān):
[zarra@node1 ~]$sudo ipmitool lan set 1 ipsrc static
[zarra@node1 ~]$sudo ipmitool lan set 1 ipaddr 10.0.0.10
[zarra@node1 ~]$sudo ipmitool lan set 1 netmask 255.255.255.0
[zarra@node1 ~]$sudo ipmitool lan set 1 arp respond on
[zarra@node1 ~]$sudo ipmitool lan set 1 arp gernerate on
[zarra@node1 ~]$sudo ipmitool lan set 1 arp interval 5
設(shè)置IPMI用戶和口令:
[zarra@node1 ~]$sudo ipmitool lan set 1 user
[zarra@node1 ~]$sudo ipmitool lan set 1 access on
[zarra@node1 ~]$sudo ipmitool user list
ID Name Callin Link Auth IPMI Msg Channel Priv Limit
2 root true true true ADMINISTRATOR
[zarra@node1 ~]$sudo ipmitool user set password 2
Password for user 2: Enter your password
Password for user 2: Enter your password
設(shè)置好IPMI地址和用戶后，可以使用命令來(lái)檢查服務(wù)器的基本狀態(tài)，例如:
[zarra@node1 ~]$sudo ipmitool -H 10.0.0.1 -I lan -U root -P password power status
IPMI的設(shè)置基本就是如上過(guò)程，兩臺(tái)服務(wù)其的IPMI 具體參數(shù)如下:
[zarra@node1 ~]$ sudo ipmitool lan print 1
Set in Progress : Set Complete
Auth Type Support : NONE MD2 MD5 PASSWORD
Auth Type Enable : Callback : MD2 MD5
: User : MD2 MD5
: Operator : MD2 MD5
: Admin : MD2 MD5
: OEM : MD2 MD5
IP Address Source : Static Address
IP Address : 10.0.0.10
Subnet Mask : 255.255.255.0
MAC Address : 00:22:19:d6:05:38
SNMP Community String : public
IP Header : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10
Default Gateway IP : 10.10.71.240
Default Gateway MAC : 00:00:00:00:00:00
Backup Gateway IP : 0.0.0.0
Backup Gateway MAC : 00:00:00:00:00:00
802.1q VLAN ID : Disabled
802.1q VLAN Priority : 0
Cipher Suite Priv Max : Not Available
[zarra@node2 ~]$ sudo ipmitool lan print 1
Set in Progress : Set Complete
Auth Type Support : NONE MD2 MD5 PASSWORD
Auth Type Enable : Callback : MD2 MD5
: User : MD2 MD5
: Operator : MD2 MD5
: Admin : MD2 MD5
: OEM : MD2 MD5
IP Address Source : Static Address
IP Address : 10.0.0.11
Subnet Mask : 255.255.255.0
MAC Address : 00:22:19:d5:f4:61
SNMP Community String : public
IP Header : TTL=0x40 Flags=0x40 Precedence=0x00 TOS=0x10
Default Gateway IP : 10.10.71.240
Default Gateway MAC : 00:00:00:00:00:00
Backup Gateway IP : 0.0.0.0
Backup Gateway MAC : 00:00:00:00:00:00
802.1q VLAN ID : Disabled
802.1q VLAN Priority : 0
Cipher Suite Priv Max : Not Available
修改兩節(jié)點(diǎn)服務(wù)器上的 /etc/hosts 文件,內(nèi)容如下:
[zarra@node1 ~]$ cat /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
10.0.0.1 node1.test.com node1
10.0.0.2 node2.test.com node2
10.10.71.42 cluster.test.com cluster
節(jié)點(diǎn)一服務(wù)器上的網(wǎng)絡(luò)設(shè)置:
[zarra@node1 ~]$ cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=localhost.localdomain
GATEWAY=10.10.71.240
[zarra@node1 ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express
DEVICE=eth0
BOOTPROTO=static
HWADDR=00:22:19:D6:05:36
ONBOOT=yes
NETMASK=255.255.255.0
IPADDR=10.0.0.1
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
[zarra@node1 ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth1
# Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express
DEVICE=eth1
BOOTPROTO=static
HWADDR=00:22:19:D6:05:37
ONBOOT=yes
IPADDR=10.10.71.40
NETMASK=255.255.255.0
TYPE=Ethernet
節(jié)點(diǎn)二服務(wù)器上的網(wǎng)絡(luò)設(shè)置:
[zarra@node2 ~]$ cat /etc/sysconfig/network
NETWORKING=yes
NETWORKING_IPV6=no
HOSTNAME=localhost.localdomain
GATEWAY=10.10.71.240
[zarra@node2 ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express
DEVICE=eth0
BOOTPROTO=none
HWADDR=00:22:19:D5:F4:5F
ONBOOT=yes
IPADDR=10.0.0.2
NETMASK=255.255.255.0
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
[zarra@node2 ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth1
# Broadcom Corporation NetXtreme BCM5721 Gigabit Ethernet PCI Express
DEVICE=eth1
BOOTPROTO=static
HWADDR=00:22:19:D5:F4:60
IPADDR=10.10.71.41
NETMASK=255.255.255.0
ONBOOT=yes
TYPE=Ethernet
如果在系統(tǒng)安裝是選擇了Clustering支持，那么就會(huì)自動(dòng)安裝RHCS集群套件。如果想手動(dòng)安裝，可以執(zhí)行如下命令:
[zarra@node1 ~]$ sudo yum groupinstall clustering
安裝完成后，檢查 ricci 服務(wù)狀態(tài)，應(yīng)該如下所示:
[zarra@node1 ~]$ chkconfig ricci --list
ricci 0:關(guān)閉 1:關(guān)閉 2:啟用 3:啟用 4:啟用 5:啟用 6:關(guān)閉
如果沒(méi)有設(shè)為啟動(dòng)，執(zhí)行如下命令:
[zarra@node1 ~]$ sudo chkconfig ricci on
然后啟動(dòng) ricci 服務(wù):
[zarra@node1 ~]$ sudo service ricci start
在node2 上執(zhí)行同樣的操作，以完成集群套件的安裝。
在 RedHat ES 5.0 系統(tǒng)中提供了Conga這一Web界面的集群設(shè)置工具，方便了集群設(shè)置。Conga工具最好是安裝在集群外的一臺(tái)服務(wù)器上。在準(zhǔn)備安裝Conga的服務(wù)器上執(zhí)行:
[zarra@localhost ~]$ sudo yum install luci
安裝完成后設(shè)置Luci的admin用戶口令，執(zhí)行命令:
[zarra@localhost ~]$ sudo luci_admin init
然后依據(jù)提示設(shè)置口令。完成后即可開啟luci服務(wù):
[zarra@localhost ~]$ sudo chkconfig luci on
[zarra@localhost ~]$ sudo service luci start
Starting luci: [確定]
Point your web browser to https://node1.test.com:8084 to access luci
依據(jù)提示登陸 https://node1.test.com:8084 進(jìn)行集群設(shè)置。
打開Web瀏覽器登陸Conga服務(wù)器，如圖所示：
進(jìn)入“Cluster”標(biāo)簽，點(diǎn)擊 “Create a New Cluster” 按鈕，進(jìn)行建立集群操作，如圖所示：
點(diǎn)擊 “Add a Failover Domain”按鈕，建立實(shí)效域，如圖
點(diǎn)擊 “Add a Resource” 按鈕，建立兩個(gè)IP地址資源，完成后Resource頁(yè)面如圖所示：
點(diǎn)擊 “Add a Service”按鈕，建立iptables服務(wù)，完成后應(yīng)如圖所示：
依次配置各節(jié)點(diǎn)服務(wù)器的Fence設(shè)備，完成后如圖所示：
Node1
Node2
設(shè)置完成后 cluster.conf 應(yīng)該如下所示:
[zarra@node1 ~]$ sudo cat /etc/cluster/cluster.conf
<?xml version="1.0"?>
<cluster alias="cluster_test" config_version="17" name="cluster_test">
<fence_daemon clean_start="0" post_fail_delay="0" post_join_delay="3"/>
<clusternodes>
<clusternode name="node1.test.com" nodeid="1" votes="1">
<fence>
<method name="1">
<device name="node1_ipmi"/>
</method>
</fence>
</clusternode>
<clusternode name="node2.test.com" nodeid="2" votes="1">
<fence>
<method name="1">
<device name="node2_ipmi"/>
</method>
</fence>
</clusternode>
</clusternodes>
<cman expected_votes="1" two_node="1"/>
<fencedevices>
<fencedevice agent="fence_ipmilan" ipaddr="10.0.0.11" login="root" name="node2_ipmi" passwd="adaptor"/>
<fencedevice agent="fence_ipmilan" ipaddr="10.0.0.10" login="root" name="node1_ipmi" passwd="adaptor"/>
</fencedevices>
<rm>
<failoverdomains>
<failoverdomain name="firewall" nofailback="0" ordered="1" restricted="1">
<failoverdomainnode name="node1.test.com" priority="1"/>
<failoverdomainnode name="node2.test.com" priority="10"/>
</failoverdomain>
</failoverdomains>
<resources>
<ip address="10.10.71.42" monitor_link="1"/>
<ip address="10.0.0.4" monitor_link="1"/>
</resources>
<service autostart="1" domain="firewall" exclusive="0" name="iptables" recovery="relocate">
<ip ref="10.10.71.42"/>
<ip ref="10.0.0.4"/>
</service>
</rm>
</cluster>