Master storage
From ISPWiki
Contents |
Prerequisite
Disk partition for DRBD
Before we proceed with installation of a master storage server, we should decide whether the cluster will contain a slave storage server. If you are not going to use one, you may jump this step.
DRBD is a block device that allows to create a RAID1-array through the network. Each DRBD storage connects a virtual device /dev/drbdX (Х is a device number) with a local disk partition. Information is stored on the /dev/drbdX device of the master storage server that passes these data to a real disk partition and sends it to a backup storage. If the master storage fails, a backup one will be applied automatically.
To create a DRBD-based storage you need to create two additional partitions on the hard drive for
- meta data (see [1]);
- storage.
Customizing internal network
If the second network interface is not present on you current server, jump this step.
Open the /etc/network/interfaces file
# nano /etc/network/interfaces
and add the lines
auto ethX
iface ethX inet static
address 10.0.0.2
netmask 255.0.0.0
where X is a network interface number.
Run the command below to apply the cahnges:
# /etc/init.d/networking restart
Setting up and configuring disk quotas
Install a user utility that is necessary for disk quotas:
# apt-get install quota
If you do not use DRBD, open the /etc/fstab file
# nano /etc/fstab
and add the grpquota and usrquota options to the storage partition
... /dev/sdaX / ext4 errors=remount-ro,grpquota,usrquota 0 1 ...
where X is a disk partition number.
Mount the partition
# mount -o remount,grpquota,usrquota /dev/sdaX
Installing and customizing libnss-extrausers
Install an additional module for the NSS service that provides the system with user and group information from additional files: /var/lib/extrausers/passwd, /var/lib/extrausers/shadow and /var/lib/extrausers/groups.
# apt-get install libnss-extrausers
Open /etc/nsswitch.conf
# nano /etc/nsswitch.conf
and change the passwd, group and shadow options
... passwd: compat extrausers group: compat extrausers shadow: compat extrausers ...
Installing and customizing NFS-server
Run the command below to install the NFS-server:
# apt-get install nfs-kernel-server
Open /etc/default/nfs-common
# nano /etc/default/nfs-common
and change the NEED_STATD and NEED_IDMAPD options
... NEED_STATD="yes" ... NEED_IDMAPD="yes" ...
Open /etc/idmapd.conf
# nano /etc/idmapd.conf
and change the Domain options in the General section and Method in the Translation section
[General] Domain = cluster.lan ... [Translation] Method = nsswitch ...
Open /etc/exports
# nano /etc/exports
and add the line
/storage 10.0.0.0/8(rw,async,secure,no_root_squash,no_all_squash,no_subtree_check,fsid=root,no_acl)
Restart the NFS-server with the command:
# /etc/init.d/nfs-common restart
Installing and customizing DRBD
If you do not use the DRBD, jump this step.
Install the modules and utilities for DRBD support:
# apt-get install drbd8-utils drbd8-modules-`uname -r`
Open /etc/drbd.conf
# nano /etc/drbd.conf
and change as follows
common {
protocol C;
syncer {
rate 20M;
}
startup {
wfc-timeout 1;
}
}
resource storage {
on H {
address 10.0.0.2:7802;
disk /dev/sdbX;
flexible-meta-disk /dev/sdbY;
device /dev/drbdZ;
}
on localhost {
address 127.0.0.1:65535;
disk /dev/sdbX;
flexible-meta-disk /dev/sdbY;
device /dev/drbdZ;
}
}
where H is local host name, X is a storage device number, Y is a meta data device number, Z is a DRBD device number.
Boot the DRBD module:
# modprobe drbd
Purge the meta data storage device:
# dd if=/dev/zero of=/dev/sdbY bs=1M
where Y is a meta data device.
Create a DRBD device:
# drbdadm -- --force create-md storage
Connect the DRBD device:
# drbdadm up storage
Move the device to the master storage:
# drbdadm -- --overwrite-data-of-peer primary storage
Create a file system:
# mkfs.X /dev/drbdY
where X is a file system and Y is a DRBD device number.
Installing and customizing Heartbeat
If you do not use DRBD, jump this step.
Install heartbeat with the command:
# apt-get install heartbeat
Create and open the /etc/ha.d/authkeys file
# touch /etc/ha.d/authkeys && nano /etc/ha.d/authkeys
and add the following lines
auth 1 1 md5 X
where X is a secret phrase for authentication.
Set access privileges to /etc/ha.d/authkeys:
# chmod 0600 /etc/ha.d/authkeys
Create and open the /etc/ha.d/ha.cf file
# touch /etc/ha.d/ha.cf && nano /etc/ha.d/ha.cf
and add the following line
logfile /var/log/ha-log keepalive 1 warntime 2 deadtime 10 initdead 30 bcast ethX ucast ethX 10.0.0.2 auto_failback off autojoin none node H fake stonith_host H external/fake
where X is a network interface number and H is a local host name.
Create and open the /usr/lib/stonith/plugins/external/fake file
# touch /usr/lib/stonith/plugins/external/fake && nano /usr/lib/stonith/plugins/external/fake
and add the following lines
#!/bin/sh exit 0
Set access privileges to /usr/lib/stonith/plugins/external/fake:
# chmod +x /usr/lib/stonith/plugins/external/fake
Create and open the /etc/ha.d/haresources file
# touch /etc/ha.d/haresources && nano /etc/ha.d/haresources
add the following line
H IPaddr2::10.0.0.1/32/ethX drbddisk::storage Filesystem::/dev/drbdY::/storage::Z::usrquota,grpquota nfs-kernel-server
where H is a local host name, X is a network interface number, Y is the DRBD device number and Z is the DRBD file system.
Note that the 10.0.0.1 IP-address is used rather than 10.0.0.2. This is a "floating" IP-address that will be automatically assigned to a slave storage server, when the master one fails. When installing a Master-node and an HTTP-backend, You will need to specify a floating IP-address.
Restart heartbeat:
# /etc/init.d/heartbeat restart
Installing a base system into the storage
Install the debootstrap utility:
# apt-get install debootstrap
Verify disk quotas:
# quotacheck -ugm /storage
Install the GNU/Linux Debian "Lenny" base system into the storage:
# debootstrap lenny /storage
Add symlimks to the /etc/passwd and /etc/group files into /var/lib/extrausers/:
# ln -s /storage/etc/passwd /storage/etc/group /var/lib/extrausers/
Create and open the /storage/etc/mtab file
# touch /storage/etc/mtab && nano /storage/etc/mtab
and add the line
/storage / X rw,usrquota,grpquota 0 0
where X id the DRBD file system.
Create and open the /storage/etc/adduser.conf file
# cp /storage/usr/share/adduser/adduser.conf /storage/etc/ && nano /storage/etc/adduser.conf
and change the FIRST_SYSTEM_UID и FIRST_SYSTEM_GID options:
... FIRST_SYSTEM_UID=501 ... FIRST_SYSTEM_GID=501 ...
