GNBD installation

From Gfswiki

This procedure is for getting a basic GNBD GFS configuration running with three nodes ÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂÃÂ one as the server and two for clients. Before using this procedure, please make sure that you have successfully built and installed GFS on all of the hosts which are to be members of the GFS cluster.

Note: You can obtain significant performance benefits by partitioning your block devices and running pool on top of GNBD, so this configuration procedure assumes that you want to use pool on top of GNBD.

[edit]

Hardware Requirement

node which will be the GNBD server and the GFS IP-based lock server (running lock_gulmd). This node has a 32GB SCSI device labeled /dev/sdb which will be used for GFS data.

Normally you would want to run lock_gulmd on a node that is neither a GNBD server or one of the GFS clients. The main reasons for this are better performance and increased reliability. Unfortunately, since only three (3) hosts are used in this example, it is necessary to double up functionality on one of the nodes.

host-a 192.168.1.2

Two (2) GNBD client nodes which will mount the GFS file system.
- host-b 192.168.1.3
- host-c 192.168.1.4

One (1) 32GB SCSI device attached to host-a and recognized as sdb. This drive has five partitions; one is about four megabytes and the other four are equally sized and use the rest of the drive space.

sdb1 = 4MB sdb2, sdb3, sdb4 and sdb5 = 7.9 GB

The small partition will be used for the CIDEV. It can be larger, but 4 MB is all that is needed. The other four partitions must be of equal size, so that pool can stripe across them.

[edit]

Userland

cd cluster/magma
./configure --with-kernel=/usr/src/linux-2.6.7
make; make install

cd ../gnbd
./configure --with-kernel=/usr/src/linux-2.6.7
make; make install

[edit]

Setup Procedure

On host-a (the GNBD server), export the five sdb partitions as five GNBD's. The global names of the GNBD's are gfscfdata and gfsdata[1-4]. The first will be used for the CIDEV and the rest will be used for the GFS file system.

host-a$ modprobe gnbd
host-a$ gnbd_export -e gfscfdata -d /dev/sdb1
host-a$ gnbd_export -e gfsdata1 -d /dev/sdb2
host-a$ gnbd_export -e gfsdata2 -d /dev/sdb3
host-a$ gnbd_export -e gfsdata3 -d /dev/sdb4
host-a$ gnbd_export -e gfsdata4 -d /dev/sdb5

On host-b and host-c (the GNBD clients), import the five GNBDs. The gnbd_import program will import all GNBDs being exported from host-a.

host-b$ modprobe gnbd
host-b$ gnbd_import -i host-a
host-c$ modprobe gnbd
host-c$ gnbd_import -i host-a

The five GNBDs should now be accessible as:

/dev/gnbd/gfscfdata /dev/gnbd/gfsdata1
/dev/gnbd/gfsdata2 /dev/gnbd/gfsdata3
/dev/gnbd/gfsdata4

At this point, setting up Pool and GFS on these shared devices is no different than the other configuration procedures.

Load kernel modules on GFS gnbd client nodes host-b and host-c.

host-b$ modprobe pool
host-b$ modprobe lock_gulm
host-b$ modprobe gfs
host-c$ modprobe pool
host-c$ modprobe lock_gulm
host-c$ modprobe gfs

Create a pool for the file system. On host-b create a file named pool0.cf.

host-b$ cat > pool0.cf
poolname pool0
subpools 1
subpool 0 128 4 gfs_data
pooldevice 0 0 /dev/gnbd/gfsdata1 0
pooldevice 0 1 /dev/gnbd/gfsdata2 0
pooldevice 0 2 /dev/gnbd/gfsdata3 0
pooldevice 0 3 /dev/gnbd/gfsdata4 0
^D

Create pool0.

host-b$ pool_tool pool0.cf
Pool labels successfully written.

Load the pool on all nodes.

host-b$ pool_assemble
Added pool0.
host-c$ pool_assemble
Added pool0.

At this point, a pool device exists that is striped across the four gnbd partitions. This allows gnbd to deal with multiple requests at once, significantly increasing performance

Create the file system.

host-b$ gfs_mkfs -p lock_gulm -t gfsgnbd -j 2 /dev/pool/pool0
Device: /dev/pool/pool0
Blocksize: 4096
Filesystem Size: ...
Journals: 2
Resource Groups: ...
Locking Pr

Retrieved from "http://gfs.wikidev.net/GNBD_installation"

GNBD installation

From Gfswiki

Hardware Requirement

Userland

Setup Procedure

Views

Personal tools

Navigation

Search

Toolbox