Lustre 1.8.1.1 安裝在 ASUS EeeBox

Lustre的原碼和RPM皆可於此下載。
RPM的方式則需要以下檔案(這邊是X86的示範)：

• kernel-lustre-2.6.18-128.7.1.el5_lustre.1.8.1.1.20091003130007.i686.rpm
• kernel-ib-1.4.2-2.6.18_128.7.1.el5.i686.rpm
• kernel-ib-1.4.2-2.6.18_128.7.1.el5_lustre.1.8.1.1.20091003130007.i686.rpm
• lustre-modules-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1.i686.rpm
• lustre-ldiskfs-3.0.9-2.6.18_128.7.1.el5_lustre.1.8.1.1.i686.rpm
• lustre-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1.i686.rpm
• e2fsprogs-1.41.6.sun1-0redhat.rhel5.i386.rpm

安裝順序如同上面的列表，如果出現警告則可rpm -Uvh luste-xxx.rpm --force強制安裝，在Red Hat 5.2上安裝必須注意的地方有兩點：

1. SELINUX必須關閉
   vi /etc/sysconfig/selinux
   

   #SELINUX=enforcing
   SELINUX=disabled
   # SELINUXTYPE= type of policy in use. Possible values are:
   # targeted - Only targeted network daemons are protected.
   # strict - Full SELinux protection.
   #SELINUXTYPE=targeted

2. 關閉防火牆
   [root@mds01 ~]# /etc/init.d/iptables stop
   

檢查lustre mod是否掛載成功
[root@mds01 ~]# modprobe -v lustre
[root@mds01 ~]# lustre_rmmod
[root@mds01 ~]# modprobe -v lustre

Lustre中的OST和MDS一定需要安裝以上的元件，client懶得找需要的元件就照做吧XD。

在EeeBox上安裝rhel 5.2所使用的預設網卡驅動為r8169，其實是有問題的，可用 lspci -v |grep Eth來檢查網卡型號，安裝適合的網卡驅動，這邊的例子為r8168，編譯網卡驅動程式步驟：
rpm -ivh kernel-lustre-devel-2.6.18-128.7.1.el5_lustre.1.8.1.1.20091003130007.i686.rpm
tar xvf r8168-8.014.00.tar(自行下載對應的驅動)
cd r8168-8.014.00
make clean modules
make install
depmod -a
insmod ./src/r8168.ko
vi /etc/modprobe.conf

alias eth0 r8168(將網卡相對應到適合的驅動模組)

echo "blacklist r8169" >> /etc/modprobe.d/blacklist-network(停用r8169驅動模組)

可用iperf來測試網路能力與壓力測試。

接下來是Lustre的配置：

1. MDS configure
   [root@mds01 dev]# mkfs.lustre --fsname=test --mdt --mgs --device-size=100000 /tmp/mdt
   

   Permanent disk data:
   Target: lustre-MDTffff
   Index: unassigned
   Lustre FS: lustre
   Mount type: ldiskfs
   Flags: 0x75
   (MDT MGS needs_index first_time update )
   Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
   Parameters: mdt.group_upcall=/usr/sbin/l_getgroups
   
   checking for existing Lustre data: not found
   2 6 18
   formatting backing filesystem ldiskfs on /dev/loop0
   target name lustre-MDTffff
   4k blocks 25000
   options -i 4096 -I 512 -q -O dir_index,extents,uninit_groups -F
   mkfs_cmd = mke2fs -j -b 4096 -L lustre-MDTffff -i 4096 -I 512 -q -O dir_index,extents,uninit_groups -F /dev/loop0 25000
   Writing CONFIGS/mountdata

   [root@mds01 dev]# mkdir /mnt/mdt
   [root@mds01 dev]# mount -t lustre -o loop /tmp/mdt /mnt/mdt
   [root@mds01 dev]# df -h
   

   Filesystem Size Used Avail Use% Mounted on
   /dev/mapper/VolGroup00-LogVol00
   141G 1.5G 132G 2% /
   /dev/sda1 99M 24M 70M 26% /boot
   tmpfs 1010M 0 1010M 0% /dev/shm
   /dev/loop0 86M 4.2M 77M 6% /mnt/mdt

2. OST configure
   [root@ost02 ~]# fdisk -l
   

   Disk /dev/sda: 160.0 GB, 160041885696 bytes
   255 heads, 63 sectors/track, 19457 cylinders
   Units = cylinders of 16065 * 512 = 8225280 bytes
   
   Device Boot Start End Blocks Id System
   /dev/sda1 * 1 258 2072353+ 83 Linux
   /dev/sda2 259 2869 20972857+ 83 Linux
   /dev/sda3 19201 19457 2064352+ 82 Linux swap / Solaris
   /dev/sda4 2870 19200 131178757+ 83 Linux
   
   Partition table entries are not in disk order

   [root@ost02 ~]# mkfs.lustre --ost --reformat --fsname=test --mgsnode=140.134.21.147@tcp /dev/sda4
   

   Permanent disk data:
   Target: lustre-OSTffff
   Index: unassigned
   Lustre FS: lustre
   Mount type: ldiskfs
   Flags: 0x72
   (OST needs_index first_time update )
   Persistent mount opts: errors=remount-ro,extents,mballoc
   Parameters: mgsnode=140.134.21.147@tcp
   
   device size = 128104MB
   2 6 18
   formatting backing filesystem ldiskfs on /dev/sda4
   target name lustre-OSTffff
   4k blocks 0
   options -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F
   mkfs_cmd = mke2fs -j -b 4096 -L lustre-OSTffff -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F /dev/sda4
   Writing CONFIGS/mountdata

   [root@ost02 ~]# mkdir -p /mnt/ost2
   [root@ost02 ~]# mount -t lustre /dev/sda4 /mnt/ost2
   
   
3. client mount
   [root@mds02 lustre]# mkdir -p /mnt/lustre
   [root@mds02 lustre]# mount -t lustre 140.134.21.147@tcp:/test /mnt/lustre
   [root@mds02 lustre]# df -h