2009年11月27日 星期五

Lustre 1.8.1.1 安裝在 ASUS EeeBox

Lustre的原碼和RPM皆可於此下載
RPM的方式則需要以下檔案(這邊是X86的示範):
  • kernel-lustre-2.6.18-128.7.1.el5_lustre.1.8.1.1.20091003130007.i686.rpm
  • kernel-ib-1.4.2-2.6.18_128.7.1.el5.i686.rpm
  • kernel-ib-1.4.2-2.6.18_128.7.1.el5_lustre.1.8.1.1.20091003130007.i686.rpm
  • lustre-modules-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1.i686.rpm
  • lustre-ldiskfs-3.0.9-2.6.18_128.7.1.el5_lustre.1.8.1.1.i686.rpm
  • lustre-1.8.1.1-2.6.18_128.7.1.el5_lustre.1.8.1.1.i686.rpm
  • e2fsprogs-1.41.6.sun1-0redhat.rhel5.i386.rpm
安裝順序如同上面的列表,如果出現警告則可rpm -Uvh luste-xxx.rpm --force強制安裝,在Red Hat 5.2上安裝必須注意的地方有兩點:
  1. SELINUX必須關閉
    vi /etc/sysconfig/selinux
    #SELINUX=enforcing
    SELINUX=disabled
    # SELINUXTYPE= type of policy in use. Possible values are:
    # targeted - Only targeted network daemons are protected.
    # strict - Full SELinux protection.
    #SELINUXTYPE=targeted
  2. 關閉防火牆
    [root@mds01 ~]# /etc/init.d/iptables stop
檢查lustre mod是否掛載成功
[root@mds01 ~]# modprobe -v lustre
[root@mds01 ~]# lustre_rmmod
[root@mds01 ~]# modprobe -v lustre

Lustre中的OST和MDS一定需要安裝以上的元件,client懶得找需要的元件就照做吧XD。

在EeeBox上安裝rhel 5.2所使用的預設網卡驅動為r8169,其實是有問題的,可用 lspci -v |grep Eth來檢查網卡型號,安裝適合的網卡驅動,這邊的例子為r8168,編譯網卡驅動程式步驟:
rpm -ivh kernel-lustre-devel-2.6.18-128.7.1.el5_lustre.1.8.1.1.20091003130007.i686.rpm
tar xvf r8168-8.014.00.tar(自行下載對應的驅動)
cd r8168-8.014.00
make clean modules
make install
depmod -a
insmod ./src/r8168.ko
vi /etc/modprobe.conf
alias eth0 r8168(將網卡相對應到適合的驅動模組)
echo "blacklist r8169" >> /etc/modprobe.d/blacklist-network(停用r8169驅動模組)

可用iperf來測試網路能力與壓力測試。

接下來是Lustre的配置:
  1. MDS configure
    [root@mds01 dev]# mkfs.lustre --fsname=test --mdt --mgs --device-size=100000 /tmp/mdt
    Permanent disk data:
    Target: lustre-MDTffff
    Index: unassigned
    Lustre FS: lustre
    Mount type: ldiskfs
    Flags: 0x75
    (MDT MGS needs_index first_time update )
    Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
    Parameters: mdt.group_upcall=/usr/sbin/l_getgroups

    checking for existing Lustre data: not found
    2 6 18
    formatting backing filesystem ldiskfs on /dev/loop0
    target name lustre-MDTffff
    4k blocks 25000
    options -i 4096 -I 512 -q -O dir_index,extents,uninit_groups -F
    mkfs_cmd = mke2fs -j -b 4096 -L lustre-MDTffff -i 4096 -I 512 -q -O dir_index,extents,uninit_groups -F /dev/loop0 25000
    Writing CONFIGS/mountdata
    [root@mds01 dev]# mkdir /mnt/mdt
    [root@mds01 dev]# mount -t lustre -o loop /tmp/mdt /mnt/mdt
    [root@mds01 dev]# df -h
    Filesystem Size Used Avail Use% Mounted on
    /dev/mapper/VolGroup00-LogVol00
    141G 1.5G 132G 2% /
    /dev/sda1 99M 24M 70M 26% /boot
    tmpfs 1010M 0 1010M 0% /dev/shm
    /dev/loop0 86M 4.2M 77M 6% /mnt/mdt
  2. OST configure
    [root@ost02 ~]# fdisk -l
    Disk /dev/sda: 160.0 GB, 160041885696 bytes
    255 heads, 63 sectors/track, 19457 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot Start End Blocks Id System
    /dev/sda1 * 1 258 2072353+ 83 Linux
    /dev/sda2 259 2869 20972857+ 83 Linux
    /dev/sda3 19201 19457 2064352+ 82 Linux swap / Solaris
    /dev/sda4 2870 19200 131178757+ 83 Linux

    Partition table entries are not in disk order
    [root@ost02 ~]# mkfs.lustre --ost --reformat --fsname=test --mgsnode=140.134.21.147@tcp /dev/sda4
    Permanent disk data:
    Target: lustre-OSTffff
    Index: unassigned
    Lustre FS: lustre
    Mount type: ldiskfs
    Flags: 0x72
    (OST needs_index first_time update )
    Persistent mount opts: errors=remount-ro,extents,mballoc
    Parameters: mgsnode=140.134.21.147@tcp

    device size = 128104MB
    2 6 18
    formatting backing filesystem ldiskfs on /dev/sda4
    target name lustre-OSTffff
    4k blocks 0
    options -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F
    mkfs_cmd = mke2fs -j -b 4096 -L lustre-OSTffff -J size=400 -i 16384 -I 256 -q -O dir_index,extents,uninit_groups -F /dev/sda4
    Writing CONFIGS/mountdata
    [root@ost02 ~]# mkdir -p /mnt/ost2
    [root@ost02 ~]# mount -t lustre /dev/sda4 /mnt/ost2

  3. client mount
    [root@mds02 lustre]# mkdir -p /mnt/lustre
    [root@mds02 lustre]# mount -t lustre 140.134.21.147@tcp:/test /mnt/lustre
    [root@mds02 lustre]# df -h

沒有留言:

張貼留言