This posting will help you configuring multipathing on RHEL 5.3 for LUNs carved from a NetApp SAN. For this guide I am using a C-Class blade system with QLogic HBA cards.
1) Make sure you have the packages needed by RHEL, otherwise install them.
|
1 2 3 4 |
rpm -q device-mapper rpm -q device-mappermultipath yum install device-mapper yum install device-mapper-multipath |
2) Install QLogic Drivers if needed, or utilize RHEL drivers. In my case I am using HP C-Class blades with Qlogic HBA cards. HP drivers can be found at the HP site, driver is called hp_sansurfer. I am utilizing RHEL built in drivers, but you can install the HP/QLogic drivers as follows:
|
1 |
rpm -Uvh hp_sansurfer-5.0.1b45-1.x86_64.rpm |
3) If Qlogic HBA, install the SanSurfer CLI, this is very useful program for doing things with QLogic HBA cards, it can be downloaded at QLogic website, install as follows:
|
1 |
rpm -Uvh scli-1.7.3-14.i386.rpm |
4) Install NetApp Host Utilities Kit, the package is a tar.gz file, you can find it at the now site http://now.netapp.com.
Open it and run the install shell script
|
1 |
netapp_linux_host_utilities_5_0.tar.gz |
5) Once Everything is installed on the host, create the LUN and ZONE it from the NetApp, Brocade(SAN Fabric),Host
|
1 2 3 |
To find your WWPNs, use the scli as follows: # scli –i all // Use the WWPN numbers for the iGroup and Brocade Aliases |
6) Once it has been Zoned and mapped correctly, verify if your RHEL host can see it.
|
1 2 3 4 5 6 7 |
// Rescan HBA for new SAN Luns # modprobe -r qla2xxx # modprobe qla2xxx // Check the kernel can see it # cat /proc/scsi/scsi # fdisk –lu |
7) Utilize NetApp tools to see LUN connectivity
|
1 2 3 4 5 6 7 8 9 10 |
// Check your host and utilities see the LUNs [root@server ~]# sanlun lun show controller: lun-pathname device filename adapter protocol lun size lun state NETAPPFILER: /vol/servervol/serverlun /dev/sdf host6 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sda host4 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sde host6 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sdc host5 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sdd host5 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sdb host4 FCP 100g (107374182 400) GOOD . |
8 ) Utilize NetApp tools to check multipathing, not set yet
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
[root@server ~]# sanlun lun show -p NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled Protocol: FCP Controller Partner: NETAPPFILER2 Multipath-provider: NONE --------- ---------- ------- ------------ --------------------------------------------- --------------- sanlun Controller Primary Partner path Path /dev/ Host Controller Controller state type node HBA port port --------- ---------- ------- ------------ --------------------------------------------- --------------- GOOD primary sdf host6 0c -- GOOD secondary sda host4 -- 0c GOOD secondary sde host6 -- 0c GOOD secondary sdc host5 -- 0d GOOD primary sdd host5 0d -- GOOD primary sdb host4 0c -- |
Time to configure multipathing
9) Start the multipath daemon
|
1 |
# service multipathd start |
10) Find you WWID, this will be needed in the configuration if you want to alias it.
Comment out the blacklist in the default /etc/multipath.conf, otherwise you will NOT see anything.
|
1 2 3 |
#blacklist { # devnode "*" #} |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
// Show your devices and paths, and record the WWID of the LUN # multipath -v3 ... ... ===== paths list ===== uuid hcil dev dev_t pri dm_st chk_st vend/pr 360a98000486e576748345276376a4d41 4:0:0:0 sda 8:0 1 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 4:0:1:0 sdb 8:16 4 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 5:0:0:0 sdc 8:32 1 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 5:0:1:0 sdd 8:48 4 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 6:0:0:0 sde 8:64 1 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 6:0:1:0 sdf 8:80 4 [undef][ready] NETAPP, ... ... |
11) Now you are ready to configure /etc/multipath.conf
Exclude (blacklist) all the devices that do not correspond to any
LUNs configured on the storage controller and which are mapped to
your Linux host. There are 2 methods:
Block by WWID
Block by devnode
In this case I am blocking by devnode since I am using HP and know my devnode RegEx
Also configure the device and alias(optional).
The full /etc/multipath.conf will look like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
defaults { user_friendly_names yes max_fds max queue_without_daemon no } blacklist { ###devnode "*" devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z]" devnode "^cciss!c[0-9]d[0-9]*" # Note the cciss, usual in HP } multipaths { multipath { wwid 360a98000486e57674834527533455570 # You found this alias netapp # This is how you want to name the device in your host # server LUN on NETAPPFILER } } devices { device { vendor "NETAPP" product "LUN" getuid_callout "/sbin/scsi_id -g -u -s /block/%n" prio_callout "/sbin/mpath_prio_ontap /dev/%n" features "1 queue_if_no_path" hardware_handler "0" path_grouping_policy group_by_prio failback immediate rr_weight uniform rr_min_io 128 path_checker directio flush_on_last_del yes } } |
12) Restart multipath and make sure it starts automatically:
|
1 2 3 4 5 |
// Restart multipath # service multipathd restart // Add to startup # chkconfig --add multipathd # chkconfig multipathd on |
13) Verify multipath is working
|
1 2 3 4 5 6 7 8 9 10 11 12 |
<em><strong><span style="color: #000000;">//RHEL tools </span></strong></em> [root@server scli]# multipath -l netapp (360a98000486e576748345276376a4d41) dm-2 NETAPP,LUN [size=100G][features=1 queue_if_no_path][hwhandler=0][rw] \_ round-robin 0 [prio=0][active] \_ 4:0:1:0 sdb 8:16 [active][undef] \_ 5:0:1:0 sdd 8:48 [active][undef] \_ 6:0:1:0 sdf 8:80 [active][undef] \_ round-robin 0 [prio=0][enabled] \_ 4:0:0:0 sda 8:0 [active][undef] \_ 5:0:0:0 sdc 8:32 [active][undef] \_ 6:0:0:0 sde 8:64 [active][undef] |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
<em><strong><span style="color: #000000;">//NetApp utilities Tool </span></strong></em> [root@server scli]# sanlun lun show -p NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled Protocol: FCP Controller Partner: NETAPPFILER2 DM-MP DevName: netapp (360a98000486e576748345276376a4d41) dm-2 Multipath-provider: NATIVE --------- ---------- ------- ------------ --------------------------------------------- --------------- sanlun Controller Primary Partner state type node HBA port port --------- ---------- ------- ------------ --------------------------------------------- --------------- GOOD primary sdb host4 0c -- GOOD primary sdd host5 0d -- GOOD primary sdf host6 0c -- GOOD secondary sda host4 -- 0c GOOD secondary sdc host5 -- 0d GOOD secondary sde host6 -- 0c ... |
14) Now you can access the LUN by using the mapper
|
1 2 3 4 5 6 |
[root@server scli]# ls -l /dev/mapper total 0 crw------- 1 root root 10, 63 Sep 12 12:32 control brw-rw---- 1 root disk 253, 2 Sep 16 10:54 netapp brw-rw---- 1 root disk 253, 0 Sep 12 16:32 VolGroup00-LogVol00 brw-rw---- 1 root disk 253, 1 Sep 12 12:32 VolGroup00-LogVol01 |
15) Format it to your liking and mount it
|
1 2 3 4 5 6 7 8 9 |
# mkdir /mnt/netapp # mkfs -t ext3 /dev/mapper/netapp # mount /dev/mapper/netapp /mnt/netapp/ //verify it mounted # mount ... ... /dev/mapper/netapp on /mnt/netapp type ext3 (rw) ... |
16 ) If you want it to be persistent after reboots put it on /etc/fstab and make sure multipathd start automatically.
|
1 2 3 4 |
# cat /etc/fstab ... ... /dev/mapper/netapp /mnt/netapp ext3 defaults 0 0 |
17) If possible reboot to check it mounts correctly after reboots.
