This posting will help you configuring multipathing on RHEL 5.3 for LUNs carved from a NetApp SAN. For this guide I am using a C-Class blade system with QLogic HBA cards.
1) Make sure you have the packages needed by RHEL, otherwise install them.
rpm -q device-mapper rpm -q device-mappermultipath yum install device-mapper yum install device-mapper-multipath
2) Install QLogic Drivers if needed, or utilize RHEL drivers. In my case I am using HP C-Class blades with Qlogic HBA cards. HP drivers can be found at the HP site, driver is called hp_sansurfer. I am utilizing RHEL built in drivers, but you can install the HP/QLogic drivers as follows:
rpm -Uvh hp_sansurfer-5.0.1b45-1.x86_64.rpm
3) If Qlogic HBA, install the SanSurfer CLI, this is very useful program for doing things with QLogic HBA cards, it can be downloaded at QLogic website, install as follows:
rpm -Uvh scli-1.7.3-14.i386.rpm
4) Install NetApp Host Utilities Kit, the package is a tar.gz file, you can find it at the now site http://now.netapp.com.
Open it and run the install shell script
netapp_linux_host_utilities_5_0.tar.gz
5) Once Everything is installed on the host, create the LUN and ZONE it from the NetApp, Brocade(SAN Fabric),Host
To find your WWPNs, use the scli as follows: # scli –i all // Use the WWPN numbers for the iGroup and Brocade Aliases
6) Once it has been Zoned and mapped correctly, verify if your RHEL host can see it.
// Rescan HBA for new SAN Luns # modprobe -r qla2xxx # modprobe qla2xxx // Check the kernel can see it # cat /proc/scsi/scsi # fdisk –lu
7) Utilize NetApp tools to see LUN connectivity
// Check your host and utilities see the LUNs [root@server ~]# sanlun lun show controller: lun-pathname device filename adapter protocol lun size lun state NETAPPFILER: /vol/servervol/serverlun /dev/sdf host6 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sda host4 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sde host6 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sdc host5 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sdd host5 FCP 100g (107374182 400) GOOD NETAPPFILER: /vol/servervol/serverlun /dev/sdb host4 FCP 100g (107374182 400) GOOD .
8 ) Utilize NetApp tools to check multipathing, not set yet
[root@server ~]# sanlun lun show -p
NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD
Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled
Protocol: FCP Controller Partner: NETAPPFILER2
Multipath-provider: NONE
--------- ---------- ------- ------------ --------------------------------------------- ---------------
sanlun Controller Primary Partner
path Path /dev/ Host Controller Controller
state type node HBA port port
--------- ---------- ------- ------------ --------------------------------------------- ---------------
GOOD primary sdf host6 0c --
GOOD secondary sda host4 -- 0c
GOOD secondary sde host6 -- 0c
GOOD secondary sdc host5 -- 0d
GOOD primary sdd host5 0d --
GOOD primary sdb host4 0c --
Time to configure multipathing
9) Start the multipath daemon
# service multipathd start
10) Find you WWID, this will be needed in the configuration if you want to alias it.
Comment out the blacklist in the default /etc/multipath.conf, otherwise you will NOT see anything.
#blacklist {
# devnode "*"
#}
// Show your devices and paths, and record the WWID of the LUN # multipath -v3 ... ... ===== paths list ===== uuid hcil dev dev_t pri dm_st chk_st vend/pr 360a98000486e576748345276376a4d41 4:0:0:0 sda 8:0 1 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 4:0:1:0 sdb 8:16 4 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 5:0:0:0 sdc 8:32 1 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 5:0:1:0 sdd 8:48 4 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 6:0:0:0 sde 8:64 1 [undef][ready] NETAPP, 360a98000486e576748345276376a4d41 6:0:1:0 sdf 8:80 4 [undef][ready] NETAPP, ... ...
11) Now you are ready to configure /etc/multipath.conf
Exclude (blacklist) all the devices that do not correspond to any
LUNs configured on the storage controller and which are mapped to
your Linux host. There are 2 methods:
Block by WWID
Block by devnode
In this case I am blocking by devnode since I am using HP and know my devnode RegEx
Also configure the device and alias(optional).
The full /etc/multipath.conf will look like this:
defaults
{
user_friendly_names yes
max_fds max
queue_without_daemon no
}
blacklist {
###devnode "*"
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^cciss!c[0-9]d[0-9]*" # Note the cciss, usual in HP
}
multipaths {
multipath {
wwid 360a98000486e57674834527533455570 # You found this
alias netapp # This is how you want to name the device in your host
# server LUN on NETAPPFILER
}
}
devices
{
device
{
vendor "NETAPP"
product "LUN"
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_ontap /dev/%n"
features "1 queue_if_no_path"
hardware_handler "0"
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
rr_min_io 128
path_checker directio
flush_on_last_del yes
}
}
12) Restart multipath and make sure it starts automatically:
// Restart multipath # service multipathd restart // Add to startup # chkconfig --add multipathd # chkconfig multipathd on
13) Verify multipath is working
//RHEL tools
[root@server scli]# multipath -l
netapp (360a98000486e576748345276376a4d41) dm-2 NETAPP,LUN
[size=100G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][active]
\_ 4:0:1:0 sdb 8:16 [active][undef]
\_ 5:0:1:0 sdd 8:48 [active][undef]
\_ 6:0:1:0 sdf 8:80 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:0:0 sda 8:0 [active][undef]
\_ 5:0:0:0 sdc 8:32 [active][undef]
\_ 6:0:0:0 sde 8:64 [active][undef]
//NetApp utilities Tool
[root@server scli]# sanlun lun show -p
NETAPPFILER:/vol/servervol/serverlun (LUN 0) Lun state: GOOD
Lun Size: 100g (107374182400) Controller_CF_State: Cluster Enabled
Protocol: FCP Controller Partner: NETAPPFILER2
DM-MP DevName: netapp (360a98000486e576748345276376a4d41) dm-2
Multipath-provider: NATIVE
--------- ---------- ------- ------------ --------------------------------------------- ---------------
sanlun Controller Primary Partner
state type node HBA port port
--------- ---------- ------- ------------ --------------------------------------------- ---------------
GOOD primary sdb host4 0c --
GOOD primary sdd host5 0d --
GOOD primary sdf host6 0c --
GOOD secondary sda host4 -- 0c
GOOD secondary sdc host5 -- 0d
GOOD secondary sde host6 -- 0c
...
14) Now you can access the LUN by using the mapper
[root@server scli]# ls -l /dev/mapper total 0 crw------- 1 root root 10, 63 Sep 12 12:32 control brw-rw---- 1 root disk 253, 2 Sep 16 10:54 netapp brw-rw---- 1 root disk 253, 0 Sep 12 16:32 VolGroup00-LogVol00 brw-rw---- 1 root disk 253, 1 Sep 12 12:32 VolGroup00-LogVol01
15) Format it to your liking and mount it
# mkdir /mnt/netapp # mkfs -t ext3 /dev/mapper/netapp # mount /dev/mapper/netapp /mnt/netapp/ //verify it mounted # mount ... ... /dev/mapper/netapp on /mnt/netapp type ext3 (rw) ...
16 ) If you want it to be persistent after reboots put it on /etc/fstab and make sure multipathd start automatically.
# cat /etc/fstab ... ... /dev/mapper/netapp /mnt/netapp ext3 defaults 0 0
17) If possible reboot to check it mounts correctly after reboots.
