Technologist

Tech stuff about Cloud, DevOps, SysAdmin, Virtualization, SAN, Hardware, Scripting, Automation and Development

Browsing Posts tagged netapp

In this guide I will go through the process of upgrading a NetApp cluster’s Data OnTap, RLM, disk and shelf firmware in a non-disruptive manner.

The following process is for a FAS3040 cluster, but it should work on other series.

Environment:
FAS3040 cluster
OS: DOT 8.0.3P2 7-mode
shelves:
– DS14MK2 (both FC and SATA)
– DS4243 (both SAS and SATA)

Information gathering
Do a sysconfig -v and check for the following:

Usually when I perform an upgrade of OnTap, I take the opportunity (or it may be a requirement) to update disk and shelf firmware.
You need to get the disk, shelf and RLM/SP firmware from netapp’s site support.netapp.com

Steps:
1) Upgrade your RML/SP
Download the latest RLM/SP (4.1) from: https://support.netapp.com/NOW/download/tools/rlm_fw/

Check your RLM/SP version (in this case it is RLM)

Place the RLM_FW.zip on the NetApp controller, under $etc/software, then:

When the system prompts you to reboot the RLM, enter y to continue.

Verify:

2) Upgrade your disk firmware for all the disks that are outdated(do this the night before the DOT upgrade)
To do the disk FW upgrade on the background, check the following is enabled:

toaster> options raid.background_disk_fw_update.enable

From the ‘sysconfig -v‘:
11.22: NETAPP X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect)
Disk X308_HMARK03TSSM with firmware NA01 needs to be upgraded to NA04

Download the latest firmware from: http://support.netapp.com/NOW/download/tools/diskfw/
Place the .LOD file under $etc/disk_fw

You will see that they will start upgrading on the background non-disruptively

3) Upgrade your shelf firmware (same day as DOT upgrade)

Download the latest firmware from: https://support.netapp.com/NOW/download/tools/diskshelf/
Copy the .SFW file and the .FVF file if present to the $etc/shelf_fw and .AFW and its .FVF file to the $etc/acpp_fw directory.

4) Upgrade OnTap
Download ontap from NetApp’s site- in this case 8.1.2
Check its md5 checksum against what netapp posts on their download page to make sure you image is good.

Since we are doing a NDU(non-disruptive-upgrade), please make sure one system can handle your load

On both NetApp controllers:
Download the system files for 8.1.2 (812_q_image.tgz) from the Support Site. Be sure to download the system files that match your node model.
If you are performing a Data ONTAP NDU (or backout), you must perform this step on both nodes before performing the takeover and giveback steps.

Copy 812_q_image.tgz to $etc/software

Make sure that it is there:

Let NetApp know you are starting the NDU upgrade:

Start the upgrade (-r prevents automatic reboot)

Check the version

Now, use this opportunity to update the shelf firmware

Perform the same process on the other NetApp controller

Now that both controllers have the 8.1.2 DOT version, it is time for takeover in a NDU manner, which will reboot the controller

From controller1 (toaster)

You should wait about 10 minutes before giving back to give the clients an opportunity to stabilize.
On the other controller, you will see (after a reboot)

…After 10 minutes

Check the second controller(toaster2) to ensure that it is running 8.1.2

Wait about 10 minutes, then from toaster2 takeover toaster

You will see on toaster

Now is time to giveback services
On toaster2:

Check the controller to ensure that it is running 8.1.2

Let NetApp know you are done:

That is it, RLM, disk fw, shelf fw and DOT were upgraded in a non-disruptive manner. You can check by running ‘sysconfig -v’

Many companies buy wildcard certificates for many reasons: price, management, flexibility, etc.

The following guide shows how to install a wildcard certificate from DigiCert on your NetApp controllers.

You will need the following 3 files in PEM format:
DigiCertCA.pem // This is the Certificate Authority, in this case from DigiCert
wildcard_example_com.pem // This is the wildcard certificate
wildcard_example_com_key.pem // This is the private key

1) Stop SSL on the NetApp controller
filer> secureadmin disable ssl

Now From a Linux/Unix system:

2) mount the NetApp’s vol0
LinuxStation# mkdir /mnt/filer
LinuxStation# mount filer.example.com:/vol/vol0 /mnt/filer

3) Go to the keymgr folder and backup the current certificate and key.

# Backup Certificate
LinuxStation# cd /mnt/filer/etc/keymgr/cert/
LinuxStation:/mnt/filer/etc/keymgr/cert/# mv secureadmin.pem secureadmin.pem.bak

# Backup Key
LinuxStation# cd /mnt/filer/etc/keymgr/key/
LinuxStation:/mnt/filer/etc/keymgr/key/# mv secureadmin.pem secureadmin.pem.bak

4) Create the new files based on the wildcard certificate files, assuming you placed them on /opt/certificates

# Create Certificate
LinuxStation# cd /opt/certificates/
LinuxStation:/opt/certificates/# cat wildcard_example_com.pem DigiCertCA.pem > secureadmin_cert.pem
LinusStation# mv /opt/certificates/secureadmin_cert.pem /mnt/filer/etc/keymgr/cert/secureadmin.pem

# Create Key
LinuxStation# cd /opt/certificates/
LinuxStation:/opt/certificates/# cat wildcard_example_com_key.pem > secureadmin_key.pem
LinusStation# mv /opt/certificates/secureadmin_key.pem /mnt/filer/etc/keymgr/key/secureadmin.pem

5) On the NetApp controller, add the new cert:
filer> secureadmin addcert ssl /etc/keymgr/cert/secureadmin.pem

6) Enable SSL
filer> secureadmin enable ssl

NetApp Appliances support Link Aggregation of their network interfaces, they call the Link Aggregation a VIF (Virtual Interface) and this provides Fault Tolerance, Load Balancing and higher throughput.

NetApp supports the following Link Aggregation modes:

From the NetApp documentation:
Single-mode vif
In a single-mode vif, only one of the interfaces in the vif is active. The other interfaces are on standby, ready to take over if the active interface fails.
Static multimode vif
The static multimode vif implementation in Data ONTAP is in compliance with IEEE 802.3ad (static). Any switch that supports aggregates, but does not have control packet exchange for configuring an aggregate, can be used with static multimode vifs.
Dynamic multimode vif
Dynamic multimode vifs can detect not only the loss of link status (as do static multimode vifs), but also a loss of data flow. This feature makes dynamic multimode vifs compatible with high-availability environments. The dynamic multimode vif implementation in Data ONTAP is in compliance with IEEE 802.3ad (dynamic), also known as Link Aggregation Control Protocol (LACP).

In this guide I will set up a Dynamic multimode vif between the NetApp system and the Cisco switches using LACP.

I am working with following hardware:

  • 2x NetApp FAS3040c in an active-active cluster
    With Dual 10G Ethernet Controller T320E-SFP+
  • 2x Cisco WS-C6509 configured as one Virtual Switch (using VSS)
    With Ten Gigabit Ethernet interfaces

Cisco Configuration:

Port-Channel(s) configuration:
// I am using Port-Channel 8 and 9 for this configuration
// And I need my filers to be in VLAN 10

!
interface Port-channel8
description LACP multimode VIF for filer1-10G
switchport
switchport access vlan 10
switchport mode access
!
interface Port-channel9
description LACP multimode VIF for filer2-10G
switchport
switchport access vlan 10
switchport mode access
!

Interface Configuration:
// Since I am using VSS, my 2 Cisco 6509 look like 1 Virtual Switch
// For example: interface TenGigabitEthernet 2/10/4 means:
// interface 4, on blade 10, on the second 6509

!
interface TenGigabitEthernet1/10/1
description “filer1_e1a_net 10G”
switchport access vlan 10
switchport mode access
channel-group 8 mode active
spanning-tree portfast
!
!
interface TenGigabitEthernet2/10/1
description “filer1_e1b_net 10G”
switchport access vlan 10
switchport mode access
channel-group 8 mode active
spanning-tree portfast
!
!
interface TenGigabitEthernet1/10/2
description “filer2_e1a_net 10G”
switchport access vlan 10
switchport mode access
channel-group 9 mode active
spanning-tree portfast
!
!
interface TenGigabitEthernet2/10/2
description “filer2_e1b_net 10G”
switchport access vlan 10
switchport mode access
channel-group 9 mode active
spanning-tree portfast
!

Check the Cisco configuration

NetApp Configuration:

filer1>vif create lacp net10G -b ip e1a e1b
filer1>ifconfig net10G 10.0.0.100 netmask 255.255.255.0
filer1>ifconfig net10G up

filer2>vif create lacp net10G -b ip e1a e1b
filer2>ifconfig net10G 10.0.0.200 netmask 255.255.255.0
filer2>ifconfig net10G up

Don’t forget to make the change persistant

Filer1:: /etc/rc
hostname FILER1
vif create lacp net10G -b ip e1b e1a
ifconfig net hostname-net mediatype auto netmask 255.255.255.0 partner net10G
route add default 10.0.0.1 1
routed on
options dns.domainname example.com
options dns.enable on
options nis.enable off
savecore

Filer2:: /etc/rc
hostname FILER2
vif create lacp net10G -b ip e1b e1a
ifconfig net hostname-net mediatype auto netmask 255.255.255.0 partner net10G
route add default 10.0.0.1 1
routed on
options dns.domainname example.com
options dns.enable on
options nis.enable off
savecore

Check the NetApp configuration

This posting will help you configuring multipathing on RHEL 5.3 for LUNs carved from a NetApp SAN. For this guide I am using a C-Class blade system with QLogic HBA cards.

1) Make sure you have the packages needed by RHEL, otherwise install them.

2) Install QLogic Drivers if needed, or utilize RHEL drivers. In my case I am using HP C-Class blades with Qlogic HBA cards. HP drivers can be found at the HP site, driver is called hp_sansurfer. I am utilizing RHEL built in drivers, but you can install the HP/QLogic drivers as follows:

3) If Qlogic HBA, install the SanSurfer CLI, this is very useful program for doing things with QLogic HBA cards, it can be downloaded at QLogic website, install as follows:

4) Install NetApp Host Utilities Kit, the package is a tar.gz file, you can find it at the now site http://now.netapp.com.

Open it and run the install shell script

5) Once Everything is installed on the host, create the LUN and ZONE it from the NetApp, Brocade(SAN Fabric),Host

6) Once it has been Zoned and mapped correctly, verify if your RHEL host can see it.

7) Utilize NetApp tools to see LUN connectivity

8 ) Utilize NetApp tools to check multipathing, not set yet

Time to configure multipathing

9) Start the multipath daemon

10) Find you WWID, this will be needed in the configuration if you want to alias it.

Comment out the blacklist in the default /etc/multipath.conf, otherwise you will NOT see anything.

11) Now you are ready to configure /etc/multipath.conf

Exclude (blacklist) all the devices that do not correspond to any
LUNs configured on the storage controller and which are mapped to
your Linux host. There are 2 methods:
Block by WWID
Block by devnode
In this case I am blocking by devnode since I am using HP and know my devnode RegEx
Also configure the device and alias(optional).
The full /etc/multipath.conf will look like this:

12) Restart multipath and make sure it starts automatically:

13) Verify multipath is working

14) Now you can access the LUN by using the mapper

15) Format it to your liking and mount it

16 ) If you want it to be persistent after reboots put it on /etc/fstab and make sure multipathd start automatically.

17) If possible reboot to check it mounts correctly after reboots.