In this guide I will go through the process of upgrading a NetApp cluster’s Data OnTap, RLM, disk and shelf firmware in a non-disruptive manner.
The following process is for a FAS3040 cluster, but it should work on other series.
Environment:
FAS3040 cluster
OS: DOT 8.0.3P2 7-mode
shelves:
– DS14MK2 (both FC and SATA)
– DS4243 (both SAS and SATA)
Information gathering
Do a sysconfig -v and check for the following:
...
System Storage Configuration: Multi-Path HA /// This tells you that your system is multipathed from a controller to shelf perspective
...
Remote LAN Module Status: Online
Part Number: 110-XXXXX
Revision: XX
Serial Number: XXXXX
Firmware Version: 4.0 // It is very important to use the latest RLM/SP version (this is your out of band access to the system)
Mgmt MAC Address: XXXXXXXXX
Ethernet Link: up
Using DHCP: no
...
...
60: NETAPP X267_HKURO500SSX AB0A 423.1GB (976642092 512B/sect) // Check your disk firmware (AB0A)
61: NETAPP X267_HKURO500SSX AB0A 423.1GB (976642092 512B/sect)
Shelf 1: AT-FCX Firmware rev. AT-FCX A: 38 AT-FCX B: 38 // Check your module version: frimware (AT-FCX A: 38) for FC-connnected shelves
Shelf 2: AT-FCX Firmware rev. AT-FCX A: 38 AT-FCX B: 38
...
...
11.22: NETAPP X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect) // Check your disk firmware (NA01)
11.23: NETAPP X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect)
Shelf 0: IOM3 Firmware rev. IOM3 A: 0132 IOM3 B: 0132 // Check your module version: firmware (IOM3 A: 0132) for SAS-connnected shelves
Shelf 10: IOM3 Firmware rev. IOM3 A: 0132 IOM3 B: 0132
...
...
Usually when I perform an upgrade of OnTap, I take the opportunity (or it may be a requirement) to update disk and shelf firmware.
You need to get the disk, shelf and RLM/SP firmware from netapp’s site support.netapp.com
Steps:
1) Upgrade your RML/SP
Download the latest RLM/SP (4.1) from: https://support.netapp.com/NOW/download/tools/rlm_fw/
Check your RLM/SP version (in this case it is RLM)
toaster> rlm status Remote LAN Module Status: Online Part Number: 110-xxx Revision: xx Serial Number: xxxxx Firmware Version: 4.0 Mgmt MAC Address: xxxxxxxxxxxxx Ethernet Link: up Using DHCP: no IPv4 configuration: IP Address: xxxxxxxxx Netmask: xxxxxxxxx Gateway: xxxxxxxxx
Place the RLM_FW.zip on the NetApp controller, under $etc/software, then:
toaster> software list .. RLM_FW.zip ... toaster> software install RLM_FW.zip toaster> priv set advanced toaster*> rlm update -f Note: You must enter the -f option. ...The update takes approximately 30 minutes. ...
When the system prompts you to reboot the RLM, enter y to continue.
Verify:
toaster> rlm status Remote LAN Module Status: Online Part Number: 110-xxx Revision: xx Serial Number: xxxxx Firmware Version: 4.1 Mgmt MAC Address: xxxxxxxxxxxxx Ethernet Link: up Using DHCP: no IPv4 configuration: IP Address: xxxxxxxxx Netmask: xxxxxxxxx Gateway: xxxxxxxxx
2) Upgrade your disk firmware for all the disks that are outdated(do this the night before the DOT upgrade)
To do the disk FW upgrade on the background, check the following is enabled:
toaster> options raid.background_disk_fw_update.enable
From the ‘sysconfig -v‘:
11.22: NETAPP X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect)
Disk X308_HMARK03TSSM with firmware NA01 needs to be upgraded to NA04
Download the latest firmware from: http://support.netapp.com/NOW/download/tools/diskfw/
Place the .LOD file under $etc/disk_fw
You will see that they will start upgrading on the background non-disruptively
3) Upgrade your shelf firmware (same day as DOT upgrade)
Download the latest firmware from: https://support.netapp.com/NOW/download/tools/diskshelf/
Copy the .SFW file and the .FVF file if present to the $etc/shelf_fw and .AFW and its .FVF file to the $etc/acpp_fw directory.
4) Upgrade OnTap
Download ontap from NetApp’s site- in this case 8.1.2
Check its md5 checksum against what netapp posts on their download page to make sure you image is good.
Since we are doing a NDU(non-disruptive-upgrade), please make sure one system can handle your load
sysstat -c 10 -x 3 // Check the CPU to make sure it does not go over 50%
toaster> sysstat -c 10 -x 3
CPU NFS CIFS HTTP Total Net kB/s Disk kB/s Tape kB/s Cache Cache CP CP Disk OTHER FCP iSCSI FCP kB/s iSCSI kB/s
in out read write read write age hit time ty util in out in out
5% 0 0 0 85 0 0 803 11 0 0 26 92% 5% Tf 7% 0 58 0 283 831 0 0
4% 0 0 0 101 0 0 1147 3140 0 0 26 94% 17% : 10% 0 101 0 449 996 0 0
4% 1 0 0 105 0 0 576 11 0 0 26 87% 0% - 6% 0 104 0 315 140 0 0
3% 1 0 0 59 0 0 371 8 0 0 26 91% 0% - 7% 0 58 0 379 844 0 0
6% 0 0 0 111 0 0 2383 4595 0 0 1 93% 37% T 10% 1 83 0 260 28 0 0
3% 0 0 0 36 0 0 349 8 0 0 1 91% 0% - 8% 0 36 0 149 902 0 0
4% 1 0 0 38 0 0 480 11 0 0 1 90% 0% - 16% 0 37 0 312 853 0 0
4% 1 0 0 98 0 0 379 11 0 0 1 92% 0% - 7% 0 70 0 347 1107 0 0
5% 0 0 0 65 0 0 1483 3224 0 0 1 95% 24% T 12% 0 65 0 334 897 0 0
4% 0 0 0 77 0 0 349 11 0 0 1 86% 0% - 6% 0 77 0 235 33 0 0
On both NetApp controllers:
Download the system files for 8.1.2 (812_q_image.tgz) from the Support Site. Be sure to download the system files that match your node model.
If you are performing a Data ONTAP NDU (or backout), you must perform this step on both nodes before performing the takeover and giveback steps.
Copy 812_q_image.tgz to $etc/software
Make sure that it is there:
toaster> software list ... 812_q_image.tgz ...
Let NetApp know you are starting the NDU upgrade:
toaster> options autosupport.doit "Staring_NDU 8.1.2"
Start the upgrade (-r prevents automatic reboot)
toaster> software update 812_q_image.tgz -r software: You can cancel this operation by hitting Ctrl-C in the next 6 seconds. software: Depending on system load, it may take many minutes software: to complete this operation. Until it finishes, you will software: not be able to use the console. cmd = ngsh -c system image update -node local -package file://localhost/mroot/etc/software/812_q_image.tgz -setdefault true ... ... Installed MD5 checksums pass Installing diagnostics and firmware files Installation complete. image1 updated on node TOASTER image1 has been set as the default Tue Febsoftware: installation of 812_q_image 26 11:.tgz completed. Please type "reboot" for the changes to take effect. // DO NOT TYPE REBOOT, WE WILL TAKEOVER
Check the version
toaster> version -b /cfcard/x86_64/freebsd/image1/kernel: OS 8.1.2 /cfcard/x86_64/freebsd/image2/kernel: OS 8.0.3 ... ...
Now, use this opportunity to update the shelf firmware
toaster> storage download shelf Downloading disk shelf firmware may take up to 10 minutes, but will NOT disrupt client access during that time. Are you sure you want to continue with shelf firmware update? yes ... ... helf]: Firmware file IOM3.0152.SFW Tue Feb 26 11:06:11 EST [toaster: sdownloafu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM3.0152.SFW downloaded on 2c.shelf0. Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSummary:info]: Shelf firmware updated on 3 shelves. ded on 2c.shelf11. Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM3.0152.SFW downloaded on 2c.shelf10. Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM3.0152.SFW downloaded on 2c.shelf0. Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSummary:info]: Shelf firmware updated on 3 shelves. // You are done toaster>
Perform the same process on the other NetApp controller
toaster2> software update 812_q_image.tgz -r .. toaster2> version -b .. toaster2> storage download shelf [storage download shelf]: No shelves eligible for update // You already did this on the other controller, this is to verify
Now that both controllers have the 8.1.2 DOT version, it is time for takeover in a NDU manner, which will reboot the controller
From controller1 (toaster)
toaster> cf status Cluster enabled, toaster2 is up.
toaster> cf takeover .. .. toaster(takeover)>
You should wait about 10 minutes before giving back to give the clients an opportunity to stabilize.
On the other controller, you will see (after a reboot)
Waiting for giveback...(Press Ctrl-C to abort wait)
…After 10 minutes
toaster> cf giveback ... ...
Check the second controller(toaster2) to ensure that it is running 8.1.2
toaster2> version toaster2> sysconfig
Wait about 10 minutes, then from toaster2 takeover toaster
toaster2> cf takeover -n // The option -n allows takeover when the onTap versions are incompatible, in this case 8.0.3 and 8.1.2 cf: ignoring version mismatch as part of NDU takeover cf: takeover initiated by operator ... ...
You will see on toaster
Waiting for giveback...(Press Ctrl-C to abort wait)
Now is time to giveback services
On toaster2:
toaster2> cf giveback ... ...
Check the controller to ensure that it is running 8.1.2
toaster2> version toaster2> sysconfig
Let NetApp know you are done:
toaster> options autosupport.doit "finishing_NDU 8.1.2"
That is it, RLM, disk fw, shelf fw and DOT were upgraded in a non-disruptive manner. You can check by running ‘sysconfig -v’
Comments
Leave a comment Trackback