In this guide I will go through the process of upgrading a NetApp cluster’s Data OnTap, RLM, disk and shelf firmware in a non-disruptive manner.

The following process is for a FAS3040 cluster, but it should work on other series.

Environment:
FAS3040 cluster
OS: DOT 8.0.3P2 7-mode
shelves:
– DS14MK2 (both FC and SATA)
– DS4243 (both SAS and SATA)

Information gathering
Do a sysconfig -v and check for the following:

...
System Storage Configuration: Multi-Path HA   /// This tells you that your system is multipathed from a controller to shelf perspective
...
Remote LAN Module           Status: Online
		Part Number:        110-XXXXX
		Revision:           XX
		Serial Number:      XXXXX
		Firmware Version:   4.0       // It is very important to use the latest RLM/SP version (this is your out of band access to the system)
		Mgmt MAC Address:   XXXXXXXXX
		Ethernet Link:      up
		Using DHCP:         no
...
...
                60: NETAPP   X267_HKURO500SSX AB0A 423.1GB (976642092 512B/sect) // Check your disk firmware (AB0A)
                61: NETAPP   X267_HKURO500SSX AB0A 423.1GB (976642092 512B/sect) 
                Shelf 1: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38     // Check your module version: frimware (AT-FCX A: 38) for FC-connnected shelves
		Shelf 2: AT-FCX  Firmware rev. AT-FCX A: 38  AT-FCX B: 38
...
...
                11.22: NETAPP   X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect) // Check your disk firmware (NA01)
                11.23: NETAPP   X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect)
		Shelf   0: IOM3  Firmware rev. IOM3 A: 0132 IOM3 B: 0132      // Check your module version: firmware (IOM3 A: 0132) for SAS-connnected shelves
		Shelf  10: IOM3  Firmware rev. IOM3 A: 0132 IOM3 B: 0132

...
...

Usually when I perform an upgrade of OnTap, I take the opportunity (or it may be a requirement) to update disk and shelf firmware.
You need to get the disk, shelf and RLM/SP firmware from netapp’s site support.netapp.com

Steps:
1) Upgrade your RML/SP
Download the latest RLM/SP (4.1) from: https://support.netapp.com/NOW/download/tools/rlm_fw/

Check your RLM/SP version (in this case it is RLM)

toaster> rlm status
	Remote LAN Module           Status: Online
		Part Number:        110-xxx
		Revision:           xx
		Serial Number:      xxxxx
		Firmware Version:   4.0
		Mgmt MAC Address:   xxxxxxxxxxxxx
		Ethernet Link:      up
		Using DHCP:         no
	IPv4 configuration:
		IP Address:         xxxxxxxxx
		Netmask:            xxxxxxxxx
		Gateway:            xxxxxxxxx

Place the RLM_FW.zip on the NetApp controller, under $etc/software, then:

toaster> software list
..
RLM_FW.zip
...

toaster> software install RLM_FW.zip

toaster> priv set advanced

toaster*> rlm update -f

Note: You must enter the -f option.
...The update takes approximately 30 minutes.
...

When the system prompts you to reboot the RLM, enter y to continue.

Verify:

toaster> rlm status
	Remote LAN Module           Status: Online
		Part Number:        110-xxx
		Revision:           xx
		Serial Number:      xxxxx
		Firmware Version:   4.1
		Mgmt MAC Address:   xxxxxxxxxxxxx
		Ethernet Link:      up
		Using DHCP:         no
	IPv4 configuration:
		IP Address:         xxxxxxxxx
		Netmask:            xxxxxxxxx
		Gateway:            xxxxxxxxx

2) Upgrade your disk firmware for all the disks that are outdated(do this the night before the DOT upgrade)
To do the disk FW upgrade on the background, check the following is enabled:

toaster> options raid.background_disk_fw_update.enable

From the ‘sysconfig -v‘:
11.22: NETAPP X308_HMARK03TSSM NA01 2538.5GB (5860533168 512B/sect)
Disk X308_HMARK03TSSM with firmware NA01 needs to be upgraded to NA04

Download the latest firmware from: http://support.netapp.com/NOW/download/tools/diskfw/
Place the .LOD file under $etc/disk_fw

You will see that they will start upgrading on the background non-disruptively

3) Upgrade your shelf firmware (same day as DOT upgrade)

Download the latest firmware from: https://support.netapp.com/NOW/download/tools/diskshelf/
Copy the .SFW file and the .FVF file if present to the $etc/shelf_fw and .AFW and its .FVF file to the $etc/acpp_fw directory.

4) Upgrade OnTap
Download ontap from NetApp’s site- in this case 8.1.2
Check its md5 checksum against what netapp posts on their download page to make sure you image is good.

Since we are doing a NDU(non-disruptive-upgrade), please make sure one system can handle your load

sysstat -c 10 -x 3  // Check the CPU to make sure it does not go over 50%
toaster> sysstat -c 10 -x 3
 CPU    NFS   CIFS   HTTP   Total     Net   kB/s    Disk   kB/s    Tape   kB/s  Cache  Cache    CP  CP  Disk   OTHER    FCP  iSCSI     FCP   kB/s   iSCSI   kB/s
                                       in    out    read  write    read  write    age    hit  time  ty  util                            in    out      in    out
  5%      0      0      0      85       0      0     803     11       0      0    26     92%    5%  Tf    7%       0     58      0     283    831       0      0
  4%      0      0      0     101       0      0    1147   3140       0      0    26     94%   17%  :    10%       0    101      0     449    996       0      0
  4%      1      0      0     105       0      0     576     11       0      0    26     87%    0%  -     6%       0    104      0     315    140       0      0
  3%      1      0      0      59       0      0     371      8       0      0    26     91%    0%  -     7%       0     58      0     379    844       0      0
  6%      0      0      0     111       0      0    2383   4595       0      0     1     93%   37%  T    10%       1     83      0     260     28       0      0
  3%      0      0      0      36       0      0     349      8       0      0     1     91%    0%  -     8%       0     36      0     149    902       0      0
  4%      1      0      0      38       0      0     480     11       0      0     1     90%    0%  -    16%       0     37      0     312    853       0      0
  4%      1      0      0      98       0      0     379     11       0      0     1     92%    0%  -     7%       0     70      0     347   1107       0      0
  5%      0      0      0      65       0      0    1483   3224       0      0     1     95%   24%  T    12%       0     65      0     334    897       0      0
  4%      0      0      0      77       0      0     349     11       0      0     1     86%    0%  -     6%       0     77      0     235     33       0      0

On both NetApp controllers:
Download the system files for 8.1.2 (812_q_image.tgz) from the Support Site. Be sure to download the system files that match your node model.
If you are performing a Data ONTAP NDU (or backout), you must perform this step on both nodes before performing the takeover and giveback steps.

Copy 812_q_image.tgz to $etc/software

Make sure that it is there:

toaster> software list
...
812_q_image.tgz
...

Let NetApp know you are starting the NDU upgrade:

toaster> options autosupport.doit "Staring_NDU 8.1.2"

Start the upgrade (-r prevents automatic reboot)

toaster> software update 812_q_image.tgz -r
software: You can cancel this operation by hitting Ctrl-C in the next 6 seconds.
software: Depending on system load, it may take many minutes
software: to complete this operation. Until it finishes, you will
software: not be able to use the console.
cmd = ngsh -c system image update -node local -package file://localhost/mroot/etc/software/812_q_image.tgz -setdefault true
...
...
Installed MD5 checksums pass
Installing diagnostics and firmware files
Installation complete. image1 updated on node TOASTER

image1 has been set as the default

Tue Febsoftware: installation of 812_q_image 26 11:.tgz completed.
Please type "reboot" for the changes to take effect.     // DO NOT TYPE REBOOT, WE WILL TAKEOVER

Check the version

toaster> version -b
/cfcard/x86_64/freebsd/image1/kernel: OS 8.1.2
/cfcard/x86_64/freebsd/image2/kernel: OS 8.0.3
...
...

Now, use this opportunity to update the shelf firmware

toaster> storage download shelf
Downloading disk shelf firmware may take up to 10 minutes,
but will NOT disrupt client access during that time.

Are you sure you want to continue with shelf firmware update? yes
...
...
helf]: Firmware file IOM3.0152.SFW Tue Feb 26 11:06:11 EST [toaster: sdownloafu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM3.0152.SFW downloaded on 2c.shelf0.
Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSummary:info]: Shelf firmware updated on 3 shelves.
ded on 2c.shelf11.
Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM3.0152.SFW downloaded on 2c.shelf10.
Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSuccess:info]: [storage download shelf]: Firmware file IOM3.0152.SFW downloaded on 2c.shelf0.
Tue Feb 26 11:06:11 EST [toaster: sfu.downloadSummary:info]: Shelf firmware updated on 3 shelves.  // You are done
toaster> 

Perform the same process on the other NetApp controller

toaster2> software update 812_q_image.tgz -r
..
toaster2> version -b
..
toaster2> storage download shelf
[storage download shelf]: No shelves eligible for update   // You already did this on the other controller, this is to verify

Now that both controllers have the 8.1.2 DOT version, it is time for takeover in a NDU manner, which will reboot the controller

From controller1 (toaster)

toaster> cf status
Cluster enabled, toaster2 is up.
toaster> cf takeover
..
..
toaster(takeover)>

You should wait about 10 minutes before giving back to give the clients an opportunity to stabilize.
On the other controller, you will see (after a reboot)

Waiting for giveback...(Press Ctrl-C to abort wait)

…After 10 minutes

toaster> cf giveback
...
...

Check the second controller(toaster2) to ensure that it is running 8.1.2

toaster2> version
toaster2> sysconfig

Wait about 10 minutes, then from toaster2 takeover toaster

toaster2> cf takeover -n   // The option -n allows takeover when the onTap versions are incompatible, in this case 8.0.3 and 8.1.2
cf: ignoring version mismatch as part of NDU takeover
cf: takeover initiated by operator
...
...

You will see on toaster

Waiting for giveback...(Press Ctrl-C to abort wait)

Now is time to giveback services
On toaster2:

toaster2> cf giveback
...
...

Check the controller to ensure that it is running 8.1.2

toaster2> version
toaster2> sysconfig

Let NetApp know you are done:

toaster> options autosupport.doit "finishing_NDU 8.1.2"

That is it, RLM, disk fw, shelf fw and DOT were upgraded in a non-disruptive manner. You can check by running ‘sysconfig -v’