DELL EMC AX4-5 vault drive upgrade

Ah, the DELL rebranded EMC Clariion AX4 (AX4-5SC, AX4-5, etc). It's awesome, but man do those Vault drives (and DELL/EMC support) make it so unfriendly.I’ll skip most of the background, and just get right to it.

First and foremost: This is an account of my experiences. Your experience will probably differ, and you’ll probably lose all of your data. Try this at your own risk.

We picked up a couple of these devices through an acquisition and they simply didn’t have drives large enough for our intended use. Upgrading the non-Vault drives is simple… pull out, plug in. Of course, you have to use EMC formatted drives (non-EMC SAS/SATA/FC drives will NOT work), which are pretty hard to procure from EMC directly without a valid support contract. It turns out that DELL is actually pretty well aligned to get these. Some of our other suppliers had month+ waits, but DELL had them to us in a few days.

Now, the Vault drives: The Vault drives are the first 4 drives in the AX4-5 (0,1,2,3). You can identify them by the little yellow stickers that warn you not to move the drives from their location. And you shouldn’t. The Vault drives contain OS, config and other information. If you pull these drives from the system, it will not fully boot (you can still connect via serial). If these drives fail in a certain sequence, the same fate will ensue. Simply put, you need them to be healthy.

If you do happen to trash the Vault drives for any reason, you can buy the 4-pack formatted and pre-loaded through DELL/EMC, but there is a premium, and finding someone who can actually get the part numbers is difficult at best.

Upgrading the Vault drives is generally a taboo subject for the end user, and I don’t condone it either. I’m sure it voids your warranty, lights Churches on fire, and brings Justin Bieber to haunt you in your sleep… but if you’re not using them in a production environment, or you buy them second hand on eBay, what the heck, right? I’m in.

This is not about upgrading Vault drives by using new Vault drives, as I don’t have any experience with that. When we put in a new 4-pack of Vault drives, we just swap them entirely and destroy any arrays/pools that are already created because we generally are putting in larger disks and want to be able to utilize the entire space.

EMC Powerlink is EMC’s customer portal. http://emc.powerlink.com

You can sign up for a free account, register your AX4-5, and then access firmware, downloads, support, etc.


Anyway, upgrading the Vault drives by using non-Vault drives:

1)     The AX4-5 should already be fully initialized. If it’s not, there is a tool that comes with the system to do this. It can also be downloaded from the EMC Powerlink site. Navisphere Storage System Initialization Utility.

2)     I’m 99% sure that all of the Vault drives have to use the same interface as each other, and you probably want to match the old Vault drives too. If you have SAS vault drives, use new SAS drives. SATA = SATA.

3)     The size of the new drive has to be greater than or equal to the old Vault drive size.

4)     DISCONNECT all hosts attached to the device via Fiber or iSCSI.

5)     Boot the system and bring everything online.

6)     Your array should not have any warnings (no orange lights). For instance, one time when I did this procedure, Storage Processor B was turned off. When I was done, I couldn’t get Storage Processor B to reinitiate no matter what I tried, and we got all kinds of “software cannot talk to storage processor” type errors.

7)     If available, upgrade the system firmware. Can be downloaded from EMC Powerlink site. Follow all directions and wait for system upgrade to finish completely.

8)     Pull out the Vault drive marked #3. The light by the drive may turn a solid orange at this point. Insert the new drive to be used as a Vault drive.

9)     The lights will flash blue (and possibly orange). Let the lights do their thing. At this point, the drive is replicating data from the remaining Vault drives. In Navisphere, you will see the new drive show up as “transitioning”. When the blue lights have stopped flashing completely or Navisphere shows the drive status as healthy, it’s done. Repeat step 8-9 for the Vault drive marked #2, then #1, then #0.

If the orange light next to the drive comes on solid, or flashes orange/blue every second for more than a few minutes, something is wrong. Put the old Vault drive back in.

10)  If you must abort the process, perform steps 8-9 backwards by pulling out the new drive and putting in the original drive. One drive at a time, allow time to complete.

11)  After you have swapped out the last Vault drive, all lights should be blue, and all components should be operational.

12)  If you decide to put all of the original Vault drives back in at a later date, you have to repeat the process all over again using the original Vault drives. You cannot just swap out all of the drives at one time.

Good luck,

Craig