alexdawson.net

O frabjous day! Callooh! Callay! ONTAP 8.3 is out, and with it, the long promised demise of the dedicated root aggregate for lower end systems!

To re-cap – NetApp has always said – have a dedicated root aggregate. But until Clustered ONTAP, that was more of a recommendation, like, say, brush your teeth morning, noon and night. When you only have 24 drives in a system, throwing away 6 of them to boot the thing seems like a silly idea. The lower-end (FAS2xxx) systems represent a very large number of NetApp’s sales by controller count, and for these systems, Clustered ONTAP was not a great move because of it. With 8.3 being Clustered ONTAP only, there had to be a solution to this pretty serious and valid objection, and there is – Advanced Disk Partitioning (ADP).

What is ADP? Basically it’s partitioning drives, and being able to assign partitions to RAID groups and aggregates. Cool, right? Well, yes, mostly. ADP can be used on All-Flash-FAS (AFF), but that is out of scope for this post. There are some important things to be aware of for these lower end systems.

1. Systems using ADP need an ADP formatted spare, and then non-ADP spares for any other drives
2. ADP can only be used for internal drives on a FAS2[2,5]xx system
3. ADP drives can only be part of a RAID group of ADP drives
4. SSD’s can now be pooled between controllers!

If a system is only using the internal drives, chances are, it is going to be a smaller system, and most of these don’t matter. The issue comes when it is time to add a disk shelf. Consider the following ADP layout system, assuming one data aggregate per controller:

If we were to add a shelf of 24 disks, and split it evenly between controllers, we would need to do some thinking first. We can’t add it to the ADP RG, and we need a non-ADP spare, for each controller. With ADP, and our 42 (18+24) SAS drives (21 per controller), we have used them like this:

• N1_aggr0
• N1_aggr1_rg0 – 6 data, 2 parity
• N1_aggr1_rg1 – 9 data, 2 parity
• N1 ADP Spare – 1
• N1 Non ADP Spare – 1
• N2_aggr0
• N2_aggr1_rg0 – 6 data, 2 parity
• N2_aggr1_rg1 – 9 data, 2 parity
• N2 ADP Spare – 1
• N2 Non ADP Spare – 1

For a total of:

• 8 parity
• 4 spare
• 30 data

If we didn’t use ADP, we’d be using them like this:

• N1_aggr0 – 1 root, 2 parity
• N1_aggr1_rg0 – 15 data, 2 parity
• N1 Non ADP Spare – 1
• N2_aggr0 – 1 root, 2 parity
• N2_aggr1_rg0 – 15 data, 2 parity
• N2 Non ADP Spare – 1

For a total of:

• 8 parity
• 4 spare
• … annnd 30 data

I toyed with running the numbers on moving the SSD drives to the shelf, meaning we could have larger ADP partitions used in RAID groups, but that still bites you in the end, as you will end up with the same number of RAID groups, but less balanced sizes as more shelves are added.

If we move to 2 shelves – 66 (18+24+24) SAS drives, we could use them like this with ADP:

• N1_aggr0
• N1_aggr1_rg0 – 6 data, 2 parity
• N1_aggr1_rg1 – 9 data, 2 parity
• N1_aggr1_rg2 – 10 data, 2 parity
• N1 ADP Spare – 1
• N1 Non ADP Spare – 1
• N2_aggr0
• N2_aggr1_rg0 – 6 data, 2 parity
• N2_aggr1_rg1 – 9 data, 2 parity
• N2_aggr1_rg2 – 10 data, 2 parity
• N2 ADP Spare – 1
• N2 Non ADP Spare – 1

For a total of:

• 12 parity
• 4 spare
• 50 data

Or this without ADP:

• N1_aggr0 – 1 root, 2 parity
• N1_aggr1_rg0 – 15 data, 2 parity
• N1_aggr1_rg1 – 10 data, 2 parity
• N1 Non ADP Spare – 1
• N2_aggr0 – 1 root, 2 parity
• N2_aggr1_rg0 – 15 data, 2 parity
• N2_aggr1_rg1 – 10 data, 2 parity
• N2 Non ADP Spare – 1

For a total of:

• 12 parity
• 50 data
• 2 spare

At 3 shelves, the story changes.. 90 (18+24+24+24) SAS drives, we could use them like this with ADP:

• N1_aggr0
• N1_aggr1_rg0 – 6 data, 2 parity
• N1_aggr1_rg1 – 9 data, 2 parity
• N1_aggr1_rg2 – 10 data, 2 parity
• N1_aggr1_rg3 – 10 data, 2 parity
• N1 ADP Spare – 1
• N1 Non ADP Spare – 1
• N2_aggr0
• N2_aggr1_rg0 – 6 data, 2 parity
• N2_aggr1_rg1 – 9 data, 2 parity
• N2_aggr1_rg2 – 10 data, 2 parity
• N2_aggr1_rg3 – 10 data, 2 parity
• N2 ADP Spare – 1
• N2 Non ADP Spare – 1

For a total of:

• 16 parity
• 4 spare
• 70 data

Or this without ADP:

• N1_aggr0 – 1 root, 2 parity
• N1_aggr1_rg0 – 19 data, 2 parity
• N1_aggr1_rg1 – 18 data, 2 parity
• N1 Non ADP Spare – 1
• N2_aggr0 – 1 root, 2 parity
• N2_aggr1_rg0 – 19 data, 2 parity
• N2_aggr1_rg1 – 18 data, 2 parity
• N2 Non ADP Spare – 1

For a total of:

• 12 parity
• 74 data
• 2 spare

So, a couple of conclusions:

1. ADP is good for internal shelf only systems
2. ADP is neutral for 1 or 2 shelf systems
3. ADP is bad for 3+ shelf systems
4. ADP is awesome for Flashpools (not really a conclusion from this post, but trust me on it? 😉

As a footnote: savvy readers will notice I’ve got unequally sized RAID groups in some of these configs. With ONTAP 8.3, the Physical Storage Management Guide (page 107) now says:

All RAID groups in an aggregate should have a similar number of disks. The RAID groups do not have to be exactly the same size, but you should avoid having any RAID group that is less than one half the size of other RAID groups in the same aggregate when possible.

This is in comparison to ONTAP 8.2 Physical Storage Management Guide (page 91) which says:

All RAID groups in an aggregate should have the same number of disks. If this is impossible, any RAID group with fewer disks should have only one less disk than the largest RAID group.