ONTAP – Why and why not to have one LIF per NFS volume

LIFs, or logical interfaces, are the interfaces from outside world to the storage of a NetApp system. There is a many to one relationship of LIFs to ports. From the early days of Clustered ONTAP, NetApp has given advice to have one LIF per datastore on VMware. There are more general purpose use-cases for this as well.

But it’s not always worth it.

The justification for a 1:1 LIF to volume mapping has been to allow a volume to move between nodes, and to move the LIF to the other node, to avoid indirect access for longer than a few moments.

Indirect access is when IP traffic comes into one node (for example N1), while the volume is on another node (say N2 – but it could be on another HA pair in another cluster). This means the data is pulled off disk on N2, goes over the cluster interconnect switching network, and then out of N1. This adds front end latency, and increases congestion on the cluster network, which in turn can delay cluster operations.

So, it seems like a good idea, right? Ok, if you have three datastores for VMware, for example – there are minimal overheads for having three IPs. But then – if you only have three datastores, how likely are you to move 1/3rd of the VMs from one node to the other? So that’s an argument for not doing it. But with 7 datastores, it’s much more likely to come up, and still, 7 to 10 IPs isn’t too bad. But if you have 50 datastores, it’s probably more than two nodes, so putting them all in place, managing the mapping datastores to LIFs.. there’s a lot of overhead.

Let’s have a look at WHY you might move a volume:

  1. Aggregate full – no more aggregates on original home node
  2. Controller CPU/IO high – balance workloads to another controller
  3. Equipment replacement – Moving off old equipment onto new equipment

In the third case, indirect access is ok, because it is temporary, so there’s no need for additional LIFs for that. For the other two cases, especially for VMware, there’s always the options of doing a storage vMotion to move all the VMs. For non VM workloads, it’s obviously going to be a different scenario – so the decision to weigh up is – how often do you as an admin think you’ll need to move only one or two volumes at a time? There is always an option of unmounting off a LIF on the source node and remounting from an IP on the destination.

So for my money – more than three datastores and less than ten, one LIF per datastore is probably fine. For anything else, I’d suggest just one NFS LIF per node (per SVM), and deal with preventing indirect access through other means. But I also don’t think it’s a “hard and fast” rule.


3 thoughts on “ONTAP – Why and why not to have one LIF per NFS volume

  1. hi Alex
    Old article but curious how you suggest “preventing indirect access through other means?”

    Also my suggestion would be 2 LIFs per SVM (1 for each node)

    1. Hi! No worries at all for comment on an old article! Reminds me I need to update some stuff on here!

      Well, depending on the workload type, you might be able to change what IP the storage is being mounted on from the clients. This doesn’t work so great for vmware of course, but for other workloads it might.

      The downside of only going with one LIF per node (remember we can do 24 node NAS clusters 😉 is that you still tie the workload through that node. For many environments, this is absolutely fine!

      .. but I have dealt with several multi-petabyte vmware environments on NetApp, so my advice is aiming to cover the whole of the spectrum of use cases, which is why it’s not quite so clearcut.

      Hope this helps!

      1. Thank you Sir!
        Have not had the pleasure of working with multi-petabyte environments.
        I do always try to provide at least 2 ip’s per SVM to prevent indirect traffic warnings on VMware.
        Lots of storage admins I’ve met don’t seem to care and are happy to give out just one ip per svm.

Leave a Reply

Your email address will not be published. Required fields are marked *