Demystifying IO Operation Readouts in ESXi

This doesn’t come up very often these days, but every once and awhile it does and every time it does, I look to see if we have documentation on it and there never seems to be. After writing this post I did find a forum post where my friend Drew answers it there too. Well […]

IO Operations

This doesn’t come up very often these days, but every once and awhile it does and every time it does, I look to see if we have documentation on it and there never seems to be. After writing this post I did find a forum post where my friend Drew answers it there too. Well anyways let’s quickly explain the situation with IO operations.

Most block vendors these days tell customers to change their path switching policy for their storage in ESXi from the default of Round Robin (1,000) to 1. This makes ESXi switches logical paths for a given device after every I/O instead of every 1,000. The reason I say this doesn’t come up much anymore is that in modern version of ESXi (6.0 express patch+, 6.5 U1+ and 6.7+) we (Pure) have rules in ESXi that makes sure this is set by default without any user configuration. Many other vendors do as well.

Anyways, when using VMware tools to see if a device is configured properly, depending on how it is set, it can readout differently.

 

So if I run the following command:

I see two devices that have slightly different multipathing configurations (or so it seems):

Notice a difference? Well there isn’t much of one except for this part. policy=rr and policy=iops. But both say IOPS=1. What does that mean and how is it different?

Well let’s look at the devices in another way.

Notice the Limit Type property. One says IOPS and the other says Default. This is a little clearer.

What if I create a custom (a.k.a. user) SATP rule and then provision a new device?

Note I made the IO Operation limit 10 instead of 1 so I know that it hit that rule. It still says Default.

So. If you see Default or RR, you know that device was configured according to a default SATP rule or a custom one. If you see IOPS, then you know it was because someone manually changed that device.