Queue depth calculation for target and initiators.

What is Queue depth?
It is the number of I/O operations that can run in parallel on a LUN/blocks. Generally, it specifies the number of outstanding requests per LUN. It is the number of I/O requests that the initiator can send and receive per LUN. Each I/O request from the host’s initiator HBA to the storage target adapter ingests a queue entry. Typically, a higher queue depth equates to improved performance. However, if the storage controller’s maximum queue depth is reached, that storage controller discards incoming I/O request by returning queue full response to the initiator, causing bottlenecks and latency issues.

The following general recommendations can be made about "tuning" queue depths.

• For small to mid-size systems, use a HBA queue depth of 32.
• For large systems, use a HBA queue depth of 128.
• For exception cases or performance testing, use a queue depth of 256 to avoid possible queuing problems.
• All hosts should have the queue depths set to similar values to give equal access to all hosts.
• Ensure that the storage controller target FC port queue depth is not exceeded to avoid performance penalties or errors.

 Its most import that before design the connectivity and provisioning LUN's to avoid performance and latency issues later after deployments

Now! how to calculate Queue Depth?
Its not a mathematically complex calculations, Just we need to do multiplications and addtions. 

Before proceeding any configuration of queue depth on initiator we have to count the total number of FC initiators in all the hosts/physical servers that connect to one FC target port - "storage port" . I have describe it in a looping structure.

do 
Multiply by 128 with number of hosts
If 
the result is less than 2,048
then
set the queue depth for all initiators to 128
else
if the value is more  2,048
then
Add more FC target ports.(or) Redistribute your FC initiators.

Example:

You have 15 hosts with one initiator connected to each of two target ports on the storage controller. 15 x 128 = 1,920. 1,920 is less than the total queue depth limit of 2,048, so you can set the queue depth for all your initiators to 128.

You have 30 hosts with one initiator connected to each of two target ports on the storage controller. 30 x 128 = 3,840. 3,840 is greater than the total queue depth limit of 2,048, so you should Add more FC target ports.(or) Redistribute your FC initiators.

when to add more FC target ports?

The desired queue depth of 3,840 exceeds the available queue depth per port. To remedy this, you can add a two-port FC target adapter to each controller, then rezone your FC switches so that 15 of your 30 hosts connect to one set of ports, and the remaining 15 hosts connect to a second set of ports. The queue depth per port is then reduced to 15 x 128 = 1,920.

Formula:

To estimate the queue depth needed to achieve a certain I/O per second throughput

Needed queue depth = (Number of I/O per second) x (Response time)

if you need 40,000 I/O per second with a response time of 3 milliseconds, the needed queue depth = 40,000 x (.003) = 120.

 How to set queue depth on host?

That depends on iop's load generated by the hosts, we have to segregate hosts depends on the vm's and its application how much load it generates.

Lets take, The desired queue depth of 3,840 exceeds the available queue depth per port. You have 10 "large" hosts that have high storage I/O needs, and 20 "small" hosts that have low I/O needs. Set the initiator queue depth on the "large" hosts to 128 and the initiator queue depth on the "small" hosts to 32. 

Your resulting total queue depth is (10 x 128) + (20 x 32) = 1,920. which is under the maximum capacity!

Finally, Queue depth is a most important for SAN administrator who are really struggle with performance and latency issues :)