Originally posted by: PontiacGeronimo

is it enough to set queue_depth for virtual HBAs on client LPAR only or also queue_depth for physical adapters which are used in background on VIOS should also have some higher queue_depth set before they are configured in "NPIV"?

are there any others physical HBA attributes to be adjusted on VIO Servers to avoind i/o bottlenecks on client LPARs?

the memory on VIO servers is enough (I calculated 300MB per single virtual HBA - recommendations I found were 140MB for each)
thanks in advance for any tips

Originally posted by: SystemAdmin

queue_depth isn't a HBA attribute, but a disk attribute.
Assuming you are mapping storage LUN to LPAR threw vscsi adapter on VIO, it is a good practice to have the same queue_depth on both VIO and LPAR for a hdisk.
NPIV is passthru, so disks are not seen by the VIO, only by LPAR, so queue_depth is only set on LPAR side.

For an HBA, you should have a look at both attributes :

• num_cmd_elems
• max_xfer_size

Originally posted by: PontiacGeronimo

sure, my mistake...sorry.

but what about:

For an HBA, you should have a look at both attributes :
•num_cmd_elems
•max_xfer_size
Should they be also set for the physical HBA's devices on VIOS or it doesn't matter what settings are on fcsX/fscsiX on VIOS HBA's? Is it enough to tune them only on client LPAR side on virtual fcsX'ses / fscsiX'es?

Originally posted by: SystemAdmin

If the bottleneck is on the (virtual) fc adapters, you should increase this parameters on LPAR, and increase VIO values also to match LPAR settings.
Use fcstat command and monitor following statistics :

• No DMA Resource Count: 75
• No Adapter Elements Count: 0
• No Command Resource Count: 642427

Originally posted by: KarlM

In this article:
http://www.ibmsystemsmag.com/aix/tipstechniques/systemsmanagement/Disk-I-O-and-the-Network/
in IBM Systems Mag Jaqui Lynch says:

When using VIO servers, max_xfer_size and num_cmd_elems should be set on the VIO servers and, if using N_Port ID Virtualization (NPIV), they’ll also need to be set on the NPIV client LPARs. Don’t set the values on the NPIV client LPAR higher than the VIO servers; I tried this and my LPAR wouldn’t boot, which was probably lucky, as I am sure there would have been overruns.

HTH

Originally posted by: PontiacGeronimo

Hello,

what is the right procedure to increase those values on running systems?

Please let me know I it would be ok (doing sequential change - one-by-one adapter on VIOS and Client LPARs) in following order:
(LUNs are served by default AIX MPIO)

Example for one adapter

Note: fcs0 on each LPAR is "linked" with physical fcs0 on VIOS1 (vfchost0)

1) on all LPARs involved, disable paths via fcs0
chpath -l hdisk0 -p fscsi0 -s disable
chpath -l hdisk1 -p fscsi0 -s disable
.....
(I don't know if chpath can disable all fscsi0's paths in one command) -- maybe "rmdev -l fcs0 -R" would do the trick.

2) on VIOS1 - unmap the virtual fibre channel
vfcmap -vadapter vfchost0 -fcp

3) on VIO Server:

oem_setup_env
rmdev -l fcs0 -R
chdev -l fcs0 -a num_cmd_elems=1000 -a max_xfer_size=0x200000
cfgmgr -l fcs0 #to enable physical HBA again on VIOS and login to SAN
exit to padmin
vfcmap -vadapter vfchost0 -fcp fcs0

4) on client LPARs:
rmdev -l fcs0 -R
chdev -l fcs0 -a num_cmd_elems=1000 -a max_xfer_size=0x200000
cfgmgr -l fcs0 #to enable virtual HBA again on client LPAR and login to SAN

is that ok? how to you do such changes online (without reboot)?
are such operations safe - no WWN change on virtual FC adapters will happen?

Originally posted by: UNIX-OS

I would suggest this:

1. on VIO servers set the value in the ODM using chdev -l ... -P option. reboot one VIO at atime.
2. change LPAR value and reboot the LPAR.

Thanks