Originally posted by: SystemAdmin

Hi,
i have the following cenario on an AIX 5.3 TL04-CSP:

• i'm using an HP XP1200 external SAN disk connected to AIX through 2 FC adapters;
• i'm using the mpio device driver (devices.fcp.disk.HP.xparray.mpio.rte) and i noticed that if i use the default hdisk algorithm that is fail_over and reserve_policy single_path i get as twice as the performance when reading data that when i change the hdisk algorithm to round_robin and reserve_policy to no_reserve.

Does anyone have any idea why does this happens? Shouldn't i get a better performance when using two FC adapters simultaneously than just using only one at a time?

thanks
#AIX-Forum

Originally posted by: nagger

Hi, This does not sound right.
The connection method should not make any difference at all - unless you are approaching the theoretical limit of the adatper/cable speed.
To get to this limit would require very serious testing is this a disk test that you are going to run for a couple of weeks or a "quicky" to just give it a go.
If you are running 2 Gbit FC then you need to be getting above 150 MB/sec before the second FC would make any different.
You are far more likely to be hitting limits elsewhere and as its a read test - do you know for sure where the data is being read from AIX cache, adapter caches, HP unit cache, disk caches.
What about all your other settings and in particular the queue depth.
I suggest chaning the algorithm might be effecing the way the I/O is being sent to the SAN and might be bottlenecking.
Also how are you generating the disk I/O - I hope it is not dd but lots of concurrent processes and is this sequential or random. This will also effect how the HP SAN system gets the disk I/O requests and it might not be clever enough to spot for example, senquential disk I/o if it arrives via two paths.

Hope this helps, N


#AIX-Forum

Originally posted by: SystemAdmin

Hi Nigel,
I'm sending you more info:
I'm using a simple dd read operation to /dev/null, reading directly from the raw disk so not using AIX cache.
Attributes of a specific HP SAN disk:
root@host /> lsattr -El hdisk1
PCM PCM/friend/xparray Path Control Module False
PR_key_value none Reserve Key True
algorithm round_robin Algorithm True
clr_q no Device CLEARS its Queue on error True
hcheck_cmd inquiry Health Check Command True
hcheck_interval 60 Health Check Interval True
hcheck_mode nonactive Health Check Mode True
location Location Label True
lun_id 0x1000000000000 Logical Unit Number ID False
lun_reset_spt yes SCSI LUN reset True
max_transfer 0x40000 N/A True
node_name 0x50060e8004f2de00 Node Name False
pvid 00c1737b15823ea20000000000000000 Physical Volume ID False
q_err yes Use QERR bit False
q_type simple Queue TYPE True
queue_depth 2 Queue DEPTH True
reassign_to 120 REASSIGN time out True
reserve_policy no_reserve Reserve Policy True
rw_timeout 60 READ/WRITE time out True
scsi_id 0xb80000 SCSI ID False
start_timeout 60 START UNIT time out True
ww_name 0x50060e8004f2de00 FC World Wide Name False

root@host />chdev -l hdisk1 -aalgorithm=fail_over -areserve_policy=single_path
root@host /> date;dd if=/dev/rhdisk1 of=/dev/null bs=8k count=50000;date
Seg 19 Mar 12:14:23 2007
50000+0 records in.
50000+0 records out.
Seg 19 Mar 12:14:36 2007
root@host /> chdev -l hdisk1 -aalgorithm=round_robin -areserve_policy=no_reserve
hdisk1 changed
root@host /> date;dd if=/dev/rhdisk1 of=/dev/null bs=8k count=50000;date
Seg 19 Mar 12:15:11 2007
50000+0 records in.
50000+0 records out.
Seg 19 Mar 12:15:37 2007
root@host />
As it can be noticed it takes almost about two times more the same operation with round_robin enabled !!!
I'm using 2x2Gbit/s FC adapters, doing sequential I/O.
The disk queue_depth value is 2 that is the default ODM setting from the driver. I don't know if i should change it or the impact of doing this.

I just want to know if is anything on AIX that i can change before speaking to HP about the array configuration.

Thanks in advance for your help.
#AIX-Forum

Originally posted by: SystemAdmin

We having same issue. Do you able to resolve this issue. If reslved, couls please give me the soluation

Logan
#AIX-Forum

Originally posted by: jvk

Running your test on my system:

1. oslevel -r

5300-05

DS6000 disks with default MPIO.
2 FC adapters in AIX box.

1. lsattr -El hdisk33

PCM PCM/friend/fcpother
algorithm fail_over
queue_depth 1
reserve_policy single_path

1. lspath -l hdisk33

Enabled hdisk33 fscsi0
Enabled hdisk33 fscsi0
Enabled hdisk33 fscsi1
Enabled hdisk33 fscsi1

1. date;dd if=/dev/rhdisk33 of=/dev/null bs=8k count=50000;date

Sat Jul 28 23:19:29 DFT 2007
50000+0 records in.
50000+0 records out.
Sat Jul 28 23:19:39 DFT 2007

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk33 93.2 41564.0 5195.5 124848 0

Paths: % tm_act Kbps tps Kb_read Kb_wrtn
Path3 0.0 0.0 0.0 0 0
Path2 0.0 0.0 0.0 0 0
Path1 0.0 0.0 0.0 0 0
Path0 93.2 41564.0 5195.5 124848 0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk33 86.9 41441.5 5180.2 124480 0

Paths: % tm_act Kbps tps Kb_read Kb_wrtn
Path3 0.0 0.0 0.0 0 0
Path2 0.0 0.0 0.0 0 0
Path1 0.0 0.0 0.0 0 0
Path0 86.2 41441.5 5180.2 124480 0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk33 89.4 39400.7 4925.1 119384 0
.
Paths: % tm_act Kbps tps Kb_read Kb_wrtn
Path3 0.0 0.0 0.0 0 0
Path2 0.0 0.0 0.0 0 0
Path1 0.0 0.0 0.0 0 0
Path0 88.1 39400.7 4925.1 119384 0

1. chdev -l hdisk33 -aalgorithm=round_robin -areserve_policy=no_reserve

hdisk33 changed

1. date;dd if=/dev/rhdisk33 of=/dev/null bs=8k count=50000;date

Sat Jul 28 23:21:56 DFT 2007
50000+0 records in.
50000+0 records out.
Sat Jul 28 23:22:06 DFT 2007

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk33 87.8 38475.0 4809.4 123168 0

Paths: % tm_act Kbps tps Kb_read Kb_wrtn
Path3 20.3 9618.7 1202.3 30792 0
Path2 22.2 9618.7 1202.3 30792 0
Path1 19.4 9618.7 1202.3 30792 0
Path0 23.1 9618.7 1202.3 30792 0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk33 91.3 37443.1 4680.4 112376 0

Paths: % tm_act Kbps tps Kb_read Kb_wrtn
Path3 20.0 9361.4 1170.2 28096 0
Path2 26.7 9358.8 1169.8 28088 0
Path1 22.7 9361.4 1170.2 28096 0
Path0 20.7 9361.4 1170.2 28096 0

Disks: % tm_act Kbps tps Kb_read Kb_wrtn
hdisk33 90.3 33658.0 4207.2 101016 0

Paths: % tm_act Kbps tps Kb_read Kb_wrtn
Path3 17.3 8415.2 1051.9 25256 0
Path2 24.3 8415.2 1051.9 25256 0
Path1 22.7 8412.5 1051.6 25248 0
Path0 24.3 8412.5 1051.6 25248 0
From above you can see that one path (single_path/fail_over) can manage
same or even slightly more than 4 paths but in general my results are
in both cases 10 seconds.
#AIX-Forum