Originally posted by: Stephan B.

Dear all,

background: I noticed rather slow disk-to-tape migration performance with TSM. To test if TSM itself is the source of the problem I tested disk read performance with ndisk64 of IBMs nstress toolset . I've noticed that the read throughput and IOPS roughly grow linearly with the amount of parallel processes and I don't understand why. I've tested this on different power machines with different storage systems in totally different environments but I always saw this behaviour. Though I mainly tested with 256k blocksize and random read this seems to be also true for other blocksizes and sequential read. Tests were done with a single large file on a  single filesystem on a single FC LUN with nearly no other I/O on the storage system in each environment. First I thought that maybe some buffers or queue depth may be the source of this behaviour but after re-thinking I came to the conclusion that this should have rather no correlation because if queues/buffer were sized too small this should impact performance regardless of the number of parallel IO threads. IMO they get filled regardless where the IO comes from. The storage system on the other hand also seems not to be the problematic component because as one can see with several parallel reading processes the storagesystem is capable of handling much higher IOPS and delivering data with much higher throughput than one can see with a single reading process. I find this quite puzzling... 

Here are the measurements for illustration purpose:

-------- 1 PROCESS: -------

root@aix020:/opt/nstress# ./ndisk64 -f /mksysb/bigfile  -r100 -b 256K -t 10 -R
Command: ./ndisk64 -f /mksysb/bigfile -r100 -b 256K -t 10 -R
        Synchronous Disk test (regular read/write)
        No. of processes = 1
        I/O type         = Random
        Block size       = 262144
        Read-Write       = Read Only
        Sync type: none  = just close the file
        Number of files  = 1
        File size to be determined from the file(s) (raw devices will fail)
        Run time         = 10 seconds
        Snooze %         = 0 percent
----> Running test with block Size=262144 (256KB) .
Determining the size of file:/mksysb/bigfile
File /mksysb/bigfile size 8594128896 bytes
Proc - <-----Disk IO----> | <-----Throughput------> RunTime
 Num -     TOTAL   IO/sec |    MB/sec       KB/sec  Seconds
   1 -      1022    102.1 |     25.54     26149.94  10.01
 

-------- 2 PROCESSES: -------

root@aix020:/opt/nstress# ./ndisk64 -f /mksysb/bigfile  -r100 -b 256K -t 10 -R -M2
Command: ./ndisk64 -f /mksysb/bigfile -r100 -b 256K -t 10 -R -M2
        Synchronous Disk test (regular read/write)
        No. of processes = 2
        I/O type         = Random
        Block size       = 262144
        Read-Write       = Read Only
        Sync type: none  = just close the file
        Number of files  = 1
        File size to be determined from the file(s) (raw devices will fail)
        Run time         = 10 seconds
        Snooze %         = 0 percent
----> Running test with block Size=262144 (256KB) ..
Determining the size of file:/mksysb/bigfile
File /mksysb/bigfile size 8594128896 bytes
Determining the size of file:/mksysb/bigfile
File /mksysb/bigfile size 8594128896 bytes
Proc - <-----Disk IO----> | <-----Throughput------> RunTime
 Num -     TOTAL   IO/sec |    MB/sec       KB/sec  Seconds
   1 -       991     99.0 |     24.75     25344.83  10.01
   2 -       951     95.1 |     23.78     24347.79  10.00
TOTALS      1942    194.1 |     48.53     49692.62
- Random procs=  2 read=100% bs=256KB
 

-------- 4 PROCESSES: -------

root@aix020:/opt/nstress# ./ndisk64 -f /mksysb/bigfile  -r100 -b 256K -t 10 -R -M4
Command: ./ndisk64 -f /mksysb/bigfile -r100 -b 256K -t 10 -R -M4
        Synchronous Disk test (regular read/write)
        No. of processes = 4
        I/O type         = Random
        Block size       = 262144
        Read-Write       = Read Only
        Sync type: none  = just close the file
        Number of files  = 1
        File size to be determined from the file(s) (raw devices will fail)
        Run time         = 10 seconds
        Snooze %         = 0 percent
----> Running test with block Size=262144 (256KB) ....
Determining the size of file:/mksysb/bigfile
File /mksysb/bigfile size 8594128896 bytes
Determining the size of file:/mksysb/bigfile
Determining the size of file:/mksysb/bigfile
Determining the size of file:/mksysb/bigfile
File /mksysb/bigfile size 8594128896 bytes
File /mksysb/bigfile size 8594128896 bytes
File /mksysb/bigfile size 8594128896 bytes
Proc - <-----Disk IO----> | <-----Throughput------> RunTime
 Num -     TOTAL   IO/sec |    MB/sec       KB/sec  Seconds
   1 -       904     90.5 |     22.62     23167.49   9.99
   2 -       948     94.9 |     23.72     24286.37   9.99
   3 -       924     92.5 |     23.13     23681.59   9.99
   4 -       917     91.5 |     22.88     23429.26  10.02
TOTALS      3693    369.4 |     92.35     94564.71
- Random procs=  4 read=100% bs=256KB

Can anyone explain why the beaviour is like that and how one can reach a higher performance on a single process? TSM always migrates with a single process (at least per tape drive) so high parallel reading speed does not help me at all in this case :-/

Thanks in advance for all hints!

regards,
Stephan