Dear Glenn
Hundreds of short-lived QP0ZSPWP jobs is indicative of the run-time performance issue and I suspect this is by design in IBM i's support for Spawn() and Fork() when running UNIX shell script in PASE. One basic and crucial thing you must do is to make sure that, in WRKSYSSTS screen, you set MAX ACT for pool 2 to not less than 1000 (because a lot of jobs need high "activity level" in the memory pool in which they run) and allocate sufficient amount of memory to it. And also allocate memory to pool 1 to at least twice the amount shown in its "Reserved Size".
When you run ADDPJE for QP0ZSPWP jobs, you also need to set its Initial Number of Jobs to a high value (such as 100) and Additional Number of Jobs to a high value as well (such as 30). If you did not do this, use CHGPJE command to change it.
If you can run your script 3 times consecutively in IBM i, does the resulting run-time the same for each run? If the first run takes the longest, then it should be indicative of the nature of PASE support in IBM i. .
By the way, what is the IFS directory path you put your files in IBM i? Just want to check if, by any chance, you happen to put your files in an improper IFS file system such as /QDLS. The safe default should be under /QOpenSys file system.
Another thing you can try is to change the script to reduce spawn() and fork() but I do not know enough to be more specific on this. There are many IBM i functions running in PASE but definitely not as shell scripts. I suspect IBM i PASE is not optimized for running shell scripts, more for programs and procedures. Hope there is an IBM i developer who takes care of deploying PASE-based functions respond to your question. If none does, what you can try is to open a PMR to IBM i WW Support and ask your question.
------------------------------
Satid Singkorapoom
------------------------------
Original Message:
Sent: Tue June 07, 2022 09:40 AM
From: Glenn Robinson
Subject: PASE performance question
I have just made the changes to SSHD to use the QP0ZSPWT pre tart jobs as described here
This has reduced the run time by about 1 minute but it's still significantly slower than RHEL or AIX.
------------------------------
Glenn Robinson
Original Message:
Sent: Tue June 07, 2022 09:27 AM
From: Glenn Robinson
Subject: PASE performance question
Satid,
My apologies, I should have made this clearer.
There are no other users or batch jobs running on the IBM i LPAR apart from my own ssh client session and default IBM jobs. In fact, this LPAR runs native IBM i workloads without any performance problems.
Neither the IBM i or RHEL LPAR have any obvious performance constraints. Both LPARs share the same storage pool and the FS7200 shows blazing fast volume response times.
I just tried running the script with the same text files on an AIX 7.1 LPAR on the same Power9 system as the RHEL and IBM i LPAR and that ran in about 8 seconds.
I can't see anything in the docs you kindly sent that I haven't read before.
What I can see on WRKACTJOB is that the script generated hundreds of shirt lived QP0ZSPWP jobs on the system.
------------------------------
Glenn Robinson
Original Message:
Sent: Tue June 07, 2022 08:51 AM
From: Satid Singkorapoom
Subject: PASE performance question
Dear Glenn
Was you job the only job running in IBM i LPAR when you performed the task? If not, how much other workload was at that time? It may not be easy to answer your question if many other jobs are also running.
Are all the LUNs from FS7200 allocated to both IBM i and RHEL LPARs carved out of the same storage pool? If not, then we need to know if IBM i LPAR has as good a disk response time as RHEL LPAR or not.
How many additional jobs were launched by your loop? If more than a few, then this is one major cause of the performance drag. In such a case, make sure *BASE memory pool (pool number 2) in your IBM i has sufficient memory allocated and its MAX ACTIVE parameter should be set at a value of least 1000. Please use WRKSYSSTS command and press F10 every 10 seconds or so and observe the memory faulting rate of pool 2 when your script is running (please ignore "Pages" value). If the faulting rate is as high as some 500 or more consistently during the script run, this can be an issue and you may try adding more memory to pool 2 to see if it reduces the faulting rate or not.
Did you run the Bash script from an SSH client? A tutorial article on running Bash shell in IBM i PASE recommends this as you can read here : https://www.itjungle.com/2014/09/17/fhg091714-story01/
A few more articles on using Bash in IBM i but not sure if they will be useful or not : https://jbh.github.io/categories/ibm%20i/
------------------------------
Satid Singkorapoom
Original Message:
Sent: Tue June 07, 2022 08:01 AM
From: Glenn Robinson
Subject: PASE performance question
I have bash script which manipulates some text file using many grep, awk and sed statements.
When I run this on my V7R3 LPAR it takes around 6 minutes to run the script.
When I run the same script on my RHEL ppc64le LPAR it takes just 5.8 seconds.
Both LPARs are connected to the same FS7200 storage.
Both LPARs have have the same CPU and Memory assigned.
I'm guessing that when I use pipes or execute commands as in put to loops this is kicking off additional IBM i jobs which is why this takes so long comapred to RHEL.
Any suggestions/comments on how I can make PASE more performant so that I can get closer to RHEL performance?
Thanks
Glenn
------------------------------
Glenn Robinson
------------------------------