IBM i Global

IBM i Global

Connect, learn, share, and engage with IBM Power.

 View Only
  • 1.  Performance issue mainly with batch jobs

    Posted 2 days ago

    Hello, 

      In typical if-it-isn't-broken-don't-fix-it fashion, we've waited until P8's and V7R3 are no longer in regular support, we loaded all patches to bring us up from TR5 to TR13 in preparation to migrate to new P10's.  However, batch jobs that sleep a while(like an hour) and wake up and run repeatedly all day, have started spewing hundreds of joblog spool files when they do run.  These programs did not do this previously.  We suspect it has something to do with embedded SQL in RPG programs, but are not sure.  Has anyone seen this behavior before or have any suggestions on what to investigate?  It chews up time and resources, and I'd like to overcome it before we migrate to more powerful processing.

    Thank you.  Mike Overlander  The Dixie Group



    ------------------------------
    Mike Overlander
    ------------------------------


  • 2.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago

    Maybe you should start posting an extract of your joblog.



    ------------------------------
    Paul Nicolay
    ------------------------------



  • 3.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago

    Certainly easy, but it will less than spectacular.  The last page from 3 consecutive spool files, 191 or 192 pages long each:

    # 1: 

      5770SS1 V7R3M0 160422                           Job Log                            TDGSADTS  08/04/25  07:25:4
       Job name . . . . . . . . . . :   EDIB2B832       User  . . . . . . :   EDIONR       Number . . . . . . . . . 
       Job description  . . . . . . :   CAMSEDIB2B      Library . . . . . :   CAMS         Section number . . . . . 
     MSGID      TYPE                    SEV  DATE      TIME             FROM PGM     LIBRARY     INST     TO PGM    
                                          To module . . . . . . . . . :   EDCC832BB                                 
                                          To procedure  . . . . . . . :   EDCC832BB                                 
                                          Statement . . . . . . . . . :   7200                                      
                                          Message . . . . :     7200 - CLRPFM FILE(EDP832BD7R)                      
     CPC3101    Completion              00   08/04/25  03:31:31.260068  QDBCLRPF     QSYS        02B4     EDCC832BB 
                                          To module . . . . . . . . . :   EDCC832BB                                 
                                          To procedure  . . . . . . . :   EDCC832BB                                 
                                          Statement . . . . . . . . . :   7200                                      
                                          Message . . . . :   Member EDP832BD7R file EDP832BD7R in CAMSWORK cleared.
     *NONE      Command                      08/04/25  03:31:31.260206  QCADRV       QSYS        041C     EDCC832BB 
                                          To module . . . . . . . . . :   EDCC832BB                                 
                                          To procedure  . . . . . . . :   EDCC832BB                                 
                                          Statement . . . . . . . . . :   7300                                      
                                          Message . . . . :     7300 - CLRPFM FILE(EDP832BD8)                       

    #2:

                                          Statement . . . . . . . . . :   11700                                      
                                          Message . . . . :   No records copied from file EDP832BD0 in CAMSWORK.     
     *NONE      Command                      08/04/25  03:32:54.801789  QCADRV       QSYS        041C     EDCC832BB  
                                          To module . . . . . . . . . :   EDCC832BB                                  
                                          To procedure  . . . . . . . :   EDCC832BB                                  
                                          Statement . . . . . . . . . :   12100                                      
                                          Message . . . . :    12100 - CPYF FROMFILE(EDP832BD1) TOFILE(EDP832BHD1)   
                                            MBROPT(*ADD)                                                             
     CPC2957    Completion              00   08/04/25  03:32:54.802823  QCPEXCON     QSYS        0BBD     EDCC832BB  
                                          To module . . . . . . . . . :   EDCC832BB                                  
                                          To procedure  . . . . . . . :   EDCC832BB                                  
                                          Statement . . . . . . . . . :   12100                                      
                                          Message . . . . :   No records copied from file EDP832BD1 in CAMSWORK.     
     *NONE      Command                      08/04/25  03:32:54.802981  QCADRV       QSYS        041C     EDCC832BB  
                                          To module . . . . . . . . . :   EDCC832BB                                  
                                          To procedure  . . . . . . . :   EDCC832BB                                  
                                          Statement . . . . . . . . . :   12500                                      
                                          Message . . . . :    12500 - CPYF FROMFILE(EDP832BD2) TOFILE(EDP832BHD2)   
                                            MBROPT(*ADD)                                                             

    #3:

     MSGID      TYPE                    SEV  DATE      TIME             FROM PGM     LIBRARY     INST     TO PGM     
                                          Message . . . . :     5900 - ADDLIBLE LIB(CAMSWORK) POSITION(*AFTER CAMS)  
     CPF2103    Escape                  40   08/04/25  03:34:17.150800  QLICUSRL     QSYS        01A3     EDCC832BB  
                                          To module . . . . . . . . . :   EDCC832BB                                  
                                          To procedure  . . . . . . . :   EDCC832BB                                  
                                          Statement . . . . . . . . . :   5900                                       
                                          Message . . . . :   Library CAMSWORK already exists in library list.       
     *NONE      Command                      08/04/25  03:34:17.150885  QCADRV       QSYS        041C     EDCC832BB  
                                          To module . . . . . . . . . :   EDCC832BB                                  
                                          To procedure  . . . . . . . :   EDCC832BB                                  
                                          Statement . . . . . . . . . :   6200                                       
                                          Message . . . . :     6200 - CLRPFM FILE(EDP832BH1)                        
     CPC3101    Completion              00   08/04/25  03:34:17.239948  QDBCLRPF     QSYS        02B4     EDCC832BB  
                                          To module . . . . . . . . . :   EDCC832BB                                  
                                          To procedure  . . . . . . . :   EDCC832BB                                  
                                          Statement . . . . . . . . . :   6200                                       
                                          Message . . . . :   Member EDP832BH1 file EDP832BH1 in CAMSWORK cleared.   
     *NONE      Command                      08/04/25  03:34:17.240107  QCADRV       QSYS        041C     EDCC832BB  

      The only CPF severity 40 is on an ADDLIBLE command where the library is already in the lib list.  The only CPF severity 10 is on file opens, CPF4028, otherwise there is no "errors" of any kind.  You can see that in roughly 3 minutes it created 3 spool files 191+ pages long, 32 in total for this job.  Others, quite a few more.

      Thank you.



    ------------------------------
    Mike Overlander
    ------------------------------



  • 4.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago

    This is not related to installing a TR, neither about embedded SQL but is just based on the severity level on your job.



    ------------------------------
    Paul Nicolay
    ------------------------------



  • 5.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago

    Well, that could be true, but since this and other jobs ran with exactly the same run parameters over the past decade without the problems they now encounter, something changed in the operating system somewhere because of these PTF's.  

      Thank you.



    ------------------------------
    Mike Overlander
    ------------------------------



  • 6.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago

    No idea if IBM ever changes QDFTJOBD, but you could check if it still has...

    Message logging:                           
      Level  . . . . . . . . . . . .   4       
      Severity . . . . . . . . . . .   00      
      Text . . . . . . . . . . . . .   *NOLIST 
    Log CL program commands  . . . .   *NO     

    ... but I would recommend using a custom job description anyway.



    ------------------------------
    Paul Nicolay
    ------------------------------



  • 7.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago
      |   view attached

    Dear Mike 

    Is your P8 server still uses all spinning HDD?  If you use all SSD, this issue should not be as serious any more.  

    My guess is that the certain TR PTFs may disrupt job log pending feature previously enabled (which is another guess).  You just enable back the job log pending feature.  Or you can choose to use job log server instead (and set only one such job log server job to minimize overall performance impact)  You may want to enable detachment of job logs from jobs as well for minimal negative performance impact to job run time performance.  I attach the information file on how to do this herewith. 



    ------------------------------
    Satid S
    ------------------------------

    Attachment(s)



  • 8.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago
    Edited by Michael Mayer 2 days ago

    Every 6 months, on all of our partitions, I run 3 items and save the output to our LAN. This is especially handy if things like this rear up. Comparison can be made to what was vs what currently is. Those items are:

    PRTSYSINF

    A custom PRTJOBD program that outputs every JOBD on the partition. I can send that along if anyone wants a copy.

    Using QMGTOOLS, we run a SYSSNAP.

    Info on the MG TOOLKIT (free) can be located on my IBM award winning blog IBM i Reference Pages on

    the "More Tips by Category" page.

    Respectfully,

    Michael Mayer
    IBM i on Power System Admin
    IBM i Award Winning Blog Site: https://ibmireference.blogspot.com

    Awarded IBM 2024/2025 Advocate, Contributor, Influencer and IBM i Ready Badges.     



    ------------------------------
    Michael Mayer
    IBM i on Power System Admin
    ERMCO-ECI
    Dyersburg, Tennessee 38024
    518.641.8906
    2024 IBM Badge Advocate, Contributor and Influencer.
    Blog: https://ibmireference.blogspot.com
    ------------------------------



  • 9.  RE: Performance issue mainly with batch jobs

    Posted 2 days ago

    2 things come to mind - the values in the jobd are different than what you expect (used to) and/or the size of the message queue is smaller as a result and the job is doing a *PRTWRAP for when the job's message queue becomes 'full' . This does not appear to have anything to do with RPG/SQL - your logging of that job either changed manually after it was submitted or the jobd used for this job is logging more than you desire...



    ------------------------------
    Rich Malloy
    Principal Systems Engineer - IBMi
    Cox Automotive
    Draper UT
    ------------------------------