IBM i Global

 View Only
  • 1.  Rpgle Multithread

    Posted Thu March 17, 2022 10:32 AM

    HI all, 

    I need some help, I have a multithread application (multithread because many clients open a socket connection to this applications, so I can separate send thread and receive thread).

    Now here's my problem, when a client connect to us I call a procedure than make some check and read some table, all the programs called by one thread are set to thread(*concurrent), and close with return (not *inlr).
    So image client number one make his connection, my thread call an external program open some table, and go ahead, the same as for client number 2, 3, 4 and so on. Every times my thread call this external program it open a new access path, and I think this reserve some memory. during the working day the client problably will disconnect and connect. At the end of the day this job will have somthing like 1500 open file, but they are always the same...

    My question is how can I reclame all the open files when a client will disconnect?

    I made a little test case to explain my problem:

    **FREE
    Ctl-Opt  DftActGrp(*no)
      ActGrp(*StgMdl)
      StgMdl(*SNGLVL)
      Thread(*Concurrent)
      Option(*NoUnRef :*SrcStmt :*NoDebugIo)
      DatFmt(*Iso) TimFmt(*Iso)
      Debug(*Constants)
      AlwNull(*UsrCtl)
      DftName(TESTTHREAD)
      Text('Test multithread');
    // ___________________________________________________________________
    /Include QSYSINC/QRPGLESRC,PTHREAD
    
    Dcl-Pr TESTTHREAD ExtPgm('TESTTHREAD');
    End-Pr TESTTHREAD;
    
    Dcl-Pi TESTTHREAD;
    End-Pi TESTTHREAD;
    
    Dcl-Pr TESTTHD1 ExtPgm('TESTTHD1');
    End-Pr TESTTHD1;
    
    Dcl-Pr USleep   Int(10) ExtProc('usleep');
      MilliSeconds Uns(10) Value;
    End-Pr USleep;
    
    Dcl-Pr Sleep   Int(10) ExtProc('sleep');
      Seconds Uns(10) Value;
    End-Pr Sleep;
    // ____________________________________________________________________________
    Dcl-Ds thread LikeDs(pthread_t) Dim(MAX) Static(*allthread);
    
    Dcl-S Rc                  Int(10);
    Dcl-S IdxMain             Int(10) ;
    Dcl-S IdxTemp             Int(10) Static(*allThread);
    
    Dcl-C MAX                 1500;
    
    // ____________________________________________________________________________
    
    IdxMain = 1;
    
    For IdxMain = 1 to MAX;
      // Open a new thread (simulate a client connection
      Rc = pthread_create(thread(IdxMain) : *Omit : %PAddr(TestOpenFile) : %Addr(IdxMain));
      // Immediately after I detach this thread (so when the thread end every varialble
      //   defined in this will be reclaimed in the procedure/external program
      Rc = pthread_detach(thread(IdxMain));
      // Set a name to the thread
      Rc = pthread_setname_np(thread(IdxMain) :'TEST-' + %Char(IdxMain));
    EndFor;
    
    
    // Wait until all thread are finished
    Dow (IdxTemp < 500);
      USleep(100000);
    EndDo;
    
    // Now if you check you will see every thread are finished, the memory is reclamed,
    //  but there are 1501 open data path to the same file
    Sleep(20);
    
    Return;
    // ____________________________________________________________________________
    Dcl-Proc TestOpenFile;
      Dcl-Pi TestOpenFile;
        PtrIdx Pointer value;
      End-Pi TestOpenFile;
    
      Dcl-S TestVar Char(16000000);
      Dcl-S IdxThread1      Int(10) Based(PtrIdx);
      Dcl-S IdxThread       Int(10);
      // ____________________________________________________________________________
    
      IdxThread = IdxThread1;
    
      // Call an external program that open a file
      TESTTHD1();
    
      IdxTemp +=1;
    
      Return;
    
    End-Proc TestOpenFile;
    

    And the external program:

    **FREE
    Ctl-Opt  DftActGrp(*no)
      ActGrp(*StgMdl)
      StgMdl(*SNGLVL)
      Thread(*Concurrent)
      Option(*NoUnRef :*SrcStmt :*NoDebugIo)
      DatFmt(*Iso) TimFmt(*Iso)
      Debug(*Constants)
      AlwNull(*UsrCtl)
      DftName(TESTTHD1)
      Text('Check client');
    // ___________________________________________________________________
    /Include QSYSINC/QRPGLESRC,PTHREAD
    
    Dcl-F Employee Keyed ExtDesc('SAMPLE/EMPLOYEE') Qualified UsrOpn;
    
    Dcl-Pr TESTTHD1 ExtPgm('TESTTHD1');
    End-Pr TESTTHD1;
    
    Dcl-Pi TESTTHD1;
    End-Pi TESTTHD1;
    
    Dcl-Pr Sleep   Int(10) ExtProc('sleep');
      Seconds Uns(10) Value;
    End-Pr Sleep;
    
    // ____________________________________________________________________________
    Dcl-Ds DsEmployee LikeRec(Employee.Employee :*ALL);
    
    // ____________________________________________________________________________
    
    If (Not %Open(EMPLOYEE));
      Open EMPLOYEE;
    EndIf;
    
    Chain '000010' Employee DsEmployee;
    
    Sleep(10);
    
    Return;


    If someone could help me... 

    Many thanks



    ------------------------------
    Paolo Salvatore
    ------------------------------


  • 2.  RE: Rpgle Multithread

    IBM Champion
    Posted Thu March 17, 2022 06:18 PM
    How do you want it to work?

    You could use the CLOSE opcode to close the file after you're done with it.  Is that what you're looking for?

    Personally, I would serialize access to the database... and keep the file open...  or maybe limit it to a pool of threads that keep the db open... rather than opening another copy in each thread.  The cost of opening/closibg the file each time is significant, so it seems to me you might improve performance that way, even though the threads are serialized.

    I'm not familiar with your application or what your goal is, so it's hard to know how to answer this.

    ------------------------------
    Scott Klement
    Director
    Profound Logic Software
    Oak Creek WI
    ------------------------------



  • 3.  RE: Rpgle Multithread

    Posted Fri March 18, 2022 08:46 AM
    Hi Paolo,

    Does the program get called more than one time in the thread?

    If not, I think you can just return with *INLR = *ON.

    If the program gets called many times in the thread, and the calling program knows when the thread will end, then you could call the program one final time to tell it to end, so it can close the files by returning with *INLR on, or by using the CLOSE opcode.

    To tell the program that it should end:
    • If it has parameters, call it without any parameters, and if %PARMS = 0, just set on *INLR and return.
    • If it does not have parameters, add a parameter, and if the parameter was passed, just set on *INLR and return.


    ------------------------------
    Barbara Morris
    ------------------------------



  • 4.  RE: Rpgle Multithread

    Posted Fri March 18, 2022 08:48 AM
    But I agree with Scott that you should investigate serializing access to the database.

    ------------------------------
    Barbara Morris
    ------------------------------



  • 5.  RE: Rpgle Multithread

    Posted Fri March 18, 2022 01:25 PM
    Hi, 

    thank you Scott and Barbara, I'm investigating to serialize the call to the procedure and I will let know.

    Many thanks.

    ------------------------------
    Paolo Salvatore
    ------------------------------