IBM Sterling Transformation Extender

Sterling Transformation Extender

Come for answers, stay for best practices. All we're missing is you.


#Sterling
#Supplychain
 View Only
Back to discussions
Expand all | Collapse all

Launcher, map and shell synchronization

  • 1.  Launcher, map and shell synchronization

    Posted 11/24/14 10:45 AM

    Originally posted by: Bhoju


    Hi All, 

    We have a typical problem in our production system as described below. Request an expert review of this problems and solution suggestions from the group. 
     
     
    1. There is a launcher system file which listen on input queue and get triggered by appearance of message on the issue. This system file reads the message from queue, do some processing with the data from mq message and APPEND the content of the message to a another file on file system. (Say File X) 
     
    2. There is another luancher system file which runs every 1 minutes and execute a shell script which counts number of records in the file X (Being created and modified by step 1 launcher system) and once the record count reaches to certain number renames the file X to say file Y 
     
    3. This file Y is moved to another location and processed to a database. 
     
    The issue is sometimes (Not very often) the second system file (in Step 2) renames the file while first system file (In step 1) is halfway writing it.  In short these two watches are not sychronized and caused issue sometimes. 
     
    I have analyzed and thought of different solutions to synchronize these two watches to make sure that when system file 1 is writing the file, the other system file waits for writing to be completed before processing it further. Problem is i cannot replicate the issue on test server even though it happens sometimes in production.  
     
    Could the group please advise a way to replicate such situation using launcher system file, maps and shell scripts, if its replicated once, i can think of multiple options to solve this. 
     
    Thanks in advance. 

    #DataExchange
    #IBM-Websphere-Transformation-Extender
    #IBMSterlingTransformationExtender


  • 2.  Re: Launcher, map and shell synchronization

    Posted 11/25/14 03:10 AM

    Originally posted by: Laurent(Satisco)


    Good morning.

    <Warning: below is more "aloud thinking" than proofed design, but I hope it helps>

    I'm afraid the best I could offer would be to review the design.

    At least, this is what I'd prefer if I was to design such an interface.

    So, IF it was me, I'd rather use a queue for intermediate storage, so that messages are piling in it until it's time to "dump" them somewhere else. The MQ adapter options would allow reading the desired number of messages to create a file of the right size.

     

    If I was not given the choice about the intermediate storage, I'd redesign the second map to make it need a write access to the file, so that the Launcher Resources Manager would take care of the concurrent access (my guess is that the issue lies there, as if the map does not need write access to the file, then the launcher will not manage the concurrency).  A possible solution would be to make the map write a dummy (empty) record to the file, simply to make it take temporary ownership of the map. Ideally, the file would be a map input but it may be a challenge to avoid erasing the file when it's too small without failing the map.

     

    Hope this helps.

    Laurent


    #IBM-Websphere-Transformation-Extender
    #IBMSterlingTransformationExtender
    #DataExchange


Related Content