IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

SFTP Remote folder, and pub.client.sftp:get by older modified date

  • 1.  SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 06:07 AM

    Hi community,

    please i need your help , what we want to do is :

    1-get all files frome remote folder using sftp
    2-put those file in my local folder
    3-loop files in my local folder by date and pattern (example : Loop–get files which name like yyyy-dd-mm_name.txt and yyyy-dd-mm correspond to the older date–)
    4-get data from file and execute some services
    5-remove files frome remote folder
    6-move files frome local folder to archive

    [b]Any suggestion or help how to proceed would be greatly apprecaited.

    Thanks, [/b]


    #Integration-Server-and-ESB
    #webMethods


  • 2.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 06:32 AM

    Hi Abrabri,

    see following code snippet (partially Pseudo-Code (might be incomplete regarding StreamToString transfortmations etc.)):

    
    pub.client.sftp:login
    pub.client.sftp:ls
    LOOP Start
    pub.client.sftp:get
    pub.file:writeToFile
    pub.client.sftp:rm
    LOOP End
    pub.client.sftp:logout
    
    Get File-List according to Pattern
    LOOP Start
    pub.file:getFile
    execute logic
    pub.file:delete
    LOOP End

    Remember to update WmPublic/config/fileAccessControl.cnf with the path to your localFolder and reload the Packages to use the pub.file:*-Services.

    Regards,
    Holger


    #Integration-Server-and-ESB
    #webMethods


  • 3.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 06:42 AM

    thank you Holger for yr reply,
    but i dont have pub.client.sftp:list in my built services??

    i think by doing * in remode folder can load all files but in my local folder how procceding them according to Pattern and by the older date???

    Regards,
    abrabri


    #Integration-Server-and-ESB
    #webMethods


  • 4.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 06:49 AM

    Abrabri – You can ls command to list the files, redirect the file names to some file and loop over it. Give a try.

    Thanks,


    #Integration-Server-and-ESB
    #webMethods


  • 5.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 06:51 AM

    Hi Abrabri,

    which Version of wM are you using?

    The mentioned SFTP-Services are available from wM 9.5 onwards.

    Previous versions of wM do not support SFTP out-of-the-box, only FTPS.
    FTPS can be handled by pub.client.ftp-Services.

    For selecting files by Pattern you can use JZooWildcard-Filter from JavaZoo.
    It is public domain open source.

    Regards,
    Holger


    #webMethods
    #Integration-Server-and-ESB


  • 6.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 06:57 AM

    thank you MR as173d i didn’t understand “redirect the file names to some file”,

    think you Holger : the version of my WM is 9.7 attatched all services i have under sftp,there is no pub.client.sftp:list .

    Regards

    SFTPServices.png


    #webMethods
    #Integration-Server-and-ESB


  • 7.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 07:02 AM

    what I mean is when you use ls command you will get file names in that directory ( Use ls to get files of that pattern) and re-direct to some file, see below one.

    ls > FileNames.txt

    Now you have all file details in FileNames.txt. Now loop over this file and perform your business logic.

    Thanks,


    #webMethods
    #Integration-Server-and-ESB


  • 8.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 07:19 AM

    thank you,
    what i want is :

    ls all files thas respect a specific pattern
    and move returned files(respecting pattern) of ls command to my local folder, and then get one by one using theire date of modification.

    and finnaly use the content of each file : i dont have proble with this step.

    any suggestion.
    Regards


    #Integration-Server-and-ESB
    #webMethods


  • 9.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 07:37 AM

    Use the below code:

    1. ls <file_name>_yyyy_mm_dd> > FileNames.txt
    2. use java streams to read above file line by line
    3. Move each file to your local
    4. As you have file names in FileNames.txt and files in your local, loop over FileNames.txt using java streams and perform your logic.

    Thanks,


    #Integration-Server-and-ESB
    #webMethods


  • 10.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 07:39 AM

    thank you MR as173d ,
    i will give it a try and tell you if it works or not;
    Regards,


    #webMethods
    #Integration-Server-and-ESB


  • 11.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 07:48 AM

    Sure, I gave only pseudo code, you should implement it.

    Thanks,


    #webMethods
    #Integration-Server-and-ESB


  • 12.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 11:35 AM

    Hi,

    here is the description from the IS Build-In-Services Guide for pub.client.sftp:ls:

    There is no need to redirect the Output of ls to a new file, the service will return a list of matching files in parameter dirList.
    You can use this List for your loop.

    Regards,
    Holger


    #Integration-Server-and-ESB
    #webMethods


  • 13.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 11:43 AM

    hi,
    thank you for yr reply,

    but i need to use the content of each file returned by ls, the content dosnt returned by ls???

    and copy every filed returned by ls to local folder

    ps: idon’t find yet how to use ls with pattern date , when i do namefile* it work good but its not what i need, what i need is to do like shell command : ls namefile which respect general pattern name_yyyy-mm-dd.txt

    Regards,


    #webMethods
    #Integration-Server-and-ESB


  • 14.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 11:53 AM

    Hi Abrabri,

    please check the pseudo code from my first post (meanwhile corrected with ls instead of list).

    What will be the value of yyyy-mm-dd when executing ls?

    If this cannot be provided to ls during runtime (i.e. 2016-01-25) you will have to do a branch inside the loop to see if the current filename from the list matches the pattern or not. If not just skip it, if yes read from remote and write to local.

    Regards,
    Holger


    #Integration-Server-and-ESB
    #webMethods


  • 15.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 12:05 PM

    hi Holger von Thomsen thank you for yr reply,

    thats exactly what i did for the first step but i dont know what to don in remotefile of get,
    for the pattern could you please show me how to use branche to test if the pattern match or not,
    attatched my code
    Regards,


    #Integration-Server-and-ESB
    #webMethods


  • 16.  RE: SFTP Remote folder, and pub.client.sftp:get by older modified date

    Posted Mon January 25, 2016 12:06 PM

    hi Holger von Thomsen thank you for yr reply,

    thats exactly what i did for the first step but i dont know what to don in remotefile of get,
    for the pattern could you please show me how to use branche to test if the pattern match or not,

    attatched my code.
    Regards,


    #webMethods
    #Integration-Server-and-ESB