Managed File Transfer

 View Only
Expand all | Collapse all

Zip files and Routing Channel Templates

  • 1.  Zip files and Routing Channel Templates

    Posted Tue July 14, 2020 11:47 AM
    I'm fairly new to SFG and still struggling with a lot of it. This may be an over-simple question for this group, for which I apologize.
    Current issue: Daily, we create files that we need to drop on a client's mailbox. The files are in a subfolder (usually named v1, but sometimes v2) of a folder that is the date the work was done (not necessarily today's date unfortunately).
    I haven't been able to get SFG to create the directories and move the files with a Template (is this possible?), so I thought I could zip them and let SFG unzip them for me. I am unclear on how to create a template for this though, especially given the documentation says SFG looks at the first level below the file level and tries to match that to the template.
    The zip file has these layers (using today's date as an example):
    20200714.zip
       20200714                  (a folder)
          v1                           (a folder)
              daily_totals.csv
              exceptions.txt
    Given the first level is a date that always changes, do I need to build a custom Business Process to make this work?

    ------------------------------
    Dana Jenner
    ------------------------------

    #DataExchange
    #filetransfer


  • 2.  RE: Zip files and Routing Channel Templates

    Posted Wed July 15, 2020 12:41 PM

    Hi Dana,

    I'm trying to find a subject matter expert that will help answer your question. 

    Meanwhile, have you looked at the resources and documents in our Knowledge centre for SFG here?



    ------------------------------
    Tanvi Kakodkar
    Supply Chain Community Manager
    IBM
    ------------------------------



  • 3.  RE: Zip files and Routing Channel Templates

    Posted Thu July 16, 2020 02:05 AM
    Hi Dana Jenner,

    This is a very good usecase. I can think an option to tackle this requirement as there is no direct out-of-box approach to handle such files.

    Option:
    - Add a custom protocol on the consumer
    - Unzip the files using a custom business process
    - Create the mailbox dynamically based on your folder structure on the consumer mailbox (I assume you are looking for the same mailbox structure as the folder structure in the zip file 20200714->v1)
    - Add the messages to the newly created mailbox

    I believe producer custom file layer option also exist but it might become little complex.

    Note: I recommend to have your house keeping schedule to delete all the old mailboxes (Ex: 20200714 etc) that are no longer required.


    ------------------------------
    KALYAN PAVAN KISHORE CHAKRAVARTHULA
    ------------------------------



  • 4.  RE: Zip files and Routing Channel Templates

    Posted Fri July 17, 2020 09:33 AM
    Does client need those folder ? Other option is to write custom bp and set as custom delivery protocol. BP Unzips files and puts in consumers /Inbox. Set message name as folder-folder-file.. i.e. 20200714-V1-daily_totals.csv and 20200714-V1-exceptions.txt. Or can you ask client to take zip files ?

    Write Generic Custom Delivery Protocol, which can take any BP as input and run it inline.

    ------------------------------
    Ajit Rathod
    ------------------------------



  • 5.  RE: Zip files and Routing Channel Templates

    Posted Fri July 17, 2020 10:52 AM
    Thanks for the quick replies. 
    It turns out the zip file structure doesn't have the v1 directory.
    If I don't zip the files, I have a variable date-named directory containing a number of files -
    20200717
        file1.txt
        file2.csv
        etc.
    But that doesn't help. You're both saying that there's no way to create that directory with a Routing Channel Template?
    I'll need both a custom protocol and a bp to accomplish this?.

    ------------------------------
    Dana Jenner
    ------------------------------



  • 6.  RE: Zip files and Routing Channel Templates

    Posted Fri July 17, 2020 04:24 PM
    Am I missing something or is there no file structure for folders (directories?).

    ------------------------------
    Dana Jenner
    ------------------------------



  • 7.  RE: Zip files and Routing Channel Templates

    Posted Mon July 20, 2020 04:34 AM
    Hi Dana,

    You can extract "facts" from the producer filename which can then be used to define the structure of both the consumer filename and the consumer mailbox path e.g. the following takes the first part of the name of the zip file uploaded e.g. "20200719.zip" to create the consumer mailbox path e.g. /Inbox/20200719 and deliver all the files in the zip to that folder, keeping the original filenames. You could extend this to include your v1/v2 folders for example by uploading as e.g. 20200719_v1.zip and have your RegEx as (.+)_(.+).zip and adding a 2nd fact to the producer zip structure e.g. myDirectory, myVersion and then specifying your consumer mailbox path as say ${ConsumerName}/${myDirectory}/${myVersion} (or however you want the path structured). Make sure you check the box to have the consumer mailbox created dynamically.
    Routing Channel Template to extract files from zip and add to consumer mailbox path based on zip filename


    ------------------------------
    RICHARD CROSS
    ------------------------------



  • 8.  RE: Zip files and Routing Channel Templates

    Posted Mon July 27, 2020 11:45 AM
    Hi Richard,
    I was out on vacation last week.
    I created a similar Template this morning and it works like a charm.
    I had thought facts need to be defined when the Routing Channel is created, which obviously wouldn't work here (at least for the directory name). Being able to extract them when the Channel runs is a great help.
    Thank you so much!

    ------------------------------
    Dana Jenner
    ------------------------------



  • 9.  RE: Zip files and Routing Channel Templates

    Posted Mon July 27, 2020 06:42 PM
    That's great Dana, Richard.

    This is assuming zip file name has the required  "folder structure" and zip doesn't contain any folders within ?

    ------------------------------
    Ajit Rathod
    ------------------------------



  • 10.  RE: Zip files and Routing Channel Templates

    Posted Tue July 28, 2020 08:13 AM
    Hi Ajit, I don't think SFG supports folder structures within Zip files.

    This is based on creating the zip with a filename that provides the elements you need to build the consumer mailbox path. Any element you surround with braces () in the RegEx is extracted to a fact in the "File name pattern group fact names" list in the corresponding order

    You could also use provisioning facts that are defined in the template and specified at routing channel creation e.g. to implement the v1/v2 you could alternatively add myVersion as a provisioning fact in the template and include that in the Producer mailbox path as ${ProducerName}/${myVersion}. Then create two channels for the same producer/consumer but specifying v1 or v2 for your myVersion fact which will create the corresponding sub-mailboxes for the producer. You'll then know whether it's a v1 or v2 file based on which sub-mailbox the producer uploads it to.

    ------------------------------
    RICHARD CROSS
    ------------------------------



  • 11.  RE: Zip files and Routing Channel Templates

    Posted 18 days ago

    I am trying to create a Routing Channel Template to Unzip a GZIP file. I receive the following message when I try to save it: 

    "Layers nested within GZIP or PGP layers must specify ".*" for the regular expression and cannot specify any facts. Structure is:

    GZIP{(.+)[.](gz|gzip|GZ|GZIP)}/Unknown{(.+)}."

    Any help you could provide would be greatly appreciated.

    Thank You !



    ------------------------------
    Len Sasso
    System Administrator Senior
    GDIT (General Dynamics Information Technology)
    EAST GREENBUSH NY
    (518) 894-0879
    ------------------------------