webMethods

 View Only
Expand all | Collapse all

Flat files Via FTP

  • 1.  Flat files Via FTP

    Posted Wed June 22, 2005 10:37 AM

    Hi,
    i have created the Fixed length Flat file schema in the developer.For this schema document i have to assign values from the incoming flat files via FTP.
    Please help me to write a flow service to receive the flat file via FTP.And let me know how can we get the filename of the incoming flatfile.It is urgent.
    Thanks,
    Viji.


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 2.  RE: Flat files Via FTP

    Posted Wed June 22, 2005 03:32 PM

    Vijaya,

    FTPing a flatfile say (ff.txt)your service input should be ffdata(object–stream)then your sample flow steps are
    streamTobytes(convert ffdata to bytes)
    bytesToString(convert bytes to flatfile string)
    pub.flatfile:convertToValues(This will parse the flatfile based on the FFSchema you have created…gives output ffValues (IDATA Document))
    start Mapping…

    To get the incoming flatfile orginal name use the service
    pub.flow:getTransportInfo see the output transport/ftp/filename and extract it from the pipeline.

    Pls do Search this forum if you have more queries on flatfile processing,many threads discussed on this procedure.

    HTH,
    RMG


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 3.  RE: Flat files Via FTP

    Posted Thu June 23, 2005 05:56 AM

    Hi RMG,
    Thanks to ur valuable reply.i have some doubt in this.As u said when we design the flow service to accept the flat file from IS ,it should inlcude all the steps what you have told above.
    What is the use "pub.client.ftp " whether it is used to send Flat file to Is or receive the flat file from IS.
    I think,i have to include this service by setting the Command as “GET” as a first step in the service which includes all of ur above steps.
    If it is not So, clarify me.

    Thanks,
    Viji.


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 4.  RE: Flat files Via FTP

    Posted Thu June 23, 2005 06:35 AM

    I am using IS and Developer 4.6.
    I am very new to webMethos and want to write a small service which can read a XML file and copy the content of the XML file to SQL Server database. I am able to write the service which reads the XML file from d: and copy the record to SQL Server 2000. Now I want to do the same by supplying the XML file thru ftp.
    Till now what I did is

    1. I made my service as startup service
    2. When I am putting my XML file to [url]ftp://localhost:7777/ns/DBAccess/LoadTrueFTP[/url] folder, I am able to see the file name, type, loginID using pub.flow:getTransportInfo.
    3. I am using pub.io:streamToBytes to read the content of teh file, but I am unable to do that. the revices I have used is –

    pub.io:streamToBytes
    pub.string:bytesToString
    pub.web:stringToDocument
    pub.web:queryDocument

    With no input defined for service. Please help me how to do this. Remamber I am using version 4.6.


    #webmethods-Protocol-and-Transport
    #webMethods
    #Integration-Server-and-ESB


  • 5.  RE: Flat files Via FTP

    Posted Thu June 23, 2005 01:58 PM

    pub.client:ftp is a client service. This means that you use it to access information from an external FTP server.

    The steps that Ram gave are for setting up an FTP port on your Integration Server. This makes your Integration Server appear very much like any other FTP server except you may be using another port instead of the usual 21.

    When an external client, for example, FTP at a Windows command-line prompt (And yes, it could even be an Integration Server executing pub.client:ftp) opens a session with your integration server their commands actually execute a service even though it looks to the client like a file is being accessed.

    Please take a look at the Administrator and Developer guides as they explain how this works very well.

    Regards


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 6.  RE: Flat files Via FTP

    Posted Thu June 23, 2005 03:29 PM

    Vijaya,
    What is the use "pub.client.ftp " whether it is used to send Flat file to Is or receive the flat file from IS.
    I think,i have to include this service by setting the Command as “GET” as a first step in the service which includes all of ur above steps.

    yes,using GET it will pull the files from the remote location or shared network folder and then using PUT command place the files directly to the ns/service and follow the above steps.

    HTH,
    RMG


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 7.  RE: Flat files Via FTP

    Posted Thu June 23, 2005 03:37 PM

    Vikash,

    When you ftp PUT xml or any files directly to the ns/service just set the service input as contentStream(object)which will be in the pipeline and this makes streamTobytes works as expected.But in IS6.x if you specify content-type “text/xml” in the filepolling section the service input should be node(object).

    goodluck,

    HTH,
    RMG


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 8.  RE: Flat files Via FTP

    Posted Mon June 27, 2005 10:30 AM

    Hi RMG,
    Thanks for your reply. I have added an Object type Input in InputOutput tab with the name contentStream and mapped the ‘stream’ service-in parameter of the streamToBytes service with contentStream. But still I am unable to do the load. Please help me how to make the streamToBytes service work in Developer 4.6. It is urgent.


    #webmethods-Protocol-and-Transport
    #webMethods
    #Integration-Server-and-ESB


  • 9.  RE: Flat files Via FTP

    Posted Mon June 27, 2005 11:28 AM

    Hi RMG,
    Thanks to ur reply.I will use the same steps in my service for receiving ,incoming Flatfile(ff.txt) from IS to the developer.I have created the Fixed length Flat file schema with the format like,
    the schema has 3 records PurchaeseOrder,Header,Lines.The purchaseOrder contains header and line records only.The hierarchy of the schema is
    ----- Purchase Order[0] (Main rec-repeatable)
    ---------Header (unique)
    -----------Account
    -----------PO Number
    -----------Customer’s PO
    -----------Currency Code
    -----------Bill to date
    ---------Lines[0] (repeatable)
    -----------POnumber
    -----------Line number
    -----------ModelNAme
    -----------ModelCode
    ----------- .
    ----------- .
    ----------- .

    ----PurchaseOrder[1]
    ------- Header
    -----------.
    -----------.
    --------Lines[1]
    -----------.
    -----------.
    The output of this record values must be,

    PurchaseOrder[0]HeaderAcc100PO200CusPO200Curcod2013/03/2005Lines[0]PO200Line300Model1Code34Lines[1]…

    For this schema,
    How the ff.txt file should be looking like?Can you please let me know the content of this file so that i can pass values and test the service.

    Thanks in advance,
    Viji.


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 10.  RE: Flat files Via FTP

    Posted Mon June 27, 2005 03:15 PM

    Vijaya,

    Use any texteditor like ultraedit and put the data in the fixed positions including RecordDefinitions as per defined in the schema and make sure the data is the right positions,so that FFSchema parses successfully.

    HTH,
    RMG


    #webmethods-Protocol-and-Transport
    #webMethods
    #Integration-Server-and-ESB


  • 11.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 11:11 AM

    Hi RMG,
    The attached one is the sample of Flat file Format which to be sent to the Integration Server via FTP . I have created the Fixed length flat file schema in the wMDeveloper by setting the record length of 1169 .
    By referring this POFormat document each record occupies 1169 characters Length.The parent record PurchaseOrder include the child records Header and Line.
    The problem is this record contains 1156 blanks following the name PurchaseOrder so that it fits the entire width of the record(1169).
    When sending the flat file to IS , the user has to design the flat file with 1156 blankspaces after entering the name PurchaseOrder.Is it so?
    can u pls send me the sample value flat file(ff.txt) for this schema?flatfile


    POFormat.doc (24.6 k)


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 12.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 04:09 PM

    Vijaya,
    yes the user has to design the flat file with 1156 blankspaces when sending the flatfile and also in the FFSchema you should add Blank FieldDefinition(contains 1156 spaces)under the PurchaseOrder RecordDefinition.So this makes parsing successful.

    Hope you are clear on this.

    HTH,
    RMG


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 13.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 06:48 PM

    Hi RMG,
    Please help me also.
    Thanks in advance.
    –Vikash


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 14.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 07:09 PM

    Vikash,

    Please elaborate on your FTP processing flow?

    Actually contentStream input should work.For debugging purpose try to do this way…After the StreamTobytes service call getTransportInfo and savepipeline service and later restore it check what is coming in the pipeline.

    Note:Stream objects cannot be saved and it will show up with different notation but atleast you will know whats going on when ftp transport puts file on the ns/service.

    HTH,
    RMG.


    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport
    #webMethods


  • 15.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 07:28 PM

    Hi RMG,
    Thanks for your reply.

    I am using developer 4.6 I want to insert the records in SQL Server from an XML/Flat file. I want that the XML/Flat file should should be submitted thru FTP. For this I have configured the ftp thru IS and my ftp server is working on port 7777. At presently I am using single machine for all. Now I have created a Package Vik, under that I have a Folder DBAccess with in that I have 5 services. I setup the startup service to LoadThruFTP as this is my mail service. Within this service I am calling streamToBytes as the first service. After the flow is

    pub.io:streamToBytes
    getTransportInfo
    SavePipelinetofile
    pub.string:bytesToString
    pub.web:stringToDocument
    pub.web:queryDocument
    pub.string:bytesToString
    pub.web:stringToDocument
    pub.web:queryDocument

    If I it in the above sequence then no result get stored into the pipeline file.But if i remove the streamToBytes service then the transport information is getting stored into the pipeline file. It seems that the streamToBytes service is not working perfectly. I tried using contentStream also. For this I have added the contentStream into Input to the service as Object type and mapped it to content variable of the service-in of StreamToBytes.
    I tried all but no sucess. Please Guide.

    Thanks


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 16.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 07:39 PM

    What variables you are seeing in the transportinfo pipeline?
    First tell me are you ftping the file directly to ns/service directly?
    Are you setting content-type=“text/xml” in the ftp process.

    HTH,
    RMG


    #Integration-Server-and-ESB
    #webMethods
    #webmethods-Protocol-and-Transport


  • 17.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 08:41 PM


  • 18.  RE: Flat files Via FTP

    Posted Tue June 28, 2005 08:47 PM

    Try setting node(object)in the service input and eliminate streamToString and use documentToRecord.

    HTH,
    RMG


    #Integration-Server-and-ESB
    #webMethods
    #webmethods-Protocol-and-Transport


  • 19.  RE: Flat files Via FTP

    Posted Wed June 29, 2005 05:46 AM

    Hai RMG,
    Thanks to ur cooperation.Again i have one problem.I hope you will help to me.
    How can we set the default value(blank spaces)for a field definition of fixed position 13-1169.In the property panel i have selected the position property in thati have selected the “default
    value” checkbox.But it is not assigned.
    And also clarify me,i have a field called trailer which should have the total number of records as its value.How can I receive the total records and assign to trailer.
    Please reply ,it is very urgent,.
    Thanks,
    Viji


    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport
    #webMethods


  • 20.  RE: Flat files Via FTP

    Posted Wed June 29, 2005 05:48 AM


  • 21.  RE: Flat files Via FTP

    Posted Wed June 29, 2005 05:20 PM

    Vijaya,

    i have a field called trailer which should have the total number of records as its value.How can I receive the total records and assign to trailer.

    You should build this logic in the mapping once you parsed the flatfile.

    Regarding Blank field select Start and End positions not default…and specify the index.

    HTH,
    RMG


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 22.  RE: Flat files Via FTP

    Posted Thu June 30, 2005 06:29 AM

    RMG,
    Thanks to ur timely help.
    Can i proceed without setting this blank field definiftion.I think in both ways( with and without blank field)the user has to enter the blanks of 1156 charlength in the flat file (ff.txt).Anyhow the schema is parsed the file successfully.So why should we add this field definition in that Parent record?IS it necessary?

    Iam using IS 6.0 and iam trying to create the service which is used to receive the ff.txt from IS via FTP.
    As u said,I think these are the steps to be followed in the service.
    FTPing a flatfile say (ff.txt)your service input should be ffdata(object–stream)then your sample flow steps are

    1. pub.client.ftp (set command=GET)
      2.streamTobytes(convert ffdata to bytes)
      3.bytesToString(convert bytes to flatfile string)
      4.pub.flatfile:convertToValues(This will parse the flatfile based on the FFSchema you have created…gives output ffValues (IDATA Document))
      start Mapping…
      IS IT RIGHT?
      Thanks,
      Viji

    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB
    #webMethods


  • 23.  RE: Flat files Via FTP

    Posted Thu June 30, 2005 03:17 PM

    So why should we add this field definition in that Parent record?IS it necessary?
    If the blank is the showstopper then Blank fielddefinition is required otherwise,you can ignore it.

    If you are ftping the file(ff.txt) directly to the service the service input should be ffdata(stream object)and these are the steps to follow
    1.streamTobytes(convert ffdata to bytes)
    2.bytesToString(convert bytes to flatfile string)
    3.pub.flatfile:convertToValues(This will parse the flatfile based on the FFSchema you have created…gives output ffValues (IDATA Document))
    start Mapping…

    How the user is sending the file?
    If the user stores the file in a directory and you are pulling it then…

    1. pub.client.ftp (set command=GET)
      3.bytesToString(convert content(bytes) to flatfile string)
      4.pub.flatfile:convertToValues(This will parse the flatfile based on the FFSchema you have created…gives output ffValues (IDATA Document))
      start Mapping…

    I hope you are clear on this.

    HTH,
    RMG


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 24.  RE: Flat files Via FTP

    Posted Tue July 05, 2005 01:06 PM

    Thanks RMG,
    Now iam working on the same service.I will use the steps.I hope it will be running properly.
    Thanks,
    Viji


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 25.  RE: Flat files Via FTP

    Posted Wed July 06, 2005 03:49 PM

    let us know if still problem persists.

    Goodluck,


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 26.  RE: Flat files Via FTP

    Posted Thu July 07, 2005 10:55 AM

    Hi,
    I am not able to ftp my file from server on to webMethods FTP port without changing its extension to .txt. when i execute put command from ftp> prompt
    ftp> put ORDRS1.2331320.x12
    Error message : " Cannot write".

    Can u help me to execute this ftp command directly.
    when I execute this as
    ftp> put ORDRS1.2331320.x12.txt
    CWD successful.

    Pls help me in this regard.

    Thanks
    Amit


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 27.  RE: Flat files Via FTP

    Posted Thu July 07, 2005 03:03 PM

    Amit,

    I beleive you can only use with extension as .txt or .dat or .xml or .csv etc… then ftp will be successful.since webMethods contenthandler will accept any of the above.Otherthan above then you have create own custom contenthandler in WM a bit of more work.

    HTH,
    RMG


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 28.  RE: Flat files Via FTP

    Posted Fri July 08, 2005 11:26 AM

    You can put files with any other extension but the ‘default’ ones such as .txt, .xml etc to the Integration Server by explicitly passing the content-type in the PUT command:

    put [localfilename] [remotefilename];[remotecontenttype]

    Replace the ‘/’ in the content type with ‘:’.

    For example, an xml file with extension .exm can be put to the integration server using the command:

    put test.exm test.exm;text:xml

    Regards,

    Koen


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 29.  RE: Flat files Via FTP

    Posted Fri July 15, 2005 08:24 AM

    RMG,
    I have created the service in 6.1 to receive the flat file from FTP.
    In this i have inluded the steps

    1. pub.client.ftp (set command=GET)
      3.bytesToString(convert content(bytes) to flatfile string)
      4.pub.flatfile:convertToValues(This will parse the flatfile based on the FFSchema you have created…gives output ffValues (IDATA Document)) .
      But i didn’t declare any input for this service in the pipeline(like stream or node).
      And it is not working ,
      Do i have to establish the FTP Sessions thru the service?
      I just logged into FTP .
      Can you please clarify me in FTP handling with Developer?
      Thanks

    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport
    #webMethods


  • 30.  RE: Flat files Via FTP

    Posted Fri July 15, 2005 03:43 PM

    What is the error you are getting?

    pub.client.ftp(GET)do your user have enough access to connect that remote FTP system.

    please elaborate.

    HTH,
    RMG


    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport
    #webMethods


  • 31.  RE: Flat files Via FTP

    Posted Fri July 15, 2005 03:46 PM

    One more thing,If you are using ftp command get extract the content/bytes from the serviceoutput and so there is no need to set input since your whole code is wrapped in one flowservice right?

    HTH,
    RMG


    #Integration-Server-and-ESB
    #webMethods
    #webmethods-Protocol-and-Transport


  • 32.  RE: Flat files Via FTP

    Posted Tue July 19, 2005 12:07 PM

    Yes.I have enclosed all steps into a single service.And i removed the input from the service.
    Using PUT service i have placed a flat file in FTP under one
    folder.when i try to use GET with the same directory path , it is giving File not found exception.please help me to get rid out of it.

    Thanks


    #webMethods
    #Integration-Server-and-ESB
    #webmethods-Protocol-and-Transport


  • 33.  RE: Flat files Via FTP

    Posted Tue July 19, 2005 12:22 PM

    Hi guyz

    I am having in the production server. If we try to hit the url http://hostname:5555 we don’t get the usual login dialog-box. It show some funny page with “Listing of” then Type, filename, date modified etc below that; and the name of the html page is “Directory of ()”. Well these are seemingly entries from the dir.html file but I have no idea what is happening or what is the cause for all these! Worst, this is the production machine! Help please!

    Thanking you in advance!

    Aqaba


    #Integration-Server-and-ESB
    #webMethods
    #webmethods-Protocol-and-Transport


  • 34.  RE: Flat files Via FTP

    Posted Tue July 19, 2005 02:57 PM

    More info on the login screen problem. I am 100% positive that what is displayed is the content of the “IntegrationServer\lib\dir.html” file but no idea how it is called and what is calling it??? !!!


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB


  • 35.  RE: Flat files Via FTP

    Posted Tue July 19, 2005 03:11 PM

    Amon,

    Looks like the IS index.dsp file may got corrupted,but if you use [url]http://hostname:5555/WmRoot[/url] then the login box will show up.so this wont stop you work on.

    But i am not sure why the index file is corrupted,even we have the same problem seen in one of our QA system.

    HTH,
    RMG


    #webMethods
    #webmethods-Protocol-and-Transport
    #Integration-Server-and-ESB