B2B Integration

 View Only
  • 1.  How can I use SFG to pull a file from a website with a GUI?

    InnerCircle
    Posted Tue November 09, 2021 12:28 PM
    How can I use SFG to pull a file from a website with a GUI?


    Thank You and Please Be Safe!

    Len Sasso
    Systems Administrator Senior
    CSRA, A General Dynamics Information Technology Company
    327 Columbia TPKE
    Rensselaer, NY 12144

    TEAM: Together Everyone Achieves More

    Office Hours: M-F  7 AM - 3:45 PM

    Vacation: ?
                              
    Phone: (518) 257-4209
    Cell: (518) 894-0879
    Fax: (518) 257-4300

     
     


  • 2.  RE: How can I use SFG to pull a file from a website with a GUI?

    Posted Tue November 09, 2021 06:00 PM
    Hi Len,

    SFG has HTTP services which can perform the HTTP operations like GET and POST. But SFG cannot replicate the functionality like what a human user does by accessing a website, browse and download a file. If you know the HTTP URL then SFG can get the file.

    I hope this answers your question.

    Regards,

    ------------------------------
    BHARAT BALOTHIA
    ------------------------------



  • 3.  RE: How can I use SFG to pull a file from a website with a GUI?

    InnerCircle
    Posted Wed November 10, 2021 07:27 AM
    Hi Bharat Balothia!

    I know the HTTP URL (https://epf.usps.gov).

    The site requires an email address and password.


    Thank You and Please Be Safe!

    Len Sasso
    Systems Administrator Senior
    CSRA, A General Dynamics Information Technology Company
    327 Columbia TPKE
    Rensselaer, NY 12144

    TEAM: Together Everyone Achieves More

    Office Hours: M-F  7 AM - 3:45 PM

    Vacation: ?
                              
    Phone: (518) 257-4209
    Cell: (518) 894-0879
    Fax: (518) 257-4300

     
     





  • 4.  RE: How can I use SFG to pull a file from a website with a GUI?

    Posted Mon November 15, 2021 01:23 AM
    Hi Len,

    I don't think that's possible. In case of SFG, if you can download a file from a website using cURL tool then you can also do the same with SFG.

    The use case which you are looking for is to automate the manual process using SFG which requires logging in username and password and then clicking on some link to download the file. Which is kind of building a BOT.

    Regards,

    ------------------------------
    BHARAT BALOTHIA
    ------------------------------



  • 5.  RE: How can I use SFG to pull a file from a website with a GUI?

    Posted Tue November 16, 2021 07:20 AM
    You don't necessarily need to emulate the full GUI as this site uses fairly basic static HTML. In this case you can just bypass the landing page and go directly to the login. The page does a POST to https://epf.usps.gov/index.cfm with the form data in the POST containing 3 fields:
    action = "login"
    email
    password.
    You can use the HTTP POST service to do that. You'd then need to look at the HTML returned from the login and figure out what subsequent GETs or POSTs you might need to achieve what you want including passing the session cookie (or whatever) returned from the initial login POST plus probably parsing the HTTP response body to find the relevant links to then use the HTTP GET client to download from. You could test using a tool such as Postman. Basically though as Bharat says you are building a BOT.

    Having said that it looks like USPS might offer some APIs you could instead call directly which would be far simpler and less likely to break if USPS update the website: https://postalpro.usps.com/address-quality

    ------------------------------
    RICHARD CROSS
    ------------------------------



  • 6.  RE: How can I use SFG to pull a file from a website with a GUI?

    InnerCircle
    Posted Wed December 01, 2021 12:47 PM
    Hi Richard !

    I'm new to this, you stated: 'Having said that it looks like USPS might offer some APIs you could instead call directly which would be far simpler and less likely to break if USPS update the website: https://postalpro.usps.com/address-quality"  

    How would SFG "call directly" these APIs?  Please explain.

    ------------------------------
    Len Sasso
    System Administrator Senior
    GDIT (General Dynamics Information Technology)
    EAST GREENBUSH NY
    (518) 894-0879
    ------------------------------



  • 7.  RE: How can I use SFG to pull a file from a website with a GUI?

    Posted Fri December 03, 2021 11:05 AM
    Hi Len,

    Being a Brit I'm not too familiar with the USPS unfortunately ;-)

    Typically the APIs will use REST in which case the method to access them is not too dissimilar from scraping a web page e.g. you can use the HTTP client adapter to GET, POST etc. but without having to worry about all the presentation elements of a web page. With SFG v6.x there is a REST API Client Service (see https://www.ibm.com/docs/en/b2b-integrator/6.0.2?topic=z-rest-api-client-service) which makes it far easier to call a REST API, combined with the new JSON Transform service as typically the REST services use JSON rather than XML. You'll need to contact USPS for more detail of their APIs as the web site says to contact their licensing team for the developer docs (so I guess you'd also need to discuss licensing access).

    Best regards,
    Richard.

    ------------------------------
    RICHARD CROSS
    ------------------------------