IBM webMethods Hybrid Integration

IBM webMethods Hybrid Integration

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.


#TechXchangePresenter
 View Only
Expand all | Collapse all

Read CSV from URL

  • 1.  Read CSV from URL

    Posted Wed January 19, 2022 01:26 PM

    Hello!

    Im new in the integration Service world and need some help.

    Im trying to read a CSV file directly from a URL but its not working.

    Here is what im doing inside my service:

    pub.client:http (https:\myserver.com\file.csv, get method)
    pub.flatfile:convertToValue (created FFSchema here)

    When i run the server i get the error:
    com.wm.app.b2b.server.ServiceExpection:java.io.IOExpection:java.net.SocketTimeoutException: Read Ttimed out

    The URL/file is accessible inside the IS server

    Can someone help me?


    #webMethods
    #Service-Designer
    #Integration-Server-and-ESB


  • 2.  RE: Read CSV from URL

    Posted Thu January 20, 2022 01:41 AM

    Hi Fabio,

    The URL you are trying to access is HTTPS…did you try setting the secure parameters in the HTTP client service? It would be good if you can put the entire stack trace of the error that you are facing.

    Thanks,
    Sree


    #Service-Designer
    #Integration-Server-and-ESB
    #webMethods


  • 3.  RE: Read CSV from URL

    Posted Thu January 20, 2022 01:43 AM

    “Read Timed Out” - Possible of the socket connection is getting closed.

    Remember pub.client:http will set the SSL timeout before calling the target if incase its https hosts. So verify the watt.net.ssl.server.clientHandshakeTimeout extended property and set accordingly.


    #Integration-Server-and-ESB
    #webMethods
    #Service-Designer


  • 4.  RE: Read CSV from URL

    Posted Thu January 20, 2022 09:00 AM

    how do i check/set this property?

    we are using Integration Service 10.5, onpremisse installation.


    #webMethods
    #Integration-Server-and-ESB
    #Service-Designer


  • 5.  RE: Read CSV from URL

    Posted Thu January 20, 2022 11:18 AM

    Hi Fabio,

    in IS Admin under Settings → Extended Settings.
    There is a link to “Show and Hide …” which opens a list with known properties, search for the mentioned property, mark the checkbox in front of it an save.
    After that the property will show up in the list of extended settings in the UI.
    Via the “Edit …” link you can then modify the property.
    After saving you might need to restart the IS to apply the modification.
    See IS Administrators Guide for further informations about this.

    Regards,
    Holger


    #Integration-Server-and-ESB
    #Service-Designer
    #webMethods


  • 6.  RE: Read CSV from URL

    Posted Tue January 25, 2022 10:49 AM

    Hi Fabio -

    If you’re getting a “read timed out,” you may have already gotten past the SSL handshake. That error means you’re waiting to receive data from the server, but no data arrived in time. There’s a “timeout” input variable in the pub.client:http service that you can play with. If you don’t specify a value for that variable, the IS will use the value from watt.net.timeout so you could also configure it there if you prefer to set it globally (see Reverb)

    Having said this, in my experience, timeouts are typically due to a more deeply rooted issue, so here are some things you can try to find the root cause:

    1. Ensure you’re using the correct URL. I noticed the URL you shared in your post had back-slashes instead of forward-slashes. I’m sure that was a typo, but just in case, make sure the URL is correct.

    2. If the server allows it, temporarily change to simple HTTP instead of HTTPS to eliminate TLS as a variable.

    3. Try pulling a different file from the server to determine if this problem is specific to the file (e.g. try pulling a small text file).

    4. Try pulling the file down via a different mechanism. In other words, from the same server where the IS is running, can you download the file via a browser or via curl?

    5. Try pulling the file from a different Integration Server (e.g. one running on your local machine).

    6. If the server you’re pulling the file from is behind a load balancer, can you go directly to the server to determine if it’s an issue with the load balancer?

    7. Does anything get logged in the target server when you attempt your request?

    Trying these different things should help you get closer to determining the root cause of your timeouts.

    Hope this helps,
    Percio


    #Integration-Server-and-ESB
    #webMethods
    #Service-Designer


  • 7.  RE: Read CSV from URL

    Posted Mon January 31, 2022 06:14 AM

    Hi Guys!

    I have some new info to share about this problem:

    1 - Setting this property doesnt workwatt.net.ssl.server.clientHandshakeTimeout;
    2 - I can access the service/csv URL from the server browser (https, its a external, public URL, .gov)
    3 - I had to set our internet proxy inside the IS
    3.1 Set the proxyAlias inside pub.client:http

    URL:
    https://www4.bcb.gov.br/Download/fechamento/20220105.csv

    Now im having a different error while running the service with just the pub.client:http

    com.wm.app.b2b.server.ServiceException: java.io.IOException: javax.net.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider;certpath.SunCertPathBuilderException: unable to find valid certification path to requested target


    #Integration-Server-and-ESB
    #Service-Designer
    #webMethods


  • 8.  RE: Read CSV from URL

    Posted Mon January 31, 2022 06:17 AM

    Additional info:

    Already inclued our certfile in Security->Certificates Client Certificates and Security → Keystore


    #Integration-Server-and-ESB
    #Service-Designer
    #webMethods


  • 9.  RE: Read CSV from URL

    Posted Mon January 31, 2022 08:24 AM

    Additional info:

    tryed this command on both IS Server and Desginer machine:

    keytool -import -trustcacerts -file certificate_path.crt -alias “Certificate” -keystore Software AG_directory\jvm\jvm160_32\jre\lib\security\cacerts

    same error


    #Integration-Server-and-ESB
    #Service-Designer
    #webMethods


  • 10.  RE: Read CSV from URL

    Posted Thu March 03, 2022 02:24 PM

    @fabio.morciani I apologize for the late reply. Hopefully, you’ve figured this out by now. In case you haven’t, the certificate that you added to the truststore, was that the CA certificate from the target HTTPS endpoint? It wasn’t clear from the post.

    Also, the Integration Server defines a separate truststore via the DEFAULT_IS_TRUSTSTORE alias. If I recall correctly, you have to add the CA cert there instead of cacerts. Better yet, you should create your own truststore file and reconfigure the DEFAULT_IS_TRUSTSTORE alias to point there. Similarly, you should create your own keystore and point DEFAULT_IS_KEYSTORE there.

    Percio


    #webMethods
    #Integration-Server-and-ESB
    #Service-Designer