Cloud Pak for Integration

 View Only
Expand all | Collapse all

Binary communication between dataPower and external system

  • 1.  Binary communication between dataPower and external system

    Posted Mon July 11, 2022 02:12 PM
    My team has started to integrate with an HSM Tales 8000/9000 system. They could generate the command message in an array as a buffer, send the command (which has the first two bytes as the message length in binary). They could get the response which has the same format (first two bytes are binary of the length) only when they configure the MPGW response as passthrough. Then they could get the response at the caller (the PostMann tool) and see the place of the two nonprintable first two bytes and the rest of the printable response.

    They cannot read the message within the MPGW to process it and take the result of the HSM process in the policy.
    Any help please to correct the configuration so that they can get and process the response!

    ------------------------------
    Hisham Ibrahim
    ------------------------------


  • 2.  RE: Binary communication between dataPower and external system

    Posted Thu July 14, 2022 03:22 PM
    Hi Hisham,

    If the goal is sent a binary message with the format of 2-bytes length prefix followed by payload, you can use the net-prefix protocol for either the URL Opener or as a backend URL for a MPGW (a TLS version is also available). In order to process the message, the best method would be to use GatewayScript and read the response with readAsBuffer if the contents are binary. If the contents are in some other format, you can use a different method.

    Please note that there is no provision for accepting messages in this format into DataPower at this time but you can generate them and parse the response from a backend system. So you can use HTTP to send the payload into DataPower and have DataPower and have it attach the binary length prefix before sending it to the backend.

    Hope this helps.

    Francisco

    ------------------------------
    Francisco Moraes
    ------------------------------



  • 3.  RE: Binary communication between dataPower and external system

    Posted Fri July 15, 2022 09:55 AM
    Thank you Francisco. 

    Will discuss with the team (in holiday today)  and get back. 

    As far as I know,  we could send the request to the BE.  However,  not able to get the response using the gatewayscript (not sure if they tried the readasbuffer()).  I believe the issue is that the dp does not expect the binary length bytes in the response and therefore gives exception with no details. 

    When the team (has not much experience) set the response config. to passthru,  the caller (postman tool) gets the response and show it as text (with the first non-printable two bytes). 


    Best Regards,

    Hesham Soultan





  • 4.  RE: Binary communication between dataPower and external system

    Posted Mon July 18, 2022 08:45 AM
    ++Adding the DP team..

    @Riham Lamei please handle with Francisco.

    Best Regards,

    Hesham Soultan, Ph. D.
    (AI, IOT, Business Integration)
    BVS Chief Technology Officer
    Tel: 00201557367399





  • 5.  RE: Binary communication between dataPower and external system

    Posted Tue July 19, 2022 09:59 AM
    Dear Francisco,
    I tried to use the below script and use the "net-prefix" including the following parameters: (x-dp-length-field-byte, x-dp-length-field-overhead).
    There is still an issue states that the "=" is illegal character as in the attached snapshot "Logs" (I am trying to fix; or appreciate your assisting).
    If we removed the entire string of the parameters, we get other issure stating that "illegal character" on the first binary byte of the response length (see the attachement "MPGWlogs").  
    On the other hand, I could not understand your statement "Please note that there is no provision for accepting messages in this format into DataPower at this time". Does this mean it is impossible mission to get the response and process it within the MPGW?
    Any ideas to solve the issue are highly appreciated.
    =====================================================
    <xsl:stylesheet version="1.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:dp="http://www.datapower.com/extensions"
    xmlns:codecs="http://www.tibco.com/asg/functions/codecs"
    extension-element-prefixes="dp"
    >
    <xsl:variable name="hsmResponse" select="/"/>
    <xsl:variable name="targetURL" select="'net-prefix://10.80.165.25:1500/?x-dp-length-field-byte=1&x-dp-length-field-overhead=0'" />
    <xsl:variable name="response">
    <dp:url-open target="$targetURL" response="binaryNode">
    <xsl:value-of select="$hsmMsg"/>
    </dp:url-open>
    </xsl:variable>
    </xsl:stylesheet>


    ------------------------------
    Riham Lamei
    ------------------------------



  • 6.  RE: Binary communication between dataPower and external system

    Posted Tue July 19, 2022 10:20 AM

    Hi Riham,

    I recommend using GatewayScript instead of XSLT to process binary data. If you really want to do it in XSLT, you will need to change the action to processing mode to be binary and you shouldn't have the INPUT connected to it or it will expect XML at the input. I can ask a colleague to chime in if you really need XSLT processing.

    Here's an example:

    // test for the raw url opener
    
    var urlopen = require('urlopen');
    var sm = require('service-metadata');
    
    function testPost(data) {
        var options = {
            target: 'net-prefix://127.0.0.1:4567/?x-dp-length-field-bytes=2&x-dp-length-little-endian=false',
            data: data
        };
    
        console.error("urlopen options: " + JSON.stringify(options));
        console.error("test data: " + JSON.stringify(data));
    
        urlopen.open(options, function (error, response) {
            if(error) {
                session.output.write("urlopen connect error: " + JSON.stringify(error));
            } else {
                console.error("Reading response as buffer: " + JSON.stringify(response));
                response.readAsBuffer(function(error, data) {
                    console.error("Got response: " + JSON.stringify(error) + " " + JSON.stringify(data));
                    if(error)
                        session.output.write(error);
                    else
                        session.output.write(data);
                });
    
            }
        });
    }
    
    console.error("Reading test data from input context");
    
    session.input.readAsBuffer(function(error, buffer) {
        if(error)
            session.output.write("context read error: " + error);
        else
            testPost(buffer);
    });
    


    Regarding my statement about DataPower support, it just means that you cannot accept a request into DataPower in this format (like an HTTP server) but you can send a request to a server and get a response back. In other words, DataPower cannot act as the backend server.

    Francisco



    ------------------------------
    Francisco Moraes
    ------------------------------



  • 7.  RE: Binary communication between dataPower and external system

    Posted Thu August 11, 2022 07:07 AM
    Dear Francisco,

    It worked this way, Many thanks for your effort.

    ------------------------------
    Riham Lamei
    ------------------------------



  • 8.  RE: Binary communication between dataPower and external system

    Posted Fri August 12, 2022 11:15 AM
    Hi Riham,

    Glad it worked.

    Francisco

    ------------------------------
    Francisco Moraes
    ------------------------------



  • 9.  RE: Binary communication between dataPower and external system

    Posted Tue December 20, 2022 10:15 AM
    it worked for me as well , thanks a lot , could you please help that i need to send attachment for backend as a rest api  call , payload type json with attachment and i need to process response with attachment as well . i appreciate that if you provide snippet of the code.

    ------------------------------
    santhoshkumar surisetty
    ------------------------------



  • 10.  RE: Binary communication between dataPower and external system

    Posted Mon January 09, 2023 10:31 AM
    This last question doesn't seem related to the original one, so please make a new post so that someone with more knowledge on that area can help.

    ------------------------------
    Francisco Moraes
    ------------------------------