DataPower

DataPower

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
Expand all | Collapse all

DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424

  • 1.  DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424

    Posted Thu October 07, 2021 08:07 AM

    We have to implement transactional logging from Multi-Protocol Gateway processing rule to a remote syslog sevrer.

    We did use ResultsAction, because LogAction wraps transactional data (XML structure, <AuditMessage>...more XML.. </AuditMessage> ) into SOAP XML envelope and adds additional information, what we do mot need..what remote server does not accept.

    We have to create syslog message format as described in https://datatracker.ietf.org/doc/html/rfc5424 ..

    The data, send to remote syslog sever has to look like:

    xxxxxxxxxxxxxxxxx schnipp xxxxxxxxxxxxxxxxxx

    1851 <110>1 2021-09-21T16:00:00Z HOSTNAME,open,10688 1 <AuditMessage>...more XML.. </AuditMessage>

    xxxxxxxxxxxxxxxxx schnapp xxxxxxxxxxxxxxxxxx

    So we have to implement transactional logging "xml over syslog-tcp".

    Is there anybody out there, who can report on how to realize it?

    Thanks to all, Eckehard



    #DataPower
    #Support
    #SupportMigration


  • 2.  RE: DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424
    Best Answer

    Posted Fri October 08, 2021 01:53 PM

    Hi Eckehard,

    I think if your messages are not very long the easiest solution would be to create a custom log category and emit the audit messages to DataPower log using the newly created category. You can then create a syslog log target and subscribe to the category and get only the selected log entries to the syslog server. Sending messages directly to the syslog target using processing policy action is a bit cumbersome. I think GatewayScript might be a better choice than XSLT if you want to create the message yourself.

    --HP



    #DataPower
    #Support
    #SupportMigration


  • 3.  RE: DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424
    Best Answer

    Posted Fri October 08, 2021 04:45 PM

    Hello HP,

    thanks a lot for your reply....

    Using xsl for event logging is limited to 2kByte, see https://www.ibm.com/support/pages/node/145167... so it is no solution because my transactional data exceeds this limit for sure.

    My customer needs transactional logging, so I have to forward/log the context of DataPower supported LogAction or ResultsAction during processing…to a remote syslog Server.

    LogAction:

    The problem (for me ) is that selecting destination protocol (in my case syslog-tcp) does not care creating protocol specific headers as specified in RFC 5424 (syslog).

    It results in wraping Input context of LocAction in a SOAP XML envelope, which is not intended.

    In https://manualzz.com/doc/o/kr6yv/datapower-soa-appliance-administration-chapter-6.-logging you can find this behaviour (wrapping in SOAP-XML envelope) is documented.

    So I am Looking for a solution, that datapower cares about all required syslog protocol requirements with attached syslog message content (transaction data)

    Seems to be verry uncommon, but is customer requirement.

    Regards

    Eckehard



    #DataPower
    #Support
    #SupportMigration


  • 4.  RE: DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424
    Best Answer

    Posted Wed October 13, 2021 05:26 AM

    Ok, yes the limitations are quite restricting.

    For the format and the headers you can try the following GatewayScript. It worked for the tests I did in my own environment, although syslog-tcp connection seemed to time out every time. But the format seems to be correct. All the lines are not needed, the timestamp handling might be a bit crude but didn't have the time to refine all the bits and pieces.


    //Load modules

    var transform = require('transform');


    // Get the service metadata

    var sm = require('service-metadata');


    // Create timestamp

    var currentdate = new Date();


    var datetime = currentdate.getFullYear() + '-'

    + addZero(currentdate.getMonth()) + '-'

    + addZero(currentdate.getDate()) + 'T'

    + addZero(currentdate.getHours()) + ':'

    + addZero(currentdate.getMinutes()) + ':'

    + addZero(currentdate.getSeconds()) + '.'

    + currentdate.getMilliseconds() + 'Z';


    // Extract DP service name and transaction id

    var service = sm.processorName;

    var transactionid = sm.transactionId


    //Create audit message XML

    var auditmessage = '<AuditMessage><element1>some audit value 1</element1><element2><element3>some audit value 2</element3></element2></AuditMessage>';


    //Create XPATH for extracting hostname

    var xpathOptions = { "expression" : "//*[local-name()='device-name']/text()","xmldom" : XML.parse(sm.system.ident)};


    transform.xpath(xpathOptions,function(err, devicename) {

    if (err) {

    session.out.write(err);

    } else {

    var stringifyOptions = {omitXmlDeclaration: true};

    // Create the syslog message payload, 13 is for notice

    // level log entry

    var syslog = '<13>1 '

    + datetime + ' '

    + XML.stringify(stringifyOptions,devicename) + ' '

    + service + ' '

    + transactionid + ' '

    + transactionid + ' - '

    + auditmessage;

    //Calculate size of the message

    var size = Buffer.byteLength(syslog); 

    // Write payload into output

    session.output.write(size + ' ' + syslog);


    }

    });


    function addZero(i) {

    if (i < 10) {

    i = "0" + i;

    }

    return i;

    }





    #DataPower
    #Support
    #SupportMigration


  • 5.  RE: DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424
    Best Answer

    Posted Wed October 13, 2021 04:27 PM

    Hi Hermanni,

    thank you very much for the time you spend on my problem... you made my day :-).


    I took your GatewayScript, customized some lines, added code to feed the script with transactional XML data from input context of the gateway script action and...it works great.


    It results in the following output:

    Support Member++ schnipp ++++++

    1916 <85>1 2021-09-13T17:11:43.812Z dp48prmue605 IHE-Notification d2fbc1876166f72f01733332 IHE+RFC-3881 - <AuditMessage><EventIdentification EventActionCode="C" EventDateTime="2021-10-13T17:11:43+02:00" EventOutcomeIndicator="0"><EventID csd-code="110107" codeSystemName="DCM" originalText="Import"/><EventTypeCode csd-code="ITI-53" codeSystemName="IHE Transactions" originalText="Document Metadata Notify"/></EventIdentification>....

    Support Member++ schnapp ++++++


    Upps, comparing value of syslog TIMESTAMP , created using currentdate.getMonth() (in above gateway script and some magic) to value of XML attribute EventDateTime , created using exslt date-time() (in an prefixed transform action /xsl stylesheet), shows, that value of month differ by one month....strange....


    I have to investigate to fix this missing month to come back to present ....


    Because I used netcat to simulate syslog server, I will deploy the code to my integration enviroment, connected to the real syslog server at the customers environment.

    I hope the customers syslog server can process the syslog message, although the wrong value for month.....


    I will come back with my findings soon....


    Thanks a lot...

    Eckehard



    #DataPower
    #Support
    #SupportMigration


  • 6.  RE: DataPower, transactional logging, LogAction/ResultsAction, protocol syslog-tcp, RFC5424
    Best Answer

    Posted Wed October 13, 2021 07:57 PM

    Hi again,

    found the bug... I could catch up one month ...

    See https://www.w3schools.com/jsref/jsref_getmonth.asp

    Support MemberSupport Member++++schnippSupport MemberSupport Member++

    The getMonth() method returns the month (from 0 to 11)

    Support MemberSupport Member++++schnappSupport MemberSupport Member++

    So using :

    + addZero(currentdate.getMonth()+1) + '-'

    instead of

    + addZero(currentdate.getMonth()) + '-'


    beams me back to present.


    Eckehard




    #DataPower
    #Support
    #SupportMigration