API Connect

API Connect

Join this online group to communicate across IBM product users and experts by sharing advice and best practices with peers and staying up to date regarding product enhancements.

 View Only
  • 1.  Customizing analytics data

    Posted Tue February 01, 2022 09:15 AM
    Hi All,

    We are currently offloading our API Connect gateway analytics to Elasticserch. However, we have some additional fields that need to go into analytics as these will be useful for troubleshooting.

    Add the following in APIC analytics:

    • Authenticated user (It currently logs in request headers as base64 encoded)
    • Backend/Invoke URL
    • Gateway cluster name (e.g. Internal EMEA)
    • Expand Request/Response headers and latency info (so that it can be used as filters on Elastic Search)
    • Same Transaction ID (Global transaction ID assigned is different between DMZ and Internal gateways correctly)
    I couldn't find any specific documentation on the website on how to do this. I would be grateful if anyone has an Idea on how to achieve this or pointers to documentation.


    ------------------------------
    Arjun Pilli
    ------------------------------


  • 2.  RE: Customizing analytics data

    Posted Wed February 02, 2022 04:45 AM
    Hello Arjun,

    It sounds like the following pages of the documentation might help you:

    Modifying your analytics data

    Sample filters for modifying analytics data

    (These are for v10.0.1.x of API Connect analytics).

    You should add the required filter configuration to the `external.offload.filter` section of your analyticscluster Custom Resource if you wish the fields to be added to the offloaded data.

    If you need more detail, analytics uses LogStash filters and so the Elastic documentation is also a useful reference. For example:

    Ruby filter plugin | Logstash Reference [7.17] | Elastic

    Let me know if you have any further questions and I'll do my best to answer.

    ------------------------------
    Mark Taylor
    ------------------------------



  • 3.  RE: Customizing analytics data

    Posted Fri February 11, 2022 07:55 AM
    Thank you Mark. We will follow these documents and update the output here. 

    Do you think it will be possible to modify the default message sent to the analytics server? Our infra is on VMWare. 

    Regards,
    Arjun

    ------------------------------
    Arjun Pilli
    API Specialist
    DSV
    Johannesburg
    +27 0840611655
    ------------------------------



  • 4.  RE: Customizing analytics data

    Posted Fri February 18, 2022 05:06 AM
    You're welcome Arjun.

    I should have also highlighted the following page, especially the warning around what fields you should not modify in the default message:

    https://www.ibm.com/docs/en/api-connect/10.0.x?topic=pyad-planning-modify-your-analytics-data

    You do have full control over the default message before it is stored in analytics, but analytics filters only have access to the data that the gateway includes in the message. I believe it is possible to modify the message the gateway sends by using the Log policy described here: https://www.ibm.com/docs/en/api-connect/10.0.x?topic=policies-log

    I don't believe the fact that your infra is VMWare has much impact on any of the functionality I have highlighted.

    Kind regards,
    Mark

    ------------------------------
    Mark Taylor
    ------------------------------



  • 5.  RE: Customizing analytics data

    Posted Fri March 11, 2022 02:35 AM

    Thank you so much Mark. I have got it working as per our requirement following your inputs. Very much appreciated.

     

    Best regards,
     
    Arjun Pilli, External, API Specialist

    EXT.Arjun.Pilli@dsv.com  |  Office +27 102480076  |  Mobile +27 0840611655  |  www.dsv.com