Hi All,

We are currently offloading our API Connect gateway analytics to Elasticserch. However, we have some additional fields that need to go into analytics as these will be useful for troubleshooting.

Add the following in APIC analytics:

• Authenticated user (It currently logs in request headers as base64 encoded)
• Backend/Invoke URL
• Gateway cluster name (e.g. Internal EMEA)
• Expand Request/Response headers and latency info (so that it can be used as filters on Elastic Search)
• Same Transaction ID (Global transaction ID assigned is different between DMZ and Internal gateways correctly)

I couldn't find any specific documentation on the website on how to do this. I would be grateful if anyone has an Idea on how to achieve this or pointers to documentation.


------------------------------
Arjun Pilli
------------------------------

Hello Arjun,

It sounds like the following pages of the documentation might help you:

Modifying your analytics data

Sample filters for modifying analytics data

(These are for v10.0.1.x of API Connect analytics).

You should add the required filter configuration to the `external.offload.filter` section of your analyticscluster Custom Resource if you wish the fields to be added to the offloaded data.

If you need more detail, analytics uses LogStash filters and so the Elastic documentation is also a useful reference. For example:

Ruby filter plugin | Logstash Reference [7.17] | Elastic

Let me know if you have any further questions and I'll do my best to answer.

------------------------------
Mark Taylor
------------------------------

Thank you Mark. We will follow these documents and update the output here. 

Do you think it will be possible to modify the default message sent to the analytics server? Our infra is on VMWare. 

Regards,
Arjun

------------------------------
Arjun Pilli
API Specialist
DSV
Johannesburg
+27 0840611655
------------------------------

Original Message:
Sent: Wed February 02, 2022 04:44 AM
From: Mark Taylor
Subject: Customizing analytics data

Hello Arjun,

It sounds like the following pages of the documentation might help you:

Modifying your analytics data

Sample filters for modifying analytics data

(These are for v10.0.1.x of API Connect analytics).

You should add the required filter configuration to the `external.offload.filter` section of your analyticscluster Custom Resource if you wish the fields to be added to the offloaded data.

If you need more detail, analytics uses LogStash filters and so the Elastic documentation is also a useful reference. For example:

Ruby filter plugin | Logstash Reference [7.17] | Elastic

Let me know if you have any further questions and I'll do my best to answer.

------------------------------
Mark Taylor
------------------------------

You're welcome Arjun.

I should have also highlighted the following page, especially the warning around what fields you should not modify in the default message:

https://www.ibm.com/docs/en/api-connect/10.0.x?topic=pyad-planning-modify-your-analytics-data

You do have full control over the default message before it is stored in analytics, but analytics filters only have access to the data that the gateway includes in the message. I believe it is possible to modify the message the gateway sends by using the Log policy described here: https://www.ibm.com/docs/en/api-connect/10.0.x?topic=policies-log

I don't believe the fact that your infra is VMWare has much impact on any of the functionality I have highlighted.

Kind regards,
Mark

------------------------------
Mark Taylor
------------------------------

Original Message:
Sent: Fri February 11, 2022 07:54 AM
From: Arjun Pilli
Subject: Customizing analytics data

Thank you Mark. We will follow these documents and update the output here. 

Do you think it will be possible to modify the default message sent to the analytics server? Our infra is on VMWare. 

Regards,
Arjun

------------------------------
Arjun Pilli
API Specialist
DSV
Johannesburg
+27 0840611655
------------------------------


Original Message:
Sent: Wed February 02, 2022 04:44 AM
From: Mark Taylor
Subject: Customizing analytics data

Hello Arjun,

It sounds like the following pages of the documentation might help you:

Modifying your analytics data

Sample filters for modifying analytics data

(These are for v10.0.1.x of API Connect analytics).

You should add the required filter configuration to the `external.offload.filter` section of your analyticscluster Custom Resource if you wish the fields to be added to the offloaded data.

If you need more detail, analytics uses LogStash filters and so the Elastic documentation is also a useful reference. For example:

Ruby filter plugin | Logstash Reference [7.17] | Elastic

Let me know if you have any further questions and I'll do my best to answer.

------------------------------
Mark Taylor