Introduction:
IBM Cloud Pak for AIOps is a leading platform for managing IT operations with advanced AI capabilities and automation. Providing the capability to collect operational data, such as events and alerts from different sources.
The Instana Connector enables integration between an Instana Instance and AIOps – a for the collection and processing of metric and event data. Here, we will focus more on the event side of things.
To get a general consensus on what the connector is, let me provide a brief overview. The connector implements a pull approach to retrieve events from the Instana Instance it is connected to, generating alerts to be ingested by AIOps.
This is mainly achieved by the following steps:
-
Instana Connector calls an Instana API to retrieve all events for a given time window.
-
Events received are then transformed and processed into a structure that is compatible with AIOps.
-
These are then sent to the AIOps policy engine for processing.
Now that we’re all up to date – let's outline the changes and extra features we have coming up in the latest release 4.11.
Alert Transformation (JSONata):
Previously, the connector did not provide options to customize mappings for the alerts that it generates – the alerts had a fixed structure, not providing any flexibility to change the values for certain fields based on a customer’s preference.
We’ve addressed this by introducing JSONata to the Instana Connector providing it with the ability to take in whatever event-to-alert mapping is desired. Not only do we provide this, but we’ve also included custom functions; discarding/filtering out s and data enrichment with data. Snapshots are Instana’s representation of the entity associated with a given event; such as a host, a process or a kubernetes container.
These can be used for the following:
-
nterested only in certain events with specific field value(s)? Taking ‘entityType’ as an example - simply modify your JSONata mapping to have ‘$discard’, like so:
( $.entityType in [“SERVICE”, “SYNTHETICS”] ? $discard() : { <jsonata_mapping>} )
-
Alerts can be enriched with snapshot data with the addition of ‘$snapshotGet’. We’ve made it easier to grab what may be needed from the snapshot response when querying a snapshot using snapshotId for a given event. ‘$snapshotGet’ requires 2 parameters, the first is the snapshotId (recommended to keep as ‘$.snapshotId’), and the second parameter is the field_name.
“clusterLabel”: $snapshotGet($.snapshotId, “clusterLabel”)
This can also be used in conjunction with the ‘$hasSnapshot’ function which checks whether that snapshotId exists – this takes 1 parameter, which is the snapshotId (recommended to keep as ‘$.snapshotId’).
“clusterLabel”: $hasSnapshot($.snapshotId) ? $snapshotGet($.snapshotId, “clusterLabel”) : undefined
NOTE: SOME SNAPSHOT DATA DOES NOT INCLUDE CERTAIN FIELDS (POSSIBLE FOR SOME ALERTS NOT TO HAVE MAPPED FIELD)
Event and Alert Migration (Upgrading to 4.11):
Initially, users of the Instana Connector created alerts through fixed data transformation process. Due to this, alerts in AIOps that were created from this process will not have the same structure as what will be generated through JSONata as these alerts can be customized.
The Instana Connector can deal with both the old and new approaches at the same time. New events from Instana will go through the JSONata approach, while the events that created an alert using the old way will continue to do so.
With this workflow, alerts that were created with the old approach will be purged, eventually leading to only the JSONata approach being used.
Cases where this is applicable to are the following:
Synchronization of AIOps & Instana Instance:
, the Instana connector emitted AIOps events for each Instana issue, relying on the standard deduplication processing to create alerts. Now, the alerts are directly managed by the connector, ensuring all alerts in AIOps always match the issues in Instana.
With that in hand, the connector is capable of querying the alerts on the Instana Instance to validate if those alerts are indeed active or closed – updating a close for them on AIOps if they are closed on the target system.
Another refinement that we have incorporated into the connector is to trigger the hard resynch functionality in the case of a configuration change via UI while the connector is running – which was not possible on previous versions.
There are two types of resynch that the connector can do, these are SOFT and HARD resync:
NOTE: THE HARD RESYNCH ONLY OCCURS IN THE EVENT WHERE THE POD RESTARTS AND A CONFIGURATION CHANGE IN THE UI.
Dynamically Caching Snapshot Data (Data Enrichment):
Modifications were also made on the connector to address performance while providing more flexibility in terms of options for the snapshot data. This was made possible thanks to our new JSONata approach when processing events - it will only store the required fields in the connector’s internal memory cache, preventing storage memory issues.
-
Connector is now capable of sending status ERRORED, when an issue occurs in the target system with the ability to update back to RUNNING status once resolved.
i.e. Target System Down, the connector will update its status to ERRORED but will update to RUNNING once recovered.
The Instana Connector's 4.11 release delivers significant improvements focused on flexibility, reliability, and performance. Key enhancements include:
These changes create a more robust integration between Instana and IBM Cloud Pak for AIOps, providing greater confidence in monitoring data while reducing operational overhead.