AIOps

Many deployments of Cloud Pak for AIOps (AIOps) are done where an existing Netcool environment already exists. Part of the migration over to AIOps is the deactivation of event correlation in the Netcool layer, and reimplementation in AIOps. This is necessary because the way AIOps creates groupings is very different between Netcool and AIOps and the representation and linkages are not compatible:

• Netcool uses either real or synthetic parent events whose children are linked via a key linkage field - often @ParentIdentifier
• AIOps stores grouping information as metadata within each correlated event - parent events are only virtual and are rendered in the Alerts viewer

The major advantage of AIOps over Netcool is that events can be a member of any number of groups, either of the same or different kinds of groups. Whereas Netcool requires a linkage between a parent event and a child event via a static field, AIOps stores this meta-data within the alert payload. There is no programmatic limit on how much meta-data can be stored in an AIOps alert, so it results in a much more flexible grouping model in AIOps.

There are three main grouping capabilities provided in both Netcool Operations Insight (NOI) and AIOps. The following outlines how these can be migrated from the Netcool layer to the AIOps layer:

• Temporal
  Temporal groupings found in either the Netcool/Impact based event analytics or the newer Cloud Native Event Analytics (CNEA) engine are not transferable to AIOps. AIOps will need to ingest event data from Netcool (and other sources) and re-train on the data internally. Any temporal grouping in NOI should be deactivated prior to enabling temporal grouping in AIOps.
• Topology-based
  NOI includes the same topology capability as AIOps and also supports topology-based event correlation. It works the same in both and needs to be set up in AIOps to match the configuration in NOI. Any topology-based grouping in NOI should be deactivated prior to enabling topology-based grouping in AIOps.
• Scope-based
  NOI includes a similar scope-based grouping capability as AIOps. By default, the Netcool Connector maps @ScopeID in NOI to resource.scopeId in AIOps. The default AIOps policy called Default Netcool scope-based grouping - with a rolling window of 15 minutes will automatically detect any inbound events with the resource.scopeId attribute set and will do scope-based grouping on it. As the name suggests, this is done with a rolling 15 minute window. This provides a convenient like-for-like function and makes migration from Netcool to AIOps relatively easy in this respect. You can of course disable this default policy and create your own, if you have more specific or complex requirements - for example, you have a variety of time windows required for different correlation scenarios.

NOTE: Netcool/OMNIbus in the absence of Netcool Operations Insight only has scope-based event correlation, if enabled:

https://www.ibm.com/docs/en/netcoolomnibus/8.1?topic=netcoolomnibus-enabling-scope-based-event-grouping

Custom correlations in Netcool

Many implementations of Netcool include custom correlations where one or more child events are linked to a parent event. Even though the grouping engine itself is eventually deactivated at the Netcool layer, in a combined Netcool/AIOps deployment, the correlation logic can remain in-place, and the grouping instead done at the AIOps layer. This section outlines a method of achieving this.

These are the basic steps:

• Identify the parent-to-child relationship linkage in your event set
• Identify the unique key attribute in each grouping that the child events have in common
• Map this value to a custom AIOps attribute that you will use to group on
• Create an AIOps scope-based grouping policy against the custom correlation field with appropriate time window settings
• The set of events will be correlated together in AIOps

The following is an example of how this might be implemented.

IDENTIFY PARENT TO CHILD RELATIONSHIP

The first step to mapping custom correlation to AIOps is identifying how the groups are defined on the Netcool side. This is commonly done by linking the Identifier field value in the parent event to the ParentIdentifier field in the child events. Hence, the child events in a grouping will all share a common value in the ParentIdentifier field, that being the Identifier field value of the parent event. We can simply use this common attribute to group on therefore.

MAP CUSTOM CORRELATION FIELD

The next step is to update the mapping of your Netcool ObjectServer Connector mapping to include this attribute. Below is a sample mapping which would work for the above scenario:

    ...
    "state": alert.@Severity = 0 ? "clear" : "open",
    "acknowledged": alert.@Acknowledged = 1 ? true : false,
    "expirySeconds": alert.@ExpireTime = 0 ? undefined : alert.@ExpireTime,
    "details": {
        "customCorrelation1": alert.@ParentIdentifier != "" ? alert.@ParentIdentifier : undefined
    },
    ...

The above mapping says:

• If the alert ParentIdentifier is not null, use its value to populate customCorrelation1;
• Else leave it undefined.

INCLUDE PARENT EVENTS

If your grouping is still active in the Netcool layer and you want to include the synthetic parent events in the same grouping as the child events within AIOps, then you can extend your JSONATA definition to include them also. The following example assumes that the synthetic parent events in Netcool have AlertGroup = 'CustomParent':

    ...
    "state": alert.@Severity = 0 ? "clear" : "open",
    "acknowledged": alert.@Acknowledged = 1 ? true : false,
    "expirySeconds": alert.@ExpireTime = 0 ? undefined : alert.@ExpireTime,
    "details": {
        "customCorrelation1": alert.@AlertGroup = "CustomParent" ? alert.@Identifier : \
                alert.@ParentIdentifier != "" ? alert.@ParentIdentifier : undefined
    },
    ...

The above mapping says:

• If the alert AlertGroup field is 'CustomParent', set the customCorrelation1 attribute to be the value of the Identifier field;
• Else if the alert ParentIdentifier is not null, use its value to populate customCorrelation1;
• Else leave it undefined.

Since the grouping logic in Netcool populates child events with the Identifier of the parent, then if we use the parent's Identifier field value as the correlation key, it should correlate parent events together with their children in AIOps.

CREATE AIOPS CORRELATION POLICY

The final step is to create a scope-based grouping automation policy in AIOps to group the incoming event stream by your newly created attribute.

• Give your policy a meaningful name
• Set the priority
• Select "Before an alert is created"
• Add a condition that the policy fires when your custom attribute is "not empty"
• Under "Create a scope-based grouping", enter your custom attribute
• Specify the desired time window
• Specify if the time window is rolling or fixed

Here is an example AIOps automation policy snapshot showing an example condition set:

You should now start to see your custom correlation working in AIOps. Remember, this policy will only work for new events, not existing events.

One of the major advantages in AIOps over Netcool is that you can set up as many of these scope-based grouping automations as needed. You can therefore use this process to migrate all your custom Netcool correlation automations over to AIOps.