AIOps

 View Only

Introduction to IBM Cloud Pak for AIOps gRPC (Remote Procedure Call) Connectors.

By CHEE YEN MAU posted Thu May 25, 2023 04:19 AM

  

Introduction to IBM Cloud Pak for AIOps gRPC (Remote Procedure Call) Connectors.

If you are new to IBM Cloud Pak for AIOps, your first question is: How do we ingest data into AIOps? The answer is there are various ways to ingest data, and one of the method is using connectors.

Simply put, a connector is an integration between AIOps and a particular data source. AI Manager can connect to multiple, diverse data sources at multiple levels of your IT infrastructure using connectors. In this topic, I am going to give a brief introduction to gRPC connectors. This is a data flow diagram of the connectors:


  • AIOps gRPC connector is packaged as container image that starts an OpenLiberty server running a connector application, which is acting as a gRPC client that communicates back to the AIOps main cluster via a gRPC channel. In practices, the connectors are created for different data sources and data types. You can find a list of data sources and data types from: https://www.ibm.com/docs/en/cloud-paks/cloud-pak-watson-aiops/3.7.1?topic=connections-connection-types. If we take Instana connector as an example, it collects 3 data types: Events, Metrics and Topology and then ingest them into AIOps.

    The connector is composed of a connector container, processor containers and a messaging system, Kafka.
    •    Connector container is container image that runs a gRPC client, running either in-cluster for local deployment or remotely for micro-edge deployment. Connectors integrate with the data sources either via RESTFUL API or other integrations such as Kafka to pull data and encapsulates it in CloudEvents and then sends them to the AIOps connector bridge.
    •    Processor container runs in-cluster. Each connector must have at least one component that interacts with its collected data, which we call a processor. There are many types of processors, depending on their purpose.  There are processors for each data types, ie: Events, Metrics and Topology.
    •    Kafka to communicate between connector containers and processor containers. Connector (gRPC) bridge writes messages to certain Kafka topics so that they are consumed by the respective processors.

AIOPs developers published a connector SDK that can be used to create new connectors. You can find more details in: https://github.com/IBM/cp4waiops-connectors. The repository provides more in-depth technical information, template source code as well as an example connector. You can also use the Java connector template if you would like to start developing a Java based connector. Although the preferred programming model is via OpenLiberty, any gRPC compatible model such as NodeJs can also be used to develop a connector. 


#automation-featured-area-1
0 comments
179 views

Permalink