This article will demonstrate the configuration of new capability added in ACE 12.0.7.0 which supports OpenTelemetry trace for an integration server and observe the transactions on Jaeger.
This article also explains how OpenTelemetry can be configured in the integration runtime of an ACE operator in Red Hat Openshift and view the spans in Jaeger on Red Hat Openshift. It will also show how to configure OpenTelemetry in the integration runtime of an ACE On-Prem and view the spans in Jaeger On-Prem.

Configuring OpenTelemetry trace for an integration server (On-Prem)

OpenTelemetry can be enabled for an integration server by editing the “ResourceManagers” section in the server.conf.yaml file:

ResourceManagers:
OpenTelemetryManager:
openTelemetryEnabled: true # Enable or disable OpenTelemetry tracing.
exporterOStreamFilename: 'C:\tele\OpenTelemetryTrace_Simple.txt'
openTelemetrySpanProcessor: 'simple' # Specify the OpenTelemetry span processor ['batch' or 'simple'].
#exporterOtlpGrpcEndpoint: '' # Specify a GRPC endpoint to which OpenTelemetry span data is sent. For example, '<hostname>:4317'.
exporterOtlpHttpUrl: 'http://localhost:4318/v1/traces' # Specify a HTTP Url to which OpenTelemetry span data (json) is sent. For example, 'http://<hostname>:<port>/v1/traces'.
#openTelemetryServiceName: '' # Override the Service Name attribute for telemetry spans. Defaults to integration server name.
#openTelemetryHostName: '' # Override the value of the Hostname attribute for telemetry spans.
#openTelemetryTruststoreType: '' # The type of the open telemetry truststore [PKCS12, PEM or JKS]. Only used for the GRPC exporter.
#openTelemetryTruststoreFile: '' # The location of the open telemetry truststore. Only used for the GRPC exporter.
#openTelemetryTruststorePass: '' # Credential or resource alias containing the truststore password. Only used for the GRPC exporter.
#openTelemetryTrustAlias: '' # The alias of the trust certificate in the PKCS12 or JKS truststore.
#openTelemetrySpanProcessor: 'batch' # Specify the OpenTelemetry span processor ['batch' or 'simple'].
#openTelemetryBatchSpanOptions: '' # Specify the OpenTelemetry batch span processor options.

Sending the data to a file

The following entries in the template will enable OpenTelemetry and the data will be written to a file in JSON format:

OpenTelemetryManager:
openTelemetryEnabled: true
exporterOStreamFilename: ''C:\tele\OpenTelemetryTrace_Simple.txt'

The flow consists of a simple HTTPInput  HTTPReply node :

Sending the Data to the Jaeger Backend

Jaeger is a OpenSource version of an OpenTelemetry collector/viewer and Jaeger can support both GRPC and HTTP/JSON formats. The following entries in the server.conf.yaml will send the data to Jaeger.

ResourceManagers:
HTTPConnector:
ListenerPort: 7802
OpenTelemetryManager:
openTelemetryEnabled: true # Enable or disable OpenTelemetry tracing.
OpenTelemetry is only available on the Linux/x86-64 platform
exporterOStreamFilename: 'C:\tele\OpenTelemetryTrace_Simple.txt'
openTelemetrySpanProcessor: 'simple' # Specify the OpenTelemetry span processor ['batch' or 'simple'].
exporterOtlpHttpUrl: 'http://localhost:4318/v1/traces' # Specify a HTTP Url to which OpenTelemetry span data (json) is sent. For example, 'http://:/v1/traces'.

Here Jaeger is running on the same host as my IntegrationServer and the Jaeger UI is running on http://localhost:16686 which is the default port.

The Jaeger program “jaeger-all-in-one.exe “ can be started with the following arguments:

jaeger-all-in-one.exe --collector.otlp.enabled=true --collector.otlp.grpc.host-port=":4317" --collector.otlp.http.host-port=":4318"

The application is an end-to-end HTTP application, which has multiple flows:

Selecting the Operation “Opentelemetry.HTTPFlow” shows 3 spans and the time duration:

So the 3 spans explain that;

1. it hits the HTTPInput node
2. then the HTTPRequest node is invoked
3. the HTTPRequest node invokes the backend Server ( i.e HTTPInput node)

The 3 Spans are explained as below:

SPAN 1: Indicates the flow hits the HTTPInput node :

SPAN 2: It shows the details for HTTPRequest node:

SPAN 3: This Span shows the details for HTTPInput node that is being invoked by the HTTPRequest node:

Configuring OpenTelemetry trace for an integration server (CP4I)

To configure OpenTelemetry trace for an integration server and send the data to Jaeger, we need to install the operators Jaeger and OpenTelemetry in the CP4I environment:

Install the operator as shown below:

Once the Operator is installed, create the Jaeger instance :

The pod for the Jaeger instance can be found running as show below:

To get the Jaeger UI , Navigate to Networking  Routes and get the location as shown below :

We can see below the Jaeger UI on CP4I:

Now we have Jaeger installed. The next step is to enable OpenTelemetry in ACE and configure ACE with the hostname where the application will send the data.

Launch the App Connect Dashboard:

Create the server.conf.yaml configuration on the Dashboard to enable OpenTelemetry and specify the endpoint where the data should be sent :

ResourceManagers:
OpenTelemetryManager:
openTelemetryEnabled: true # Enable or disable OpenTelemetry tracing. OpenTelemetry is only available on the Linux/x86-64 platform
openTelemetrySpanProcessor: 'simple' # Specify the OpenTelemetry span processor ['batch' or 'simple'].
exporterOtlpGrpcEndpoint: 'http://jaeger-all-in-one-inmemory-collector.preeghos.svc.cluster.local:4317' # Specify a GRPC endpoint to which OpenTelemetry span data is sent. For example, ':4317'.
#exporterOtlpHttpUrl: 'http://jaeger-all-in-one-inmemory-collector.preeghos.svc.cluster.local:4317' # Specify a HTTP Url to which OpenTelemetry span data (json) is sent. For example, 'http://:/v1/traces'.

The endpoint “exporterOtlpGrpcEndpoint” can be found Jaeger service running on CP4I:

The next step is to deploy the integration and configure on the ACE Dashboard:

Select the Configuration “server.conf.yaml” as mentioned above and apply to the integration:

Once the Integration is ready and configured with the server.conf.yaml file, the opentelemetry is configured and enabled:

Testing

Get the URL location for the integration server deployed with the flow as shown below:

To trigger the flow send the message to the URL “http://is-02-http-http-preeghos.apps.acel3-swatd-squad.cp.fyre.ibm.com/OTeleCP4I”

The Data is sent to the Jaeger UI in the CP4I environment as shown below:

Acknowledgement and thanks to Trevor Lobban and Amar Shah for providing valuable input to this article.