IBM Sterling Transformation Extender

View Only

ITX integration with remote Kafka server as a Producer.

SAMBASIVA RAO Jilludumudi posted Fri December 05, 2025 02:24 AM

Hi All

I am facing issue when trying to establish a connection between ITX and Kafka servers.

=PUT("KAFKA", "-SRV XXXX:X0X2 -CID XXXX-EDI-API-INTEGRATION -TP OMS-LSS-PO -SP SASL_SSL -ACK ALL -SM PLAIN -TV -LCFL /opt/kafka/kafka_client_jaas.conf", input)

Contents of Kafka_client_jaas.conf

# File: /opt/itx/kafka/login.conf

# Purpose: JAAS entries for Kafka Producer/Consumer using SASL/PLAIN over TLS

KafkaClient {

org.apache.kafka.common.security.plain.PlainLoginModule required

username="XXXX"

password="$$$$$$";

};

snippet of trace file

<13584-6992-12/05/25 12:43:26>: [MAdapterImpl.validateProperties]

<13584-6992-12/05/25 12:43:26>: | Created adapter instance: com.hcl.hip.adapters.m4kafka.MAdapterImpl@94dc126b

<13584-6992-12/05/25 12:43:26>: [MAdapterImpl.validateProperties] (rc = 0) OK

<13584-19048-12/05/25 12:43:26>: [MConnectionImpl.connect]

<13584-19048-12/05/25 12:43:26>: | connect() connection instance: com.hcl.hip.adapters.m4kafka.MConnectionImpl@22e59b4

<13584-19048-12/05/25 12:43:26>: | Connecting to the cluster: XXXX:X0X2

<13584-19048-12/05/25 12:43:26>: | Using security protocol: SASL_SSL

<13584-19048-12/05/25 12:43:26>: | Using SASL mechanism: PLAIN

<13584-19048-12/05/25 12:43:26>: | [MConnectionImpl.createProducer]

<13584-19048-12/05/25 12:43:26>: | | Producer settings: {security.protocol=SASL_SSL, bootstrap.servers=XXXX:X0X2, value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer, buffer.memory=33554432, retries=0, key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer, client.id=XXXX-EDI-API-INTEGRATION, linger.ms=0, sasl.mechanism=PLAIN, batch.size=16384, enable.idempotence=false, acks=all, compression.type=none}

<13584-19048-12/05/25 12:43:26>: | | Setting new context class loader.

<13584-19048-12/05/25 12:43:26>: | | Creating producer...

<13584-19048-12/05/25 12:43:26>: | | Producer created successfully.

<13584-19048-12/05/25 12:43:26>: | | Restoring old context class loader.

<13584-19048-12/05/25 12:43:26>: | [MConnectionImpl.createProducer] (rc = 0) OK

<13584-19048-12/05/25 12:43:26>: | Connected successfully to the cluster.

<13584-19048-12/05/25 12:43:26>: [MConnectionImpl.connect] (rc = 0) OK

<13584-19048-12/05/25 12:43:26>: [MAdapterImpl.onNotify]

<13584-19048-12/05/25 12:43:26>: | Notification: Adapter[com.hcl.hip.adapters.m4kafka.MAdapterImpl@94dc126b] ID[MPIN_ADAPTER_PUTSTART] intParam[0] objParam[MPI_OBJTYPE_CONNECTION]

<13584-19048-12/05/25 12:43:26>: [MAdapterImpl.onNotify] (rc = 0) OK

<13584-19048-12/05/25 12:43:26>: [MAdapterImpl.beginTransaction]

<13584-19048-12/05/25 12:43:26>: | beginTransaction() adapter instance: com.hcl.hip.adapters.m4kafka.MAdapterImpl@94dc126b

<13584-19048-12/05/25 12:43:26>: [MAdapterImpl.beginTransaction] (rc = 0) OK

<13584-19048-12/05/25 12:43:26>: [MAdapterImpl.put]

<13584-19048-12/05/25 12:43:26>: | put() adapter instance: com.hcl.hip.adapters.m4kafka.MAdapterImpl@94dc126b

<13584-19048-12/05/25 12:43:26>: | [MAdapterImpl.putNextRecord]

<13584-19048-12/05/25 12:43:26>: | | Payload size: 26

<13584-19048-12/05/25 12:43:26>: | | Using futures.

<13584-19048-12/05/25 12:44:26>: | | Published message of payload size: 26 bytes.

<13584-19048-12/05/25 12:44:26>: | [MAdapterImpl.putNextRecord] (rc = 0) OK

<13584-19048-12/05/25 12:44:26>: [MAdapterImpl.put] (rc = 0) OK

<13584-19048-12/05/25 12:44:26>: [MAdapterImpl.endTransaction]

<13584-19048-12/05/25 12:44:26>: | endTransaction() adapter instance: com.hcl.hip.adapters.m4kafka.MAdapterImpl@94dc126b

<13584-19048-12/05/25 12:44:26>: | Transaction action: MPI_COMMIT

<13584-19048-12/05/25 12:44:26>: | Commit producer scenario.

<13584-19048-12/05/25 12:44:26>: | The broker reported error when acknowledging message: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic OMS-LSS-PO not present in metadata after 60000 ms.

<13584-19048-12/05/25 12:44:26>: | Caught exception: java.util.concurrent.ExecutionException

org.apache.kafka.common.errors.TimeoutException: Topic OMS-LSS-PO not present in metadata after 60000 ms.: Topic OMS-LSS-PO not present in metadata after 60000 ms.

java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic OMS-LSS-PO not present in metadata after 60000 ms.

at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:1320)

at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:989)

at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:889)

at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:775)

at com.hcl.hip.adapters.m4kafka.MAdapterImpl.putNextLogicalRecord(MAdapterImpl.java:779)

at com.hcl.hip.adapters.m4kafka.MAdapterImpl.put(MAdapterImpl.java:587)

Caused by: org.apache.kafka.common.errors.TimeoutException: Topic OMS-LSS-PO not present in metadata after 60000 ms.

<13584-19048-12/05/25 12:44:26>: | Error committing transaction: org.apache.kafka.common.errors.TimeoutException: Topic OMS-LSS-PO not present in metadata after 60000 ms.