Configuring Kafka with Kerberos Authentication in IBM Integration Bus V10 and App Connect Enterprise V11

1. Download the IIBKafkaPubTestV2.jar

   IIBKafkaPubTestV2.jar

2. Create a standalone Kafka client properties file with the name “producer.properties” and configure the options below based on your Kafka Server settings

1. Execute the IIBKafkaPubTestV2.jar with below syntax and collect the result.
   Java -Djavax.net.ssl.trustStore=<path> - Djavax.net.ssl.trustStorePassword=<trust_password> -Djavax.net.ssl.keyStore=<path> - Djavax.net.ssl.keyStorePassword=<keystore_password> -jar IIBKafkaPubTestV2.jar

1. Check kafkalog.txt logs to identify the status of Kafka Publish using the Kerberos authentication. On successful publish you will see the below message

   “successfully published”

   If it throws any error related to SSL Handshake, Kerberos Authentication or any other errors related to Kafka with Kerberos setup then fix the issue until you receive “successfully published” from the IIBKafkaPubTestV2.jar execution.

   Once you have “successfully published” then you are ready to incorporate the setup in to your IIB configuration.

Once you have a successful run of the tool, use the steps below to adopt the arguments used in the tool to configure Kafka in IBM Integration Bus.

1. Set the Kerberos configuration file. It is the second argument used while running the tool on Integration server
2. Set the producer.properties file.

   mqsichangeproperties IBNODE -c ConnectorProviders -o Kafka -n property1 -v  <full path to
   producer.properties file>

   (Optional) For Kafka consumer nodes if they are using the same properties file, please run

   mqsichangeproperties IBNODE -c ConnectorProviders -o Kafka -n property2 -v <full path to 
   producer.properties file>

   
3. Configure the Kafka node with the following properties

   • Specify the topic name on the node property as <publish topic>

   • Set the 'Bootstrap servers' node property as <bootstrap server addr>

   • Set node property 'Security protocol' as <security protocol>

   
4. (Optional) If you have used -Djavax.net.ssl.trustStore or -Djavax.net.ssl.keyStore while running the tool, then you need to configure the same keystore and truststore in IBM integration Bus using the following commands.

Commands to configure truststore under the Integration sever.

1. mqsichangeproperties IBNODE -e IBSERVER -o ComIbmJVMManager -n truststoreFile -v 
   <Truststore value used to run the program IIBKafkaPubTestV2.jar >

2. mqsichangeproperties IBNODE -e IBSERVER -o ComIbmJVMManager -n truststorePass -v 
   kafkaTrust::password

3. mqsisetdbparms IBNODE  -n kafkaTrust::password -u temp -p <truststore password used to run
   the program IIBKafkaPubTestV2.jar >

Commands to configure keystore under the Integration server.

1. mqsichangeproperties IBNODE -e IBSERVER -o ComIbmJVMManager -n keystoreFile -v <keystore
   value used to run the program IIBKafkaPubTestV2.jar >

2. mqsichangeproperties IBNODE -e IBSERVER -o ComIbmJVMManager -n keystorePass  -v 
   kafkaKeystore::password

3. mqsisetdbparms IBNODE  -n kafkaKeystore::password -u temp -p <keystore  password used to 
   run the program IIBKafkaPubTestV2.jar >

You will need App Connect Enterprise version 11.0.0.5 or higher for Kafka nodes to use Kerberos authentication.

1. Define the Kerberos configuration file

   a) For Integration node, run the command

   mqsichangeproperties NODE -e SERVER -o ComIbmJVMManager -n kerberosConfigFile -v <full path to 
   krb5.conf file>

   
2. Set the producer.properties file using environment variable

   MQSI_KAFKA_PRODUCER_PROPERTIES_FILE and MQSI_KAFKA_CONSUMER_PROPERTIES_FILE

   export MQSI_KAFKA_PRODUCER_PROPERTIES_FILE=<full path to producer.properties file>

   b) (optional) setting the consumer.properties file for KafkaConsumer node

   export MQSI_KAFKA_CONSUMER_PROPERTIES_FILE=<full path to consumer.properties file>

3. Configure the Kafka node with following properties

   • Specify the topic name on the node property as <publish topic>
   • Set the 'Bootstrap servers' node property as <bootstrap server addr>
   • Set node property 'Security protocol' as <security protocol>

The above procedure is to configure Kerberos authentication with Kafka using the producer.properties file. For those who are using plain username and password authentication instead of Kerberos authentication, you can use the same stand-alone program for testing in the following way;

1. Change the sasl.jaas.config in the producer.properties file in the following form;

   sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="username" password="password";

2. Run the program. You can use a dummy file for the argument <full path to krb5.conf file>
3. Once you have a successful result, configure the username and password credentials as per the following Knowledge Center page for Kafka nodes to pick them.
   https://www.ibm.com/support/knowledgecenter/en/SSMKHH_10.0.0/com.ibm.etools.mft.doc/bz91050_.htm

The stand alone program is built using Java KafkaProducer class. You can refer to the Javadoc for further information;
https://kafka.apache.org/0102/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html

Acknowledgement and thanks to Amar Shah for his technical support and advice in the construction of this article.