Data Integration

  • 1.  cdc KAFKA with SASL_PLAINTEXT 5522_linux build

    Posted Thu April 23, 2020 10:29 AM
    Could some one please guide how to setup CDC instance/subscription for KAFKA. I have JAAS, Keytab and KRB5 files. I don't see documentation which i can follow.

    ------------------------------
    Abhishek Kumar

    ------------------------------


    #DataReplication
    #DataIntegration


  • 2.  RE: cdc KAFKA with SASL_PLAINTEXT 5522_linux build

    Posted Thu April 23, 2020 10:49 AM
    Hello,
    There are several documents on the IDR Wiki (http://ibm.biz/idrwiki#Kafka) that you may find helpful.

    CDC for Kafka installation and configuration Installation and configuration instructions for Kafka, including specialized topics such as Kerberos configuration.
    Configuring the CDC Replication Engine for Kafka to use SSL security Configure CDC to use SSL and SASL authentication when connecting to Apache Kafka.
    Troubleshooting Kerberos You can perform the steps presented in this document to validate Kerberos configuration outside of IDR CDC product. It is a good thing to do before you attempt to set up the replication, as it reduces the overall complexity of the task.

    Best Regards,
    Sarah
    IBM Data Replication development team

    ------------------------------
    SARAH ORVIS
    ------------------------------



  • 3.  RE: cdc KAFKA with SASL_PLAINTEXT 5522_linux build

    Posted Fri April 24, 2020 06:10 AM
    when i am trying to bring instance up by adding java runtime option in dmts64.vmargs
    i get below error-
    Error :--Djava.security.auth.login.config=/dbrscdcsst1/dbrs/kafka/installs/scripts/certs/kafka_client_jaas.conf is an invalid option.
    Use -? for help.

    below is configuration of vmargs

    -Dlog4j.configuration=file:conf/log4j.properties -Dorg.apache.commons.logging.Log=com.datamirror.ts.util.trace.TraceSimpleLog -Dcom.datamirror.ts.instance=%TSINSTANCE% com.datamirror.ts.commandlinetools.script.Startup -Djava.security.auth.login.config=/dbrscdcsst1/dbrs/kafka/installs/scripts/certs/kafka_client_jaas.conf -Djava.security.krb5.conf=/dbrscdcsst1/dbrs/kafka/installs/scripts/certs/krb5.conf

    ------------------------------
    Abhishek Kumar
    Fidelity Investments
    ------------------------------



  • 4.  RE: cdc KAFKA with SASL_PLAINTEXT 5522_linux build

    Posted Fri May 15, 2020 03:38 PM
    Hi Abhishek Kumar,

    please follow command :

    -Djava.security.auth.login.config=/dbrscdcsst1/dbrs/kafka/installs/scripts/certs/kafka_client_jaas.conf

    noted :
    1. KafkaServer is a section name in the JAAS file used by each broker. This section tells the broker which principal to use and the location of the keytab where this principal is stored. It allows the broker to login using the keytab specified in this section.
    2. The Client section is used to authenticate a SASL connection with ZooKeeper. It also allows the brokers to set ACLs on ZooKeeper nodes which locks these nodes down so that only the brokers can modify it. It is necessary to have the same primary name across all brokers. If you want to use a section name other than Client, set the system property zookeeper.sasl.client to the appropriate name (e.g. -Dzookeeper.sasl.client=ZkClient).
    3. ZooKeeper uses zookeeper as the service name by default. If you want to change this, set the system property zookeeper.sasl.client.username to the appropriate name (e.g. -Dzookeeper.sasl.client.username=zk).



    Thanks,
    Richard

    ------------------------------
    Richard Richard
    ------------------------------



  • 5.  RE: cdc KAFKA with SASL_PLAINTEXT 5522_linux build

    Posted Wed May 20, 2020 01:56 AM
    Thanks Richard and Sarah. 

    I was able to replicate data.   Documentation links are for older version wrt security setup.

    ------------------------------
    Abhishek Kumar
    Fidelity Investments
    ------------------------------