Cloud Pak for Integration

 View Only
  • 1.  MQ Connector for Confluent on CP4I 2021.2.1

    IBM Champion
    Posted Wed July 28, 2021 09:09 AM
    Hi,
    I have installed Confluent on CP4I 2021.2.1. I need to configure MQ Connector for Confluent (both installed in CP4I cluster). Is there any details steps or demo available?

    ------------------------------
    Thanks,
    Santhosh Ramanathan
    ------------------------------


  • 2.  RE: MQ Connector for Confluent on CP4I 2021.2.1

    IBM TechXchange Speaker
    Posted Fri July 30, 2021 09:43 AM
    Hi Santosh,

    We talked about it and the Confluent docs for MQ would be the best resource right now:  https://docs.confluent.io/kafka-connect-ibmmq-source/current/overview.html

    If you have specific questions when doing this, post them here and we can find some one to answer or will refer you to support if necessary.  We'll also put this down as a requested demo. 

    Thanks!

    ------------------------------
    Stephanie Wilkerson
    IBM
    ------------------------------



  • 3.  RE: MQ Connector for Confluent on CP4I 2021.2.1

    IBM Champion
    Posted Mon August 02, 2021 02:54 AM
    Hi Stephanie,

    Thanks for the response. I am using confluent community edition on openshift CP4I v2021.2.1. I am finding bit difficulty in understanding the documentation related to copying the jar files and configuring the connector in confluent. Is there any video reference to configure MQ connector for confluent available?

    ------------------------------
    Thanks,
    Santhosh Ramanathan
    ------------------------------



  • 4.  RE: MQ Connector for Confluent on CP4I 2021.2.1

    Posted Wed August 04, 2021 08:34 AM
    Hi Santhosh,

    Here're the steps:

    1. Create a dockerfile that will include MQ-Source Connector. For example:

    Ps. line 4 is how to add the JAR files to the connector.
    FROM confluentinc/cp-server-connect-operator:6.1.0.0
    USER root
    RUN confluent-hub install  --no-prompt confluentinc/kafka-connect-ibmmq:11.0.7
    COPY *.jar /usr/share/confluent-hub-components/confluentinc-kafka-connect-ibmmq/lib
    USER 1001​


    2. Build and Push the container image to your container registry. For example:

    docker build -t <container-registry>/<image-name>:<image-tag> . -f <filename.Dockerfile>
    docker push <container-registry>/<image-name>:<image-tag>


    3. Navigate to the Connect Custom Resource (CR) objects. For example:

    connect:
      spec:
        image:
          application: <container-registry>/<image-name>:<image-tag>


    4. Wait for a few second, and a new Connect pod will spin up. You can validate the deployment by investigate the files in the following path (within the connect pod):

    /usr/share/confluent-hub-components


    5. Finally, we are ready to configure the Kafka Connect to source MQ messages, and here's an example of the configurations:

    {
      "name": "IbmMQSourceConnectorConnector_0",
      "connector.class": "io.confluent.connect.ibm.mq.IbmMQSourceConnector",
      "tasks.max": "1",
      "mq.hostname": "<mq-service-name>.<namespace>.svc.cluster.local",
      "mq.port": "1414",
      "mq.transport.type": "client",
      "mq.queue.manager": "QMPAYMENT",
      "mq.channel": "ACE.TO.MQ",
      "jms.destination.name": "NEWORDER.MQ",
      "jms.destination.type": "QUEUE",
      "kafka.topic": "NEWORDERS"
    }


    For your reference:
    1. https://docs.confluent.io/operator/current/co-configure-connect.html
    2. https://docs.confluent.io/kafka-connect-ibmmq-source/current/overview.html

    ------------------------------
    Rashid Aljohani
    ------------------------------



  • 5.  RE: MQ Connector for Confluent on CP4I 2021.2.1

    IBM Champion
    Posted Thu August 05, 2021 10:03 AM
    Hi Rashid,

    Thanks for sharing the info. It was really helpful and its working

    ------------------------------
    Thanks,
    Santhosh Ramanathan
    ------------------------------