IBM Event Streams and IBM Event Automation

 View Only

  • 1.  Event streams Connectors, CP4I

    Posted Tue December 07, 2021 03:39 AM
    Hi All,

    I have been trying to setup Kafka Connect and adding connectors in CP4I Event streams but facing issues while adding mongodb and postgresql connectors...
    When checked logs, can see below error. I am not able to connect to container terminal to debug further since pod itself failed with crashloopbackoff error. Please suggest the fix for this

    021-12-03 06:54:49,658 INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed) [main]
    2021-12-03 06:54:49,697 INFO Loading plugin from: /opt/kafka/plugins/debezium-connector-mongodb-1.2.0.Final.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader) [main]
    2021-12-03 06:54:49,935 ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed) [main]
    java.lang.NoClassDefFoundError: com.mongodb.MongoException
    at java.base/java.lang.J9VMInternals.newInstanceImpl(Native Method)
    at java.base/java.lang.Class.newInstance(Class.java:2290)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:392)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:362)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:334)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:268)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:260)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:229)
    at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:206)
    at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:61)
    at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:93)
    at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:80)
    Caused by: java.lang.ClassNotFoundException: com.mongodb.MongoException
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:605)
    at java.base/java.lang.ClassLoader.loadClassHelper(ClassLoader.java:1188)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1103)
    at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
    at java.base/ja

    Note: Tried IBM MQ source connect and it worked fine without any issues. We are using CP4I 2021.3.1 version which is running openshift 4.7

    ------------------------------
    Bhanu Prakash Desakuru
    ------------------------------


  • 2.  RE: Event streams Connectors, CP4I

    Posted Tue December 07, 2021 04:32 AM
    The exception you've seen (java.lang.NoClassDefFoundError: com.mongodb.MongoException) suggests that there is a problem in the way that you provided the MongoDB connector jar to your Kafka Connect pod (e.g. maybe you accidentally used the wrong jar, or you put it in the wrong place, that sort of thing). 

    To help narrow down the problem, would you mind describing the steps you went through to add the MongoDB connector jar to your Connect setup, please?

    (Note that the Kafka Connect image provided with Event Streams comes with the IBM MQ connector jars already baked in, so it does still make sense that you wouldn't have this problem with the MQ Connectors - unlike any other connectors where you have to provide the jars yourself).

    If it helps, in my recent blog post at https://community.ibm.com/community/user/integration/blogs/dale-lane1/2021/12/03/migrating-to-event-driven-architectures I happened to include a video demo of setting up Connect with third-party connector jars - so you may find the video (the one in the section called "Make existing APIs available as events") useful to compare with what you did.

    ------------------------------
    Dale Lane
    ------------------------------

    Original Message


  • 3.  RE: Event streams Connectors, CP4I

    Posted Tue December 07, 2021 09:28 AM
    Hi Dale Lane,

    Thank you for the help.

    I have rebuilt the image with all dependencies jar of mongodb and postgresql connectors. and deployed. it worked successfully.

    ------------------------------
    Bhanu Prakash Desakuru
    ------------------------------

    Original Message