Cognos Analytics with Watson

 View Only
Expand all | Collapse all

XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

  • 1.  XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted Wed October 26, 2022 09:05 AM
    I'm trying to create a JDBC connection to the DATABRICKS/SPARK cluster using CA11.2.1 and  I'm getting this error:

    Driver: DatabricksJDBC42.jar
    Class: com.databricks.client.jdbc.Driver
    Error: "XQE-JDB-0012 The driver "class com.databricks.client.jdbc.Driver" could not be loaded because it is not certified for use with this product."

    Driver: SparkJDBC42.jar
    Class: com.simba.spark.jdbc.Driver
    Error: "XQE-JDB-0012 The driver "class com.simba.spark.jdbc.Driver" could not be loaded because it is not certified for use with this product."

    Why is this connection not yet certified?
    Any way this connection can be certified in the future?
    Is there any other way to create a JDBC connection to DATABRICKS or SPARK cluster using CA?

    Thanks!



    ------------------------------
    Paulo Klein
    ------------------------------

    #CognosAnalyticswithWatson


  • 2.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    IBM Champion
    Posted Wed October 26, 2022 10:07 AM

    Hi,

    From the Supported Environments page for your version (IBM Cognos Analytics on Premises 11.2.x: Supported Software Environments) it looks like Spark SQL Thrift Server is a supported JDBC connection type. It looks like this has been an option since the 11.0.8 release - the notes for this say:
    Cognos Analytics supports the Spark SQL 2.1 Thrift server through the SIMBA (Magnitude) JDBC driver for Spark SQL
    How exactly are you trying to configure the JDBC connection in CA - are you creating a Data Server connection and selecting "Spark SQL" from the list of supported connection types?

    Cheers!

    MF.



    ------------------------------
    Mark Fry
    Technical Consultant
    ------------------------------



  • 3.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted Wed October 26, 2022 10:32 AM
    Hello @Mark Fry,
    Yes,
    I'm triyng to create a new Data Server connection using Spark SQL connection.
    I've created an cluster on https://community.cloud.databricks.com/ to test this connection.
    Using DBeaver the connection works fine. Using CA shows the error message




    ------------------------------
    Paulo Klein
    ------------------------------



  • 4.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    IBM Champion
    Posted Wed October 26, 2022 10:48 AM
    Edited by System 12 days ago

    Hi,

    Just as a sanity check - you are definitely using the 2.1 version of the driver? Have you copied the driver into the Drivers folder in your CA install location, then re-started Cognos Analytics?

    What's slightly puzzling here is that the error you posted refers to com.simba.spark.jdbc.Driver but the class the normal connection uses is com.simba.spark.jdbc4.Driver. Are you sure you haven't accidentally changed the class?

    Cheers!

    MF.



    ------------------------------
    Mark Fry
    Technical Consultant
    ------------------------------



  • 5.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted Wed October 26, 2022 01:21 PM
    Hello,
    Thanks for your answer.

    Using DBeaver I can check the class name in the jar file so I am using the same class name in CA and DBeaver.
    I'm using the same configuration in CA in DBeaver, including the jar file, and both have the same error - works in DBeaver, error message in CA

    ------------------------------------------------------
    SIMBA JDBC DRIVER:
    SimbaSparkJDBC42-2.6.22.1040.zip
    com.simba.spark.jdbc.Driver
    JDBC URL: jdbc:spark://community.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/XX/XX;AuthMech=3

    CA Error:
    XQE-JDB-0012 Could not load the driver "class com.simba.spark.jdbc.Driver" because it is not certified for use with this product.

    ------------------------------------------------------
    DATABRICKS JDBC DRIVER:
    DatabricksJDBC42-2.6.29.1051.zip
    com.databricks.client.jdbc.Driver
    JDBC URL: jdbc:databricks://community.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/XX/XX;AuthMech=3

    CA Error:
    XQE-JDB-0012 Could not load the driver "class com.databricks.client.jdbc.Driver" because it is not certified for use with this product.

    ------------------------------
    Paulo Klein
    ------------------------------



  • 6.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    IBM Champion
    Posted Wed October 26, 2022 01:54 PM

    Hi,

    The error Could not load the driver "class com.simba.spark.jdbc.Driver" because it is not certified for use with this product is telling us that Cognos Analytics does not support a driver with this class. You need to be using the jdbc4 driver. If your current JDBC driver doesn't support this, you need to look for the correct driver that does. If I was in your position, I'd look to see if I could find another jdbc driver.

    Cheers!

    MF.



    ------------------------------
    Mark Fry
    Technical Consultant
    ------------------------------



  • 7.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted Wed October 26, 2022 03:39 PM
    Hello,
    Thanks for your answer.
    Using DBeaver I'm able to scan the Class Name from jar file, so I'm using the same Class Name in CA and DBeaver.
    I'm trying to use the same configuration on CA em DBreaver, including the jar file, and both has same error - works on DBeaver, error message on CA,

    ###########
    SIMBA JDBC
    SimbaSparkJDBC42-2.6.22.1040.zip
    SparkJDBC42.jar
    com.simba.spark.jdbc.Driver
    JDBC URL: jdbc:spark://community.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/xxx/xxx;AuthMech=3

    ###########
    DATABRICKS JDBC JDBC:
    DatabricksJDBC42-2.6.29.1051.zip
    DatabricksJDBC42.jar
    JDBC URL:jdbc:databricks://community.cloud.databricks.com:443/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/xxx/xxx;AuthMech=3







    ------------------------------
    Paulo Klein
    ------------------------------



  • 8.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    IBM Champion
    Posted Wed October 26, 2022 04:08 PM

    Hi,

    I understand what you're saying - you posted this in your previous reply. You need to be using a driver that supports the com.simba.spark.jdbc4.Driver class in Cognos Analytics, though. There is no way around this. Cognos Analytics only allows you to use supported drivers, and if you use an unsupported driver with a class it doesn't recognise (as you appear to be doing here) it simply won't work.

    As I suggested previously, I think your best option is to look for a driver that is supported in Cognos Analytics.

    Cheers!

    MF.



    ------------------------------
    Mark Fry
    Technical Consultant
    ------------------------------



  • 9.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted Wed October 26, 2022 04:48 PM
    Hello,
    Did you know where can I found the jdbc4 driver for SPARK or Databricks ?
    Thanks!

    ------------------------------
    Paulo Klein
    ------------------------------



  • 10.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    InnerCircle
    Posted Tue December 27, 2022 09:36 AM
    We were able to successfully connect to databricsk using both ODBC Driver and JDBC Drvier. For JDBC you need to contact Simba support for "version 2.6.16 of Spark JDBC driver" which has ""com.simba.spark.jdbc4.Driver" driver class. You can reach them at connectivity@magnitude.com

    ------------------------------
    Vishwanatha Chilla
    ------------------------------



  • 11.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted 27 days ago

    Connectivity to Databricks is supported as of CA 11.2.4, but not in previous releases.

    Please refer to the releases notes, and note the minimum version of the Databricks driver that must be used.

    https://www.ibm.com/docs/en/cognos-analytics/11.2.0?topic=administration-databricks-support​

    With respect to the reply #10 and JDBC.  This is not supported regarding working with Databricks.​



    ------------------------------
    NIGEL CAMPBELL
    ------------------------------



  • 12.  RE: XQE-JDB-0012 - JDBC SPARK/DATABRICKS CONNECTION FAILURE

    Posted Mon October 31, 2022 11:56 AM

    Databricks is not a supported product in 11.2.3 or prior releases.

    Similarly nor is the driver that Databricks distribute which is developed for them by SIMBA.

    I recommend your company contact your IBM account rep with regards our roadmap.



    ------------------------------
    NIGEL CAMPBELL
    ------------------------------