For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu
------------------------------

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu
------------------------------

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Thank you for that suggestion Tulasi. I tried that, but got the exact same errors as before.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

Original Message:
Sent: Mon June 13, 2022 08:53 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Hi Rob,

It has been long since I used Hadoop, I will setup  again, test and get back to you.

Thanks,

------------------------------
Tulasi Das Uppu
------------------------------

Original Message:
Sent: Mon June 13, 2022 12:18 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you for that suggestion Tulasi. I tried that, but got the exact same errors as before.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 08:53 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

I can't thank you enough! Most appreciated.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

Original Message:
Sent: Mon June 13, 2022 11:47 PM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

It has been long since I used Hadoop, I will setup  again, test and get back to you.

Thanks,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 12:18 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you for that suggestion Tulasi. I tried that, but got the exact same errors as before.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 08:53 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Hi Rob,

I am from Optim Development. We don't have CDH license and for some reason Cloudera stopped providing trial license CDH. I tried to configure Apache Hadoop and Hive but having some problem in setting up the environment. Resolving all those problems would take sometime.

To troubleshoot your problem I wanted to have a screen sharing session, would you be in a position to open a PMR and from there I will take it forward.

Thanks,

------------------------------
Tulasi Das Uppu
------------------------------

Original Message:
Sent: Tue June 14, 2022 06:39 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

I can't thank you enough! Most appreciated.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 11:47 PM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

It has been long since I used Hadoop, I will setup  again, test and get back to you.

Thanks,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 12:18 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you for that suggestion Tulasi. I tried that, but got the exact same errors as before.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 08:53 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Sure, that would be great. I will open a PMR and let you know.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

Original Message:
Sent: Wed June 15, 2022 01:55 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I am from Optim Development. We don't have CDH license and for some reason Cloudera stopped providing trial license CDH. I tried to configure Apache Hadoop and Hive but having some problem in setting up the environment. Resolving all those problems would take sometime.

To troubleshoot your problem I wanted to have a screen sharing session, would you be in a position to open a PMR and from there I will take it forward.

Thanks,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Tue June 14, 2022 06:39 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

I can't thank you enough! Most appreciated.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 11:47 PM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

It has been long since I used Hadoop, I will setup  again, test and get back to you.

Thanks,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 12:18 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you for that suggestion Tulasi. I tried that, but got the exact same errors as before.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 08:53 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim

Thank you again Tulasi. I just opened Case number TS009676092
Rob

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

Original Message:
Sent: Wed June 15, 2022 01:55 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I am from Optim Development. We don't have CDH license and for some reason Cloudera stopped providing trial license CDH. I tried to configure Apache Hadoop and Hive but having some problem in setting up the environment. Resolving all those problems would take sometime.

To troubleshoot your problem I wanted to have a screen sharing session, would you be in a position to open a PMR and from there I will take it forward.

Thanks,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Tue June 14, 2022 06:39 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

I can't thank you enough! Most appreciated.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 11:47 PM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

It has been long since I used Hadoop, I will setup  again, test and get back to you.

Thanks,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 12:18 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you for that suggestion Tulasi. I tried that, but got the exact same errors as before.

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 08:53 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

I see some errors in the attached trace file.
Can you please try by changing

ODBC Configuration, Click Advanced Options, Change "Default string column length: from 255 to 32767.

Regards,

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Mon June 13, 2022 08:04 AM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

Thank you Tulasi!  You make a good point -- I didn't even notice the blank data types myself.
I'm using Cloudera ODBC Driver for Apache Hive (32-bit version).  This is the driver I was given by our internal Hadoop team.  There are so many configuration options for this driver, and I'm really not familiar with any of them since this is my 1st time working with Hadoop.
I've attached my latest PR0TOOL log file, named PR0TOOL.008.txt

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH

Original Message:
Sent: Mon June 13, 2022 01:51 AM
From: Tulasi Das Uppu
Subject: Optim TDM and Hadoop Hive

Hi Rob,

What ODBC driver you are have configured?

In the attached screen shot, I see for some of the columns, data types are blank. It should have been mapped proper types. I guess it is failing because of this only.

Can you please attach pr0tool trace file.

Regards,
Tulasi

------------------------------
Tulasi Das Uppu

Original Message:
Sent: Fri June 10, 2022 01:01 PM
From: Rob Searson
Subject: Optim TDM and Hadoop Hive

For the first time, I am attempting to use distributed Optim TDM to access data within a Hadoop hive, via an ODBC DSN. I was able to set up the new DSN using the Cloudera ODBC Driver for Apache Hive, create a new DB alias within the Optim Configuration UI, and create a new access definition for a specific table within the Optim UI (PR0TOOL).  Within my new access definition, I can successfully bring up the "Table Specifications" and see the column definitions; so I know Optim can "see" the table (see attached screen print). However when I try to run an Extract using this AD, I immediately get an error. In the log file I'm seeing messages:
ErrMsg : COL_CMTransformCompile failed
Line : 4456 Function: XfbLclBuildColumnTransform Module: ..\SrcRes\XFBCMAIN.c
RetCode: XFBERR_COL_CM_TRANSFORM_COMPILE_FAILED(10250) Transform Compile Failed

My question is - does anyone here have experience with accessing a Hadoop hive with TDM, and if so can you share any information on anything special you needed to do for it? 
For example, I know that in order to access a database within an RDBMS like Oracle or SQL Server you must install a bunch of Optim stored procedures into the database. But I have not done that here -- Are there stored procedures that need to be installed into a hive, and if so what/where are they?
Thanks for any help you can provide!
Rob Searson, Progressive

------------------------------
Rob Searson
Application Developer
Progressive Insurance
Mayfield Village OH
------------------------------

#InfoSphereOptim
#Optim