Informix

 View Only

  • 1.  Issues with installing CSDK in Docker container to deploy in GCP

    Posted Thu October 17, 2024 10:41 AM

    I'm trying to set up a data pipeline from our IBM Informix database to GCP Bigquery. For that purpose, I have been trying to install the ODBC driver (within CSDK) in a Docker image to deploy the container later in Google Cloud Run Function. 

    I am stuck in the Dockerfile development and getting error messages when trying to build the container.

    Here is the Dockerfile:

    FROM ubuntu:latest

# Install Python and driver manager
RUN apt update 
RUN apt install -y python3 python3-pip unixodbc unixodbc-dev build-essential 

# Install necessary libraries
RUN apt update 
RUN apt install -y file libncurses5-dev libelf1

# Install JRE
RUN apt update 
RUN apt install -y openjdk-11-jre-headless

# Install Python modules
RUN apt update
RUN apt install -y python3-pandas python3-pyodbc

# Clean up
RUN apt clean
RUN rm -rf /var/lib/apt/lists/*

# Copy files to their destination folders
COPY ibm.csdk.4.50.FC11.LNX.64bit.tar /tmp/
COPY requirements.txt /app/requirements.txt
COPY main.py /app/main.py

# Extra then delete the two .tar files
RUN tar -xf /tmp/ibm.csdk.4.50.FC11.LNX.64bit.tar -C /tmp/
RUN tar -xf tmp/ibm.csdk.4.50.FC11.LNX.tar -C /tmp/
RUN rm /tmp/ibm.csdk.4.50.FC11.LNX.64bit.tar
RUN rm tmp/ibm.csdk.4.50.FC11.LNX.tar

# Install the SDK
RUN /tmp/installclientskd \
     -DLICENSE_ACCEPTED=TRUE \
     -DUSER_INSTALL_DIR=/opt/IBM/informix/4.50 \
     -DCHOSEN_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA \
     -DCHOSEN_INSTALL_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA \
     -DCHOSEN_INSTALL_SET=Custom \
     -DENCRYPTION_LIBRARY=GSKIT

# Set environment variable
ENV INFORMIXDIR = /opt/IBM/Informix.4.50.FC11/lib/cli

#  Populate ocbcinst.ini
RUN echo "[Informix]\n\
Description = Informix ODBC Driver\n\
Driver = /opt/IBM/Informix.4.50.FC11/lib/cli/iclit09b.so\n\
Setup = /opt/IBM/Informix.4.50.FC11/lib/cli/iclit09b.so" >> /etc/odbcinst.ini

# Set the working directory
WORKDIR /app

CMD ["sh"]

    And here is the error returned when I try building the container:

     => ERROR [19/21] RUN /tmp/installclientskd      -DLICENSE_ACCEPTED=TRUE      -DUSER_INSTALL_DIR=/opt/IBM/informix/4.50      -DCHOSEN_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SD  0.5s 
------
 > [19/21] RUN /tmp/installclientskd      -DLICENSE_ACCEPTED=TRUE      -DUSER_INSTALL_DIR=/opt/IBM/informix/4.50      -DCHOSEN_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT:
0.488 /bin/sh: 1: /tmp/installclientskd: not found
------

 1 warning found (use docker --debug to expand):
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 44)
Dockerfile:35
--------------------
HN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT:
0.488 /bin/sh: 1: /tmp/installclientskd: not found
------

 1 warning found (use docker --debug to expand):
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 44)
Dockerfile:35
--------------------
------

 1 warning found (use docker --debug to expand):
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 44)
Dockerfile:35
--------------------
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 44)
Dockerfile:35
--------------------
  34 |     # Install the SDK
--------------------
  34 |     # Install the SDK
  35 | >>> RUN /tmp/installclientskd \
  34 |     # Install the SDK
  35 | >>> RUN /tmp/installclientskd \
  35 | >>> RUN /tmp/installclientskd \
  36 | >>>      -DLICENSE_ACCEPTED=TRUE \
  36 | >>>      -DLICENSE_ACCEPTED=TRUE \
  37 | >>>      -DUSER_INSTALL_DIR=/opt/IBM/informix/4.50 \
  38 | >>>      -DCHOSEN_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA \
  39 | >>>      -DCHOSEN_INSTALL_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA \
  39 | >>>      -DCHOSEN_INSTALL_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA \
  40 | >>>      -DCHOSEN_INSTALL_SET=Custom \
  41 | >>>      -DENCRYPTION_LIBRARY=GSKIT
  42 |
--------------------
--------------------
ERROR: failed to solve: process "/bin/sh -c /tmp/installclientskd      -DLICENSE_ACCEPTED=TRUE      -DUSER_INSTALL_DIR=/opt/IBM/informix/4.50      -DCHOSEN_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-ERROR: failed to solve: process "/bin/sh -c /tmp/installclientskd      -DLICENSE_ACCEPTED=TRUE      -DUSER_INSTALL_DIR=/opt/IBM/informix/4.50      -DCHOSEN_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_FEATURE_LIST=SDK,SDK-CPP,SDK-CPP-DEMO,SDK-ESQL,SDK-ESQL-ACM,SDK-ESQL-DEMO,SDK-LMI,SDK-ODBC,SDK-ODBC-DEMO,GLS,GLS-WEURAM,GLS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT" did not complete successfully: exit code: 127
PS G:\My Drive\Python\Docker>
LS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT" did not complete successfully: exit code: 127
PS G:\My Drive\Python\Docker>

LS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT" did not complete successfully: exit code: 127
PS G:\My Drive\Python\Docker>


LS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT" did not complete successfully: exit code: 127
PS G:\My Drive\Python\Docker>


LS-EEUR,GLS-JPN,GLS-KOR,GLS-CHN,GLS-OTH,DBA-DBA      -DCHOSEN_INSTALL_SET=Custom      -DENCRYPTION_LIBRARY=GSKIT" did not complete successfully: exit code: 127

    Can someone help me to see where I went wrong?



    ------------------------------
    Clement Martin
    ------------------------------


  • 2.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Tue October 22, 2024 01:51 AM 
    0.488 /bin/sh: 1: /tmp/installclientskd: not found

    If you untar the 

    ibm.csdk.4.50.FC11.LNX.64bit.tar

    Are you sure, that 

    installclientskd

    file is located in /tmp/ directly and not it e.g. /tmp/ibm.csdk.4.50.FC11.LNX.64bit?
    Also, I would check if that file has execution privilege (or add it chmod u+x ./<filename>).



    ------------------------------
    Ondřej
    ------------------------------

    Original Message


  • 3.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Tue October 22, 2024 08:15 AM

    Also, just to state the obvious: it's "sdk", not "skd" - could it simply be a typo?



    ------------------------------
    Andreas Legner
    Dev Engineer
    HCL Software
    ------------------------------

    Original Message


  • 4.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Tue October 22, 2024 05:52 PM

    Yes, I fixed the typo, but still no luck...



    ------------------------------
    Clement Martin
    ------------------------------

    Original Message


  • 5.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Tue October 22, 2024 05:51 PM

    Yes, I checked where the installclientsdk is located and added chmod to my script, but still no luck.



    ------------------------------
    Clement Martin
    ------------------------------

    Original Message


  • 6.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Fri October 25, 2024 12:47 AM

    What is the error you see now?



    ------------------------------
    Ondřej
    ------------------------------

    Original Message


  • 7.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Wed October 23, 2024 12:30 AM
    Edited by Clement Martin Wed October 23, 2024 02:02 PM

    With some external help, we got the Dockerfile to work and successfully built the container.

    Now I am testing my app.py script to see if I can test the connection with the db. But I am having more errors.

    Here is my updated Dockerfile:

    FROM python:3.9-slim-buster

# Installs
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y \
    unixodbc \
    unixodbc-dev \
    build-essential \
    file \
    libncurses5-dev \
    libelf1 \
    openjdk-11-jre-headless \
    && apt-get clean \
    && rm -rf /var/lib/apt/lists/*

# Copy files
COPY /drivers/installclientsdk /tmp/installclientsdk
COPY /src/conn.properties /tmp/conn.properties

# Run the client SDK installer
RUN chmod +x /tmp/installclientsdk \
    && /tmp/installclientsdk -i silent -f /tmp/conn.properties || true

# Set env variable and PATH
ENV INFORMIXDIR=/opt/IBM/Informix_Client-SDK
ENV LD_LIBRARY_PATH=${INFORMIXDIR}/lib:${INFORMIXDIR}/lib/esql:${INFORMIXDIR}/lib/cli

# Populate odbcinst.ini 
RUN echo "[IBM INFORMIX ODBC DRIVER]" > /etc/odbcinst.ini && \
    echo "Description=Informix ODBC Driver" >> /etc/odbcinst.ini && \
    echo "Driver=/opt/IBM/Informix_Client-SDK/lib/cli/iclit09b.so" >> /etc/odbcinst.ini && \
    echo "Setup=/opt/IBM/Informix_Client-SDK/lib/cli/iclit09b.so" >> /etc/odbcinst.ini

# Install Python dependencies
COPY /src/requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt

COPY /src/app.py /app/app.py

WORKDIR /app

CMD ["sh"]

    Here is the app.py:

    import pyodbc

cxn_string = \
  'DRIVER={IBM INFORMIX ODBC DRIVER};\
  UID=user_name;\
  PWD=12345;\
  DATABASE=db;\
  HOST=host.com;\
  SERV=9088'\

connection = pyodbc.connect(cxn_string)

print('Successful connection')

connection.close()

    And here is the error message:

    Traceback (most recent call last):
  File "/app/app.py", line 10, in <module>
    connection = pyodbc.connect(cxn_string)
pyodbc.Error: ('H000', '[H000] [ (-11060) (SQLDriverConnect)')



    ------------------------------
    Clement Martin
    ------------------------------

    Original Message


  • 8.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Fri October 25, 2024 08:24 AM

    Can you set up a functional app/connection on some VM using the same ODBC configuration?

    Another possible issue is that Informix SDK does not officially support Debian Buster, so you could try some supported OS.



    ------------------------------
    Ondřej
    ------------------------------

    Original Message


  • 9.  RE: Issues with installing CSDK in Docker container to deploy in GCP

    Posted Mon October 28, 2024 06:15 PM

    I got it to work with a bit of external help, eventually.

    1. I needed to add Django module to the requirement.txt file -> which is odd cause I didn't need it on my local script, plus I could not find any note about django in the driver documentation.
    2. I also had to play around with the locales

    Thanks to anyone who helped.

    Clement



    ------------------------------
    Clement Martin
    ------------------------------

    Original Message