IBM Sterling Transformation Extender

Sterling Transformation Extender

Come for answers, stay for best practices. All we're missing is you.


#Sterling
 View Only
  • 1.  AWS DynamoDB and ITX

    Posted Thu April 13, 2023 09:12 AM

    Use of AWS DynamoDB has been proposed for our ITX environment.  We have a need to put a fair number of text lookup files in something besides text files to increase performance.  Our historical problem has been an unreliable DB environment.  Our ITX runns 24x7x365, but the only db we currently have access to goes down at least quarterly for maintenance.  We do not want to tie our always-on ITX activity to something that is less then always on.

    The advantage of text files is they are always present, and we have used them for years instead of db because of this reliability issue.  DynamoDB would also be always on, but we still have to connect to it.

    I'm seeing third party ODBC drivers for DynamoDB that should work in an ITX process.  I'm also seeing REST capabilities that could work too.

    Is there any plan for a native ITX adapter for DynamoDB?  Any other AWS persistent db and adapter that would be a viable alternative to DyanmoDB?



    ------------------------------
    Jim Davee
    ------------------------------


  • 2.  RE: AWS DynamoDB and ITX

    Posted Thu April 13, 2023 07:17 PM

    There is a database called SQLite.
    It comes with a JDBC driver.   ITX has a JDBC adapter in V10 but I don't remember which version.  You can give it a try.
    https://www.sqlitetutorial.net/sqlite-java/sqlite-jdbc-driver/



    ------------------------------
    Rex Chan
    ------------------------------



  • 3.  RE: AWS DynamoDB and ITX

    Posted Fri April 14, 2023 09:40 AM

    Thanks Rex!  We have considered SQLite for this use case.  The only issue is where the db locks during writes to that single user.  Multiple reads are allowed though, so it's really more of a design consideration.  If we presume this is for lookup data that does not change often then it would likely work very well as a per-server local db.



    ------------------------------
    Jim Davee
    ------------------------------



  • 4.  RE: AWS DynamoDB and ITX

    Posted Fri April 14, 2023 10:27 AM

    Did a little more research and need to understand how SQLite would work in Launcher.  I'm guessing the SQLite JDBC jar file needs to be named in the Launcher dtx.ini to be loaded in the classpath at Launcher startup.  The JVM and maps running in Launcher that call SQLite JDBC will then have access to the driver.  Am I missing anything?



    ------------------------------
    Jim Davee
    ------------------------------



  • 5.  RE: AWS DynamoDB and ITX

    Posted Fri April 14, 2023 11:27 AM

    In V10, there should be a /data/extra folder.   You can simply put your jar there.  If not, specifying it in dtx.ini should work.



    ------------------------------
    Rex Chan
    ------------------------------



  • 6.  RE: AWS DynamoDB and ITX

    Posted Fri April 21, 2023 02:29 PM

    Just confirming where the JDBC jar file goes on RHEL ITX installs.  The itx1010/data/extra isn't working, neither is the itx1010/extjar (which already has the javamail.jar file in it, so I though it would be a good alt to data/extra.

    On our Windows systems with Design Studio the /extjar location worked fine.

    On RHEL I'm not even getting a JDBC trace file so it's not finding anything at all.

    I'm going to try the dtx.ini approach, but was hoping I could just put the JDBC file somewhere already known to ITX.

    Any other suggestions?



    ------------------------------
    Jim Davee
    ------------------------------



  • 7.  RE: AWS DynamoDB and ITX

    Posted Fri April 21, 2023 02:55 PM

    I partially outsmarted myself - we use a trimmed down config/adapters.xml file.  We found that performance on RHEL 8 with the full adapter list was sluggish so we cut out ones we don't use, and JDBC was one.

    I added the JDBC adapter back in and now get a different error message:

    libjvm.so preloadLibrary(/itx1010/libomrsig.so): cannot open shared object file: cannot open shared object file: no such file or directory
    libjvm.so failed to load: omrsig

    That file does exist but under the /itx1010/java dirs.  I created a symlink to it in the root install dir /itx1010 and got a different error message:

    libjvm.so preloadLibrary(/itx1010/libj9vm29.so): cannot open shared object file: cannot open shared object file: no such file or directory
    libjvm.so failed to load: j9vm29

    That libj9vm29.so file does not exist in my itx1010 install.

    Why is the location of the "missing" so files directly under the /itx1010 install dir?



    ------------------------------
    Jim Davee
    ------------------------------



  • 8.  RE: AWS DynamoDB and ITX

    Posted Mon April 24, 2023 12:51 PM

    These are other options you can try:
    <install_dir> 
    <iinstall_dir>/jars/com.hcl.hip.adapters.m4jdbc

    If the problem is not resolved, you may have to open a ticket with support.  They probably can determine the problem through the JNI trace.



    ------------------------------
    Rex Chan
    ------------------------------