IBM Z and LinuxONE IBM Z

IBM Z

The enterprise platform for mission-critical applications brings next-level data privacy, security, and resiliency to your hybrid multicloud.

 View Only
Back to discussions
Expand all | Collapse all

Batch Performance Improvement through Multithreading in JAVA and Partitioning of DB2 Tables

  • 1.  Batch Performance Improvement through Multithreading in JAVA and Partitioning of DB2 Tables

    Posted Tue January 21, 2020 08:54 AM
    I am working on the design of JAVA JEE (Version 8) project which involves batch processing (inserts, updates, deletes) of huge volume of data (~50 Million) in DB2 database. Kindly help with the following for effective handling of such data in JAVA which leads to performance improvement.

    1. Enable multithreading in JAVA and partitioning of DB2 tables. Note: Data in all columns are dynamic and changes over period of time.
    2. Handling chunks of DB2 data in each thread in JAVA with regular intermediate commits.
    3. Proper error handling mechanism in case of abnormal termination in any of the threads.

    Thanks!

    ------------------------------
    Balaji Paramasivam
    ------------------------------


  • 2.  RE: Batch Performance Improvement through Multithreading in JAVA and Partitioning of DB2 Tables

    Posted Tue January 21, 2020 11:04 AM
    Hi Balaji.  Can you help me understand a little more about your application?

    Is this a Java application running on z/OS?

    I see you mention batch, so does that mean you are invoking this Java application by JCL, or is CICS and/or Liberty somehow involved?

    Are you driving DB2 for z/OS?

    I'm glad to see you mention proper error handling.  This is probably one of the most important and hardest functions to implement.

    ------------------------------
    Matthew Cousens
    ------------------------------

    Original Message


  • 3.  RE: Batch Performance Improvement through Multithreading in JAVA and Partitioning of DB2 Tables

    Posted Wed January 22, 2020 06:19 AM

    Hi Mathew,

    Please find below the requested details:

    1. Java Batch (.jar file) will be executed from Linux server. No MainFrame component is involved in this case.
    2. DB2 is driven on z/OS.

    PS: Spring Batch Framework will be used to implement this batch.

    Thanks!



    ------------------------------
    Balaji Paramasivam
    ------------------------------

    Original Message


  • 4.  RE: Batch Performance Improvement through Multithreading in JAVA and Partitioning of DB2 Tables

    Posted Tue January 28, 2020 08:18 AM
    Thank you for explaining that you are running your Java application from Linux.  That means that your application won't be able to rely on components of z/OS to ensure that your entire DB2 transaction is committed.  This can be the hardest thing to implement -- back to your comment about error handling before.  In the easiest scenario one thread's work is not dependent on another thread's work so if one thread dies and thus doesn't commit it can simply be tried again without worrying about the threads that did commit.

    Performance is very specific to how your data is defined to DB2.  For example things like the data type of fields,  indexing, datapools, data sharing, and on and on.  Your DBA should be able to help explain how your data is defined and give you some tips on working with it.

    The best advice I can give you is to work with your DBA to understand how the tables, tablespaces, etc. are defined to DB2 and then do a performance benchmarking on a sample of data once your application is at least partially built.

    ------------------------------
    Matthew Cousens
    ------------------------------

    Original Message


Related Content