WebSphere Application Server

JSR-352 (Java Batch) Post #101: Monitoring a Chunk Step with a ChunkListener

By David Follis posted Wed July 29, 2020 09:26 AM

  
This post is part of a series delving into the details of the JSR-352 (Java Batch) specification. Each post examines a very specific part of the specification and looks at how it works and how you might use it in a real batch application.

To start at the beginning, follow the link to the first post.

The next post in the series is here.

This series is also available as a podcast on iTunesGoogle PlayStitcher, or use the link to the RSS feed
-----

Once in a while I get involved with a batch application that has a chunk step that is not performing as well as the developer had expected (or perhaps has management had hoped).  Sometimes you can fiddle around with the chunk interval and improve things.  Partitioning may or may not be an option.  Ultimately everybody just ends up wondering what the heck is going in there…. How can you tell? 

Well, the obvious thing is to put in some message logging and see.  But developers are often reluctant to clutter up the application with monitoring code.  Fortunately, the batch programming model supports listeners that let you do that without touching the application.

It occurred to me that, with the various listeners you can configure for a chunk step, it would be possible to implement all of them (or most of them anyway) in a single Java class and accumulate metrics about what was going on in the step.  You could record elapsed time (and, depending the platform, CPU time) for each read, process, and write. 

While you could log start and end timestamps as you go, that’s going to clutter up the job log something fierce, and probably slow things down a bit.  Remember that while one logged message might not cost much, one logged message every time through a loop that runs several million times will cost a lot. 

Instead, just hang onto the information in an in-memory table and dump it all out to a file when the step ends (because there’s a step-end listener you can implement too!).  Enabling or disabling the listener(s) is just a matter of putting the elements in the JSL – no application updates are required beyond having the listener class in the deployed application .war file.  There’s no contact between the application and the listeners, so they don’t need to be in the same Java package.  And you can have multiple implementations of any listener get control in the same job, so there’s no worry about these listeners causing problems for listeners used by the application.

Then just run the job as usual and have a look at the resulting data.  Are you spending your time reading data?  Processing data?  Writing results?  Is it all the time or are there “special” records that seem to take longer?  Or maybe it is time-of-day related…perhaps every 10 minutes access to a database slows down mysteriously…something else is touching it and causing contention?  Implementing this monitoring listener won’t tell you why it is slower than you expected, but it is quite likely to at least give you some clues to pursue.

Oh, and I went ahead and wrote up the listener.  You can find it in github here:  https://github.com/WASdev/sample.batch.misc and you can find a whitepaper that goes into more detail about how it works here: 

http://www.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP102780

0 comments
8 views

Permalink