WebSphere Application Server & Liberty

 View Only

JSR-352 (Java Batch) Post #161: How-To: Write a Retryable ItemReader

By David Follis posted Thu November 11, 2021 08:28 AM

  
This post is part of a series delving into the details of the JSR-352 (Java Batch) specification. Each post examines a very specific part of the specification and looks at how it works and how you might use it in a real batch application.

To start at the beginning, follow the link to the first post.

The next post in the series is here.

This series is also available as a podcast on iTunesGoogle PlayStitcher, or use the link to the RSS feed
-----

How you do this depends a lot on how your reader functions, so this time we’re going to tie the discussion a little more closely to the actual example.  As usual, the code is all at https://github.com/follisd/batch-samples

Our JSL is defined in CodingARetryableReader.xml, but you don’t need to go look at that yet.  First let’s talk about how our reader is going to work.  We’re going to presume that we need to read records out of some database and that we’re using an integer as the primary key.  We’ll start with a value of one and increment our way up until we get to the last valid key (hard-coded as 100 in our example just to keep things simple). 

That means when our reader is called to do open processing we’ll initialize the integer index to one and as we read items we’ll increment the index.  Sounds easy and simple.  Now we’re going to cause trouble.  The processor for our sample, ThrowOnThirtyEightProcessor will throw an Illegal Argument Exception when it is passed an object from the reader for index 38.  It will only do this once, so the error will be ‘resolved’ if we retry.  We’ll add a retryable exception element to the JSL to let the batch container know we expect this exception and want to just retry it.

What happens next?  The batch container will roll back the transaction wrapping the chunk processing, close the reader and writer, and then re-open them.  The expectation is that the reader will start over at the last checkpoint.  What’s that?  Our reader didn’t provide any checkpoint data?  Oh dear.  That means when the reader is called to do open processing it will initialize the index at one just like when the job started.  We’ll reprocess the records we’ve already processed and committed.  We need to provide some checkpoint data.

To do that we need to create a serializable object to contain the reader’s checkpoint data.  You might be tempted (I was) to make an inner class within the reader to represent the data.  After all, nobody else cares about this except the reader.  However, if you do that you’ll get in trouble when the batch container tries to serialize or deserialize the data.  Make it a separate class.  Ours is called IntegerReaderState because it just contains an Integer.

After we update our reader to provide the state object as checkpoint data, there’s just one thing left.  We need to update our open processing to determine whether we’ve been passed checkpoint data or not.  When the job first starts, the parameter is null, but after a retry or restart the parameter will contain the last checkpoint data provided.  If the parameter is null we need to create the checkpoint data object, but if one was passed in then that’s the data to use so we pick up where we left off.

If you run the sample job, you should also notice that after the exception at index 38 and the retry back to the last checkpoint (at 30 because we checkpoint every 10 elements), the batch container will take a checkpoint after every record until it gets past the one that caused the problem.  So checkpoints will happen for element 31, 32, 33, etc. up to 38.  Then the container will resume checkpointing every 10 elements as before.

Permalink