WebSphere Application Server & Liberty

JSR-352 (Java Batch) Post #42: Partitions Too Fat? Get a Partition Reducer! (No, not really)

By David Follis posted Wed May 15, 2019 07:46 AM

This post is part of a series delving into the details of the JSR-352 (Java Batch) specification. Each post examines a very specific part of the specification and looks at how it works and how you might use it in a real batch application.

To start at the beginning, follow the link to the first post.

The next post in the series is here.

This series is also available as a podcast on iTunesGoogle PlayStitcher, or use the link to the RSS feed

This is kind of a weird one.  The PartitionReducer gives your batch application a chance to get control at some interesting points during the processing of a partitioned step.  The intent was to give you a chance to do transaction-like things if your application was interacting with non-transactional resources (like a file).  Let’s have a look:

The first method in the reducer is beginPartitionedStep.  This method gets control on the main thread for the step, before the PartitionMapper is called and, thus, before any partitions even start.  This is almost a listener-like point that gets control before any partition stuff has happened at all, but you know it is going to. 

The matching method that receives control when all the partitions are finished is called beforePartitionedStepCompletion.  Why isn’t it called afterPartitionedStep?  Because it doesn’t get control if any of the partitions fail.  There are actually a few different ways this can happen, but basically if an unhandled exception gets thrown out of the partition processing (including from the PartitionMapper!) then this method won’t get control. 

After beforePartitionedStepCompletion gets control, the transaction that wraps PartitionAnalyzer processing on the step thread is committed.  After the commit, another method called afterPartitionedStepCompletion is called.  These two methods (before and after step completion) wrap the commit to let you do commit-like processing for any resources touched by the analyzer that enrolled in the transaction.

On the other hand, if one of the partitions fails, then once the last call to the analyzer is done, we’re going to rollback the transaction around analyzer processing.  But before that happens, the Reducer’s rollbackPartitionedStep method gets control.  So, either the before/after completion methods get control or the rollback method does, depending on how things went with the partitions. 

This is all rather complicated, and you’ll have to think through what your partition analyzer is doing to determine if you really even care how things turned out.