WebSphere Application Server & Liberty

 View Only

JSR-352 (Java Batch) Post #1: Batch Jobs in Java? YES! Modernizing Batch Applications with JSR-352

By David Follis posted Fri June 15, 2018 02:03 PM

  

This post is part of a series delving into the details of the JSR-352 (Java Batch) specification. Each post examines a very specific part of the specification and looks at how it works and how you might use it in a real batch application.

This series is also available as a podcast on iTunesGoogle PlayStitcher, or use the link to the RSS feed

The next post in the series is here.
------
When we think about batch applications, we usually think about ancient COBOL programs running in the middle of the night during the mysterious ‘batch window’.  But a batch application is really just a fixed sequence of process steps that occur in some specified order and start on some event.  It might be a bunch of COBOL running at 3 AM, but it might also be Java invoking a rules engine to process records in a file that showed up on an FTP site. 

A Batch process generally does a lot of things.  Maybe it processes a lot of records, doing the same basic thing for every record.  Maybe it works through a complex sequence of small steps, making decisions along the way about which step is next based on what it finds as it goes.

A batch application might be launched by an Enterprise Scheduler at a particular time or triggered by some event (a file arrives).  But a batch application might be triggered from some transactional event occurring.  Suppose a servlet is processing a fund transfer and there is a complex sequence of fraud detection steps that have to occur.  That processing could occur as a batch application kicked off by the servlet – which may or may not wait for it to finish.

If you want to write a batch application in Java, a good place to start is with JSR-352.  This open standard provides a framework for developing batch applications:  multiple steps, flow control between the steps based on results the current step, checkpoint/restart capability, concurrent execution of different steps, even concurrent execution of multiple copies of the same step, and more.

Of course, you could do all of that yourself in your own Java code.  But why?  JSR-352 provides a standard way to do it that is portable across various implementations and platforms.  And more importantly, you don’t have to maintain all that infrastructure code!  Just write the parts to read from (and write to) your data sources and then focus on the business logic.  The flow control and concurrency are all described in a standardized XML file.  And the whole thing can be bundled into a .war file just like your web applications (and maybe with that servlet that starts it). 

I’ve barely touched on all the cool things that are in JSR-352.  There’s a lot more, as well as some interesting tips and techniques that we might get into in further blog posts.  There’s lots of exciting and surprising things to be found in….batch!

-----------
Index of additional posts:

JSR-352 (Java Batch) Post #2: Basics of Batchlets
JSR-352 (Java Batch) Post #3: Stopping a Running Batchlet
JSR-352 (Java Batch) Post #4: Introducing the Step Context
JSR-352 (Java Batch) Post #5: Using StepContext UserData
JSR-352 (Java Batch) Post #6: Setting Step Exit Status Values
JSR-352 (Java Batch) Post #7: Basic Step Flow Control
JSR-352 (Java Batch) Post #8: Flow Control Using Step Exit Status
JSR-352 (Java Batch) Post #9: Using the Job Context Transient Data
JSR-352 (Java Batch) Post #10: Using the Job Properties from the JobContext
JSR-352 (Java Batch) Post #11: Setting the Job Exit Status
JSR-352 (Java Batch) Post #12: An Introduction to Listeners
JSR-352 (Java Batch) Post #13: Details of the Job Listener
JSR-352 (Java Batch) Post #14: Details of the Step Listener
JSR-352 (Java Batch) Post #15: Grouping Job Steps into Flows
JSR-352 (Java Batch) Post #16: Running Flows Concurrently with Splits: Splitting Up Is Easy To Do
JSR-352 (Java Batch) Post #17: Making Job Flow Decisions with a Decider (Part 1)
JSR-352 (Java Batch) Post #18: Making Job Flow Decisions with a Decider (Part 2)
JSR-352 (Java Batch) Post #19: Specifying Job Parameters
JSR-352 (Java Batch) Post #20: Setting and Using Job Properties
JSR-352 (Java Batch) Post #21:  JSL Substitution Syntax
JSR-352 (Java Batch) Post #22: Substitution Syntax Continued: Defaults
JSR-352 (Java Batch) Post #23: Substitution Syntax - More Realistic Samples
JSR-352 (Java Batch) Post #24:  One Chunk at a Time
JSR-352 (Java Batch) Post #25: Writing a Chunk-Step Reader
JSR-352 (Java Batch) Post #26: Processing Data with an ItemProcessor
JSR-352 (Java Batch) Post #27: Writing results with an ItemWriter
JSR-352 (Java Batch) Post #28: Ka-Chunk – What’s that? – Ka-Chunk – I think I heard something.. ​​
JSR-352 (Java Batch) Post #29: To Checkpoint Now, or Not To Checkpoint Now, and if Not Now, When?
JSR-352 (Java Batch) Post #30: Go (or Checkpoint) Your Own Way
JSR-352 (Java Batch) Post #31: Listen to the Rhythm of the Falling Rain (or the Chunk Processing…)
JSR-352 (Java Batch) Post #32: Five Little Monkeys, Jumping on the Bed…. Skipping in Java Batch
JSR-352 (Java Batch) Post #33: ‘Round and ‘Round – Retry Processing
JSR-352 (Java Batch) Post #34: Get Comfortable and Go Back In Your Mind with Retry-Rollback Processing
JSR-352 (Java Batch) Post #35: JSL Syntax for Retry with and without Rollback
JSR-352 (Java Batch) Post #36: A review of Chunk Step Listeners
JSR-352 (Java Batch) Post #37: Introduction to Partitions
JSR-352 (Java Batch) Post #38: Static Partition Control
JSR-352 (Java Batch) Post #39: Creating a Partition Plan Dynamically with a PartitionMapper
JSR-352 (Java Batch) Post #40: Coding a Skippable, Retryable, Restartable, Partitionable Reader
JSR-352 (Java Batch) Post #41: Analyze This! The Partition Step Collector and Analyzer
JSR-352 (Java Batch) Post #42: Partitions Too Fat? Get a Partition Reducer! (No, not really)
JSR-352 (Java Batch) Post #43: Restarting a Partitioned Step
JSR-352 (Java Batch) Post #44: Play it again, Sam! Restarting a failed job
JSR-352 (Java Batch) Post #45: Job Restart Processing – How it works..
JSR-352 (Java Batch) Post #46: Restarting a Chunk Step
JSR-352 (Java Batch) Post #47: Job Parameters when Restarting a Job
JSR-352 (Java Batch) Post #48: The Job Operator Interface
JSR-352 (Java Batch) Post #49: Try To Remember – The Job Repository
JSR-352 (Java Batch) Post #50: Moving on….to WebSphere Liberty
JSR-352 (Java Batch) Post #51: Job Repository Implementations – Part 1
JSR-352 (Java Batch) Post #52: Job Repository Implementations – Part 2
JSR-352 (Java Batch) Post #53: The REST Interface
JSR-352 (Java Batch) Post #54: JobOperator Functions via the REST interface
JSR-352 (Java Batch) Post #55: Purging Old Jobs
JSR-352 (Java Batch) Post #56: REST Interface page and pagesize Parameters
JSR-352 (Java Batch) Post #57: Application Packaging and In-Line JSL with the REST API
JSR-352 (Java Batch) Post #58: batchManager Command Line Interface
JSR-352 (Java Batch) Post #59: WebSphere Developer Tools and Batch REST
JSR-352 (Java Batch) Post #60: SideBar: Using WDT to Develop Batch Applications
JSR-352 (Java Batch) Post #61: The Admin Center Batch Tool – Part 1
JSR-352 (Java Batch) Post #62: The Admin Center Batch Tool – Part 2
JSR-352 (Java Batch) Post #63: Enterprise Scheduler Integration
JSR-352 (Java Batch) Post #64: Enterprise Scheduler Integration with Scripts
JSR-352 (Java Batch) Post #65: The IBM Workload Scheduler
JSR-352 (Java Batch) Post #66: What About the Joblog?
JSR-352 (Java Batch) Post #67: Accessing Joblogs Using the REST Interface
JSR-352 (Java Batch) Post #68: Batch Events Overview
JSR-352 (Java Batch) Post #69: Job Instance Events
JSR-352 (Java Batch) Post #70: Job Execution Events
JSR-352 (Java Batch) Post #71: Job Log Events
JSR-352 (Java Batch) Post #72: Step Events
JSR-352 (Java Batch) Post #73: Partitioned Step Events
JSR-352 (Java Batch) Post #74: Split/Flow Events
JSR-352 (Java Batch) Post #75: Introducing Batch Server Topology Considerations
JSR-352 (Java Batch) Post #76: Sharing a Job Repository
JSR-352 (Java Batch) Post #77: The Multi-Server Batch Configuration – Dispatchers
JSR-352 (Java Batch) Post #78: The Multi-Server Batch Configuration – Executors
JSR-352 (Java Batch) Post #79: The Multi-Server Batch Configuration – Job Routing
JSR-352 (Java Batch) Post #80: The Multi-Server Batch Configuration – Work Throttling
JSR-352 (Java Batch) Post #81: My Job is Urgent – Batch Job Message Priority
JSR-352 (Java Batch) Post #82: The Multi-Server Batch Configuration – No Server? No Problem!
JSR-352 (Java Batch) Post #83: The Multi-Server Batch Configuration – Partitions
JSR-352 (Java Batch) Post #84: The Multi-Server Batch Configuration – Bad, Bad Messages
JSR-352 (Java Batch) Post #85: Security Overview
JSR-352 (Java Batch) Post #86: Securing the Job Repository and Job Queue
JSR-352 (Java Batch) Post #87: Batch Roles
JSR-352 (Java Batch) Post #88: Group Based Security
JSR-352 (Java Batch) Post #89: Batch Application Security
JSR-352 (Java Batch) Post #90: Extra Things Just for z/OS
JSR-352 (Java Batch) Post #91: The z/OS Native Command Line Interface
JSR-352 (Java Batch) Post #92: Integrating with JCL Driven Jobs
JSR-352 (Java Batch) Post #93: Using the JZOS Library and Accessing Datasets
JSR-352 (Java Batch) Post #94: Dataset Contention Issues
JSR-352 (Java Batch) Post #95: What About Utility Programs – DFSORT & IDCAMS
JSR-352 (Java Batch) Post #96: SMF Recording
JSR-352 (Java Batch) Post #97: SMF Accounting Data
JSR-352 (Java Batch) Post #98: Exploiting a Split to Stop a Chunk
JSR-352 (Java Batch) Post #99: Pipelining with a Split-Flow
JSR-352 (Java Batch) Post #100: The Retryable-Skippable Exception
JSR-352 (Java Batch) Post #101: Monitoring a Chunk Step with a ChunkListener
JSR-352 (Java Batch) Post #102: Handling Changes to Checkpoint Data
JSR-352 (Java Batch) Post #103: Creating an Aggregate Reader
JSR-352 (Java Batch) Post #104: Initialization Processing for Batch Artifacts
JSR-352 (Java Batch) Post #105: Cleaning Up After a Step
JSR-352 (Java Batch) Post #106: Batch Performance – Introduction
JSR-352 (Java Batch) Post #107: Batch Performance – Checkpoint Intervals
JSR-352 (Java Batch) Post #108: Batch Performance – Database Updates - Bulk or 1x1
JSR-352 (Java Batch) Post #109: Batch Performance – Checkpoint Data Size
JSR-352 (Java Batch) Post #110: Batch Performance – Process Result Size
JSR-352 (Java Batch) Post #111: Batch Performance – Partitions
JSR-352 (Java Batch) Post #112: Batch Performance – Batch Events
Jakarta Batch Post 113: Jakarta Batch Issues
Jakarta Batch Post 114: Batch Properties Object
Jakarta Batch Post 115: Partition Listener
Jakarta Batch Post 116: Whole Job Step Listener
Jakarta Batch Post 117: Get Partition Number
Jakarta Batch Post 118: Job Parameters in Job Context
Jakarta Batch Post 119: Flushing Persistent User Data
Jakarta Batch Post 120 – Set Persistent Data by Reference?
Jakarta Batch Post 121: Persistent User Data and Rollback
Jakarta Batch Post 122: Persisting User Data after the Partition Mapper
Jakarta Batch Post 123: Setting a Property from Earlier Results
Jakarta Batch Post 124: Multiple Readers
Jakarta Batch Post 125: Add Methods to ItemProcessor
Jakarta Batch Post 126: No Transaction Chunk
Jakarta Batch Post 127: Transactions Around Open and Close
Jakarta Batch Post 128: Optional Writer
Jakarta Batch Post 129: JSL Inheritance
Jakarta Batch Post 130: Job Definition API
Jakarta Batch Post 131: Jakarta Batch Extras - Apache BatchEE and JBeret
Jakarta Batch Post 132: Apache BatchEE - ChainProcessor/ChainBatchlet
Jakarta Batch Post 133: Apache BatchEE – FlatFileItemReader
Jakarta Batch Post 134: Apache BatchEE – FlatFileItemWriter
Jakarta Batch Post 135: Jakarta Batch Extras – Reading CSV Files
Jakarta Batch Post 136: Jakarta Batch Extras – Writing CSV Files
Jakarta Batch Post 137: Jakarta Batch Extras - JSON and XML Readers and Writers
Jakarta Batch Post 138: Jakarta Batch Extras - ArrayItemReader
Jakarta Batch Post 139: Jakarta Batch Extras – BeanIO Reader and Writer
Jakarta Batch Post 140: Extras – Reading and Writing Excel Files
Jakarta Batch Post 141: Extras – NoOp and Mock Writers
Jakarta Batch Post 142: Extras – JBeret – OsCommandBatchlet
Jakarta Batch Post 143: Extras – JdbcBatchlet
Jakarta Batch Post 144: Extras – JdbcReader
Jakarta Batch Post 145: Extras – JdbcWriter
Jakarta Batch Post 146: Extras – Using JPA Readers and Writers
Jakarta Batch Post 147: Extras – NoSQL databases
Jakarta Batch Post 148: Extras – Reading and Writing Messages
Jakarta Batch Post 149: Extras – REST Readers and Writers
JSR-352 (Java Batch) Post #150: How-To: Introduction
JSR-352 (Java Batch) Post #151: How-To: Using Exit Status to Influence Job Flow
JSR-352 (Java Batch) Post #152: How-To: Passing Data Between Steps
JSR-352 (Java Batch) Post #153: How-To: Setting the Job Exit Status
JSR-352 (Java Batch) Post #154: How-To: Using Splits and Flows
JSR-352 (Java Batch) Post #155: How-To: Use a Decider After a Split
JSR-352 (Java Batch) Post #156: How-To: Using a Checkpoint Algorithm
JSR-352 (Java Batch) Post #157: How-To: Using Skippable Exceptions to Handle Bad Records
JSR-352 (Java Batch) Post #158: How-To: Use a Partition Mapper - Part 1
JSR-352 (Java Batch) Post #159: How-To: Use a Partition Mapper - Part 2
JSR-352 (Java Batch) Post #160: How-To: Use a Partition Mapper - Part 3
JSR-352 (Java Batch) Post #161: How-To: Write a Retryable ItemReader
JSR-352 (Java Batch) Post #162: How-To: Use a Collector/Analyzer
JSR-352 (Java Batch) Post #163: Kubernetes Jobs Introduction
JSR-352 (Java Batch) Post #164: Kubernetes Jobs - Restart, Backoff, and Deadline
JSR-352 (Java Batch) Post #165: Kubernetes Jobs – One Pod, One Job
JSR-352 (Java Batch) Post #166: Kubernetes Jobs – One Pod, One Record?
JSR-352 (Java Batch) Post #167: Kubernetes Jobs – Indexed Completion Mode
JSR-352 (Java Batch) Post #168: Kubernetes Jobs – Concurrency
JSR-352 (Java Batch) Post #169: Batch Jobs in Java?
JSR-352 (Java Batch) Post #170: Java Batch – Should You Use a Framework?
JSR-352 (Java Batch) Post #171: Java Batch – An Open or Proprietary Framework?
JSR-352 (Java Batch) Post #172: Java Batch – Is Liberty a Good Place to run Jakarta Batch?
JSR-352 (Java Batch) Post #173: That's All Folks

0 comments
464 views

Permalink