IBM Application Runtimes Come for answers. Stay for best practices. All we’re missing is you. Join / Log in Ask a question
This post is part of a series delving into the details of the JSR-352 (Java Batch) specification. Each post examines a very specific part of the specification and looks at how it works and how you might use it in a real batch application.This series is also available as a podcast on iTunes, Google Play, Stitcher, or use the link to the RSS feed. The next post in the series is here.------When we think about batch applications, we usually think about ancient COBOL programs running in the middle of the night during the mysterious ‘batch window’. But a batch application is really just a fixed sequence of process steps that occur in some specified order and start on some event. It might be a bunch of COBOL running at 3 AM, but it might also be Java invoking a rules engine to process records in a file that showed up on an FTP site.
A Batch process generally does a lot of things. Maybe it processes a lot of records, doing the same basic thing for every record. Maybe it works through a complex sequence of small steps, making decisions along the way about which step is next based on what it finds as it goes.
A batch application might be launched by an Enterprise Scheduler at a particular time or triggered by some event (a file arrives). But a batch application might be triggered from some transactional event occurring. Suppose a servlet is processing a fund transfer and there is a complex sequence of fraud detection steps that have to occur. That processing could occur as a batch application kicked off by the servlet – which may or may not wait for it to finish.
If you want to write a batch application in Java, a good place to start is with JSR-352. This open standard provides a framework for developing batch applications: multiple steps, flow control between the steps based on results the current step, checkpoint/restart capability, concurrent execution of different steps, even concurrent execution of multiple copies of the same step, and more.
Of course, you could do all of that yourself in your own Java code. But why? JSR-352 provides a standard way to do it that is portable across various implementations and platforms. And more importantly, you don’t have to maintain all that infrastructure code! Just write the parts to read from (and write to) your data sources and then focus on the business logic. The flow control and concurrency are all described in a standardized XML file. And the whole thing can be bundled into a .war file just like your web applications (and maybe with that servlet that starts it).
I’ve barely touched on all the cool things that are in JSR-352. There’s a lot more, as well as some interesting tips and techniques that we might get into in further blog posts. There’s lots of exciting and surprising things to be found in….batch!
-----------Index of additional posts:JSR-352 (Java Batch) Post #2: Basics of BatchletsJSR-352 (Java Batch) Post #3: Stopping a Running BatchletJSR-352 (Java Batch) Post #4: Introducing the Step ContextJSR-352 (Java Batch) Post #5: Using StepContext UserDataJSR-352 (Java Batch) Post #6: Setting Step Exit Status ValuesJSR-352 (Java Batch) Post #7: Basic Step Flow ControlJSR-352 (Java Batch) Post #8: Flow Control Using Step Exit StatusJSR-352 (Java Batch) Post #9: Using the Job Context Transient DataJSR-352 (Java Batch) Post #10: Using the Job Properties from the JobContextJSR-352 (Java Batch) Post #11: Setting the Job Exit StatusJSR-352 (Java Batch) Post #12: An Introduction to ListenersJSR-352 (Java Batch) Post #13: Details of the Job ListenerJSR-352 (Java Batch) Post #14: Details of the Step ListenerJSR-352 (Java Batch) Post #15: Grouping Job Steps into FlowsJSR-352 (Java Batch) Post #16: Running Flows Concurrently with Splits: Splitting Up Is Easy To DoJSR-352 (Java Batch) Post #17: Making Job Flow Decisions with a Decider (Part 1)JSR-352 (Java Batch) Post #18: Making Job Flow Decisions with a Decider (Part 2)JSR-352 (Java Batch) Post #19: Specifying Job ParametersJSR-352 (Java Batch) Post #20: Setting and Using Job PropertiesJSR-352 (Java Batch) Post #21: JSL Substitution SyntaxJSR-352 (Java Batch) Post #22: Substitution Syntax Continued: DefaultsJSR-352 (Java Batch) Post #23: Substitution Syntax - More Realistic SamplesJSR-352 (Java Batch) Post #24: One Chunk at a TimeJSR-352 (Java Batch) Post #25: Writing a Chunk-Step ReaderJSR-352 (Java Batch) Post #26: Processing Data with an ItemProcessorJSR-352 (Java Batch) Post #27: Writing results with an ItemWriterJSR-352 (Java Batch) Post #28: Ka-Chunk – What’s that? – Ka-Chunk – I think I heard something.. JSR-352 (Java Batch) Post #29: To Checkpoint Now, or Not To Checkpoint Now, and if Not Now, When?JSR-352 (Java Batch) Post #30: Go (or Checkpoint) Your Own WayJSR-352 (Java Batch) Post #31: Listen to the Rhythm of the Falling Rain (or the Chunk Processing…)JSR-352 (Java Batch) Post #32: Five Little Monkeys, Jumping on the Bed…. Skipping in Java BatchJSR-352 (Java Batch) Post #33: ‘Round and ‘Round – Retry ProcessingJSR-352 (Java Batch) Post #34: Get Comfortable and Go Back In Your Mind with Retry-Rollback ProcessingJSR-352 (Java Batch) Post #35: JSL Syntax for Retry with and without RollbackJSR-352 (Java Batch) Post #36: A review of Chunk Step ListenersJSR-352 (Java Batch) Post #37: Introduction to PartitionsJSR-352 (Java Batch) Post #38: Static Partition ControlJSR-352 (Java Batch) Post #39: Creating a Partition Plan Dynamically with a PartitionMapperJSR-352 (Java Batch) Post #40: Coding a Skippable, Retryable, Restartable, Partitionable ReaderJSR-352 (Java Batch) Post #41: Analyze This! The Partition Step Collector and AnalyzerJSR-352 (Java Batch) Post #42: Partitions Too Fat? Get a Partition Reducer! (No, not really)JSR-352 (Java Batch) Post #43: Restarting a Partitioned StepJSR-352 (Java Batch) Post #44: Play it again, Sam! Restarting a failed jobJSR-352 (Java Batch) Post #45: Job Restart Processing – How it works..JSR-352 (Java Batch) Post #46: Restarting a Chunk StepJSR-352 (Java Batch) Post #47: Job Parameters when Restarting a JobJSR-352 (Java Batch) Post #48: The Job Operator InterfaceJSR-352 (Java Batch) Post #49: Try To Remember – The Job RepositoryJSR-352 (Java Batch) Post #50: Moving on….to WebSphere LibertyJSR-352 (Java Batch) Post #51: Job Repository Implementations – Part 1JSR-352 (Java Batch) Post #52: Job Repository Implementations – Part 2JSR-352 (Java Batch) Post #53: The REST InterfaceJSR-352 (Java Batch) Post #54: JobOperator Functions via the REST interfaceJSR-352 (Java Batch) Post #55: Purging Old JobsJSR-352 (Java Batch) Post #56: REST Interface page and pagesize ParametersJSR-352 (Java Batch) Post #57: Application Packaging and In-Line JSL with the REST APIJSR-352 (Java Batch) Post #58: batchManager Command Line InterfaceJSR-352 (Java Batch) Post #59: WebSphere Developer Tools and Batch RESTJSR-352 (Java Batch) Post #60: SideBar: Using WDT to Develop Batch ApplicationsJSR-352 (Java Batch) Post #61: The Admin Center Batch Tool – Part 1JSR-352 (Java Batch) Post #62: The Admin Center Batch Tool – Part 2JSR-352 (Java Batch) Post #63: Enterprise Scheduler Integration JSR-352 (Java Batch) Post #64: Enterprise Scheduler Integration with ScriptsJSR-352 (Java Batch) Post #65: The IBM Workload SchedulerJSR-352 (Java Batch) Post #66: What About the Joblog?JSR-352 (Java Batch) Post #67: Accessing Joblogs Using the REST InterfaceJSR-352 (Java Batch) Post #68: Batch Events OverviewJSR-352 (Java Batch) Post #69: Job Instance EventsJSR-352 (Java Batch) Post #70: Job Execution EventsJSR-352 (Java Batch) Post #71: Job Log EventsJSR-352 (Java Batch) Post #72: Step EventsJSR-352 (Java Batch) Post #73: Partitioned Step EventsJSR-352 (Java Batch) Post #74: Split/Flow EventsJSR-352 (Java Batch) Post #75: Introducing Batch Server Topology ConsiderationsJSR-352 (Java Batch) Post #76: Sharing a Job RepositoryJSR-352 (Java Batch) Post #77: The Multi-Server Batch Configuration – DispatchersJSR-352 (Java Batch) Post #78: The Multi-Server Batch Configuration – ExecutorsJSR-352 (Java Batch) Post #79: The Multi-Server Batch Configuration – Job RoutingJSR-352 (Java Batch) Post #80: The Multi-Server Batch Configuration – Work ThrottlingJSR-352 (Java Batch) Post #81: My Job is Urgent – Batch Job Message PriorityJSR-352 (Java Batch) Post #82: The Multi-Server Batch Configuration – No Server? No Problem!JSR-352 (Java Batch) Post #83: The Multi-Server Batch Configuration – PartitionsJSR-352 (Java Batch) Post #84: The Multi-Server Batch Configuration – Bad, Bad MessagesJSR-352 (Java Batch) Post #85: Security OverviewJSR-352 (Java Batch) Post #86: Securing the Job Repository and Job QueueJSR-352 (Java Batch) Post #87: Batch RolesJSR-352 (Java Batch) Post #88: Group Based SecurityJSR-352 (Java Batch) Post #89: Batch Application SecurityJSR-352 (Java Batch) Post #90: Extra Things Just for z/OSJSR-352 (Java Batch) Post #91: The z/OS Native Command Line InterfaceJSR-352 (Java Batch) Post #92: Integrating with JCL Driven JobsJSR-352 (Java Batch) Post #93: Using the JZOS Library and Accessing DatasetsJSR-352 (Java Batch) Post #94: Dataset Contention IssuesJSR-352 (Java Batch) Post #95: What About Utility Programs – DFSORT & IDCAMSJSR-352 (Java Batch) Post #96: SMF RecordingJSR-352 (Java Batch) Post #97: SMF Accounting DataJSR-352 (Java Batch) Post #98: Exploiting a Split to Stop a ChunkJSR-352 (Java Batch) Post #99: Pipelining with a Split-FlowJSR-352 (Java Batch) Post #100: The Retryable-Skippable ExceptionJSR-352 (Java Batch) Post #101: Monitoring a Chunk Step with a ChunkListenerJSR-352 (Java Batch) Post #102: Handling Changes to Checkpoint DataJSR-352 (Java Batch) Post #103: Creating an Aggregate ReaderJSR-352 (Java Batch) Post #104: Initialization Processing for Batch ArtifactsJSR-352 (Java Batch) Post #105: Cleaning Up After a StepJSR-352 (Java Batch) Post #106: Batch Performance – IntroductionJSR-352 (Java Batch) Post #107: Batch Performance – Checkpoint IntervalsJSR-352 (Java Batch) Post #108: Batch Performance – Database Updates - Bulk or 1x1JSR-352 (Java Batch) Post #109: Batch Performance – Checkpoint Data SizeJSR-352 (Java Batch) Post #110: Batch Performance – Process Result SizeJSR-352 (Java Batch) Post #111: Batch Performance – PartitionsJSR-352 (Java Batch) Post #112: Batch Performance – Batch EventsJakarta Batch Post 113: Jakarta Batch IssuesJakarta Batch Post 114: Batch Properties ObjectJakarta Batch Post 115: Partition ListenerJakarta Batch Post 116: Whole Job Step ListenerJakarta Batch Post 117: Get Partition NumberJakarta Batch Post 118: Job Parameters in Job ContextJakarta Batch Post 119: Flushing Persistent User DataJakarta Batch Post 120 – Set Persistent Data by Reference?Jakarta Batch Post 121: Persistent User Data and RollbackJakarta Batch Post 122: Persisting User Data after the Partition MapperJakarta Batch Post 123: Setting a Property from Earlier ResultsJakarta Batch Post 124: Multiple ReadersJakarta Batch Post 125: Add Methods to ItemProcessorJakarta Batch Post 126: No Transaction ChunkJakarta Batch Post 127: Transactions Around Open and CloseJakarta Batch Post 128: Optional WriterJakarta Batch Post 129: JSL InheritanceJakarta Batch Post 130: Job Definition APIJakarta Batch Post 131: Jakarta Batch Extras - Apache BatchEE and JBeretJakarta Batch Post 132: Apache BatchEE - ChainProcessor/ChainBatchletJakarta Batch Post 133: Apache BatchEE – FlatFileItemReaderJakarta Batch Post 134: Apache BatchEE – FlatFileItemWriterJakarta Batch Post 135: Jakarta Batch Extras – Reading CSV FilesJakarta Batch Post 136: Jakarta Batch Extras – Writing CSV FilesJakarta Batch Post 137: Jakarta Batch Extras - JSON and XML Readers and WritersJakarta Batch Post 138: Jakarta Batch Extras - ArrayItemReaderJakarta Batch Post 139: Jakarta Batch Extras – BeanIO Reader and WriterJakarta Batch Post 140: Extras – Reading and Writing Excel FilesJakarta Batch Post 141: Extras – NoOp and Mock WritersJakarta Batch Post 142: Extras – JBeret – OsCommandBatchletJakarta Batch Post 143: Extras – JdbcBatchletJakarta Batch Post 144: Extras – JdbcReaderJakarta Batch Post 145: Extras – JdbcWriterJakarta Batch Post 146: Extras – Using JPA Readers and WritersJakarta Batch Post 147: Extras – NoSQL databasesJakarta Batch Post 148: Extras – Reading and Writing MessagesJakarta Batch Post 149: Extras – REST Readers and WritersJSR-352 (Java Batch) Post #150: How-To: IntroductionJSR-352 (Java Batch) Post #151: How-To: Using Exit Status to Influence Job FlowJSR-352 (Java Batch) Post #152: How-To: Passing Data Between StepsJSR-352 (Java Batch) Post #153: How-To: Setting the Job Exit StatusJSR-352 (Java Batch) Post #154: How-To: Using Splits and FlowsJSR-352 (Java Batch) Post #155: How-To: Use a Decider After a SplitJSR-352 (Java Batch) Post #156: How-To: Using a Checkpoint AlgorithmJSR-352 (Java Batch) Post #157: How-To: Using Skippable Exceptions to Handle Bad RecordsJSR-352 (Java Batch) Post #158: How-To: Use a Partition Mapper - Part 1JSR-352 (Java Batch) Post #159: How-To: Use a Partition Mapper - Part 2JSR-352 (Java Batch) Post #160: How-To: Use a Partition Mapper - Part 3JSR-352 (Java Batch) Post #161: How-To: Write a Retryable ItemReaderJSR-352 (Java Batch) Post #162: How-To: Use a Collector/AnalyzerJSR-352 (Java Batch) Post #163: Kubernetes Jobs IntroductionJSR-352 (Java Batch) Post #164: Kubernetes Jobs - Restart, Backoff, and DeadlineJSR-352 (Java Batch) Post #165: Kubernetes Jobs – One Pod, One JobJSR-352 (Java Batch) Post #166: Kubernetes Jobs – One Pod, One Record?JSR-352 (Java Batch) Post #167: Kubernetes Jobs – Indexed Completion ModeJSR-352 (Java Batch) Post #168: Kubernetes Jobs – ConcurrencyJSR-352 (Java Batch) Post #169: Batch Jobs in Java?JSR-352 (Java Batch) Post #170: Java Batch – Should You Use a Framework?JSR-352 (Java Batch) Post #171: Java Batch – An Open or Proprietary Framework?JSR-352 (Java Batch) Post #172: Java Batch – Is Liberty a Good Place to run Jakarta Batch?JSR-352 (Java Batch) Post #173: That's All Folks