WebSphere Application Server

JSR-352 (Java Batch) Post #50: Moving on….to WebSphere Liberty

By David Follis posted Wed July 10, 2019 08:12 AM

  
This post is part of a series delving into the details of the JSR-352 (Java Batch) specification. Each post examines a very specific part of the specification and looks at how it works and how you might use it in a real batch application.

To start at the beginning, follow the link to the first post.

The next post in the series is here.

This series is also available as a podcast on iTunesGoogle PlayStitcher, or use the link to the RSS feed
-----

Well, at least for the moment, I’ve run out of things to write about the JSR-352 specification.  I’m sure there are some other things I’ve overlooked or oversimplified that might be worthy of another post.  Feel free to suggest some things in the comments.  But for now, I’m drawing a blank here.

So, I thought I’d move on to talking about the implementation of JSR-352 that I’m most familiar with.  That would be the implementation in IBM’s WebSphere Liberty.  We added support for JSR-352 as part of the Java EE 7 support and it is fully spec compliant.

But we also added a lot of extra operational enhancements to help you write, run, and manage batch applications running inside Liberty.  Some of those are pretty interesting (and, I think, useful).  I thought I would spend some time looking at some of the enhancements and why I think they are cool. 

I’ve tried to be pretty implementation neutral when talking about JSR-352, but, at least for the next series of posts, I’m going to show some pretty significant bias towards the Liberty implementation. 

I’m planning to talk about some of the extensions and enhancements in Liberty that allow you to better integrate JSR-352 batch applications with an existing enterprise batch environment. 

Since a lot of existing batch applications are running on z/OS, there are some Liberty Batch extensions that are specific to z/OS and help it better integrate into a z/OS environment (think SMF records).  We’ll have a look at those and perhaps also some of the challenges of mixing traditional (COBOL) batch applications with “modernized” Java Batch. 

And after that…. Who knows? 

===================
Alright...I wrote this for use somewhere else and it wound up not being used so it has just been resting on my laptop for a while.  It was fun to write, so I thought I'd share it somewhere...maybe just dropping it here at the end of this transitional post is a good spot..

===================

Bob settled onto the bar stool and signaled to the bartender.  Shortly he had a tall glass of his favorite beverage resting in front of him.  And there it sat, and was still sitting, the foam long settled, when his friend Joe sat down next to him.

“How’s it going Bob?”, he asked.

“Oh, just swell, I got a new assignment.”, Bob replied.

“Really?  Are you in that new AI-based fraud-detection project?”, asked Joe.

“Batch.  I’m in charge of our batch applications.  They want me to modernize them.”  You could hear the italics in Bob’s voice.

“Batch?  Really?  We still have that?  What do they mean by modernize?”, Joe asked.

“Oh, sure we do.  It is actually the core of a lot of our business processes, but good grief is it boring.  And I don’t know what they mean.  Probably anything I like as long as it sounds modern.”, Bob said.

“So…. What are you going to do?”, Joe asked.

“I’m thinking about Java.”, Bob replied.

“Java?  That’s not all that modern.  And it’s interpreted isn’t it?  That’s going to be awful for batch.”, said Joe, shaking his head.

“Well, for something so mission critical I don’t really want anything too new.  Batch is newer than what we’re using now and it is proven to be reliable.  And it actually can be pretty fast.  The Just In Time compiler will generate object code for the platform for code that is run over and over.  And if there’s one application that runs the same code over and over, it is batch.”, Bob said, trying to sound confident.

“So you’re just going to replace all that COBOL with Java and let the Enterprise Scheduler spin up JVMs?”, asked Joe.

“Well, we could, at least to start.  But I read some stuff about JSR-352.  Apparently, there’s this whole batch programming model just for Java.”, Bob said. 

“No kidding”, said Joe, signaling for another beverage.  Bob’s remained untouched.  “So how does that work?”

“You can define jobs and steps within the jobs.  There’s something called JSL, but it is really just an XML file.”, Bob said. 

“And that’s it?  Doesn’t sound like it helps much.”, said Joe, sounding skeptical.

“Well, there are regular steps that just run.  Those are batchlets.  Cute name eh?  But there’s also another step type called a chunk.”, said Bob.

“A hunk?  Sounds NSFW to me.”, chuckled Joe.

“No…a chunk.  Basically, it is just a loop where the app code can read, process, and write records.  But the environment wraps a transaction around it, so you get checkpointing.  That means if something bad happens we don’t have to start the job over from the beginning”, explained Bob.  “That’s a big deal with some of these jobs that process millions of records.  We’ve only got so much time for the job to run, and if something bad happens we don’t have time to start over from the beginning.  Being able to pick up at the last checkpoint provides the reliability we need from stuff this important to the business.”

“I thought you were dreading this assignment.  Sounds like you’re kind of getting into it, Mr. Hunk-Programmer!”, Joe said, laughing.

“Stop that.  There really is some kind of cool stuff in this spec.”, shushed Bob.

“Ok, ok.  Tell me some more cool stuff about this spec you found.  But drink your drink first and I’ll get you another one.”, said Joe. 

Several large gulps later, Bob continued, “Partitions are cool.  Think of a regular job step that just cruises through millions of records processing them one by one.  Could take hours.  But Java Batch lets you run multiple copies of the same step concurrently.  Each copy processes a different range of the data.  So instead of one thread slogging through a million records, you’d have ten threads each handling 100,000 records.  If we’ve got the horsepower and there isn’t contention, we could cut elapsed time down quite a bit.  That’s modernization the execs will like.” 

“And…”, said Joe, setting a fresh beverage down in front of Bob.

“Well…”, Bob said, taking a sip and getting foam in his moustache, “there are also split/flows.” 

“Sounds like something a river does.”, said Joe, finishing his drink and ordering another.

“Sort of..  if you have steps that aren’t dependent on each other, you can run them concurrently.  Sometimes you have steps in a job that both have to run but could run in either order.  A split/flow lets you run them concurrently on different threads.”, said Bob.  “Which means that’s another way you might reduce elapsed time.”

“Ok, so this is all great, but our batch applications aren’t just running code, they touch stuff.  How do you do that?”, Joe asked.

“Same as from any Java application.  There’s nothing magical about the application code.  You can use JDBC to touch databases, JMS for messaging, JCA for things like CICS and IMS.  There’s even the JZOS libraries for the applications that run on z/OS and have to deal with VSAM datasets and whatnot.”, said Bob.

“This stuff runs on z/OS?”, asked Joe.

“Sure.  z/OS is actually a pretty good place for Java.  And the Java Batch implementation in WebSphere Liberty has a bunch of extras to help it integrate into our production batch environment.  There’s even some z/OS-only stuff like SMF records.”, said Bob.

“SMF?  That will make the z/OS guys happy.  If it doesn’t write SMF records it isn’t really running.”, Joe laughed.

“I hope so.  I think I can get the Java Batch jobs to run in Liberty, driven by our existing Enterprise Scheduler.  Job output will go where it always goes, and the z/OS guys will have their CPU usage numbers to charge for the jobs.  I think this might actually work.”, said Bob.

“I hope it goes smoothly for you so you can get onto something you’re actually interested in”, Joe said sarcastically.  “Where did you learn all this stuff?”

“I’ve been reading a blog about it.  I’ll send you a link.  Thanks for the drink.”, Bob said, finishing his too.

https://community.ibm.com/community/user/imwuc/blogs/david-follis/2018/06/15/batch-jobs-in-java-yes

0 comments
14 views

Permalink