Brian makes many good points. Of course there is some overlap, and MANY opinions on what is best. I always advocate that you should use the right tool for the job. We can make almost any IT tool solve almost any IT problem using brute force, via code or other methods. It is the old I have a hammer so everything is a nail approach -- or always pounding the square pegs through the round hole.
So again, this is my opinion.
Application servers are designed to run your "business logic" -- to run your applications that perform your business functions. That is what it is designed to do and what it does best.
Now sure, you can do almost anything in Java. But Java and the constraints that the application server brings means that it is not the ideal location to do certain functions. There is a place for things like Java batch, but again, it should be for performing your business logic/applications. Same for messaging. Messaging within the application server should be for obtaining or distributing data that is directly produced or required by that business application.
I would argue that anything more than that kind of simple exchange should be done by your integration layer -- whatever that is.
In the case of IIB, IIB is designed from the ground up to be optimized to perform integration. It is NOT designed to host and execute business logic -- rather "integration logic" -- mediations to handle integration -- simple and especially complex -- in the movement and exchange of data between disparate applications -- including applications running in an application server.
IIB allows the creation of integration logic graphically, to save time and negate the need for coding complex integration logic. This includes creating graphical maps to mediate between different message formats (think converting between an internal custom tagged delimited format and a large complex XML Schema), handling a large variety of protocols with ease, and doing all kinds of message enrichment such as reading different databases, calling out to web services and RESTful services, gathering or updating data in applications such as SAP, etc.
And yes, as you mention, IIB can update databases, even with two-phase commit. But should it? Again, the answer is it depends. I would not propose you use IIB as an ETL tool and doing large batch updates. This is best done by an ETL tool. However, when handling real time data that is moving between endpoints, it certainly makes sense to do these kind of one at a time real time database updates inline within IIB.
I mention "real time." And that is what IIB is designed for -- doing real time integration. However, again, you can make it do batch type work. Should you? Again it depends on what you are doing. I would argue against doing many large, long running batch style work within IIB, However, large file handling again is an area where IIB exceeds. And using the right patterns, you can try to minimize any restart or rework. Since IIB is optimized for such handling, I would often suggest using IIB for doing such a thing -- it depends on what you are doing with the file contents. If you need typical integration steps, such as transformation, augmentation, and delivery to critical systems, then I would say use IIB
It is often a thought to start coding business logic in IIB -- and sure some customers do. But again, that is not what we suggest. I would argue that you call out to an application server or some service or API that performs that functionality. And while mentioning APIs, I would also argue that many APIs are best done within IIB (and of course I would argue you should use API Connect to manage and secure those APIs), because many APIs need integration. Sure if it is simple data retrieval, you can do that in Node or Java or whatever is desirable. But as soon as you start needing transformation, augmentation, correlation of data, etc., IIB does that best.
It is indeed usually a challenge for customers to determine when to use IIB and when to use ETL and when to use an application server and when to use a gateway, etc. There is overlap in almost any set of tools. So again, I go back to the mantra to use the right tool for the job -- the tool designed for that purpose (unless the requirement is very light weight and the only requirement in that area of functionality). So really, customers, maybe working with IBM, should try to set standards on what types of use cases and functional requirements should be handled by what tool.
I hope this helps. You can make arguments many ways. Here are some references that may be helpful:
Executive IBM Cloud Middleware Connectivity & Integration Solution Architect
Member of the IBM Academy of Technology
North American IBM MQ Appliance Technical Leader
Open Group Certified Distinguished IT Specialist
IBM Certified SOA Solution Designer
IBM Certified Specialist/Solutions Expert - WebSphere MQ V6/V7
IBM Certified System Administrator/Solution Designer - WebSphere Message Broker V6/V7/V8
IBM Certified Solution Designer/System Administrator - IBM Integration Bus V9.0&l