When application developers think of IBM MQ, they think of queues and topics. That’s understandable, as that’s what their application interacts with, and it’s great that they don’t really need to know any more – that’s kind of the point!

IBM MQ administrators on the other hand, typically recognise the product as something much more powerful. They see a connected set of queue managers spread across a broad IT landscape, seamlessly making queues and topics accessible locally to applications wherever they are connected. That “messaging network” can enable message transmission that spans multiple platforms, locations, and cloud providers without the applications using it needing to know how that happens. This is the real hidden beauty of IBM MQ, and one of the reasons it remains to this day the most deployed messaging network in the world.

What is a messaging network?

I want to dive into what we mean by a messaging network, why it’s so important at this particular time in the evolution of IT, and why it is that IBM MQ is so very good at it.

The idea of messaging is that once a message has been placed on a queue or topic, the application no longer needs to worry about it, and it can move onto the next task.

However, what if the recipient of that message is in a completely different IT domain? How does the message get there, and who is responsible for it whilst in transit?

A messaging network is a set of loosely connected messaging servers that enable reliable, cross-domain transmission and delivery of messages over a broad, potentially hybrid multi-cloud IT landscape.

When I defined messaging network above, I deliberately used the term “cross-domain” message transmission because I wanted to recognise an increasingly common need. Modern solutions are built across ever more heterogeneous landscapes. New innovations typically tie together functionality from very separate parts of a business (different domains), each of which has significant autonomy over their technological choices. For example, what runtimes do they use, what platforms do they run on (containers, virtual machines, mainframes), when are they available, what are their performance characteristics, and where do they run - on premises or in one of the major hyperscalers?

There is a good chance that many of the parts of a modern solution will be distant from one another both physically and technologically. Even more so if your solution uses capabilities from business partners external to your organisation. This is where the messaging network comes in, providing an asynchronous back-plane that enables applications to find each other over a suitably decoupled transport. 

For more theoretical discussion on the characteristics and benefits of a messaging network see here. However, in this post we're going to dive down into the key capabilities in IBM MQ that make it an enterprise grade messaging network. First though, lets very briefly remind ourselves why we're choosing messaging in the first place.

What about APIs?

The go-to protocol for bridging these domains is often HTTP based APIs. They certainly have their place and enable a simple programming model for many simple tasks. However, for more complex interactions, it can push a lot of complexity back into your application. Interactions are dependent on the entire chain from caller to provider being available, and performant. To make an application robust to the very likely possibility of issues in that chain, you have to build in retry logic, additional persistence, idempotence. You also have to consider the effect on your application’s performance and resources if the intervening network latency increases, or the downstream application is struggling with workload.

This is the problem that messaging was designed for. Designing your solution to interact with queues in between your components decouples them from one another’s performance and availability characteristics.

Queue Managers

In IBM MQ, queues are housed within a queue manager which makes queues available to suitably authorised clients over a variety of protocols.

A queue manager takes care of persisting the messages, performing any transactionality required, and assuring exactly once delivery. Whilst the queue manager is a lightweight component, it is also highly optimized for performance and provides platform specific high availability and disaster recovery options. Queue managers are the all critical nodes within a messaging network that provide the core asynchronous decoupling.

It's important to note at this point that although IBM MQ clearly has queues at its heart, the destination doesn’t have to be a single application. IBM MQ has also long had the notion of publish/subscribe via “topics”. These too are administered via queue managers and as such can be made available across a messaging network.

MQ Channels

If you are communicating across domain boundaries, wherever the messaging server sits, at least one side will still have to perform a request over the domain gap, which means it is still dependant on the vagaries of the intervening network connectivity.

This is where the second construct of the messaging network comes to the rescue – “MQ channels”. Each application can speak to a messaging server in its own domain then let the messaging servers (queue managers) worry about the gap. In IBM MQ they do this via an efficient, highly resilient and scalable connection known as a channel.

Channels assure exactly once delivery between the queue managers. The applications need only interact with their nearest queue, and the queue managers and channels do the rest.

To long time users of IBM MQ, this benefit from delegating transmission to queue managers and their channels will be very familiar since server-to-server communication is one of the most mature areas of IBM MQ.

Most applications talk over the local network to their queue managers because cross domain network communication can still be complex and susceptible to lag. In increasingly hybrid multi-cloud landscapes there is significant benefit in persisting messages within the local domain and delegating cross domain to queue managers and channels.

The real benefit of a channel is that once it is in place, it can be used to make any number of queues or topics visible across the gap, mars

halling messages between an ever growing number of applications on each side.

MQ Clusters

What if we want to connect lots of queue managers together? It’s not unusual for a large IBM MQ landscape to have hundreds or even thousands of queue managers. In other words, what if we want to build a messaging network? Surely this would mean configuring all the different permutations of channels between all the queue managers?

Fortunately, IBM MQ has long had an answer to this challenge in the form of an MQ “cluster”. When queue managers are grouped into a cluster, they are all aware of one another’s queues.

An application can place a message onto any queue in the cluster via their local queue manager and IBM MQ will reliably route that message to wherever in the MQ network that queue exists. If the queue doesn’t exist on that initial queue manager, messages destined for it will be persisted in a transient queue ready for transmission across a channel. The application at this point can continue with other work as it is confident that the message will get to its destination without having to know anything about the route the message will take through the cluster. It doesn’t need to know what networks it will traverse, what bandwidth might be available, or the performance and availability of the queue managers along that route. The channels between the queue managers will handle all that.

The really clever thing about MQ clusters, is that you don’t have to configure the channels at all, or even know where the queues are on the network. When a queue manager is added to a cluster, the channels are configured automatically, and dynamically adjusted if the cluster changes. Queue definitions are also automatically and efficiently shared throughout the network such that messages can always find a path to their intended queue. This means the cluster has the knowledge to dynamically route around failures, keeping your messages moving regardless of network or component outages.

MQ clusters are easy to set up, and provide huge simplifications to the overall configuration, to the extent that some sites use clusters as their default way of adding queue managers into their landscape.

The messaging network takes care of the complexities of reliable transmission of messages regardless of how separated applications are, on different networks in different sites, and from differing platform providers. Once the channel (or cluster) is set up, messages can flow. Of course, there are plenty of access controls available, so we can still govern who can send what to whom.

Reviewing the ingredients of an IBM MQ based messaging network

We have discussed that IBM MQ is extremely well suited to the creation of a messaging network. It was designed specifically for this purpose decades ago with these capabilities burnt into queue managers, channels and clusters. That’s why it underpins solutions in 100% of the top 25 banking companies in the 2024 Forbes Global 2000.

What you might not know is how much IBM MQ has continued to innovate to align with modern development principles and platforms. Did you know that you can spin up IBM MQ in a container within seconds, in a fraction of a CPU? Did you know that IBM MQ is available as a managed service? Perhaps it’s worth taking a fresh look at how you can robustly and securely join your applications together with queues and topics across a hybrid cloud landscape using the world’s most deployed messaging network?

IBM MQ, a high-performance, security-rich and assured delivery messaging solution 

Acknowledgements

Many thanks to the following for their invaluable insights and review: Anthony Beardsmore, Andy Garratt, Callum Jackson, Matt Roberts, David Ware.