An Introduction to MQ queue manager processes (Version 6)

neekrish |Jan 19 2011 Updated

(Republished old entry after correcting the links to images)
Author: Madhukar B V
 
When a queue manager is created, you get a message something like this:

Observe that 40 default objects are created when a queue manager is created. An MQ object can be a queue, a channel, a namelist, a listener (in WMQ v6.x), a process, etc. Most of these default objects act as templates for defining queue manager objects. For example, SYSTEM.DEFAULT.LOCAL.QUEUE is used as a template and its attributes are copied when a new local queue is defined using “DEFINE QLOCAL” command.
 
During queue manager creation, it starts some of the queue manager processes, which creates these objects, their equivalent files and directories under /var/mqm/qmgrs/<QMGR_NAME> directory, and other supporting directory structures such as queue manager LOG files under /var/mqm/log directory, and at the end the queue manager is ended.
 
Now, let us start a queue manager using “strmqm TECHTALK” 

  
We will discuss about output lines of “strmqm” a little later.
 
Now, if you issue:
 
In the above output, process “amqzxma0” has a parent PID of ‘1’, whereas it is the parent for most of the other processes. This is the first process created and it is called the “Execution Controller”, simply called EC.A little bit of UNIX here. Since “strmqm” exits at the end, this process will eventually be owned by the first UNIX process “init”. (Technically, “init” is not the first UNIX process. First process is pid ‘0’, the scheduler process).
 
Apart from spawning other processes that are necessary for the functioning of a queue manager, the EC does few more things:
1. Establishing connection between the connecting application process and the agent process.
2. Periodically checking the health of the WMQ internal processes.
3. Cleaning up its children processes in case of queue manager termination (normal and abnormal), etc.
 
Hey, I mentioned about “agent” process above! It is “amqzlaa0” in the above “ps” listing. It is the servant process that receives request from applications and responds back. In other words, it acts as the agent between the application and the queue manager resources.
 
Few tit-bits about agent process:
1. Agent is a threaded process and can service 64-applications at a time. Each application is connected to an agent-thread.
2. New agent will be created there are more than 64-connection requests.
3. The queue manager uses a pool of agent threads and reuses them.
4. AgentClassMap and AgentClassLimit<n> are two tuning parameters to control the number of threads an agent can spawn.
 
I also mentioned about “internal” queue manager processes. These are the processes prefixed with “amq” which are necessary for the functioning of a queue manager. External queue manager processes are the processes like runmqsc, runmqlsr, runmqchl, runmqchi, runmqdlq, etc, and these interact with the queue manager just the way queue manager applications do.
 
How do these various processes (internal qmgr processes, external qmgr processes and qmgr applicaton processes) interact?
They interacting using Inter Process Communication (IPC) facility provided by the Operating System. Pipes, Named pipes, Shared Memory, Mutexes & Semaphores, Messages Queues are 5 IPC mechanisms according to System V standards. Sockets and Streams are the other two. WMQ “largely” uses Shared Memory mechanism to share the information and access to these resources are serialized / controlled using Mutexes and Semaphores.

Figure 1. Shared memory is the preferred technique of communication between two processes.
 
In WMQ, we mainly use two terms for shared resources – SUBPOOLS and MEMORY SETS. Subpool is a big chunk of memory, which can be further divided into n-number of memory sets. Memory sets are further divided into EXTENTS. And a queue manager will have a number of SUBPOOL for different purposes – IPCC Subpool, QMGR Subpool, APPLICATION Subpool, SYSTEM Subpool, QMPERSISTENT Subpool, etc. IPCC Subpool is the one which is used for communication between an application process and an WMQ process (mostly AGENT process).
 
How are these resources created and who creates them?
When a QMGR is started, it creates a set of files such as <QMGR_DIR>/@ipcc/shmem/shm.<n> where <n> denotes the kind of subpool. The i-node of these files are mixed with the “device number” to generate a “key” that is passed to “shmget( )” call to generate an unique ID for the shared memory resource created. This unique id, SHMID, can be compared to “file descriptor” when you open a file using open( ) system call.

 
Any application that wants to connect to the IPCC subpool will generate the key by the above technique and puts a request on the IPCC Subpool. This code will be part of the MQCONN( ) API call.
 
If “shared memory” is the method of communication, we say that the application is connecting to the queue manager in STANDARD bindings. Since the address space is shared between two processes here, the queue manager is prone to corruption if the application corrupts some part of the shared memory.
 
If you replace the mode of communication with “pipes” / “sockets”, the mode of connection is termed as “ISOLATED” binding. This can be achieved by specifying MQCNO_ISOLATED_BINDING option in MQCONNX( ) call, or by specifying DefaultBindType=ISOLATED in Connection stanza of qm.ini file. This isolates the application address space completely from that of the agent address space. But, it is less efficient compared to STANDARD binding in efficiency.
 
There is one more type of binding, called FASTPATH, where the application directly access the queue manager resources, bypassing the agent process. This provides the lowest level of protection and is configured only for trusted-applications.
 
Going back to the output of “strmqm” command:
 
There are two types of messages: Persistent and Non-Persistent. A non-persistent message does not survive a queue manager restart. However, queue manager can not afford losing a persistent message. But, each time a message is PUT or GET, the message will not be written or removed from the queue file on the file system. This would be very expensive in disk I/O. Queue manager keeps a buffer in memory, called queue buffer, which is flushed into the queue file periodically. To ensure that we do not lose persistent messages, each operation with a persistent message is going to be stored in what is called “log buffer” which will be flushed into the LOG files periodically. These log files are created under /var/mqm/log/<QMGR_NAME> directory based on the settings in mqs.ini file (or arguments passed to crtmqm command).
 
“amqzmur0” and “amqzmuc0” are the two processes that are responsible for logging the transaction information into the log files. These are called restartable (mur0) and critical (muc0) processes respectively. Discussion about the functioning of LOGGER component is beyond the scope of this article. It is sufficient to understand that each and every operation (CONN, OPEN, PUT, GET, etc) gets logged in the WMQ log, and periodically the contents of the log files that are complete are “synced” with the queue file. This sync operation is also called “checkpoint”.
 
Figure 3. Log records and periodic checkpoint operations.
 
If you imagine log file as a continuous strip of information, last checkpoint record indicates the “point-of-consistency” between the log file and the queue file. In other words, information till the log file will be safe in the queue file. If a queue manager crashes, when you restart the queue manager, the log records “from” the last checkpoint will be played-forward to bring the queue manager to a state it was when it last crashed. This is how persistent messages are recovered during a crash. Even during queue manager start operation, these log records are played-forward. This is indicated by the messages:
 
You must have already been familiar with the following error:

First of all, an user should belong to “mqm” group (on UNIXes) to be able to perform system administration operations such as creating, starting qmgrs, creating qmgr objects, etc.
 
Further, for an application to connect and perform tasks such as PUT, GET, it should have appropriate authority given. This is granted by a WMQ sys-admin using “setmqaut” command. For eg., # setmqaut -m TECHTALK -t qmgr -p guest +connect provides authority to perform MQCONN( ) operation on a queue. Where is this information stored? This information is stored in a system object “SYSTEM.AUTH.DATA.QUEUE” as a message. There is a process called Object Authority Manager (OAM), amqzfuma, which reads and authorises any connecting application process (and even WMQ internal processes from accessing queue manager objects).
 
“amqzfuma” is the default OAM provided by the WMQ queue manager. This is defined in the queue manager’s qm.ini file as follows:

This means, one can write their own OAM and make WMQ use these services by making appropriate entries like the one above. It is a kind of pluggable component.
 
Till now, we have been talking about a single queue manager and an application connecting to a queue manager directly. There are other modes of communication possible – An application connecting to a queue manager over a TCP/IP channel (CLIENT connection), and an application communicating with a remote queue manager through another queue manager.
Figure 4. Application connecting over CLIENT channel or through another queue manager.Application connecting over CLIENT channel or through another queue manager.
 
In both cases, the receiving queue manager will have to listen on a port (could be TCP/IP or LU6.2, etc). The common listener process is “runmqlsr”. This listens for the incoming requests and hands over the request to a process called “amqrmppa”, also called channel pooler. This is a threaded process and based on the number of requests, more number of “amqrmppa” process will be spawned.
 
On the remote end, “amqrmppa” works on behalf of the application and returns the service. Here is the simplified flow:
 
Application interacting directly with a queue manager:
APPLICATION --> AGENT --> QMGR RESOURCES
 
Application connecting to a queue manager through another queue manager:
APPLICATION --> SENDING MCA --> RECEIVING MCA (LISTENER/AMQRMPPA) --> AGENT --> QMGR RESOURCES
 
Application connecting to a queue manager over CLIENT CHANNEL:
APPLICATION --> TCP/IP --> LISTENER/AMQRMPPA --> AGENT --> QMGR RESOURCES
 
Lastly, in situations where more number of queue managers want to communicate among each other, CLUSTERING of queue managers helps the administration overhead by automatically defining channel connections between some of the queue managers.
 
As you must be aware, a cluster is recommended to have at least TWO full-repository queue managers (Refer Figure below). A fullrepository process is going to hold information about the ENTIRE cluster, whereas a partial repository queue manager INITIALLY holds information about itself and over a period of time it builds up the information which is required for its operation. These repository information are stored in the form of messages inside SYSTEM.CLUSTER.REPOSITORY.QUEUE. “amqrrmfa” is the process, called repository process. This process is responsible for retrieving information from the repository queue, updating the
information, etc.
Figure 5. Queue manager CLUSTER and REPOSITORY process, AMQRRMFA.
 
Although I have not covered each and every process, these are the basic things one can start with.
 
Disclaimer: Each posting on this site is the view of its author and does not necessarily represent IBM’s positions, strategies or opinions.I do not guarantee correctness of the opinions or content or sample code presented here. Use it at your own risk.