As of MQ 9.1.5 we have introduced
new capabilities relating to queue file sizes to
increase the maximum size of an individual queue file and
limit the maximum size of queue files. Prior to MQ 9.1.5 on distributed platforms, the only queue attributes available to influence the maximum size a queue file can take up on disk were the following:
- MAXDEPTH : maximum number of messages on the queue
- MAXMSGL : maximum size of an individual message body
- MAXPROPL : maximum number of bytes of property data in a message
However due to possible varying quantities and sizes of messages
it was not possible to adequately guarantee the size a queue file may grow to. Prior to MQ 9.1.5 the maximum queue file size is constrained to ~2TB, however as of MQ 9.1.5 we can now dynamically manage the maximum size of an existing individual queue file with queue and qstatus attributes and specify sizes up to 255TB.
For local and model queues we have introduced the queue attribute
MAXFSIZE which is the maximum size a queue’s queue file can grow to. Valid integer parameters in multiples of 1MB are from 20 to 267386880 giving us a range of 20MB to 255TB, defining an invalid value will result in an 'AMQ8425E: Attribute value error'. There is also a parameter constant 'MAXFSIZE(DEFAULT)' which resolves to 2097152 resulting in a default maximum queue file size of 2TB.
Queue files store message data in fixed size blocks (BLKSIZE), which is calculated automatically in relation to MAXFSIZE. Block sizes range from 512B for MAXFSIZE(20) to 64KB for MAXFSIZE(267386880) and are in multiples of 512. The default block size for queue files with MAXFSIZE set to DEFAULT or lower is 512 bytes. It is possible however to set a custom block size as shown in the examples section below.
In the event that a change to
BLKSIZE is requested (whether defined explicitly or automatically calculated) on an existing queue populated with messages, there are two conditions for the new value to be initiated, firstly the queue has to be emptied, secondly the queue has to have received a message. In this scenario a new warning message 'AMQ7493 Granularity Changed' is written to indicate to the user that the queue must be emptied or cleared in order for the new block size to take effect. This may not be a non trivial operation taking a significant amount of time, in this case you may want to periodically check if the operation has completed by comparing the current maximum queue file size to the newly proposed MAXFSIZE value for equality, from MQ 9.1.5 we can do just that with the new queue status attribute
CURMAXFS.
We have also introduced another qstatus attribute
CURFSIZE to indicate the current size of the queue file on disk rounded up in megabytes. Be aware that CURFSIZE will track the actual file size on disk as the number of messages increases, however as the number of messages reduce there may be periods where CURFSIZE has not reduced concurrently, this is as expected as MQ has queue compaction algorithms that are optimised for I/O efficiency.
Demo
echo "DEFINE QLOCAL(Q1)" | runmqsc QM1
echo "DISPLAY QLOCAL(Q1) MAXFSIZE" | runmqsc QM1
- You will see the constant value 'DEFAULT', now increase MAXFSIZE to a size that will require a new BLKSIZE:
echo "ALTER QLOCAL(Q1) MAXFSIZE(3145728)" | runmqsc QM1
- Check MAXFSIZE as above, you will see our new value '3145728', now check CURMAXFS:
echo "DISPLAY QSTATUS(Q1) CURMAXFS" | runmqsc QM1
- Instead of our new value we see the default value '2088960'. Remember that even though our queue is empty, we still need a message to have been received, now put a message on the queue:
- Check CURMAXFS again, you will now see the value '3145728', now increase MAXFSIZE to a size that will NOT require a new BLKSIZE:
echo "ALTER QLOCAL(Q1) MAXFSIZE(3145729)" | runmqsc QM1
- Check CURMAXFS again, you will see the value '3145729', we have not had to drain the queue as the increase does not require a new BLKSIZE now increase MAXFSIZE to a size that will require a new BLKSIZE:
echo "ALTER QLOCAL(Q1) MAXFSIZE(5242880)" | runmqsc QM1
- Check CURMAXFS again, you will see a value smaller than '5242880', we will need to drain the queue and put a new message:
echo "CLEAR QLOCAL(Q1)" | runmqsc QM1
- Check CURMAXFS again, You will now see our new value '5242880'.
MQSC examples
Define a new queue with a maximum queue file size of 100GB:
DEFINE QL(Q1) MAXFSIZE(102400)
1: DEFINE QL(Q1) MAXFSIZE(102400)
AMQ8006I: IBM MQ queue created.
Define a huge queue with a maximum queue file size of 100TB:
DEFINE QL(Q1) MAXFSIZE(104857600)
1: DEFINE QL(Q1) MAXFSIZE(104857600)
AMQ8006I: IBM MQ queue created.
Define a queue with a default maximum queue file size of 2TB and a BLKSIZE of 1024:
DEFINE QLOCAL(Q1) CUSTOM('BLKSIZE(1024)')
1 : DEFINE QLOCAL(Q1) CUSTOM('BLKSIZE(1024)')
AMQ8006I: IBM MQ queue created.
Reduce the maximum size of a given queue file to the lowest upper bound of 20MB:
ALTER QLOCAL(Q1) MAXFSIZE(20)
1 : ALTER QLOCAL(Q1) MAXFSIZE(20)
AMQ8008I: IBM MQ queue changed.
Increase the maximum size of a given queue file to the highest upper bound of 255TB:
ALTER QLOCAL(Q1) MAXFSIZE(267386880)
1 : ALTER QLOCAL(Q1) MAXFSIZE(267386880)
AMQ8008I: IBM MQ queue changed.
Query the maximum size a given queue file can grow to:
DISPLAY QLOCAL(Q1) MAXFSIZE
1 : DISPLAY QLOCAL(Q1) MAXFSIZE
AMQ8409I: Display Queue details.
QUEUE(Q1) TYPE(QLOCAL)
MAXFSIZE(DEFAULT)
Query the current maximum file size the queue can grow to and the current size of the file on disk.
DISPLAY QSTATUS(Q1) CURMAXFS CURFSIZE
1 : DISPLAY QSTATUS(Q1) CURMAXFS CURFSIZE
AMQ8450I: Display queue status details.
QUEUE(Q1) TYPE(QUEUE)
CURDEPTH(3) CURFSIZE(1)
CURMAXFS(2088960)
Define MAXFSIZE with an invalid value:
DEFINE QLOCAL(Q1) MAXFSIZE(19)
1 : DEFINE QLOCAL(Q1) MAXFSIZE(19)
AMQ8425E: Attribute value error.