Hello Chin.
I have analyzed the error log of the MQ queue manager. The result of the analysis is that the DP itself broke the connection to the MQ by force.
The first mistake on the MQ side was:
<11>Nov 23 15:01:34 [0x8d009665][qmgr][error] qmgr(Queue Manager): [ip_address_DP]: AMQ9665E: SSL connection closed by remote end of channel 'CH.SVR.SSL'. [ArithInsert1(420), CommentInsert1(CH.SVR.SSL), CommentInsert2(gsk_secure_soc_read), CommentInsert3(ip_address_DP)]
After that, the following error appeared:
After that, there was already an attempt on the part of MQ to write to a channel that was no longer there:
<11>Nov 23 15:01:34 [0x8d009206][qmgr][error] qmgr(Queue Manager): [ip_address_DP]: AMQ9206E: Error sending data to host ip_address_DP. [ArithInsert1(32), ArithInsert2(32), CommentInsert1(ip_address_DP), CommentInsert2(TCP/IP), CommentInsert3((write))]
My only guess as to the cause of this error is that the MQ manager is performing a REFRESH SECURITY command on the MQ manager side, which results in the termination of all existing SSL connections. Then I do not understand why DP was the initiator of the connection break.
As a result of analyzing the situation, I believe that the problem occurred on the DP side.
At the network level, we have DataPower and MQ Appliance included on the same network equipment. There is no IP Firewall or security equipment for analyzing traffic (like IP Security) between them.
Since the experience of administration is still little, up to one year, understanding of some already specific situations is still little. Therefore, I hope for your support in this matter.
------------------------------
Andrii Kushneryk
------------------------------
Original Message:
Sent: Tue November 28, 2023 04:52 PM
From: Chin Sahoo
Subject: Failed to establish a backside connection error
Hello Andrii,
The source of connection broken from DP MQ client (reason code 1010) is not clear. It can happen if cache-timeout is not configured whose value should be less than the qmgr keepAlive. Since qmgr is running in the MQ appliance, it will be useful to collect packet traces from DP and MQ traces from qmgr side to debug this issue. You can also check the qmgr side error logs and see what error code is visible with respect to DP error code of 1010.
If there is any IP firewall between DP and MQ appliance, you should also check the connection idle timeout of the IP Firewall to synchronize the timeous as follows: DP mq-qm cache-timeout < IP Firewall idle timeout < MQ qmgr keepAlive timeout.
------------------------------
Chin Sahoo
Original Message:
Sent: Tue November 28, 2023 03:48 PM
From: Andrii Kushneryk
Subject: Failed to establish a backside connection error
Hello Steve.
Connection error with MQ Queue Manager occurred in a production system. We have a queue manager running on IBM MQ Appliance, so this is the first time I have encountered an error related to the disconnection of an existing connection to the queue manager. While maintaining other information systems that work through the MQ client, the loss of connection to the MQ queue manager caused by an error in the application software causes error messages on the MQ side of another type.<o:p></o:p>
Since it is now difficult to find documentation that describes the specifics of DataPower's interaction with the MQ manager, as well as any additional information on this topic, I would like to receive links to documentation or presentations on this topic.<o:p></o:p>
So the error is not of a periodic nature, we have not opened a case in IBM Support so have not yet been able to understand the nature of the occurrence of such an error.<o:p></o:p>
------------------------------
Andrii Kushneryk
Original Message:
Sent: Mon November 27, 2023 04:41 PM
From: Steve Linn
Subject: Failed to establish a backside connection error
Hi Andrii,
Given this is a production environment, have you opened a PMR/support ticket?
Best Regards,
Steve Linn
------------------------------
Steve Linn
Senior Consulting I/T Specialist
IBM
Original Message:
Sent: Mon November 27, 2023 08:31 AM
From: Andrii Kushneryk
Subject: Failed to establish a backside connection error
Good afternoon colleagues.
In recent days, problems have begun to arise in the interaction of our productive DataPower with the queue manager deployed on the IBM MQ Appliance. Errors of the following type began to appear on the DataPower side:
[0x808000f2][XXX4][error] mpgw(XXX4): trans(1018701471)[error][ip_client_1] gtid(e6b99f89655f4d233cb8269f): XSLT custom log message ' sender: TESTXXX400 signer: TESTXXX400 action:Receive msg:Код помилки: 1010 Пояснення: Failed to establish a backside connection'
[0x01130006][mpgw][error] mpgw(XXX4): trans(1018701471)[error][ip_client_1] gtid(e6b99f89655f4d233cb8269f): Failed to establish a backside connection
[0x80e0012b][mpgw][error] mpgw(XXX4): trans(1018701471)[ ip_client_1] gtid(e6b99f89655f4d233cb8269f): Backside header ('N/A') failed to parse due to: Failed to establish a backside connection, URL: dpmq://<Queue Manager>?ReplyQueue=TEST;GMO=2;TimeOut=2;ParseProperties=on
On the IBM MQ Appliance side, errors are generated stating that the initiator of the connection termination was IBM DataPower.
We could not find in the IBM DataPower documentation an error message with code 1010 that occurs when connecting to an SSL queue manager on an IBM MQ Appliance. It is also not clear why DataPower broke the existing connection with the queue manager on the IBM MQ Appliance.
------------------------------
Andrii Kushneryk
------------------------------