Thanks for your response.
This is exactly what we have mutually agreed along with Bank's IT infra (Wintel support team). We have decided to initiate tests first at DEVELOPMENT environment and then go for TEST/QA environments, each one with its own Windows MQ service account and hosted at different AD Domains (internal & external DMZ) not trusted each other.
It will take some time to "onboard" MQ service at gMSA for each environment (domain) and evaluate the results.
I will keep you posted when i have some feedback.
Original Message:
Sent: Thu September 26, 2024 02:05 AM
From: Francois Brandelik
Subject: MQ Service Compromise -> documented process for auditing?
Hi Nick,
Have you set up a gmsa test bed environment for MQ? How does it work when you change the password? How does it work when you restart the service?
------------------------------
Francois Brandelik
Original Message:
Sent: Wed September 25, 2024 05:51 AM
From: NICK DAKORONIAS
Subject: MQ Service Compromise -> documented process for auditing?
Hi Roger,
In my last reply to you, I have tried hard to make clear that my intention was to elaborate and contribute on the thread rather than to insult you. It is not my style to offend people. I have never done it and i will never do it. It is against my ethical standards. So, i assume that you just misunderstood my purpose.
Considering the above, I will take your last response as an unintended misperception of my approach and i agree to leave this (toxic) part behind us. At the end, all of us working in IT business (generally speaking), i assume we are willing to collaborate with each other, aiming to support and contribute on this business taking into consideration each individual's added value i.e. (skills, experience, etc).
Now, let's come to the "core stuff". In my last response, i was wondering about how compliant Windows MQ Service could be against AD gMSA auto pwd rotation feature, considering technical impacts such as unique SID per service account (mentioned by Luc-Michel) or whether Windows MQ service uses its pwd to re-authenticate while it is up & running.
Any comment on this by anybody that can share his thoughts will be much appreciated, considering that customer's (major local bank organization) cybersecurity/audit dpts, asks for evidences/justifications, so that Windows MQ service NOT to be "onboarded" on gMSA account.
Rgds, Nick.
------------------------------
Nick Dakoronias
Original Message:
Sent: Tue September 24, 2024 05:35 PM
From: Roger Lacroix
Subject: MQ Service Compromise -> documented process for auditing?
Hello Nick,
I'm French-Canadian & my French blood tends to make me speak bluntly, like responding to an insult with an insult (as you pointed out).
And yes, this part of the thread should end here.
Later
Roger
------------------------------
Roger Lacroix
CTO
Capitalware Inc.
London Canada
https://capitalware.com
Original Message:
Sent: Tue September 24, 2024 10:52 AM
From: NICK DAKORONIAS
Subject: MQ Service Compromise -> documented process for auditing?
Hummm. You do realize that there are a variety of people participating in this forum: IBMers, IBM Champions & regular MQ users who all do it out of their free time to help individual users like yourself. Posting a toxic response is not how to ingratiate yourself with the community. Given you are a Kyndryl consultant and this is a public forum, I'm surprised at your response. Also, explaining security to someone who creates and sells MQ security products (encryption/authentication) is the definition of mansplaining.
Your response is offensive and full of insults against me. You accused me that i am a regular MQ user trying to help individuals on my free time, making a post for newbies. I haven't seen such errogant & offending response like yours, throughout my 35 years in IT. Try to behave yourself with people you don't know.
You don't know my background and what i have been dealing with, all these years. I really don't want to further engage and elaborate on this.
Just FYI, you really misunderstood my perception. I have tried gently to explain the AD security part with intention to contribute on the thread and not to promote myself or something like that. This is not compliant with my ethical standards and the way i was raised up. Believe me, I have no idea about the background of any community user who read my post, so I have tried to explain some topics for users who might don't have such huge expertise on this topic. My response to you is also read by other community users, who might want to elaborate. So, why did you react like that?
My intention from the beginning was to find a guidance or documentation evidence on MQ service compromise, as per customer's cybersecurity/audit divisions request. But during the elaboration of responses given by Luc-Michel & Vincent (thank you both), I have also thought to ask about how compliant Windows MQ Service against AD gMSA auto pwd rotation, considering technical impacts such as unique SID per service account (mentioned by Luc-Michel) or whether Windows MQ service uses its pwd to re-authenticate while it is up & running, because if it is the case, then gMSA auto pwd rotation could not be the case.
------------------------------
Nick Dakoronias
Original Message:
Sent: Mon September 23, 2024 06:33 PM
From: Roger Lacroix
Subject: MQ Service Compromise -> documented process for auditing?
Hello Nick,
Have you paid the proper attention to my whole particular response in order to get the whole idea? (asked gently) .
As I have mentioned in that response, the specific question that you didn't understand ...
Hummm. You do realize that there are a variety of people participating in this forum: IBMers, IBM Champions & regular MQ users who all do it out of their free time to help individual users like yourself. Posting a toxic response is not how to ingratiate yourself with the community. Given you are a Kyndryl consultant and this is a public forum, I'm surprised at your response. Also, explaining security to someone who creates and sells MQ security products (encryption/authentication) is the definition of mansplaining.
Your original post was very generic and wide-ranging, asking for help regarding MQ being compromised on various distributed platforms (Windows, AIX, Linux) and I bet most people took it as a newbie looking for someone else to do their homework. Only after Luc-Michel took a shot at answering it did you get to the real/heart of your concern.
If you take Luc-Michel's suggestion of having separate MQ UserIds for each environment or Windows server and the Enterprise MSAD-SysAdmin will work with the MQAdmin regarding MQ account passwords, then only about 5% of the issue is related to MQ and 95% is related to MS AD or Windows servers/clients in general. Hence, as pointed out by numerous websites, the best solution involves strong mitigation practices. Here are several websites with excellent mitigation strategies:
later
Roger
------------------------------
Roger Lacroix
CTO
Capitalware Inc.
London Canada
https://capitalware.com
Original Message:
Sent: Mon September 23, 2024 11:35 AM
From: NICK DAKORONIAS
Subject: MQ Service Compromise -> documented process for auditing?
Have you paid the proper attention to my whole particular response in order to get the whole idea? (asked gently) .
As I have mentioned in that response, the specific question that you didn't understand (you highlighted it in blue) has been asked -exactly as is- by customer's cybersecurity & audit divisions. To be honest, I don't know how familiar are these guys with Microsoft AD technology, but their question makes sense.. Let me explain what i mean by that, by leveraging my expertise as AD Certified member of Microsoft Support Services for a long time, along with my education & skills on Cryptography / Key Management.
This question is not actually translated in the way you did, meaning that an AD enterprise/domain admin with proper access can look up the pwd store, so a hacker can do the same... No way by all means.
The question and my comments after, refer purely to unexpected external attacks, somehow relevant to the well known planned "black penetration scenarios".
It is certain that passwords are stored in AD SAM & NTDS.DIT dbases as hash digests. Microsoft names this hash algorithm OWF (One way function). The AD account pwd is stored using LM (LAN Manager) hash in older Windows OS versions and NT (New Tecnhology) hash (DES, AES-256) in recent Windows OS versions. The authentication is being performed upon kerberos authentication mechanism against KDC (Key distribution Center) hosted on Domain Controller (DC). So it is really difficult for a local AD Admin to break the hash and perform internal intrusion, unless he has has cracking skills..
But in this case we refer to external attackers, which are well skilled hackers and have developed methods of retrieving gMSA pwds.
In brief what an external attacker usually tries to achieve, is to get access to any AD machine (client, server, ..) with PowerShell installed, so that he can get the full list of service accounts with just a simple PowerShell command. If he achieves that he can then look into specific attributes in order to get gMSAs and by looking into SPNs he can find where these gMSA are installed. After that he can target the systems found in the SPNs in order to do the ...rest of the job..
FYI, there are Powershell modules -available in worldwide powershell hacking programming community - developed just to retrieve gMSA secrets.. I don't want to further elaborate on this, for obvious reasons. So, at last the specific question makes sense in a way that an external experienced / skilled attacker can decide to compromise the gMSA secrets.
But for the benefit of MQ community users, I think we should avoid to further elaborate on this and instead focus on how the frequent pwd change on gMSA can affect the stability of Windows MQ service (if they decide to use it), especially on Production MQ (Microsoft) failover setups hosting 24 x 7 flows. In other words, how compliant is the Windows MQ service against the gMSA, since as far as i know there is no such evidence in IBM documentation. If there is something relevant, pls keep me posted.
Rgds, Nick.
------------------------------
Nick Dakoronias
Original Message:
Sent: Fri September 20, 2024 03:01 PM
From: Roger Lacroix
Subject: MQ Service Compromise -> documented process for auditing?
I don't understand this comment by you:
But my question actually refers to the case where specifically the MQ Windows (admin/startup) service which is configured as service account in Active Directory domain (where MQ host is joined) is being compromised by an external attacker who will gain access to the password of this account.
Years ago, when I was learning about MS AD, if I remember correctly, I was told that the passwords to AD users were stored encrypted using a 1-way encryption mechanism. i.e. If you forgot your password, there was no way of looking it up.
So, are you saying that an AD Admin with proper credentials/authorization can look up the password of an account? Hence, because of this, a hacker, if they can gain access, can also look up passwords for any account in MS AD? Because that doesn't seem right.
later
Roger
------------------------------
Roger Lacroix
CTO
Capitalware Inc.
London Canada
https://capitalware.com
Original Message:
Sent: Thu September 19, 2024 11:57 AM
From: NICK DAKORONIAS
Subject: MQ Service Compromise -> documented process for auditing?
Hi Luc-Michel,
At First many thanks for your time and response.
I fully agree with the listed items (i.e. binaries, triggers, SSL store, qm.ini, qm config, ..) you have pinpointed in case of MQ server (hosting QMs) is compromised.
But my question actually refers to the case where specifically the MQ Windows (admin/startup) service which is configured as service account in Active Directory domain (where MQ host is joined) is being compromised by an external attacker who will gain access to the password of this account.
The above hypothetical case has been set by cybersecurity & audit divisions in a bank organization, asking for official (vendor) password recovery process documentation.
Furthermore this request is also related to another recent (5 days old) post subjected "Does MQ Windows Service Support or is it compatible with group Managed Service Account (gMSA)?" considering that their plan is to move all Windows (domain) service accounts to gMSA leveraging its auto password rotation feature..
Relying on my experience related to both Microsoft AD & MQ technologies, my perception is that not only MQ Service password can be cracked but also all Microsoft Windows Services credentials can be hacked by an external attacker who will gain access to the AD SAM & NTDS.DIT databases (in PDCs), where all these are stored. In that case it is obvious that the whole AD might be at risk of compromise and possible collapse, so MQ will be down in the priority list of business continuity.
The same applies to gMSA credentials risk for hacking, given that they are all stored in PDCs (AD Domain Controllers) KDC (Key Distribution Service) object. So my point is that everything can be hacked. There is no absolute safety when a hacker decides to perform a dark penetration.
Btw, there publicly available articles & posts (known to me) that provide full guidance & methods to compromise both service accounts & gMSA credentials..
At last, the auto pwd rotation feature provided by gMSA, is not compliant with the fact that Windows MQ service account credentials need to be provided during MQ setup process (i.e. new MQ instance). MQ admins cannot retrieve the MQ service password from gMSA store, since only Enterprise Admins and MQ Admins can access that store.
I would like to have your comments (if any) or further commendations.
Cheers, Nick.
To be honest
------------------------------
Nick Dakoronias
Original Message:
Sent: Thu September 19, 2024 08:48 AM
From: Luc-Michel Demey
Subject: MQ Service Compromise -> documented process for auditing?
Hi Nick,
First of all, I think we need to define what a 'compromised MQ service' is.
Does it mean an unauthorised change to the configuration, change in MQ rights on certain objects, modification of binaries, etc...?
If the server hosting MQ has been compromised in the classic sense of the term (intrusion by an outside party and potential modification of the configuration and/or binaries), the process is simple: delete the compromised server and deploy a new one.
In this case, of course, you need to have :
- MQ binaries
- Complete MQ configuration in MQSC format (dmpmqcfg)
- Any additions to the qm.ini file
- Certificate stores
- Scripts launched by the triggering process
Please note that the QMID of the Queue Manager will change (important if MQ cluster) and the DQM channels will have to be resynchronised.
If it is only a question of modifying certain objects, re-applying a configuration export may help, but it will not remove objects or permissions added by the 'compromision'.
In terms of preventive solutions, you need to secure access to the :
- CHLAUTH on SVRCONN channels
- MCAUSER (possibly dynamic) on SVRCONN and RECEIVER channels
- Symmetrical TLS encryption on SVRCONN and DQM channels
Enabling configuration events provides an audit trail, so you can see who changed what, and when, with a before and after image.
------------------------------
Luc-Michel Demey
DEMEY CONSULTING
lmd@demey-consulting.fr
#IBMChampion
Original Message:
Sent: Tue September 17, 2024 11:36 AM
From: NICK DAKORONIAS
Subject: MQ Service Compromise -> documented process for auditing?
Hello,
Does anybody know , whether there is a formal or at least an informal process posted for restoring MQ service (on Multiplatforms: Windows, AIX, Linux) after it has been compromised?
Such document including downtime and MQ service unavailability considerations would be valuable against any organization's cybersecurity and audit divisions claims..
P.S.
Do you have in mind whether MQ service has been actually compromised in real life?
I am asking that, because I haven't had such experience with MQ for a long time (at least 15 years..)
Furthermore, after searching in the web, i have found that IBM has addressed via Fix the denial-service-attack-cve-2023-26285 posted at URL:
https://www.ibm.com/support/pages/security-bulletin-ibm-mq-vulnerable-denial-service-attack-cve-2023-26285
Thanks in advance,
Chers, Nick.
Any advise on the following will be much appreciated.
Cheers, Nick.
------------------------------
Nick Dakoronias
------------------------------