MQ

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Hi John,

RDQM does support synchronous replication. However, note -

Performance degrades rapidly with increasing latency between data centers. IBM will support a latency of up to 5 ms for synchronous replication and 50 ms for asynchronous replication.

See: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=recovery-requirements-rdqm-dr-solution

If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well. Your choice of async or sync will depend on your application's  tolerance for potential message loss/performance. You could also mix and match depending on the application's requirements for message availability and performance; I know of some customers that turn off synchronous replication to DR where the applications favour of performance over message loss and can reconcile any message loss, but they keep it on for those that are less sensitive to latency and must not lose any data. 

Regards,

------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs
------------------------------

Original Message:
Sent: Mon August 09, 2021 09:32 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Hi Martin,
thanks for reminding me that RDQM-DR can do synchronous ! (I've read a lot recently ;-)

I'm not quite sure I understand what you're saying after that though...

"If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well"

Are you suggesting that I use native VM replication across the two DC for all the VMs - or perhaps just Node1? In which case I'm confused as to why I'd then put RDQM-DR on the top of it.

Perhaps you're suggesting that I use native Azure VM replication intra-DC so that e.g. node1-3 are replicating amongst themselves then I use the RDQM-DR to replicate the Node1->Node1' and Node1' is then replicated internally to its other two nodes (Node2'-3') in its DC? This is then ONLY using RDQM-DR and not using RDQM ? Which confuses me.

Could you clarify for me please?

many thanks,
john.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Original Message:
Sent: Tue August 10, 2021 03:51 AM
From: Martin Evans
Subject: RDMQ DR v Azure native Disk replication

Hi John,

RDQM does support synchronous replication. However, note -

Performance degrades rapidly with increasing latency between data centers. IBM will support a latency of up to 5 ms for synchronous replication and 50 ms for asynchronous replication.

See: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=recovery-requirements-rdqm-dr-solution

If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well. Your choice of async or sync will depend on your application's  tolerance for potential message loss/performance. You could also mix and match depending on the application's requirements for message availability and performance; I know of some customers that turn off synchronous replication to DR where the applications favour of performance over message loss and can reconcile any message loss, but they keep it on for those that are less sensitive to latency and must not lose any data. 

Regards,

------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs

Original Message:
Sent: Mon August 09, 2021 09:32 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Hi John,

In that scenario you would have one VM in each DC and the platform (e.g., native Azure VM replication) will provide the equivalent of RDQM-HA, you then replicate to your DR site using RDQM-DR to another highly available VM. That avoids the need for 6 VMs but relies on you having some other HA capability in each DC. Some people only have HA VMs so it's pointless running RDQM-HA on top as it will replicate an already replicated system, you might also prefer native Azure VM replication over a 3 node RDQM-HA solution.

Regards,

------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs
------------------------------

Original Message:
Sent: Tue August 10, 2021 05:07 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Martin,
thanks for reminding me that RDQM-DR can do synchronous ! (I've read a lot recently ;-)

I'm not quite sure I understand what you're saying after that though...

"If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well"

Are you suggesting that I use native VM replication across the two DC for all the VMs - or perhaps just Node1? In which case I'm confused as to why I'd then put RDQM-DR on the top of it.

Perhaps you're suggesting that I use native Azure VM replication intra-DC so that e.g. node1-3 are replicating amongst themselves then I use the RDQM-DR to replicate the Node1->Node1' and Node1' is then replicated internally to its other two nodes (Node2'-3') in its DC? This is then ONLY using RDQM-DR and not using RDQM ? Which confuses me.

Could you clarify for me please?

many thanks,
john.

------------------------------
John Hawkins
Integration Consultant

Original Message:
Sent: Tue August 10, 2021 03:51 AM
From: Martin Evans
Subject: RDMQ DR v Azure native Disk replication

Hi John,

RDQM does support synchronous replication. However, note -

Performance degrades rapidly with increasing latency between data centers. IBM will support a latency of up to 5 ms for synchronous replication and 50 ms for asynchronous replication.

See: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=recovery-requirements-rdqm-dr-solution

If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well. Your choice of async or sync will depend on your application's  tolerance for potential message loss/performance. You could also mix and match depending on the application's requirements for message availability and performance; I know of some customers that turn off synchronous replication to DR where the applications favour of performance over message loss and can reconcile any message loss, but they keep it on for those that are less sensitive to latency and must not lose any data. 

Regards,

------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs

Original Message:
Sent: Mon August 09, 2021 09:32 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Hi John,

By "native Azure VM replication" I'm assuming you mean the entire VM can move seamlessly from one physical system to another within a DC or availability zone and that the DR system would live in another region where the VM can't move to, hope that makes sense.

Regards,



------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs
------------------------------

Original Message:
Sent: Tue August 10, 2021 05:28 AM
From: Martin Evans
Subject: RDMQ DR v Azure native Disk replication

Hi John,

In that scenario you would have one VM in each DC and the platform (e.g., native Azure VM replication) will provide the equivalent of RDQM-HA, you then replicate to your DR site using RDQM-DR to another highly available VM. That avoids the need for 6 VMs but relies on you having some other HA capability in each DC. Some people only have HA VMs so it's pointless running RDQM-HA on top as it will replicate an already replicated system, you might also prefer native Azure VM replication over a 3 node RDQM-HA solution.

Regards,

------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs

Original Message:
Sent: Tue August 10, 2021 05:07 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Martin,
thanks for reminding me that RDQM-DR can do synchronous ! (I've read a lot recently ;-)

I'm not quite sure I understand what you're saying after that though...

"If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well"

Are you suggesting that I use native VM replication across the two DC for all the VMs - or perhaps just Node1? In which case I'm confused as to why I'd then put RDQM-DR on the top of it.

Perhaps you're suggesting that I use native Azure VM replication intra-DC so that e.g. node1-3 are replicating amongst themselves then I use the RDQM-DR to replicate the Node1->Node1' and Node1' is then replicated internally to its other two nodes (Node2'-3') in its DC? This is then ONLY using RDQM-DR and not using RDQM ? Which confuses me.

Could you clarify for me please?

many thanks,
john.

------------------------------
John Hawkins
Integration Consultant

Original Message:
Sent: Tue August 10, 2021 03:51 AM
From: Martin Evans
Subject: RDMQ DR v Azure native Disk replication

Hi John,

RDQM does support synchronous replication. However, note -

Performance degrades rapidly with increasing latency between data centers. IBM will support a latency of up to 5 ms for synchronous replication and 50 ms for asynchronous replication.

See: https://www.ibm.com/docs/en/ibm-mq/9.2?topic=recovery-requirements-rdqm-dr-solution

If you can use a platform that provides HA for the VM and its disks in each DC then RDQM-DR can work well. Your choice of async or sync will depend on your application's  tolerance for potential message loss/performance. You could also mix and match depending on the application's requirements for message availability and performance; I know of some customers that turn off synchronous replication to DR where the applications favour of performance over message loss and can reconcile any message loss, but they keep it on for those that are less sensitive to latency and must not lose any data. 

Regards,

------------------------------
Martin Evans
Integration Architect
IBM Expert Labs
https://www.ibm.com/products/expertlabs

Original Message:
Sent: Mon August 09, 2021 09:32 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Hi John,

I have not read your previous question but you are correct that RDQM does not offer compression (or encryption if you have not yet asked about that) but that does not necessarily mean that you need to look at DRBD Proxy, which is not supported by RDQM. If Azure provides compression at the network level that is transparent to applications such as RDQM/DRBD then you could use that. Alternatively you could do something at the Linux level, again as long as it is transparent.

I have talked to a few customers that were looking at using Azure Site Recovery in conjunction with RDQM HA but I do not know what decision any of them made.

You are also correct that RDQM DR/HA requires six VMs to be on all the time, which does have the advantage (I think) that you know that the VMs are there when you need them. One disadvantage is that you have to keep them up to date, as well as the VMs in the main site. RDQM supports an active-active model where you can have different queue managers running in each site, which obviously requires six VMs anyway, but this may not be relevant in your situation.

Lastly, in RDQM, DR failover is an explicit action, as I believe it is with Azure Site Recovery. The main reason we did this in RDQM was that customers told us that they wanted to include MQ DR failover in a broader DR failover of databases, application servers etc. so there may not necessarily be any problem with MQ/RDQM when a DR failover is triggered. If you do want to include MQ/RDQM status in the decision about whether to do a DR failover then you will have to monitor the status of your RDQM queue managers with the rdqmstatus command.

------------------------------
JOHN COLGRAVE
------------------------------

Original Message:
Sent: Mon August 09, 2021 09:32 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------

Thanks John - all very useful and clear - as always ;-)
I think I'm leaning towards RDQM for HA intra-DC. Then Azure Native VM replication across DC. Seems to provide me with a well-known and Azure Admin visible method for DR and some slightly more hidden but reliable and MQ centric HA.

thanks again - too all the folks who've contributed here. It's good to know that I was right to think that it was not a clear-cut decision. 

John.  

------------------------------
John Hawkins
Integration Consultant
------------------------------

Original Message:
Sent: Tue August 10, 2021 04:56 AM
From: JOHN COLGRAVE
Subject: RDMQ DR v Azure native Disk replication

Hi John,

I have not read your previous question but you are correct that RDQM does not offer compression (or encryption if you have not yet asked about that) but that does not necessarily mean that you need to look at DRBD Proxy, which is not supported by RDQM. If Azure provides compression at the network level that is transparent to applications such as RDQM/DRBD then you could use that. Alternatively you could do something at the Linux level, again as long as it is transparent.

I have talked to a few customers that were looking at using Azure Site Recovery in conjunction with RDQM HA but I do not know what decision any of them made.

You are also correct that RDQM DR/HA requires six VMs to be on all the time, which does have the advantage (I think) that you know that the VMs are there when you need them. One disadvantage is that you have to keep them up to date, as well as the VMs in the main site. RDQM supports an active-active model where you can have different queue managers running in each site, which obviously requires six VMs anyway, but this may not be relevant in your situation.

Lastly, in RDQM, DR failover is an explicit action, as I believe it is with Azure Site Recovery. The main reason we did this in RDQM was that customers told us that they wanted to include MQ DR failover in a broader DR failover of databases, application servers etc. so there may not necessarily be any problem with MQ/RDQM when a DR failover is triggered. If you do want to include MQ/RDQM status in the decision about whether to do a DR failover then you will have to monitor the status of your RDQM queue managers with the rdqmstatus command.

------------------------------
JOHN COLGRAVE

Original Message:
Sent: Mon August 09, 2021 09:32 AM
From: John Hawkins
Subject: RDMQ DR v Azure native Disk replication

Hi Folks,
(Damn - topic typo - should be "RDQM"!)

a bit of a follow-up to my previous question on Azure HA/DR options for MQ...

Having researched this further I'm struggling to see whether to use RDQM DR in the cloud (Azure in this case) is a clear-cut argument. I say this for the following reasons...

RDQM doesn't have compression for-free (it's an add-on offering from LinBit in their "proxy" product as far as I can tell?). Therefore I'm going to be paying top-dollar for data egress from one DC to another. Is this a big deal - it depends on how big my queues physically get on disk along with the tran. log?

RDQM DR replication is asynchronous - I can get that level from Azure itself and I would rather leave disk replication to my Azure admin.

Azure can provide VM replication for the three RDQM nodes in one DC to another - which means I have limited configuration and good visibility at the Azure level.

With RDQM I need the remote VM on all the time  - which means more MQ licences (although they would be HA licences not full ones) and I also have the Azure DR VM costs. (I think I avoid both costs with Azure VM Replication?)
Having said that - I could have my test/UAT QMs on the secondary site. (albeit with full licences for one machine). That probably negates the licence issue. Indeed, this could make it cheaper than having VM replication.

Will I  have to ensure that I monitor the RDQM in order to ensure that they are all alive and nothing under-the-hood has gone wrong so that I know when to fail-over to the DR site with the Azure centric solution - does this apply to both solutions? This is DR not HA so what would I actually monitor that Azure itself doesn't know?

So, I think I'm on the fence until I work out my licence and VM costs - any thoughts anyone?

Thanks for any insights !
John.

------------------------------
John Hawkins
Integration Consultant
------------------------------