Primary Storage

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele
------------------------------

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

T Masteen's observation might be correct.
For the now legacy RTC (Real-Time Compression) in fact the benefit of one-time traversal of write data between the nodes in an Enhanced Stretched Cluster does not apply. I strongly assume this equally applies to DRP compression as well, need to verify this.
For RTC this used to be documented somewhere (IBM Docs alias Knowledge Center, IBM Redbooks...), yet I didn't get hold of this for now, will post an update once I got it.

Best regards, 

Christian Schroeder

------------------------------
Christian Schroeder
IBM SpecV Storage Support with Passion
------------------------------

Original Message:
Sent: Thu May 20, 2021 03:04 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

Hello Christian, Thanks for looking into it.

------------------------------
T Masteen
------------------------------

Original Message:
Sent: Fri May 21, 2021 08:43 AM
From: Christian Schroeder
Subject: SVC compression in DRP and Enhanched Stretched Cluster

T Masteen's observation might be correct.
For the now legacy RTC (Real-Time Compression) in fact the benefit of one-time traversal of write data between the nodes in an Enhanced Stretched Cluster does not apply. I strongly assume this equally applies to DRP compression as well, need to verify this.
For RTC this used to be documented somewhere (IBM Docs alias Knowledge Center, IBM Redbooks...), yet I didn't get hold of this for now, will post an update once I got it.

Best regards, 

Christian Schroeder

------------------------------
Christian Schroeder
IBM SpecV Storage Support with Passion

Original Message:
Sent: Thu May 20, 2021 03:04 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

Original Message:
Sent: 5/21/2021 1:44:00 PM
From: T Masteen
Subject: RE: SVC compression in DRP and Enhanched Stretched Cluster

Hello Christian, Thanks for looking into it.

------------------------------
T Masteen
------------------------------

Original Message:
Sent: Fri May 21, 2021 08:43 AM
From: Christian Schroeder
Subject: SVC compression in DRP and Enhanched Stretched Cluster

T Masteen's observation might be correct.
For the now legacy RTC (Real-Time Compression) in fact the benefit of one-time traversal of write data between the nodes in an Enhanced Stretched Cluster does not apply. I strongly assume this equally applies to DRP compression as well, need to verify this.
For RTC this used to be documented somewhere (IBM Docs alias Knowledge Center, IBM Redbooks...), yet I didn't get hold of this for now, will post an update once I got it.

Best regards, 

Christian Schroeder

------------------------------
Christian Schroeder
IBM SpecV Storage Support with Passion

Original Message:
Sent: Thu May 20, 2021 03:04 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

FCM is not an option for us because I have to work with multiple existing SSD arrays in a new SV2 cluster.
Furthermore, DRP can be a good option, but I would like to understand the impact of DRP (compression) in an ESC.
The SVC IO stack was always clear, Real time Compression was above the lower cache and virtualization layer. Now DRP (with compression) sits next to the lower cache, virtualization layer and RAID.

------------------------------
T Masteen
------------------------------

Original Message:
Sent: Sat May 22, 2021 05:55 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Actually, an Alternative method and the best way to avoid this altogether is bypassing DRP on SVC and writing directly on FCM. This allows for normal Pools, volumes, etc... 

Especially of you are not planning on deduplication. 

BR, 

Original Message:
Sent: 5/21/2021 1:44:00 PM
From: T Masteen
Subject: RE: SVC compression in DRP and Enhanched Stretched Cluster

Hello Christian, Thanks for looking into it.

------------------------------
T Masteen

Original Message:
Sent: Fri May 21, 2021 08:43 AM
From: Christian Schroeder
Subject: SVC compression in DRP and Enhanched Stretched Cluster

T Masteen's observation might be correct.
For the now legacy RTC (Real-Time Compression) in fact the benefit of one-time traversal of write data between the nodes in an Enhanced Stretched Cluster does not apply. I strongly assume this equally applies to DRP compression as well, need to verify this.
For RTC this used to be documented somewhere (IBM Docs alias Knowledge Center, IBM Redbooks...), yet I didn't get hold of this for now, will post an update once I got it.

Best regards, 

Christian Schroeder

------------------------------
Christian Schroeder
IBM SpecV Storage Support with Passion

Original Message:
Sent: Thu May 20, 2021 03:04 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

Hello T Masteen,
I scanned through my personal notes and was happy enough to find this info I once got from SpecV development wrt this very topic:

As others had correctly mentioned before, compression indeed happens in between Upper Cache and Lower Cache. 
I had assumed hence, Upper & Lower Cache would need to be copied between the nodes in an Enhanced Stretched Cluster with DRP compression.
To be 100% sure on that, I have double-checked with development team on this, and they confirmed this to be the case.

------------------------------
Christian Schroeder
IBM SpecV Storage Support with Passion
------------------------------

Original Message:
Sent: Fri May 21, 2021 01:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello Christian, Thanks for looking into it.

------------------------------
T Masteen

Original Message:
Sent: Fri May 21, 2021 08:43 AM
From: Christian Schroeder
Subject: SVC compression in DRP and Enhanched Stretched Cluster

T Masteen's observation might be correct.
For the now legacy RTC (Real-Time Compression) in fact the benefit of one-time traversal of write data between the nodes in an Enhanced Stretched Cluster does not apply. I strongly assume this equally applies to DRP compression as well, need to verify this.
For RTC this used to be documented somewhere (IBM Docs alias Knowledge Center, IBM Redbooks...), yet I didn't get hold of this for now, will post an update once I got it.

Best regards, 

Christian Schroeder

------------------------------
Christian Schroeder
IBM SpecV Storage Support with Passion

Original Message:
Sent: Thu May 20, 2021 03:04 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

In the past when RTC was in place, the usage of Compression use to lead out to double write on the N2N because of SVC internal architecture. In other words, every write needed too be doubled when used in RTC doubling the bandwidth between nodes. With DRP , currently I have no clue if it applly as well.

------------------------------
Angelo Bernasconi
------------------------------

Original Message:
Sent: Thu May 20, 2021 03:04 AM
From: Tom Van Daele
Subject: SVC compression in DRP and Enhanched Stretched Cluster

I don't really see why you would have more or less bandwidth usage on your ISL's by using compressing in SVC ESC since you are basically constantly mirroring the write cache between the split IO group.

Compression is one the last steps performed before actually commiting the write to disk, so the bandwith from SVC to your back end system could be smaller since there the data is compressed. I think mirroring your write cache happens before that.

I know there is a schema on how Spectrum Virtualize handles an IO and what steps are taken when but I can't seem to find it.

Br,
Tom

------------------------------
Tom Van Daele

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------

Fellow IBMers Barry Whyte and Andrew Martin wrote the following in their blog, which suggests that DRP still operates above the lower cache. Unfortunately this doesn't precisely state whether upper and lower cache are co-mirrored in an ESC when using DRPs.  However, I believe that the DRP engine is still considered a scarce resource compared to mirroring bandwidth, in particular with respect to streaming performance: It makes more sense to economize 50% DRP effort using 30% more mirroring bandwidth than to economize 30% bandwidth using 100% (+50%) increased DRP load.

De-duplication

With v8.1.3 you can now have Fully Allocated, Thin Provisioned, Compressed, De-duplicated and De-duplicated then Compressed volumes inside a single Data Reduction Pool. Allowing you to mix and match, and not forcing the meta-data I/O amplification on every volume on the system. This is also inline, so data is de-duplicated before it reaches lower cache. (...)

Kind Regards, Axel

------------------------------
Axel Koester/IBM
------------------------------

Original Message:
Sent: Thu April 22, 2021 02:44 PM
From: T Masteen
Subject: SVC compression in DRP and Enhanched Stretched Cluster

Hello all,

With SVC Enhanched Stretched Cluster when writing there is reduced inter-site traffic (compared with a classic stretched cluster).
However when using compression lower cache will be re-mirrored. Hence additional bandwidth.

Now we have compression in Data Reduction Pools. Is there a difference in how writes are handled in a ESC?

Thanks!

------------------------------
T Masteen
------------------------------