Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

Hello Saori, 

https://www.ibm.com/support/pages/compression-zlib-algorithm-save-commands-and-apis

• https://www.ibm.com/downloads/cas/QWXA9XKN

7.23 What are some considerations for use of data compression in IBM i? 

"The performance of any compression algorithm varies drastically based on the nature of the data that is being compressed."

• https://www.linkedin.com/posts/diegokesselman_ibmi-ibm-as400-activity-7151413744621768704-kF4J/

How much can you compress your data using ZLIB?
*ZLIB is faster than *HIGH and during my tests I got an average compression ratio of 5:1, but it depends on the data you are saving.

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

Dear Satid san

Thanks again for your comment.

>What DTACPR option did you use for compressing 2 TB in 15 minutes? 

I'll check with the customer.

>The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

So one of the sacrifices must be made. I understand.

You mean that you check for ineligible jobs. The customer commented that WRKSYSSTS checks were being performed and that there was not much CPU load.
There was no mention of ineligible jobs, so I will check.

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

One thing to point out with ZLIB.  It was added in 7.4 and yes, it does provide on average the best compression, but yea, it does come at a price. The pay off for using ZLIB is there, if you are on 7.5 AND Power10.  The Power10 processor, it has some onboard co-processor units that are specifically for ZLIB compression. So if you were to attempt the same compression test as Simon shown in the tables above with this updated environment, you would find the numbers very different. While Zlib is there. in 7.4 and 7.5 and works great, the real payoff, mixing the Power10 with the 7.5 release...

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

I want it on record that I wrote my post, mentioned above, as I was surprised to find ZLIB was possible in IBM i 7.4 running on an IBM Power 8.

My results should not be taken as being the definitive answer as to whether you should use ZLIB or not as I have been unable to test it with IBM i 7.5 and IBM Power 10. I don't have access to a LPAR running IBM i 7.5 running on IBM Power 10. When I do I cannot wait to try it.

One thing to point out with ZLIB.  It was added in 7.4 and yes, it does provide on average the best compression, but yea, it does come at a price. The pay off for using ZLIB is there, if you are on 7.5 AND Power10.  The Power10 processor, it has some onboard co-processor units that are specifically for ZLIB compression. So if you were to attempt the same compression test as Simon shown in the tables above with this updated environment, you would find the numbers very different. While Zlib is there. in 7.4 and 7.5 and works great, the real payoff, mixing the Power10 with the 7.5 release...

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

This thread makes me hope that our discussion so far would help inspire any members of this group with charitable souls who happen to run IBM i 7.5 on a Power10 machine would spare some times to conduct the test in the same way that Mr. Hutchinson did and share the result with us here as a generous and admirable charity for this community.   :-)   

I want it on record that I wrote my post, mentioned above, as I was surprised to find ZLIB was possible in IBM i 7.4 running on an IBM Power 8.

My results should not be taken as being the definitive answer as to whether you should use ZLIB or not as I have been unable to test it with IBM i 7.5 and IBM Power 10. I don't have access to a LPAR running IBM i 7.5 running on IBM Power 10. When I do I cannot wait to try it.

One thing to point out with ZLIB.  It was added in 7.4 and yes, it does provide on average the best compression, but yea, it does come at a price. The pay off for using ZLIB is there, if you are on 7.5 AND Power10.  The Power10 processor, it has some onboard co-processor units that are specifically for ZLIB compression. So if you were to attempt the same compression test as Simon shown in the tables above with this updated environment, you would find the numbers very different. While Zlib is there. in 7.4 and 7.5 and works great, the real payoff, mixing the Power10 with the 7.5 release...

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

Thanks to everyone who commented!

The discussion is lively and very informative.

>What DTACPR option did you use for compressing 2 TB in 15 minutes? 

The 15 minute results for 2TB were cases where the DTACPR was *DEV or *NO.

Customers are not wedded to *ZLIB time compression.
However, they are concerned about which components should be adjusted when using *ZLIB efficiently.
I understand that this may involve the processor or memory, am I correct?

This thread makes me hope that our discussion so far would help inspire any members of this group with charitable souls who happen to run IBM i 7.5 on a Power10 machine would spare some times to conduct the test in the same way that Mr. Hutchinson did and share the result with us here as a generous and admirable charity for this community.   :-)   

I want it on record that I wrote my post, mentioned above, as I was surprised to find ZLIB was possible in IBM i 7.4 running on an IBM Power 8.

My results should not be taken as being the definitive answer as to whether you should use ZLIB or not as I have been unable to test it with IBM i 7.5 and IBM Power 10. I don't have access to a LPAR running IBM i 7.5 running on IBM Power 10. When I do I cannot wait to try it.

One thing to point out with ZLIB.  It was added in 7.4 and yes, it does provide on average the best compression, but yea, it does come at a price. The pay off for using ZLIB is there, if you are on 7.5 AND Power10.  The Power10 processor, it has some onboard co-processor units that are specifically for ZLIB compression. So if you were to attempt the same compression test as Simon shown in the tables above with this updated environment, you would find the numbers very different. While Zlib is there. in 7.4 and 7.5 and works great, the real payoff, mixing the Power10 with the 7.5 release...

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

According to Tim's comment, is the number of processors most involved?

Thanks to everyone who commented!

The discussion is lively and very informative.

>What DTACPR option did you use for compressing 2 TB in 15 minutes? 

The 15 minute results for 2TB were cases where the DTACPR was *DEV or *NO.

Customers are not wedded to *ZLIB time compression.
However, they are concerned about which components should be adjusted when using *ZLIB efficiently.
I understand that this may involve the processor or memory, am I correct?

This thread makes me hope that our discussion so far would help inspire any members of this group with charitable souls who happen to run IBM i 7.5 on a Power10 machine would spare some times to conduct the test in the same way that Mr. Hutchinson did and share the result with us here as a generous and admirable charity for this community.   :-)   

I want it on record that I wrote my post, mentioned above, as I was surprised to find ZLIB was possible in IBM i 7.4 running on an IBM Power 8.

My results should not be taken as being the definitive answer as to whether you should use ZLIB or not as I have been unable to test it with IBM i 7.5 and IBM Power 10. I don't have access to a LPAR running IBM i 7.5 running on IBM Power 10. When I do I cannot wait to try it.

One thing to point out with ZLIB.  It was added in 7.4 and yes, it does provide on average the best compression, but yea, it does come at a price. The pay off for using ZLIB is there, if you are on 7.5 AND Power10.  The Power10 processor, it has some onboard co-processor units that are specifically for ZLIB compression. So if you were to attempt the same compression test as Simon shown in the tables above with this updated environment, you would find the numbers very different. While Zlib is there. in 7.4 and 7.5 and works great, the real payoff, mixing the Power10 with the 7.5 release...

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------

Thanks to everyone who commented!

The discussion is lively and very informative.

>What DTACPR option did you use for compressing 2 TB in 15 minutes? 

The 15 minute results for 2TB were cases where the DTACPR was *DEV or *NO.

Customers are not wedded to *ZLIB time compression.
However, they are concerned about which components should be adjusted when using *ZLIB efficiently.
I understand that this may involve the processor or memory, am I correct?

------------------------------
Saori Koinuma

Original Message:
Sent: Mon August 26, 2024 09:10 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

This thread makes me hope that our discussion so far would help inspire any members of this group with charitable souls who happen to run IBM i 7.5 on a Power10 machine would spare some times to conduct the test in the same way that Mr. Hutchinson did and share the result with us here as a generous and admirable charity for this community.   :-)   

------------------------------
Satid S

Original Message:
Sent: Mon August 26, 2024 05:45 AM
From: Simon Hutchinson
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

I want it on record that I wrote my post, mentioned above, as I was surprised to find ZLIB was possible in IBM i 7.4 running on an IBM Power 8.

My results should not be taken as being the definitive answer as to whether you should use ZLIB or not as I have been unable to test it with IBM i 7.5 and IBM Power 10. I don't have access to a LPAR running IBM i 7.5 running on IBM Power 10. When I do I cannot wait to try it.

------------------------------
Simon Hutchinson
https://www.rpgpgm.com

Original Message:
Sent: Fri August 23, 2024 09:11 AM
From: Tim Rowe
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

One thing to point out with ZLIB.  It was added in 7.4 and yes, it does provide on average the best compression, but yea, it does come at a price. The pay off for using ZLIB is there, if you are on 7.5 AND Power10.  The Power10 processor, it has some onboard co-processor units that are specifically for ZLIB compression. So if you were to attempt the same compression test as Simon shown in the tables above with this updated environment, you would find the numbers very different. While Zlib is there. in 7.4 and 7.5 and works great, the real payoff, mixing the Power10 with the 7.5 release...

------------------------------
Tim Rowe

Original Message:
Sent: Wed August 21, 2024 07:56 AM
From: 矢作 英幸
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Is there any reason to stick with *ZLIB? As others have mentioned, different compression algorithms work best for different object types, which is why multiple compression options are offered.

For reference, here is an example comparing the compression ratio and relative elapsed time (*NO = 1) by compression algorithm in the library QGPL and in a user library where the database files occupy most of the space. Clearly, *LOW is the way to go for the library QGPL. For the User Library, the choice would be *LOW, *MEDIUM, or *ZLIB, depending on whether backup time or compression ratio is more important. 

I hope this helps.

------------------------------
Hideyuki Yahagi

Original Message:
Sent: Wed August 14, 2024 02:30 AM
From: Satid S
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Dear Koinuma-san

>>>> However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB. <<<<

What DTACPR option did you use for compressing 2 TB in 15 minutes? 

ZLIB's main benefit is in reducing saved data size AT THE EXPENSE of the time it takes to do the data save.   How do I know this?  Look at this very informative article by Mr. Simon Hutchinson: Testing the new save compression ZLIB  at https://www.rpgpgm.com/2023/11/testing-new-save-compression-zlib.html  (you can see this article when you Google with "ibm i zlib").

Look at the comparison table at at the end of the article and you will see that ZLIB produces the SMALLEST saved data size but takes the SECOND LONGEST TIME to finish. It appears that ZLIB is a better choice ONLY WHEN compared to *HIGH as ZLIB is better in both dimensions. 

If the time it takes for save operation is the most important factor to you, you should use *LOW (or *MEDIUM) for DTACPR parameter of the save commands.    

The lesson here is that you cannot have it both ways.  You can achieve one (compressed data size) at the expense of the other (time).

For basic data save performance, you should also make sure that memory faulting rate is not too high during the data save period for the memory pool in which your data save job runs. Also "Ineligible" job state should not happen.  Check both of these with WRKSYSSTS screen or look at IBM i PDI chart for Memory and Faulting.  Also check that CPU % Busy is not high either. 

------------------------------
Satid S

Original Message:
Sent: Tue August 13, 2024 09:26 PM
From: Saori Koinuma
Subject: Please let me know if there is any way to reduce the processing time of ZLIB

Hello, Team

We are currently running a ZLIB compression test on a Power10 machine.
The compression rate of ZLIB is high and highly rated.
However, the compression using the virtual tape device took 15 minutes for 2 TB, but it took 2 hours when compressing with ZLIB.
Is there any way to reduce the compression time?Post

------------------------------
Saori Koinuma
------------------------------