AIX

Performance improvement in openssh with on-chip data compression accelerator in power9

By SWETHA NARAYANA posted Mon August 02, 2021 03:15 PM

  

Introduction:

The IBM Power 9 systems include on-chip data compression accelerator, which reduces the processor cycles for data compression and decompression. To exploit this accelerator, an enhanced ‘zlibNX’ library is available in IBM AIX® 7.2 with Technology Level 4 Expansion Pack, and later. Also, starting with IBM AIX® 7.2 Technology Level 5 Service Pack 2, the zlibNX installation package is available on the AIX base media in addition to the Expansion Pack.

Additional information about zlibNX library is available at:   https://www.ibm.com/docs/en/aix/7.2?topic=management-data-compression-by-using-zlibnx-library

OpenSSH provides the ssh command to establish secure connection to a host. When -C is provided as an option to ssh command then there is a request to compress all data during transfer. For this data compression, openssh was using libz.a library provided in /usr/lib/ path.

Starting with openSSH 8.1.102.2102 version, AIX Openssh has been modified to utilize enhanced ‘zlibNX’ library instead of default libz.a library while executing on AIX 7.2 levels. When this openssh version is installed in AIX 7.2 TL 4 and above levels that is running on a Power 9 hardware, there is a significant gain seen in compression performance due to the on-chip accelerator support. The detail of this gain is shared below.

 

1. Enabling in POWER 9 hardware:

 

In the Power 9 hardware, the on-chip data compression accelerator support is provided by selecting the POWER9 for the ‘Processor compatibility mode’ in the profile as shown below :

 

 

2. Enabling in OpenSSH:

 

In the ssh server, for data compression, a configuration option ‘Compression’ is provided in sshd_config. A value of ‘yes’, ‘no’ or ‘delayed’ can be provided for this option. ‘delayed’ is the default value which means compression will be delayed until the user has authenticated successfully.

After setting the configuration option, the ssh daemon needs to be restarted through the commands:

$ stopsrc -s sshd

$ startsrc -s sshd

 

In the ssh client, to perform the data compression, the -C command line option should be used with ‘ssh’ command.

 

2.1 - Addition option ‘EnableHWCompression’:

 

Till OpenSSH 8.1.102.2101, /usr/ lib/libz.a library was used in ssh server and client setup to provide compression, and there is only software implementation of compression available in this library.

 

From 8.1.102.2103, the compression functionality in openssh has been improved to make use of the enhanced libzNX.a library. This library has both software and hardware implementation of compression, wherein if the underlying hardware has the on-chip compressor accelerator support available then it makes use of this hardware capability.

 

A openssh configuration option has been provided for user to specify if they need software compression or hardware compression. This is done through option “EnableHwCompression” in both sshd_config of server and ssh_config of client. By default this configuration option is set to “no” which means only software compression is used. If this option is set to “yes” in both the config files then hardware compression is done when ssh -C is used while executing on a AIX 7.2 levels on a Power9 hardware with proper ‘Processor compatibility mode’ set.

 

Example:


 

If “EnableHwCompression” is either set to “no” ( or by default) software compression is done when -C is used in the command.

 

Example:


 

3. Performance numbers:

 

Here are the snapshots of performance numbers collected for time taken on transferring 1GB, 2GB, 4GB files on a P9 system with AIX 72Q installed (run on localhost).

Please refer the table below. First 2 options are default values where “Compression” will be set to “delayed”/”yes” in sshd_config file and “EnableHwCompression” is set to “no” in sshd_config and ssh_config both. Last 2 options are with “EnableHwCompression” been set to “yes” in both sshd_config file and ssh_config file along with “Compression” set to “yes” in sshd_config file.

 

On OpenSSH 8.1.102.2103 system, we have both hardware and software compression support. As we can see that First option with “scp -C” (marked in Blue) takes approx.  1min, 2min, 4min for 1GB, 2GB, 4GB files respectively (for software compression). Whereas for second option with “scp -C” (marked in Orange) is taking just 16 sec, 34sec, 71sec for 1GB, 2GB, 4GB files respectively(for hardware compression).

 

Here is the table and the bar graph:



 

  

References:

 

https://www.ibm.com/docs/en/aix/7.2?topic=management-nest-accelerators

 

https://community.ibm.com/community/user/power/blogs/brian-veale1/2020/11/09/power9-g[…]th-ibm-aix?CommunityKey=daa942cb-b783-4fd3-ba27-a2d7462f9530

 

https://www.ibm.com/docs/en/aix/7.2?topic=management-data-compression-by-using-zlibnx-library

 

2 comments
47 views

Permalink

Comments

19 days ago

Hello!

Update on this issue:

I´ve opened TS007478116 for the problem and we found out that 

openssh.base.client 8.1.102.2104 C F Open Secure Shell Commands
openssh.base.server 8.1.102.2104 C F Open Secure Shell Server

need to be installed for this to work, a version which wasn´t available through MRS until recently.

After upgrading to that version and restarting sshd everthing works as expected:

[YYY@ZZZZZ:/home/YYY]> scp -v -C /tmp/ESD* YYY@XXXXX:/tmp/                

Executing: program /usr//bin/ssh host XXXXX, user YYY, command scp -v -t /tmp/

OpenSSH_8.1p1, OpenSSL 1.0.2u 20 Dec 2019

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: /etc/ssh/ssh_config line 20: Applying options for *

debug1: oslevel : 7.2.0.0

debug1: using libzNX from /opt/freeware/ssh/libzNX_c.a(libz.so.1) 

debug1: init_libz_ptrs success

debug1: Failed dlopen: /usr/krb5/lib/libkrb5.a(libkrb5.a.so): Could not load module /usr/krb5/lib/libkrb5.a(libkrb5.a.so).

System error: No such file or directory

debug1: Error loading Kerberos, disabling Kerberos auth.

debug1: Connecting to XXXXX port 22.

debug1: Connection established.

...

debug1: Next authentication method: password

YYY@XXXXX's password: 

debug1: Enabling compression at level 6.

debug1: Authentication succeeded (password).

Authenticated to XXXXX:22).

debug1: channel 0: new [client-session]

debug1: Requesting no-more-sessions@openssh.com

debug1: Entering interactive session.

debug1: pledge: network

debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0

debug1: Sending command: scp -v -t /tmp/

Sending file modes: C0640 4294965248 ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz

Sink: C0640 4294965248 ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz

ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz                                                                       4% 193MB 63.1MB/s  01:01 ETA^Cdebug1: channel 0: free: client-session, nchannels 1

debug1: fd 0 clearing O_NONBLOCK

Killed by signal 2.



[YYY@ZZZZZ:/home/YYY]> scp -v /tmp/ESD* YYY@XXXXX:/tmp/   

Executing: program /usr//bin/ssh host XXXXX, user YYY, command scp -v -t /tmp/

OpenSSH_8.1p1, OpenSSL 1.0.2u 20 Dec 2019

debug1: Reading configuration data /etc/ssh/ssh_config

debug1: /etc/ssh/ssh_config line 20: Applying options for *

debug1: Failed dlopen: /usr/krb5/lib/libkrb5.a(libkrb5.a.so): Could not load module /usr/krb5/lib/libkrb5.a(libkrb5.a.so).

System error: No such file or directory

debug1: Error loading Kerberos, disabling Kerberos auth.

debug1: Connecting to XXXXX port 22.

debug1: Connection established.

...

debug1: Next authentication method: password

YYY@XXXXX´s password: 

debug1: Authentication succeeded (password).

Authenticated to XXXXX]:22).

debug1: channel 0: new [client-session]

debug1: Requesting no-more-sessions@openssh.com

debug1: Entering interactive session.

debug1: pledge: network

debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0

debug1: Sending command: scp -v -t /tmp/

Sending file modes: C0640 4294965248 ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz

Sink: C0640 4294965248 ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz

ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz                                                                       6% 248MB 124.7MB/s  00:30 ETA^Cdebug1: channel 0: free: client-session, nchannels 1

debug1: fd 0 clearing O_NONBLOCK

Killed by signal 2.

Maybe this article could be extended to add the information that .2104 version of OpenSSH is needed? Thanks!

21 days ago

Hello!

@SWETHA NARAYANA


I´ve tried this between two LPARs with the following specs:

prtconf:
System Model: IBM,9040-MR9
Processor Type: PowerPC_POWER9
Processor Implementation Mode: POWER 9
Processor Version: PV_9_Compat
Platform Firmware level: VM940_041
Firmware Version: IBM,FW940.02 (VM940_041)

oslevel -s :

​7200-04-04-2114

***

When i try ssh -C as a normal user (with zlibNX and xgzip installed) it always uses:

Executing: program /usr//bin/ssh host XXXX, user YYY, command scp -v -t /tmp/
OpenSSH_8.1p1, OpenSSL 1.0.2u 20 Dec 2019
debug1: using libz from /usr/opt/rpm/lib/libz.a(libz.so.1)
debug1: init_libz_ptrs success

and it uses software compression which equals to 20 MB/s instead of 100 MB/s without -C.

***

Changing the symlink /usr/lib/libz* doesn´t help

******

When I try it as root:

Executing: program /usr//bin/ssh host XXXX, user YYY, command scp -v -t /home/die/
OpenSSH_8.1p1, OpenSSL 1.0.2u 20 Dec 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: oslevel : 7.2.0.0

debug1: using libzNX from /opt/freeware/ssh/libzNX_c.a(libz.so.1)
debug1: init_libz_ptrs success

but it then fails:

Sink: C0640 4294965248 ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz

ESD-Toolbox_for_Linux_Apps_Common_6.1-7.2_112020_LKT010321.tar.gz

0% 0 0.0KB/s --:-- ETA

debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
Connection to XXXX closed by remote host.
Transferred: sent 164516, received 2328 bytes, in 0.1 seconds
Bytes per second: sent 2068712.3, received 29273.5
debug1: Exit status -1
debug1: compress outgoing: raw data 164151, compressed 162670, factor 0.99
debug1: compress incoming: raw data 628, compressed 629, factor 1.00
lost connection

All the options in ssh(d)_config are set as described.

Do you have a idea what I´m doing wrong?

Thanks,

With kind regards,

Steve