AIX

This Thread is created to address some security challenges with installing and checking efixes.

One of the good things is that most of efixes now have a signature file. This guarantees that the download was ok and the emgr efix package were not tampered.

That is a good thing.

There is now also a "new" AIX command emgr_sec that automatically verifies the package signature before installation.

That is also good improvement!

However there are some still some challenges: 

If you are using NIM  to install efixes there is  no NIM based command for efix_sec.

For this you can use the method below to verify the efix before installing it.

Note: that this method use the standard PSIRT public key included in AIX (since AIX 7.2 TL05 ?)

This is also a neat and proper way for verification.

Example for checking the efix package before using NIM to apply it:

Part of our script we use:

ifix_pub_key=/etc/security/certificates/AIX_PSIRT_pubkey.txt

openssl dgst -sha256 -verify $ifix_pub_key -signature ${emgr_package}/${emgr_package}.sig  ${emgr_dir}/${efix} >/dev/null 2>&1

If the return code is 0 than your efix package is save to install (package match the signature).

An alternative method would be to deploy efixes with a remote scp and ssh script to use the "new"  efix_sec command.

Second challenge:

Currently HIPER fixes have no signature files for the efix packages, for instance take a look at these HIPER fix locations, at least I could not find the .sig files there …

https://aix.software.ibm.com/aix/ifixes/ij49737/

https://www.ibm.com/support/pages/node/7131625

https://aix.software.ibm.com/aix/ifixes/ij46487/

https://aix.software.ibm.com/aix/ifixes/ij46694/

So how do we know if those important fixes are safe to install?

Maybe I have to create a case or RFE for this? Advice here is appreciated!

Another challenge:

Another issue with efix is that replaced (temporary) fixed binaries and libraries are not added with the right certtag and/or signature and hash_value in the tsd database for trustchk, such approach does not allow secure checking the content using trustchk after installation.

Sort explanation why this TE option is so important for us as a Dutch Bank.

For every AIX binary, Library a cert_tag and signature, hash is added to the TE database (tsd.dat).

In this way we can proof that the binary, executable, library is build by IBM and is not modified.

See also my blog series about TE.

Example for this in more detail:

For example IJ49883s7a does replace the binary /usr/sbin/qdaemon

Before the efix was applied the binaries are correct signed and  have a valid signature value  (as it should be) in the tsd database tsd.dat, see below:

part of the tsd.dat (stanza for this binary)

/usr/sbin/qdaemon:

        owner = root

        group = printq

        mode = TCB,SUID,SGID,550

        type = FILE

        hardlinks =

        symlinks =

        size = 103101

        cert_tag = 49424d4149583a31324331342d33314332303a324b3a41

        signature = 7c1ca9efec42f3f20bd2dda6c391e3fc4e6be12a5a88293009611a0ebab073ca00bd156f91bcd6e2542b023df488178d13983f93895b872c75a06f818bc02ce6d50340cd67cc110b6cf586858720cc35d84a0eff4a6d1893b51e6f6b3fe413ef5a430856e62e39d8f4ce032be2483dbb7fb69b2582cb28cd7039d6b4cde288d92c05b2661f17b6df82d3251fa3020033e1912cca4b158b85de76fc2bd42388ba44e12410d7cbecd02a9f182b0ac5b5ac907519bf9e3d9e33b33cf355a0b2f15d266e8f80fa77d5b23fb3f9c5f0130f9696566b4a3ad4522298ebda2886bf8e14058b9c08ce62cd0ff248ce691430263999b896d68536ffefa4bc7537794cb44a

        hash_value = 9ee51df3d744bbd5edb54259419b4f911f2f4978904c0d3ee413bbe1614b8b91

        minslabel =

        maxslabel =

        intlabel =

        accessauths =

        innateprivs =

        inheritprivs =

        authprivs =

        secflags =

After applying the efix this is modified to: 

trustchk -q /usr/sbin/qdaemon

/usr/sbin/qdaemon:

        type = FILE

        owner = root

        group = printq

        mode = SUID,SGID,550

        size = VOLATILE

        hash_value = VOLATILE

        cert_tag = VOLATILE

        signature = VOLATILE

In my humble opinion this not ok!  For each security efix or HIPER  the efix should have the  proper signature/hash values  also updated/changed in the  tsd.dat database, and NOT set this to VOLATILE.

So if these binaries were modified after installation, you will not be able to detect that, thus the security checks via trustchk are not reliable anymore! 

Why? The word VOLATILE in the tsd.dat breaks this verification, and even the size of an binary is no longer checked.

In other words a trustchk -n /usr/sbin/qdaemon in the former example will always pass.

Maybe we have to create case or RFE for this also?  again advice here is appreciated!

Automated download challenge:

Another challenge we have, with efixes / packages that we need to download, is that we cannot automate downloads of the MRS site.

https://www.ibm.com/resources/mrs/assets/packageList?source=aixbp&lang=en_US

We first have to authenticate ourselves with a IBM ID and Password. Sure this is a good thing, but makes it difficult to automate.

Does anyone know if we can download those packages on this side without authentication first, but via a public key / certificate or something?

We like to automate those downloads also, but not this seems not possible? 

Please advise me and give me feedback on this subjects. My intention is to improve and automate patch management for AIX in a secure environment.

------------------------------
Christian Sonnemans
Tactical Unix system engineer
De Volksbank
Den Bosch
------------------------------

Hi Christian, Thanks for sharing these details. We are already aware of these problems and also started working on couple of them.

NIM based command for efix_sec: We have this feature in our plan for the next TL. 

HIPER fixes with no signature: We need check on this and get more info to you. But, please go ahead and open an RFE, if you like.

VOLATILE tsd after efix installation: We agree that the tsd should not be volatile after an efix installation. We will check a bit more and update you on this. RFE is very welcome.

Automated download from MRS: We have a plan to automate the MRS downloads and include a solution in the base AIX. We are planning this activity also for next TL.

Please feel free to share any more details that would help us. 

This Thread is created to address some security challenges with installing and checking efixes.

One of the good things is that most of efixes now have a signature file. This guarantees that the download was ok and the emgr efix package were not tampered.

That is a good thing.

There is now also a "new" AIX command emgr_sec that automatically verifies the package signature before installation.

That is also good improvement!

However there are some still some challenges: 

If you are using NIM  to install efixes there is  no NIM based command for efix_sec.

For this you can use the method below to verify the efix before installing it.

Note: that this method use the standard PSIRT public key included in AIX (since AIX 7.2 TL05 ?)

This is also a neat and proper way for verification.

Example for checking the efix package before using NIM to apply it:

Part of our script we use:

ifix_pub_key=/etc/security/certificates/AIX_PSIRT_pubkey.txt

openssl dgst -sha256 -verify $ifix_pub_key -signature ${emgr_package}/${emgr_package}.sig  ${emgr_dir}/${efix} >/dev/null 2>&1

If the return code is 0 than your efix package is save to install (package match the signature).

An alternative method would be to deploy efixes with a remote scp and ssh script to use the "new"  efix_sec command.

Second challenge:

Currently HIPER fixes have no signature files for the efix packages, for instance take a look at these HIPER fix locations, at least I could not find the .sig files there …

https://aix.software.ibm.com/aix/ifixes/ij49737/

https://www.ibm.com/support/pages/node/7131625

https://aix.software.ibm.com/aix/ifixes/ij46487/

https://aix.software.ibm.com/aix/ifixes/ij46694/

So how do we know if those important fixes are safe to install?

Maybe I have to create a case or RFE for this? Advice here is appreciated!

Another challenge:

Another issue with efix is that replaced (temporary) fixed binaries and libraries are not added with the right certtag and/or signature and hash_value in the tsd database for trustchk, such approach does not allow secure checking the content using trustchk after installation.

Sort explanation why this TE option is so important for us as a Dutch Bank.

For every AIX binary, Library a cert_tag and signature, hash is added to the TE database (tsd.dat).

In this way we can proof that the binary, executable, library is build by IBM and is not modified.

See also my blog series about TE.

Example for this in more detail:

For example IJ49883s7a does replace the binary /usr/sbin/qdaemon

Before the efix was applied the binaries are correct signed and  have a valid signature value  (as it should be) in the tsd database tsd.dat, see below:

part of the tsd.dat (stanza for this binary)

/usr/sbin/qdaemon:

        owner = root

        group = printq

        mode = TCB,SUID,SGID,550

        type = FILE

        hardlinks =

        symlinks =

        size = 103101

        cert_tag = 49424d4149583a31324331342d33314332303a324b3a41

        signature = 7c1ca9efec42f3f20bd2dda6c391e3fc4e6be12a5a88293009611a0ebab073ca00bd156f91bcd6e2542b023df488178d13983f93895b872c75a06f818bc02ce6d50340cd67cc110b6cf586858720cc35d84a0eff4a6d1893b51e6f6b3fe413ef5a430856e62e39d8f4ce032be2483dbb7fb69b2582cb28cd7039d6b4cde288d92c05b2661f17b6df82d3251fa3020033e1912cca4b158b85de76fc2bd42388ba44e12410d7cbecd02a9f182b0ac5b5ac907519bf9e3d9e33b33cf355a0b2f15d266e8f80fa77d5b23fb3f9c5f0130f9696566b4a3ad4522298ebda2886bf8e14058b9c08ce62cd0ff248ce691430263999b896d68536ffefa4bc7537794cb44a

        hash_value = 9ee51df3d744bbd5edb54259419b4f911f2f4978904c0d3ee413bbe1614b8b91

        minslabel =

        maxslabel =

        intlabel =

        accessauths =

        innateprivs =

        inheritprivs =

        authprivs =

        secflags =

After applying the efix this is modified to: 

trustchk -q /usr/sbin/qdaemon

/usr/sbin/qdaemon:

        type = FILE

        owner = root

        group = printq

        mode = SUID,SGID,550

        size = VOLATILE

        hash_value = VOLATILE

        cert_tag = VOLATILE

        signature = VOLATILE

In my humble opinion this not ok!  For each security efix or HIPER  the efix should have the  proper signature/hash values  also updated/changed in the  tsd.dat database, and NOT set this to VOLATILE.

So if these binaries were modified after installation, you will not be able to detect that, thus the security checks via trustchk are not reliable anymore! 

Why? The word VOLATILE in the tsd.dat breaks this verification, and even the size of an binary is no longer checked.

In other words a trustchk -n /usr/sbin/qdaemon in the former example will always pass.

Maybe we have to create case or RFE for this also?  again advice here is appreciated!

Automated download challenge:

Another challenge we have, with efixes / packages that we need to download, is that we cannot automate downloads of the MRS site.

https://www.ibm.com/resources/mrs/assets/packageList?source=aixbp&lang=en_US

We first have to authenticate ourselves with a IBM ID and Password. Sure this is a good thing, but makes it difficult to automate.

Does anyone know if we can download those packages on this side without authentication first, but via a public key / certificate or something?

We like to automate those downloads also, but not this seems not possible? 

Please advise me and give me feedback on this subjects. My intention is to improve and automate patch management for AIX in a secure environment.

------------------------------
Christian Sonnemans
Tactical Unix system engineer
De Volksbank
Den Bosch
------------------------------

Hello Saikrishna Akkela,

First of all thank you so much for your quick and clear reply, very much appreciated!

Also very good to read IBM start working already on some of those challenges.

NIM based command for efix_sec: We have this feature in our plan for the next TL.

Will this be available in AIX 7.3 TL03 already (planned in December)?

Automated download from MRS: We have a plan to automate the MRS downloads and include a solution in the base AIX. We are planning this activity also for next TL. Same question here AIX 7.3 TL03 ?

HIPER fixes with no signature I will create a RFE for this tomorrow.

Please let me know what information you will need form me, I am more than willing to provide this.

VOLATILE tsd after efix installation: We agree that the tsd should not be volatile after an efix installation. We will check a bit more and update you on this.

Very good news!

I will create a RFE for this also if that is helpful. This would really help us to prove that efixes that were provided by IBM and signed and can't be modified when running TE in kernel mode. Also to run valid trustchk validations of binaries and libraries after those are replaced by a efix, those are now vulnerable.

Please let me know if we have to communicate via email or a via this channel.

Hi Christian, Thanks for sharing these details. We are already aware of these problems and also started working on couple of them.

NIM based command for efix_sec: We have this feature in our plan for the next TL. 

HIPER fixes with no signature: We need check on this and get more info to you. But, please go ahead and open an RFE, if you like.

VOLATILE tsd after efix installation: We agree that the tsd should not be volatile after an efix installation. We will check a bit more and update you on this. RFE is very welcome.

Automated download from MRS: We have a plan to automate the MRS downloads and include a solution in the base AIX. We are planning this activity also for next TL.

Please feel free to share any more details that would help us. 

This Thread is created to address some security challenges with installing and checking efixes.

One of the good things is that most of efixes now have a signature file. This guarantees that the download was ok and the emgr efix package were not tampered.

That is a good thing.

There is now also a "new" AIX command emgr_sec that automatically verifies the package signature before installation.

That is also good improvement!

However there are some still some challenges: 

If you are using NIM  to install efixes there is  no NIM based command for efix_sec.

For this you can use the method below to verify the efix before installing it.

Note: that this method use the standard PSIRT public key included in AIX (since AIX 7.2 TL05 ?)

This is also a neat and proper way for verification.

Example for checking the efix package before using NIM to apply it:

Part of our script we use:

ifix_pub_key=/etc/security/certificates/AIX_PSIRT_pubkey.txt

openssl dgst -sha256 -verify $ifix_pub_key -signature ${emgr_package}/${emgr_package}.sig  ${emgr_dir}/${efix} >/dev/null 2>&1

If the return code is 0 than your efix package is save to install (package match the signature).

An alternative method would be to deploy efixes with a remote scp and ssh script to use the "new"  efix_sec command.

Second challenge:

Currently HIPER fixes have no signature files for the efix packages, for instance take a look at these HIPER fix locations, at least I could not find the .sig files there …

https://aix.software.ibm.com/aix/ifixes/ij49737/

https://www.ibm.com/support/pages/node/7131625

https://aix.software.ibm.com/aix/ifixes/ij46487/

https://aix.software.ibm.com/aix/ifixes/ij46694/

So how do we know if those important fixes are safe to install?

Maybe I have to create a case or RFE for this? Advice here is appreciated!

Another challenge:

Another issue with efix is that replaced (temporary) fixed binaries and libraries are not added with the right certtag and/or signature and hash_value in the tsd database for trustchk, such approach does not allow secure checking the content using trustchk after installation.

Sort explanation why this TE option is so important for us as a Dutch Bank.

For every AIX binary, Library a cert_tag and signature, hash is added to the TE database (tsd.dat).

In this way we can proof that the binary, executable, library is build by IBM and is not modified.

See also my blog series about TE.

Example for this in more detail:

For example IJ49883s7a does replace the binary /usr/sbin/qdaemon

Before the efix was applied the binaries are correct signed and  have a valid signature value  (as it should be) in the tsd database tsd.dat, see below:

part of the tsd.dat (stanza for this binary)

/usr/sbin/qdaemon:

        owner = root

        group = printq

        mode = TCB,SUID,SGID,550

        type = FILE

        hardlinks =

        symlinks =

        size = 103101

        cert_tag = 49424d4149583a31324331342d33314332303a324b3a41

        signature = 7c1ca9efec42f3f20bd2dda6c391e3fc4e6be12a5a88293009611a0ebab073ca00bd156f91bcd6e2542b023df488178d13983f93895b872c75a06f818bc02ce6d50340cd67cc110b6cf586858720cc35d84a0eff4a6d1893b51e6f6b3fe413ef5a430856e62e39d8f4ce032be2483dbb7fb69b2582cb28cd7039d6b4cde288d92c05b2661f17b6df82d3251fa3020033e1912cca4b158b85de76fc2bd42388ba44e12410d7cbecd02a9f182b0ac5b5ac907519bf9e3d9e33b33cf355a0b2f15d266e8f80fa77d5b23fb3f9c5f0130f9696566b4a3ad4522298ebda2886bf8e14058b9c08ce62cd0ff248ce691430263999b896d68536ffefa4bc7537794cb44a

        hash_value = 9ee51df3d744bbd5edb54259419b4f911f2f4978904c0d3ee413bbe1614b8b91

        minslabel =

        maxslabel =

        intlabel =

        accessauths =

        innateprivs =

        inheritprivs =

        authprivs =

        secflags =

After applying the efix this is modified to: 

trustchk -q /usr/sbin/qdaemon

/usr/sbin/qdaemon:

        type = FILE

        owner = root

        group = printq

        mode = SUID,SGID,550

        size = VOLATILE

        hash_value = VOLATILE

        cert_tag = VOLATILE

        signature = VOLATILE

In my humble opinion this not ok!  For each security efix or HIPER  the efix should have the  proper signature/hash values  also updated/changed in the  tsd.dat database, and NOT set this to VOLATILE.

So if these binaries were modified after installation, you will not be able to detect that, thus the security checks via trustchk are not reliable anymore! 

Why? The word VOLATILE in the tsd.dat breaks this verification, and even the size of an binary is no longer checked.

In other words a trustchk -n /usr/sbin/qdaemon in the former example will always pass.

Maybe we have to create case or RFE for this also?  again advice here is appreciated!

Automated download challenge:

Another challenge we have, with efixes / packages that we need to download, is that we cannot automate downloads of the MRS site.

https://www.ibm.com/resources/mrs/assets/packageList?source=aixbp&lang=en_US

We first have to authenticate ourselves with a IBM ID and Password. Sure this is a good thing, but makes it difficult to automate.

Does anyone know if we can download those packages on this side without authentication first, but via a public key / certificate or something?

We like to automate those downloads also, but not this seems not possible? 

Please advise me and give me feedback on this subjects. My intention is to improve and automate patch management for AIX in a secure environment.

------------------------------
Christian Sonnemans
Tactical Unix system engineer
De Volksbank
Den Bosch
------------------------------