AIX

In this part I like to share some technical details and dive into some more details.

To understand more how it works first lets explain the most important stanza based database: tsd.dat

Explanation of the tsd.dat:

TSD stands for Trusted Signature Database and in this stanza based formatted file that exist locally on a AIX system or can exist on a remote Ldap server.

During the installation, the tsd.dat file is filled with every AIX executable / library / script that IBM has signed for the specified fileset that is installed. The same is done during upgrading to a TL or SP.

Note: for efixes (emergency fixes) this not NOT the case. I will explain how to deal with this a bit further for those file(sets).

For every stanza in this tsd.dat the command is trusted. See below part of the security documentation of AIX 7.3:

A trusted file is a file that is critical from the security perspective of the system, and if compromised,

can jeopardize the security of the entire system. Typically the files that match this description are the

following:

• Kernel (operating system)

• All setuid root programs

• All setgid root programs

• Any program that is exclusively run by the root user or by a member of the system group

• Any program that must be run by the administrator while on the trusted communication path (for

example, the ls command)

• The configuration files that control system operation

• Any program that is run with the privilege or access rights to alter the kernel or the system configuration

files

Every trusted file should ideally have an associated stanza or a file definition stored in the Trusted

Signature Database (TSD).

For the impatient people you can read this part later and skip to active / passive mode:

Example stanza of cfgmgr in the databasefile tsd.dat

 /usr/sbin/cfgmgr:

        owner = root

        group = system

        mode = TCB,SUID,550

        type = FILE

        hardlinks =

        symlinks =

        size = 64980

        cert_tag = 49424d4149583a30344232302d30334233303a324b3a41

        signature 4947a123bb0cb96b591074df5f7a6d97c9be0b3a00902d373516b299cfaef7b7f16625ad0c5c9fed445cf473b9bf1dfe8dc66f6a11d68749c7b0f4239e4a09a0d1ba005a552d64377816d0a641221bd6b9ce58ae7facf7eecbdac19c9

9371db3423fb294dbf1622d164699170a15db82a6d484aec71f85c4579eb79052e006a8eaa5fb8fa44f0b5a771d0abb65e51c49e79d3678a0b36222c9a54a6b88eb21a384ba09bf3307efec5f165d79c5dff3550b43163beae5bdfd9a7ba575f93136cb17e8232

7eb71cb169add8a7387febe299be4fc06485420b1fd1523969a5901fa97232cb053f93640a66bb925d06cb95c50361596d758074a83beb65d13fe4d8

        hash_value = c42915237644858ce913c9930ab7d30f2f918d5e0b6f38431f3e39819eda55ff

        minslabel =

        maxslabel =

        intlabel =

        accessauths = aix.device.config

        innateprivs = PV_AU_ADD,PV_AU_PROC,PV_DAC_R,PV_DAC_W,PV_DAC_X,PV_DAC_O,PV_DEV_CONFIG

        inheritprivs = PV_DAC_R,PV_DAC_W,PV_DAC_X,PV_DEV_CONFIG,PV_DEV_LOAD,PV_DEV_QUERY,PV_KER_VARS,PV_ROOT

        authprivs =

        secflags = FSF_EPS

        egid = 0

For this first part of TE, I will only explain a few of the fields in this example, which are important for TE, other fields are important for another security feature of AIX: advanced RBAC.

For a complete documentation and explanation of all fields please refer the link below:

https://www.ibm.com/docs/en/aix/7.3?topic=management-trusted-signature-database

See below the explanation for the stanza fields that are used for TE:

owner

Owner of the file. This value is computed by the trustchk command when the file is being added to TSD.

group

Group of the file. This value is computed by the trustchk command.

mode

Comma-separated list of values. The permissible values are SUID (SUID set bit), SGID (SGID set bit), SVTX (SVTX set bit), and TCB (Trusted Computing Base). The file permissions must be the last value and can be specified as an octal value. For example, for a file that is set with uid and has permission bits as rwxr-xr-x, the value for mode is SUID, 755. The value is computed by the trustchk command.

type

Type of the file. This value is computed by the trustchk command. The possible values are FILE, DIRECTORY, MPX_DEV, CHAR_DEV, BLK_DEV, and FIFO.

hardlinks

List of hardlinks to the file. This value cannot be computed by the trustchk command. It must be supplied by the user when adding a file to the database.

symlinks

List of symbolic links to the file. This value cannot be computed by the trustchk command. It must be supplied by the user when adding a file to the database.

size

Defines size of the file. The VOLATILE value means that the file gets changed frequently.

cert_tag

This field maps the digital signature of the file with the associated certificate that can be used to verify the signature of the file. This field stores the certificate ID and is computed by the trustchk command at the time of addition of the file to the TSD. The certificates are stored in /etc/security/certificates directory.

signature

Digital signature of the file. The VOLATILE value means that the file gets changed frequently. This field is computed by the trustchk command.

hash_value

Cryptographic hash of the file. The VOLATILE value means that the file gets changed frequently. This field is computed by the trustchk command.

Example of stanza /etc/security/tsd/lib/lib.tsd.dat

/usr/lib/libc.a/strcmp.o:

Type = OBJ

Size = 2345

Hash value

Signature =

Cert_tag =

Part of documentation that describes the lib.tsd.dat

In the /etc/security/tsd/tsd.dat file, the stanzas for the libraries are modified to reflect the type attribute as LIB and the size and signature attributes are empty. Currently the values for the dynamica attributes size, hash, signature are maintained as a VOLATILE value. Therefore, the library verification is skipped during system boot. Beginning with the release of AIX 6.1.0, the size, hash, and signature of the trusted library stanzas are computed with the .o files of a library. During installation, the tsd.dat database is populated to reflect the computed values and the corresponding .o file stanza for a trusted library is stored in the /etc/security/tsd/lib/lib.tsd.dat database.

Passive and Active mode of TE:

By default after installing AIX, TE is running in passive mode, and it will use the local stored tsd.dat file and no TE policy is enabled or defined, you can check this with the command trustchk -p

See picture below left scheme:

The TE passive form is always available and is a very good checkup for sanity your AIX system.

With this system integrity check you can always verify if your AIX binaries and Libraries are not tempered or corrupted, this with only one simple command:

trustchk -n all

Advice: always do this checkup first  before you enable the runtime mode.

This is the default mode and it’s always available also if the runtime policy is active (trustchk -p te=on).

Sometimes especially when efixes are applied it can be necessarily to modify / update the local tsd.dat file.

For efixes the signature is and hash do have old values, and you have to set them to VOLATILE into the tsd.dat.

After the efix becomes an official Apar, for example in a new SP or TL, it will have again a valid signature and hash, signed by IBM.

Another (dirty way) is remove the execuable lib or script form the tsd.dat with:

trustchk -d /complete/path/of/the/exec/script/file

Please use this as last resort first try to update the tsd.dat with the VOLATILE values.

This because even when used the VOLATILE settings, those files can be locked, and still verified for the right owner and groups.

When you have a nice consistent local tds.dat you can give it a try to enable the runtime TE.

Read the output of the command

 trustchk -n ALL

 gives no errors and gives on ouput.

Also TE in passive mode can be used to do a daily checkup for the sanity of your Lpar, but it can be cpu consuming, a nice option for this is running this process in a wpar see example below that I use in my daily checkup script:

/usr/sbin/wparexec -n trustchkexec -R CPU=2%-3%,3% /usr/sbin/trustchk -n all 2>> ${LOG}

Lets enable the runtime part of TE

First have a look at the scheme below:

Above the TE kernel logic is represented.

This scheme is in place for the runtime enabled TE and is executed for each stanza in the tsd.dat and lib.tsd.dat.

According the documentation:

To enable or disable the runtime integrity-verification function that is responsible for verifying of a file's cryptographic hash or signature before being started, use the trustchk command. To turn the Trusted Execution function on or off, use the TE -p flag.

In other words: You can enable the runtime by trustchk -p te=on

But before you execute this command let me first explain the second database file tepolicies.dat

As the name already expect, this database contains the TE policies that can be set

Both database files are stored localy on /etc/security/tsd/

For the libraries in /etc/security/tsd/lib

Table below describes each policy that can be set:

CHKEXEC

Checks the hash value or the signature of only the trusted executables before loading them in memory for execution.

CHKSHLIBS

Checks the hash value or the signature of only the trusted shared libraries before loading them in memory for execution.

CHKSCRIPTS

Checks the hash value or the signature of only the trusted shell scripts before loading them in memory.

CHKKERNEXT

Checks the hash value or the signature of only the kernel extension before loading it in memory.

SIG_VER

Enables or disables the Runtime Signature Verification policy. When both the Runtime Signature Verification policy and the TE policy are enabled, the signature is used to verify the integrity of files instead of the hash value.

STOP_UNTRUSTD

Stops loading of files that are not trusted. Only files belonging to TSD are loaded. This policy only works in combination with any of the CHK* policies mentioned above. For example, if CHKEXEC=ON and STOP_UNTRUSTD=ON, then any executable binary that does not belong to TSD is blocked from execution.

STOP_ON_CHKFAIL

Stops loading of trusted files that fails the hash value check or the signature check. This policy also works in combination with CHK* policies. For example, if CHKSHLIBS=ON and STOP_ON_CHKFAIL=ON, then any shared library not belonging to the TSD is blocked from being loaded into memory for use.

TSD_LOCK

Locks TSD so it is not available for editing.

TSD_FILES_LOCK

Locks trusted files. This does not allow opening of trusted files in write mode.

TE

Enables/Disables Trusted Execution functionality. Only when this functionality is enabled, the above configuration policies are in effect.

Trusted Execution Path and Trusted Library Path
Trusted Execution Path (TEP) defines a list of directories that contain the trusted executables. Once TEP verification is enabled, the system loader allows only binaries in the specified paths to execute. Trusted Library Path (TLP) has the same functionality, except that it is used to define the directories that contain trusted libraries of the system.

Some extra notes for the above policy TSD_FILES_LOCK.

This is a very good policy to set, and when the runtime is enabled files, executables and scripts cannot be modified / or tampered with. But for some systems this must be allowed; one example is a NIM server when this policy is set to ON, it will stop some functions of NIM such as netboot.

For those severs you set this policy to EXVOL eg. TSD_FILES_LOCK=EXVOL

The following table gives the interaction between different CHK* policies and STOP* policies when enabled:

Note: read in the above table, if a script does not exists into the tsd.dat then it will not be checked! And normal execution is allowed.

Best practices for a start with runtime enabled:

Please, always practice on a test environment first, in case you break something in the worst case you have to reboot.

Before changing anything into the /etc/security/tsd/tsd.dat files or /etc/security/tsd/lib/lib.tsd.dat make a copy to a save directory.

It’s always a good practice to clone your rootvg with alt_disk_copy -Bd diskx.

Below a good starting point for some policy settings

TE=ON

SIG_VER=OFF

CHKEXEC=ON

CHKSHLIB=OFF

CHKSCRIPT=ON

CHKKERNEXT=OFF

STOP_UNTRUSTD=OFF

STOP_ON_CHKFAIL=ON

LOCK_KERN_POLICIES=OFF

TSD_FILES_LOCK=ON

TSD_LOCK=OFF

TEP=OFF

TLP=OFF

After turning on (trustchk -p te=on) you can verify youre executable and scripts or config files, for example /etc/inittab with trustchk -n /complete path/file/exec/script

If your syslog.conf is right configured you can monitor the runtime behavior, and check on errors

How to configure syslog or rsyslog I will not cover in this blog. But sending it to a central syslog server with the right triggers set on it you have an a central monitoring tool to monitor all your Lpars.

Below a sample rsyslog output of a tampered script

2024-01-22T13:43:53.363+01:00 test_host kernel: unix: Trusted Execution: pid=19661188, euid=1244, ruid=1244: Crypto hash verification failed: /somepath/bin/script_exe_te.sh

Adding your own sets to the tsd.dat

Now I have explained a bit how the runtime works, you maybe like to add your own set of executables and files to the tsd.dat database.

Note these step are only needed to add your own sets.

Prerequisites for this are: clic.rte filesets, a valid singed or self-singed certificate stored in /etc/security/certificates

trustchk command to add the files to the tsd.dat with your own certificate.

-1 Check clic.rte:

On AIX you need the fileset CryptoLite for C library (CLiC) and the kernel extentions loaded

lslpp -l |grep clic

  clic.rte.kernext          4.10.1.0  COMMITTED  CryptoLite for C Kernel

  clic.rte.lib              4.10.1.0  COMMITTED  CryptoLite for C Library

  clic.rte.kernext          4.10.1.0  COMMITTED  CryptoLite for C Kernel

  clic.rte.lib              4.10.1.0  COMMITTED  CryptoLite for C Library

check kernel extension loaded with:

This can be checked with:

# /usr/lib/methods/loadkclic -q

/usr/lib/drivers/crypto/clickext is loaded, ID = -1609080832

Version number is 5.0

If kernel extention is not loaded then run command:

/usr/lib/methods/loadkclic –l

-2 Before you can add and sign your set to the tsd.dat you have to create a valid certificate self-signed or an official signed certificate. For now I explain how to create an self-signed.

The next step is to create a certificate, in this example I create a self-singed cert

Please follow the following steps:

use the example script or create your own certificate, but make sure that you have a private and a n public cert in .der format.

Below a simple script to create a self-signed key:

#!/bin/ksh93

# set -x

# below is old for RSA key not so save anymore!

# openssl genrsa -out privkey.pem 2048

# openssl req -new -x509 -key privkey.pem -outform DER -out cert.der -days 7300

# openssl pkcs8 -inform PEM -in privkey.pem -topk8 -nocrypt -outform DER -out privkey.der

# below method for Elliptic Curve Keys

# openssl ecparam -list_curves

# generate a private key for a curve

openssl ecparam -name prime256v1 -genkey -noout -out private-key.pem

# generate corresponding public key

openssl ec -in private-key.pem -pubout -out public-key.pem

# create a self-signed certificate

openssl req -new -x509 -key private-key.pem -outform DER -out cert.der -days 7300

openssl pkcs8 -inform PEM -in private-key.pem -topk8 -nocrypt -outform DER -out privkey.der

# copy the cert.der to /etc/security/certificates

Now that you have a valid certificate you can add your own set to the /etc/security/tsd/tsd.dat

Please make a copy before you start of the current tsd.dat database.

Go to the directory where you saved both the cert.der and the private key privkey.der

Form there you can run:

trustchk -s privkey.der -v cert.der -a /user/local/example/test.ksh93

After this command you wil notice that the /etc/security/tsd.dat is changed.

You can have a look with view /etc/security/tsd/tsd.dat and lookup your stanza you just add.

Also if everything went well you, you can verify your action with:

trustchk -n /path/to/file/just/added/test.ksh93

Note: you do not have to disable the runtime if you add new sets to the tsd.dat.

Policies can be changed with:

trustchk -p policy_name for example:

trustchk -p TSD_FILES_LOCK=ON

trustchk -p STOP_ON_CHKFAIL=ON

To lean how your set is added to the tsd.dat you can simply open it with a editor such as vi 

vi /etc/security/tsd/tsd.dat

If TE runtime is enabled with the policy TSD_FILES_LOCK=ON you will notice that you cannot write the tsd.dat file because it’s locked by this policy.

You will get a write error -1.

To unlock it you have to disable either the policy or disable the runtime completely with trustchk -p te=off

for now this was part2, please let me know it this was interesting any comments are appreciated.

In the next part, I will explain how to store the databases on an LDAP server, and what are the benefits of this.