KVM Secure -ap passthrough :

Abstract :

This white paper is to demonstrate the working of the secure -ap passthrough in a secure environment  .
The explanatory content inculcate design , architecture practical use cases and working of the secure-ap passthrough in a secure environment with the working of the crypto workloads on the secure guest .

It also unfolds and illuminate certain features of secure-ap passthrough .

Introduction :

IBM secure execution  for Linux is designed to serve the idea of having a isolation of workloads at granularity to scale  help and protect them from the internal and external threats .

 Data breaches are a serious threat and keeping confidential information secure is a fundamental priority for leading enterprises around the world. The exploitation of sensitive data by internal and external threats can result in large financial penalties, regulatory scrutiny, and company discharges. An approach with hardware-based access controls and workload isolation can help give enterprises confidence that their data will be less vulnerable to exploitation by insider threats or external parties than with traditional software-based approaches.

IBM Secure Execution encrypts the kernel image, the initial RAM file system, and the kernel parameter line. You are responsible for the application data encryption and its associated key management.

Crypto Express hardware adapters are divided into multiple domains, also called cryptographic domains or AP domains. 

Each domain acts as an independent cryptographic device with its own state, including its own master key. Two domains in the same Crypto Express adapter are completely isolated and cannot access each other's states. The maximum number of domains depends on the mainframe model and is the same for all Crypto Express adapters in that mainframe.

In Linux, virtual cryptographic devices are called AP queues. The name of an AP queue consists of two parts, the adapter ID and the domain ID, both in hexadecimal notation. For example, if cryptographic adapters with the IDs 00 and 02 are selected, and the domain IDs 0002, 0003 and 0005 have been configured on the cryptographic adapter, then the following AP queues are defined to Linux:

/sys/devices/ap/card00/00.0002

/sys/devices/ap/card00/00.0003

/sys/devices/ap/card00/00.0005

/sys/devices/ap/card02/02.0002

/sys/devices/ap/card02/02.0003

/sys/devices/ap/card02/02.0005

KVM guests that run in IBM Secure Execution mode on IBM z16 with firmware bundleS30 can use Crypto Express domains. A maximum of 12 AP queues per secure guest can be configured.

A secure Execution guest can now use crypto express adapters in EP11 or accelerator mode but not with CCA .

Submitting a secret to ultravisor :

An IBM Secure Execution guest can submit a secret to the ultravisor. The secret must be contained in an add-secret request. The add-secret request is protected with the help of the host key and a random Elliptic Curve Diffie–Hellman (ECDH) key pair. Different types of secrets can be submitted, as of now only rhel9.4,sles15.6 and rhel8.10 with null-secrets and association secrets are supported. Inserting a secret into the ultravisor of a guest is a prerequisite for associating an AP queue with the guest.

Feature details :

A Secure Execution guest can now use Crypto Express adapters in EP11 or accelerator mode. To protect secure keys, secure keys can be bound to an association secret maintained by the ultravisor on behalf of the secure guest. Therefore, you can now insert secrets. To provide structured and enhanced security .

Each AP queue must be bound to and associated with an association key .

Prerequisites :

You need a secure-execution boot image that supports the insertion of secrets into the ultravisor.

To create a secure guest  securing a guest .

The need of appropriate firmware bundles is a must without it , it’s not feasible to facilitate the functionality , Secure AP passthrough after  z16  firmware bundle 31b or later.

We use KVM host as trusted host .

Supported matrix:

Inserting a association secret is done by a utility called pvsecret .An association secret is a 32-byte value that can be referred to by a 32-byte identifier. If not specified, the command generates a random secret. The identifier is an SHA-256 hash of a string that you specify.

Setup crypto and create secure guest(s)

·       Ensure the KVM host is enabled with crypto devices.

·       Create secure guest(s)

·       Free the AP queues so they are available to the guest.

·       Create and start a mediated device for each guest.

·       Add mediated device to each guest XML.

·       Assign crypto adapters and domains to the guest(s)

Procedure :

1.)    on the guest extract the secure header file .

sudo pvextract-hdr -o guest-new-hdr.bin /boot/secure-linux1

and copy it back to the trusted host .

2.)On a trusted system, generate an add-secret request. The request is written to a file that you specify. Issue a command of the following form:

 On the host run the below command :

trusted]# pvsecret create -k <host_key_document> --hdr <secure_exe_header> -o <request_file> -C <CA_certificate> -C <IBM_signing_certificate> association <string>

For example, to use a host-key document z16.crt, a guest header se.hdr, a CA certificateDigiCert.crt, and an IBM signing key ibm-sign.crt, issue the following command on a trusted system Resource link.

Secure Execution Header for the guest

See the “Extracting an IBM Secure Execution header” section in lx23se04 doc for details.

[trusted]# pvsecret create -k z16.crt --hdr se.hdr -o addSecretReq \-C DigiCertCA.crt -C ibm-sign.crt association "myConfidentialSecret."

The command creates an add-secret request and writes it to addSecretReq. It also creates an identifier for the request, consisting of a SHA-256 hash of the string "myConfidentialSecret".

3.) Copy the files to the guest

 On the KVM guest that is running in secure-execution mode, insert the secret. Issue a command of the following form:

[se_guest]# pvsecret add <request_file>

E.g.:

[se_guest]# pvsecret add addSecretReq.

The secret is added to the store of secrets in the ultravisor. You can list all stored secrets by issuing.

4.) Validate with pvsecret list

[root@localhost ~]# pvsecret list

Total number of secrets: 2

0 Association

d8449d31b2913fea45d62e15878df240068a2f199750550ac7dbf64bcd53b4f4

1 Association:

1be41bdc5558e0d4a87183edb3ade47c75adbb75fc36ec4aaa3aa46a47d01f7d

5.) on the guest bind the domains

• chzcrypt --se-bind 16.001
• chzcrypt --se-bind 17.001

Verify with lszcrypt -V that the domains are in “bound” state

[root@localhost ~]# lszcrypt -V

CARD.DOM TYPE  MODE        STATUS     REQUESTS  PENDING HWTYPE QDEPTH FUNCTIONS  DRIVER      SESTAT

--------------------------------------------------------------------------------------------------------

16       CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4card    -

16.001e  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue   bound

17       CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4card    -

17.001e  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue   bound

SE State can show the below following :

• usable - the AP queue can be used for cryptographic requests.

• bound - the AP queue is bound but not associated.

• unbound - the AP queue is unbound and must be bound to this secure-execution guest to use it.

• illicit - the AP queue is not available for this secure-execution guest.

Reference :lx23se04

6.) On the guest: Associate the secrets with the domains

• Use pvsecret list to find the secret ids.

• chzcrypt --se-associate 0 16.001

• chzcrypt --se-associate 0 17.001

Verify with lszcrypt -V that the domains are in “usable” state

[root@localhost ~]# lszcrypt -V

CARD.DOM TYPE  MODE        STATUS     REQUESTS  PENDING HWTYPE QDEPTH FUNCTIONS  DRIVER      SESTAT

--------------------------------------------------------------------------------------------------------

16       CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4card    -

16.001e  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue   usable

17       CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4card    -

17.001e  CEX8P EP11-Coproc online            0        0     14     08 -----XN-F- cex4queue   usable

8.)Running the crypto workload :

On the guest :

We need to execute the below following commands to run the crypto workload .

a.)Display token info

pkcsconf -t

b.) set the slot and pins  

pkcsconf -c 4 -P -S 87654321 -n 01234567.

c.)Running crypto workload:

cd /opt/crypto_workloads/EP11cryptopthread-acsp

./EP11cryptopthread-acsp -s 4 -p 01234567 -r 1 -t 5

Restrictions :

Adhere to the restriction imposed by current IBM Z  firmware, preventing the use of the same association secret with two AP queues of the same adapter. This restriction aims to prevent unexpected side effects that are associated with resetting an EP11 AP queue.

Use cases :

1.)This line item  fundamentally tries to facilitate the accomplishment a secured and isolated environment for the execution of workload and prohibiting the access with added security enhancements . It facilitates each virtual machine to run in a standalone and secured environment .

2.)Secure ap passthrough ensures identity , integrity and enhanced security .

3.)After successful binding an  AP queue, you can use it to send requests and receive replies for clear key cryptography. The AP queue is now exclusively available to the secure guest. Other operating systems, including that of the KVM host, cannot access the AP queue. However, the KVM host can, when maintaining its guests and their resources, reset the cryptographic resource. The guest recognizes this as any further requests are rejected.

Diagram :

Workflow for the secure-ap passthrough :

Troubleshooting :

1.)    We have observed the presence of illicit cards is going to hamper the overall execution and the lpar goes into a hung state , so its suggested to remove the illicit cards .

E.g. Chzcrypt -d <adapter-id>

2.)    Please make sure to have different secrets for each domains as same secret is going to create failures .

References :

1.)https://ibmdocs-test.dcs.ibm.com/docs/en/linuxonibm_zdocsreview_test?topic=execution-secure-workload

2.)https://ibmdocs-test.dcs.ibm.com/docs/en/linuxonibm_zdocsreview_test?topic=execution-crypto-express-adapters

3.)https://ibmdocs-test.dcs.ibm.com/docs/en/linuxonibm_zdocsreview_test?topic=mode-secure-usage