Instructions for safely getting non-concurrent crypto card code applied while running ICSF workloads.
Assumptions:
- All LPARs are running ICSF versions HCR77B1 or higher.
- For the crypto cards configured on your machine there is at least 2 of each type at your highest co-processor level*. This is necessary so that at least one crypto card will remain online to service the ICSF workloads while the other is being updated to the latest crypto card code.
Example of possible crypto card types:
- CCA- Normal Mode (default)
- CCA- PCI-HSM Compliance Mode
- EP11 Mode
- Accelerator mode – Since accelerator workloads can run on any CCA Mode card (normal or pci-hsm compliant), 2 Accelerators are not required as long as you have a CCA card remain online.
* highest co-processor level –
For example if you have Crypto Express 5, Crypto Express 6, and Crypto Express 7 cards you want to make sure you always leave at least a Crypto Express 7 online to service the running workloads.
- The bundle with the non-current code has already been applied and pending the crypto configure offline/online.
Procedure:
To get the non-concurrent crypto code applied safely while ICSF workloads are running, perform the following steps which require issuing commands from one of your ICSF LPARs:
- Display all the lpars and crypto cards for each machine in your plex by issuing:
'D ICSF,CARDS,SYSPLEX=Y'
- Decide which crypto card(s) you want to apply the code to first.
A good rule of thumb would be to choose 1/2 of your cards where the non-concurrent code update is needed. Be sure to split up the cards so that one of each type at the highest co-processor level stays online for the workloads to continue to run on.
See the assumptions section above for the various potential crypto card types and examples of co-processor levels.
- Deactivate the crypto cards from the ICSF LPARs.
Once you've decided which card(s) you want to apply the code update to first, deactivate those cards from all ICSF LPARs they are active on (ICSF workloads will be safely moved to the remaining active cards).
You can issue the deactivate command in one of 2 ways:
-
-
- Deactivate by card serial number:
Find the serial number of card(s) you want to deactivate (as shown in results of the ‘d icsf,cards,sysplex=y’ command) and issue:
'SETICSF DEACTIVATE,SN=(93AAA1EM, 93AABC7Y),SYSPLEX=Y'
OR
-
-
- Deactivate by index number:
Find the index number of the card(s) you want to deactivate (as shown in the results of the ‘d icsf,cards,sysplex=y’ command).
For example, in display results:
CRYPTO EXPRESS7 COPROCESSOR 7C06 SERIAL#=93AABC3H LEVEL=7.2.25z
the index number is 6.
Note that when issuing the deactivate by index number command you must be careful if you add the SYSPLEX=Y option as it will apply to all cards with that index across all of the machines in your sysplex.
To avoid deactivating crypto cards on other machines in the plex that have the same index number, it is suggested that you route the following deactivate by index command to each intended LPAR instead. For example,
'ro (SYSA,SYSB), SETICSF DEACTIVATE,INDEX=(0:5)'
to deactivate a range of cards, in this case 0-5, on LPARs SYSA & SYSB.
'ro (SYSA,SYSB), SETICSF DEACTIVATE,INDEX=(0,3,5)'
to deactivate cards 0,3 & 5 on LPARs SYSA & SYSB.
* See ICSF pubs for more info on command syntax:
https://www.ibm.com/support/knowledgecenter/SSLTBW_2.4.0/com.ibm.zos.v2r4.csfb200/seticsf.htm
- Confirm all cards/lpars have been Deactivated:
Issue the 'D ICSF,CARDS,SYSPLEX=Y' command until you see that all cards you intended to deactivate show the status they are deactivated to ALL LPARs*.
* Note that this could take a few minutes depending on the workloads running to get them safely moved to other active cards.
For example, in the following display result the 7C12 card is still Active on system S2E so we need to wait until the status displayed changes to Deactivated:
CRYPTO EXPRESS7 COPROCESSOR 7C12 SERIAL#=93AABCWC LEVEL=7.1.34z
S2B DOMAIN=000 Deactivated REQ=0039402473 ACT=0000
S2C DOMAIN=084 Deactivated REQ= N/A ACT=0000
S2D DOMAIN=066 Deactivated REQ=0130716197 ACT=0000
S2E DOMAIN=008 Active REQ=0053486429 ACT=0001
S2F DOMAIN=073 Deactivated REQ= N/A ACT=0000
- Apply the crypto card code update.
Once the cards are deactivated from all ICSF LPARs, you should issue the configure offline/online for those cards on the SE to get the new code applied.
You can confirm the code has been applied via ICSF by again issuing the
'D ICSF,CARDS,SYSPLEX=Y' command which will display the updated card code LEVEL*:
-D ICSF,CARDS,SYSPLEX=Y
CSFM668I 10.27.55 ICSF CARDS 861
CPC Name = T88 CPC Sequence# = 00000000000273A8
CRYPTO EXPRESS7 COPROCESSOR 7C04 SERIAL#=93AABD0E LEVEL=7.2.24z
S2B DOMAIN=000 Active REQ=0085383340 ACT=0002
S2C DOMAIN=084 Active REQ= N/A ACT=0001
S2D DOMAIN=066 Active REQ=0343151186 ACT=0004
S2E DOMAIN=008 Active REQ=0143347629 ACT=0002
S2F DOMAIN=073 Active REQ= N/A ACT=0001
CRYPTO EXPRESS7 COPROCESSOR 7P07 SERIAL#=93AABC7Y LEVEL=07.22
S2B DOMAIN=000 Active REQ=0075511651 ACT=0004
S2C DOMAIN=084 Active REQ= N/A ACT=0000
S2D DOMAIN=066 Active REQ=0000047248 ACT=0000
S2E DOMAIN=008 Active REQ=0016683309 ACT=0002
S2F DOMAIN=073 Active REQ= N/A ACT=0000
CRYPTO EXPRESS5 COPROCESSOR 5P08 SERIAL#=93AAA1EN LEVEL=07.11
S2B DOMAIN=000 Active REQ=0042904456 ACT=0003
S2C DOMAIN=084 Active REQ= N/A ACT=0008
S2D DOMAIN=066 Active REQ=0000010862 ACT=0000
S2E DOMAIN=008 Active REQ=0000049595 ACT=0000
S2F DOMAIN=073 Active REQ= N/A ACT=0000
* if the new level is not being shown on the display after the update, you may need to refresh the crypto cards in ICSF by going into ICSF option "1 COPROCESSOR MGMT - Management of Cryptographic Coprocessors" on your current LPAR and then re-issuing the above 'D ICSF,CARDS,SYSPLEX=Y' command.
* the 'D ICSF,CARDS,SYSPLEX=Y' command was updated to refresh the current card status in ICSF APARS:
-
-
-
-
-
OA61609 zArtemis explotation, ICSF 77D1 & 77D2
-
OA61803 zArtemis toleration, ICSF C0-D0
- Once the code has been applied to the cards and new levels confirmed, activate the cards back to ICSF.
To activate the cards you will issue the same commands you did for the deactivate in step 3 above, but change the ‘DEACTIVATE’ to ‘ACTIVATE’.
For example:
'SETICSF ACTIVATE,SN=(93AAA1EM, 93AABC7Y),SYSPLEX=Y'
OR
'ro (SYSA,SYSB), SETICSF ACTIVATE,INDEX=(0:5)'
- Verify all cards are active by issuing the 'D ICSF,CARDS,SYSPLEX=Y'
- Once all updated cards are active, repeat steps 1-7 above for the remaining crypto cards still needing the crypto update to be applied.