WebSphere Application Server & Liberty

As explained in the following product manual, the admin agent uses the target application server's RSA public key for encryption:

RSA token authentication mechanism
https://www.ibm.com/docs/en/was/9.0.5?topic=mechanism-rsa-token-authentication

Below is a simplified sequence of events:

 1. The admin agent starts and runs.
 2. To manage a target application server, the admin agent retrieves the server's RSA public key using a "bootstrap" MBean request from the target.
 3. The admin agent caches the retrieved RSA public key of the target server (referred to as "target public").
 4. The admin agent uses the cached "target public" to encrypt outgoing messages.
 5. The target application server decrypts the message using its private key ("target private").

If the target server is stopped or in the process of starting when its public/private keys are renewed, the admin agent cannot retrieve the updated public key via the bootstrap MBean request. Instead, it continues to use the cached public key ("target public"). However, when the target server begins using the new private key, this discrepancy results in the JSAS0803E error.

Adminagent has a retry logic to deal with this discrepancy situation which was introduced as APAR PM66060.
https://www.ibm.com/support/pages/apar/PM66060

In the retry logic, adminagent clears the cache and retrieve the renewed one. However, as you might expect, there could be a situation where JSAS0803E is inevitable on the first attempt after the key renewal because the adminagent has not retrieved the target's renewed public key yet.
Therefore, if JSAS0803E occurs only a few times immediately after the target server's rsatoken-key.p12 file is updated, it can generally be considered ignorable, provided no other errors occur around the same time.