Cloud Pak for Business Automation

Cloud Pak for Business Automation

Come for answers. Stay for best practices. All we’re missing is you.

 View Only

How to consume CP4BA REST APIs server to server

By Jens Engelke posted Wed June 28, 2023 11:50 AM

  

IBM Cloud Pak for Business Automation (CP4BA) 21.0.3 introduced a new component in its architecture, referred to as "the Zen frontdoor". All HTTPS traffic is forced through this component, which therefore acts as a reverse proxy. In that position, it can centralize important aspects and behaviors of all Cloud Pak components, such as session management and authentication.

For browser users, this component is mostly opaque: An initial request to a user interface is processed by the frontdoor and redirected to Cloud Pak foundational service "Identity Manager" (IM) for authentication. Upon successful authentication, the frontdoor establishes a single session (tracked by cookies) and forwards the request to the relevant CP4BA component with an injected identity token.

Non-browser clients need to be registered at IM and follow one of the OAuth 2.0 flows and then exchange the IM token for a Zen token. Several flows are described at API access tokens in Cloud Pak for Automation 21.0.3. Product documentation was available for customers upgrading from User Management Services prior to 21.0.3.

This article demonstrates how non-browser clients can use the OAuth 2.0 authorization code flow and an additional API to obtain the required token for invoking CP4BA APIs.

Components in this tutorial
The (1) sample app is a JEE web app on a Liberty server. It is connected to the same (2) Identity Provider as (4) Cloud Pak foundational service "IM" - Keycloak in this sample. Ultimately, the custom app will send a request via the (3) Zen frontdoor to a (5) CP4BA component to invoke a REST API on behalf of the current end user.
The animation below demonstrates a seamless user experience that can be achieved in this approach:
  1. The user accesses an anonymous landing page of the custom app.
  2. The user selects a use case that involves an interaction of the custom app with a CP4BA API.
  3. The user is redirected to Keycloak for authentication. Note that this step establishes the session between the browser and the custom app.
  4. Upon successful authentication, the user accesses the protected page in the custom app. To serve all content for this page, the server needs to invoke a REST API in CP4BA, but there is no Zen token in the user's session yet. Therefore, the custom app immediately opens a new browser tab to obtain an access_token from Cloud Pak foundational service IM.
    1. Because Cloud Pak foundational service IM is configured with "preferred login option", the screen for selecting an authentication scheme is skipped and the browser is immediately forwarded to Keycloak for authentication.
    2. Because the browser already established a session with Keycloak, its authentication screen (password prompt) is skipped, too.
    3. Keycloak immediately redirects back to IM and IM immediately redirects back to the custom app's pre-registered callback URL to pass the authorization_code.
    4. Server side code in custom app reads the code from the callback request and invokes IM's token endpoint to exchange code for token. The same server-side code can use the IM access_token to invoke an API of the Zen frontdoor to obtain a Zen token for the same user. The Zen token is finally stored in the user's session.
  5. The sample app's callback URL uses browser side JavaScript to postMessage a notification to the original page before closing the additional tab.
  6. Upon receiving the postMessage notification, the sample app's protected page reloads to use the Zen token from its session to invoke a simple REST API and display the result on screen.
Dissecting this scenario, the following aspects can be considered in sequence:
  1. Optional: Expose a starter deployment's LDAP
  2. Optional: Set up Keycloak
  3. Connect Cloud Pak foundational service IM to Keycloak for authentication via OpenID Connect (OIDC)
    1. Optional: Configure preferred login option
  4. Set up Liberty to use Keycloak for authentication
  5. Sample app
    1. who-am-I - Liberty's view on the user
    2. Authorization_code flow: Redirecting the user to IM 
    3. Client-side code: opening and closing browser tab, notify
    4. Authorization_code flow: Exchanging code for tokens
    5. Zen: Exchange IM token for Zen token
    6. Invoke CP4BA API

1. Optional: Expose a starter deployment's LDAP

The starter deployment of IBM Cloud Pak for Business Automation provisions a set of sample users in an OpenLDAP pod in CP4BA's namespace. Using a starter pattern and its sample users can provide a jump start. You can review sample users (and passwords) in an OCP secret: icp4adeploy-openldap-customldifTo allow external access to this instance of OpenLDAP, you can create a k8s service of type LoadBalancer:

oc login --token=sha256~****--server=https://api.jefips.cp.fyre.ibm.com:6443
Logged into "https://api.jefips.cp.fyre.ibm.com:6443" as "clusteradmin" using the token provided.

oc project fips2301
Now using project "fips2301" on server "https://api.jefips.cp.fyre.ibm.com:6443".
 oc get deployment
NAME                                            READY   UP-TO-DATE   AVAILABLE   AGE
...
icp4adeploy-openldap-deploy                     1/1     1            1           39d
...

oc expose deployment icp4adeploy-openldap-deploy --type=LoadBalancer --name=ldap
service/ldap exposed

By default, random ports are assigned. For convenience, you can use oc edit service/ldap to edit the k8s service.

In my environment, all of the worker nodes are on a private network and only accessible via the infrastructure node of my cluster. There is an instance of HAProxy on the infrastructure node that can easily forward traffic after adding the required configuration and restarting the service using service haproxy reload:

frontend ldap_service_front
        mode            tcp
        bind            *:10389
        description     LDAP service
        option          tcpka
        default_backend ldap_service_back

backend ldap_service_back
        mode            tcp
        balance         leastconn
        option          ldap-check
        timeout server  10s
        timeout connect 1s
        server master0 10.22.26.103:30389
        server master1 10.22.26.108:30389
        server master2 10.22.27.109:30389
        server worker0 10.22.28.106:30389
        server worker1 10.22.29.90:30389
        server worker2 10.22.30.143:30389

The set up is easily tested using ldapsearch 

binduser=$(oc get secret ibm-dba-ums-ldap-secret -o yaml | yq .data.ldapUsername | base64 -d)
bindpass=$(oc get secret ibm-dba-ums-ldap-secret -o yaml | yq .data.ldapPassword | base64 -d)
ldapsearch -x -D $binduser -w $bindpass -H "ldap://api.jefips.cp.fyre.ibm.com:10389" -b "dc=example,dc=org" -b dc=example,dc=org uid=cp4admin
# extended LDIF
#
# LDAPv3
# base <dc=example,dc=org> with scope subtree
# filter: uid=cp4admin
# requesting: ALL
#

# cp4admin, example.org
dn: uid=cp4admin,dc=example,dc=org
uid: cp4admin
cn: cp4admin
...
mail: cp4admin@example.org

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

[top]

2. Optional: Set up Keycloak

Keycloak is a free open-source user management component - supporting standard protocols like SAML, OAuth 2.0 and OpenID Connect. A simple instance can be set up and connected to a database for persistence. For this article, a simple postgres container on the same machine will provide the database.

podman rm -f postgres
podman run -d \
  --name postgres \
  -e POSTGRES_USER=keycloakdb \
  -e POSTGRES_PASSWORD=passw0rd \
  -e PGDATA==/var/lib/postgresql/data/pgdata \
  -v $PWD/data:/var/lib/postgresql/data \
  -p 5432:5432 \
  postgres

Keycloak requires HTTPS connections for remote communication. Therefore, a new key and certificate are created:

openssl req -newkey rsa:2048 -nodes -keyout server.key.pem -x509 -days 3650 -out server.crt.pem

Key, certificate and the following configuration file are mounted into the container:

db=postgres
db-username=keycloakdb
db-password=passw0rd
db-url=jdbc:postgresql://unframed1.fyre.ibm.com/keycloakdb
metrics-enabled=true
https-certificate-file=/opt/keycloak/conf/server.crt.pem
https-certificate-key-file=/opt/keycloak/conf/server.key.pem
hostname=unframed1.fyre.ibm.com

The following script launches Keycloak:

podman stop keycloak
podman rm keycloak
podman run -d --name keycloak \
      -e KEYCLOAK_ADMIN=admin \
      -e KEYCLOAK_ADMIN_PASSWORD=password \
      -v $PWD/server.crt.pem:/opt/keycloak/conf/server.crt.pem \
      -v $PWD/server.key.pem:/opt/keycloak/conf/server.key.pem \
      -v $PWD/keycloak.conf:/opt/keycloak/conf/keycloak.conf \
      -p 8443:8443 quay.io/keycloak/keycloak start

Once Keycloak is up and running, the admin console can be accessed on the host machine at the exposed port: https://unframed1.fyre.ibm.com:8443/admin

Keycloak configuration is partitioned by "realm", hence the first step is to create a realm for this article: ocp-sample

LDAP can be added by selecting "User Federation" > "Add LDAP provider" and specifying all connection details as tested in Optional: Expose a starter deployment's LDAP. You can already sync users from LDAP to Keycloak using the Actions menu. 

Groups are not mapped automatically; hence you may want to add a group Mapper:

  

LDAP group details such as base DN or objectClasses must be configured to match your LDAP.

You can now also sync groups from LDAP to Keycloak using the Actions menu.
[top]

3. Connect Cloud Pak foundational service IM to Keycloak for authentication via OpenID Connect (OIDC)

In OpenID Connect, there is a one-time setup required to allow a reyling party (RP, "the client") to delegate authentication to an offering party (OP, "the server"): The RP is registered as a client at the OP and the same client credentials need to be configured in the RP for runtime connections to the OP. In this scenario, there are no special requirements, hence only the redirect_uri and client credentials must be configured in Keycloak:
The client_secret is available on the "Credentials" tab:
There is a subtlety about Keycloak's user management, which can cause confusion in this article's set up: All users are represented by a Keycloak specific UUID, however, CP4BA relies on finding the user by username in LDAP. The easiest way out here is to force the User Property "username" to be used for the sub claim in identity tokens:
Client Scopes > profile > Mappers > Add Mapper > By Configuration 
On the IM side, documentation mentions four specific OIDC providers and Keycloak is not among them. However, because Keycloak implements the OIDC discovery endpoint, it can be easily configured. A link is available in the Keycloak admin console under Realm Settings > Endpoints, e.g. https://unframed1.fyre.ibm.com:8443/realms/ocp-sample/.well-known/openid-configuration in my sample instance.
# from OC console's "copy login command"
token=... 
namespace=fips2301 # where CP4BA is installed

oc login --token=$token --server=https://api.jefips.cp.fyre.ibm.com:6443
oc project $namespace

apihost=https://$( oc get route -n $namespace cp-console -o jsonpath='{.spec.host}')
adminuser=$(oc get secret ibm-iam-bindinfo-platform-auth-idp-credentials -o yaml | yq .data.admin_username | base64 -d)
adminpass=$(oc get secret ibm-iam-bindinfo-platform-auth-idp-credentials -o yaml | yq .data.admin_password | base64 -d)

iamaccesstoken=$(curl -sk -X POST -H "Content-Type: application/x-www-form-urlencoded;charset=UTF-8" -d "grant_type=password&username=$adminuser&password=$adminpass&scope=openid" $apihost/idprovider/v1/auth/identitytoken | jq -r .access_token)

# GET current list of IDP configurations
curl -sk "$apihost/idprovider/v3/auth/idsource" -H "Authorization: Bearer $ACCESS_TOKEN"  | jq

# POST a new config to connect IAM to keycloak
# Mapping sub is not yet supported, hence I forced the username property into the claim in keycloak

curl -skX POST "$apihost/idprovider/v3/auth/idsource" \
 -H "Authorization: Bearer $ACCESS_TOKEN" \
 -H 'Content-Type: application/json'  \
--data-raw \
'{
      "name": "keycloak",
      "description": "keycloak OIDC",
      "protocol": "oidc",
      "type": "default",
      "idp_config": {
        "client_id": "cloudpak-iam",
        "client_secret": "0AM8tNGk9jTL6WH4LPOX1HzWkkALkyv6",
        "discovery_url": "https://unframed1.fyre.ibm.com:8443/realms/ocp-sample/.well-known/openid-configuration",
        "token_attribute_mappings": {
          "sub": "preferred_username",
          "given_name": "given_name",
          "family_name": "family_name",
          "groups": "groupIds",
          "email": "email",
          "uniqueSecurityName": "preferred_username",
          "preferred_username": "preferred_username",
          "displayName": "preferred_username"
        }
      },
      "jit": true
 }'

Documentation suggests creating a Zen group and assign a role to it to ensure that users of the identity provider can be onboarded automatically. However, in a starter deployment, all users are already known to Zen and this step can be skipped.

Some configuration takes effect after recycling IM and Zen pods:

pods=$(oc get pod --no-headers=true --output=custom-columns=NAME:.metadata.name  -l component=usermgmt)
for pod in ${pods[@]};do
  oc delete pod $pod
done

pods=$(oc get pod --no-headers=true --output=custom-columns=NAME:.metadata.name  -l 'app in (platform-identity-provider, platform-identity-management, platform-auth-service)')

for pod in ${pods[@]};do
  oc delete pod $pod
done

[top]

3.1. Optional: Configure preferred login option

With the configuration in place so far Cloud Pak foundational services will always present a screen to select your login option, such as Enterprise LDAP, OpenID Connect or others.

In an environment, where all your users always use the identity provider for authentication, you can set the preferred login options.

oc patch cm platform-auth-idp -p '{"data": {"PREFERRED_LOGIN": "OIDC"}}'

The change should take effect immediately and skip the login option selection in future login attempts.

[top]

4. Set up Liberty to use Keycloak for authentication

Just as Cloud Pak foundational service IM, we must register Liberty as a OIDC RP of Keycloak. In the Keycloak admin console in realm ocp-sample, create a new client and "clientapp" and specify a redirect URI following step 2 of Configuring an OpenID Connect Client in Liberty , e.g. https://localhost:9443/oidcclient/redirect/keycloaksample.

The liberty server will also require a truststore with certificate of all the systems it connects to: Keycloak, IM, and Zen. In a development set up that builds the sample app using gradle, the Liberty server "defaultServer" was located in build/wlp. With that, the following script can create a keystore, obtain the signer certificates from remote machines and import them into the same keystore. Copying the keystore to a different location was only required in the gradle set up:

keystore=build/wlp/usr/servers/defaultServer/resources/security/key.p12
rm $keystore
rm src/main/liberty/config/resources/security/key.p12
build/wlp/bin/securityUtility createSSLCertificate --server=defaultserver --password=passw0rd
keytool -printcert -sslserver unframed1.fyre.ibm.com:8443 -rfc > keycloak.pem
keytool -printcert -sslserver cp-console-fips2301.apps.jefips.cp.fyre.ibm.com:443 -rfc > iam.pem
keytool -printcert -sslserver cpd-fips2301.apps.jefips.cp.fyre.ibm.com:443 -rfc > zen.pem
keytool -import -keystore $keystore -storepass passw0rd -file keycloak.pem -alias keycloak -noprompt
keytool -import -keystore $keystore -storepass passw0rd -file iam.pem -alias iam -noprompt
keytool -import -keystore $keystore -storepass passw0rd -file zen.pem -alias zen -noprompt
cp $keystore src/main/liberty/config/resources/security/key.p12

The final Liberty configuration

  • configures features for the sample app (webProfile-8.0, passwordUtilities1.0, jca-1.7 allowing to read credentials from server config)
  • configures the openidConnectClient-1.0 feature for interacting with Keycloak
  • configures a httpEndpoint to listen to port 9443 on all hostnames 
  • lines 12-19: configures Liberty as a client of Keycloak. The most important item is the discoveryEndpointUrl to download most of the options of the Keycloak server itself. In addition to that, Liberty needs client specific settings, such as clientId and clientSecret.
    • The audiences claim is set by Keycloak to restrict the set of servers that accept the token.
    • The realmName and userIdentifier settings are residuals from experiments to find the best configuration matching the token contents.
    • The signatureAlgorithm claim tells Liberty to obtain a public key from Keycloak's JWK endpoint to verify identity tokens.
  • lines 23-29: configures the web application with a set of protected URLs that require authentication.
  • line 31: configures an authentication alias with the client_id and client_secret to be used by the sample app interacting with Cloud Pak foundational service IM, not Keycloak. The application will use this configuration to exchange the authorization code for tokens.
With this configuration in place, a user who attempts to access a protected URL of the sample app is redirected to Keycloak for authentication.

[top]

5. Sample app

The clientapp is available in the library of this community. It is not fully self-contained as it assumes a sample OCP environment and Keycloak, but it can help you getting started.

unzip clientapp.zip
rm clientapp.zip
chmod +x prepare-certs.sh gradlew
./gradlew build deploy && ./prepare-certs.sh && ./gradlew libertyStart

The sample app implements three use cases to allow walking through the sequence more easily. All serious Java code is collected in com.ibm.test.tokensample.clientapp.TokenHelper for easier editing and reuse.

index.html is the anonymous landing page that allows navigating to dedicated pages for the three use cases. 

src/main/liberty/config is the copied by the liberty gradle plugin into a dev server. build.gradle mostly follows the tutorial Building a web application with Gradle

[top]

5.1 who-am-I - Liberty's view on the user

This page is not actually related to interacting with CP4BA, however, it is a great utility to learn about Keycloak tokens and Liberty APIs for accessing the security context. The three methods in use are

<h1>Welcome <%=TokenHelper.getCurrentUsername(request)%></h1>
This is just a wrapper around the JEE standard method for obtaining a username from context: request.getUserPrincipal().getName()
<% for (String key : TokenHelper.getIdTokenClaims().keySet()) {  %>
The getIdTokenClaims() methods extracts the id_token, which was obtained during OIDC authorization code flow from Keycloak, from the security context and provides all claims of the token in a Map.
<%=TokenHelper.getSecurityContext().toString() %>
This is a wrapper around a Liberty API to obtain the current user's security context: WSSubject.getRunAsSubject() , see javadoc.

[top]

5.2 Authorization_code flow: Redirecting the user to IM

The following sections of this article deal with the same code that is either invoked "lazily" (by the click of a button) by cp4ba-on-demand.jsp or "eagerly" (automatically) by cp4ba-client.jsp.

When you go through the spec of the OAuth 2.0 authorization code flow, the first step is to redirect the browser to the /authorize endpoint of the OAuth 2.0 authorization server (Cloud Pak foundational service IM in this case). A number of parameters have to be passed by the browser with redirect.

The sample app obtains the IM URL prefix from configuration in web.xml and appends the parameters as follows:

    public static final String calculateIamAuthorizeUrl(HttpServletRequest request) throws Exception {
        final String METHOD_NAME = "calculateIamAuthorizeUrl";
        LOG.entering(CLASS_NAME, METHOD_NAME);

        // 1. calculate random state and store it in session for later comparision
        String state = UUID.randomUUID().toString();
        request.getSession().removeAttribute("state");
        request.getSession().setAttribute("state", state);

        StringBuilder authorizeUrl = new StringBuilder(request.getServletContext().getInitParameter("iam-urlprefix"));
        authorizeUrl.append("/idprovider/v1/auth/authorize");
        authorizeUrl.append("?response_type=code");
        authorizeUrl.append("&client_id=").append(getIamClientId(request));
        authorizeUrl.append("&scope=openid");
        authorizeUrl.append("&state=").append(state);
        authorizeUrl.append("&redirect_uri=").append(
                URLEncoder.encode(calculateRedirectUri(request), java.nio.charset.StandardCharsets.UTF_8.toString()));

        LOG.exiting(CLASS_NAME, METHOD_NAME, authorizeUrl.toString());
        return authorizeUrl.toString();
    }

The redirect_uri parameter is must be a pre-registered value as described in Configuring an OpenID Connect Client in Liberty step 2. It is the callback URL implemented by token-consumer.jsp.

The state parameter allows a client to verify that that a callback received at the redirect_uri is actually a "response" to a request for authentication that the application has initiated. It can also help single page applications to restore state after redirecting back and force. The value should be random and verified upon receiving the callback, hence it is stored in the session by this sample app.

The resulting URL may look like this sample: https://cp-console-fips2301.apps.jefips.cp.fyre.ibm.com/idprovider/v1/auth/authorize?response_type=code&client_id=qp4kwb51tg31f5zjd33bq4p16gp9nnf2&scope=openid&state=c2e67e40-17d7-4095-a448-1a8f73e75bae&redirect_uri=https%3A%2F%2Flocalhost%3A9443%2Fclientapp%2Fprotected%2Ftoken-consumer.jsp 

The task at hand is now to let the browser visit that URL and ultimately come back to the pre-registered and provided redirect_uri. There are two common patterns to achieve that:

  1. You may choose to store relevant navigational information (the page that the user actually attempted to go to) in the user's session, a cookie or a dedicated cache using the state parameter value as key. You can then use the current browser tab to visit the authorize endpoint by sending a 302 redirect from the server or using window.location from client side script code. Upon receiving the response, retrieving and storing all relevant tokens in the user's session, your token consumer code must redirect the user to the target page.
  2. You may choose to keep all state of your current browser tab and open the authorize URL in a dedicated tab. This approach is detailed in the following section. Upon receiving the response, retrieving and storing all relevant tokens in the user's session, your client side code must close the additional tab and notify the originating tab about the successful completion of the authorization code flow.

[top]

5.3 Client-side code: opening and closing browser tab, notify

You can open another browser tab using the following client side JavaScript:

window.open("<%=TokenHelper.calculateIamAuthorizeUrl(request)%>", '_blank').focus();

This approach works well upon user interaction (e.g. the click of a button). However, for security reasons, browsers may block this automatic tab opening without user interaction unless explicitly enabled by the user (or policy).

The token-consumer page later needs to close its own tab, which work without any special permissions in my observation:

window.close();

If you choose the multi-tab approach, you need to somehow communicate the presence of required tokens in the user's session back to the opening page. The browser feature for that is postMessage().

The sending page can send an event which includes the real payload as well as an origin (typically, the server's URL prefix):

window.opener.postMessage("zen token available", "https://localhost:9443");

The receiving page must register an event handler to consume the event:

//register an event listener, so that the token-consumer.jsp page can notify once a token is available in the session.
function init() {
	console.log("registering event listener");
	window.addEventListener("message",(event) => {
	if (event.origin !== "https://localhost:9443") {
		console.log("received event for " + event.origin + " ... ignoring." );
		return;
	} 
	console.log("event received: " + event.data);
	window.location.reload();
	}, false);
}

The sample code above ignores events in case the origin is unexpected. For an expected event, the page just reloads using  window.location.reload(); without even looking at the event's payload. In a single page application, you may choose to instead set some state that enables buttons and use cases, which require the CP4BA token in the session.

[top]

5.4 Authorization_code flow: Exchanging code for tokens

As defined in OAuth 2.0 authorization code flow, there is a callback to the pre-registered and provided redirect_uri - https://localhost:9443/clientapp/protected/token-consumer.jsp in the sample. The server-side code, which serves this URL can now validate the state parameter, extract the code from a request parameter and call out to IM's token endpoint to exchange code for token.

You can of course use your favorite REST client framework for this task, but the sample app avoids introducing additional dependencies. There is also no need to store all this debugging information in the session, but it may help following the flow during your own implementation.

[top]

5.5 Zen: Exchange IM token for Zen token

There is a Zen REST API to exchange IM tokens for Zen tokens. It expects the username and a matching IM token to be passed as request headers and returns a Zen token in response.

Again, use the REST client framework of your choice and the only data you actually want to store in the session is the Zen token itself.

[top]

5.6 Invoke CP4BA API

Now, with the required token in place, the final step is to invoke a CP4BA REST API and pass the token as in a HTTP request header Authorization: Bearer <token>.

[top]

0 comments
76 views

Permalink