Cloud Pak for Business Automation

Cloud Pak for Business Automation

Come for answers. Stay for best practices. All we’re missing is you.

 View Only

CP4BA tuning recommendations for large workloads

By FRANK LU posted Wed April 16, 2025 10:43 AM

  

IBM Cloud Pak for Business Automation tuning recommendations for large workloads

This article is an introduction to admin users on what & how to tune to improve your CP4BA (Cloud Pak for Business Automation) performance.

Tuning is essential to improve performance. Unlike others, you will have a set of clear & simple instructions when you finish reading this article. As a starting point, let me brief my case study definition & result.

Large workloads use case:

  • OpenShift 4.14.27 - 3 master nodes and 37 worker nodes
  • Each of the 40 nodes possesses 16 cores of CPU, and 64Gb of memory
  • CP4BA 24.0.0 runtime was deployed to a single namespace with components: ADS, BAW, FNCM, and ODM. Additionally, EDB PostgreSQL database was used (instead of MongoDB).
  • 625 users were simulated and completed more than 36,000 operations within an hour (which is equivalent to 10 operations per second).  The operations are based on mortgage and navigator use cases.

“Tuned 24.0.0” verses “24.0.0” (out-of-box) :

  • the overall throughput increased by 1.2%
  • the average performance shows 14% improvement

Please note the improvements above are observed in the lab-controlled environment. The results may vary case to case.

To achieve this impressive performance result, I tuned the three layers below:

  1. OpenShift (OCP)
  2. Cloud Pak Foundational Services (CPFS)
  3. Cloud Pak for Business Automation (CP4BA)

1. OpenShift (OCP)

Recommendations to consider:

Node sizing

  • Each node should have sufficient CPU, memory, and storage to handle workloads without bottlenecks. Use multiple larger nodes to improve fault tolerance and load distribution.
  • Utilize persistent storage for stateful applications. Implement storage classes to definite different performance characteristics.

Network configuration

  • Evaluate different network plugins in a staging environment to determine the best fit for your workloads.
  • Utilize dedicated network interfaces for different traffic types. Implement Quality of Service (QoS) policies to prioritize critical traffic. Regular monitor network performance to address network issues early.
  • Router configuration to ensure router pods are correctly sized.
    • Ingress-controller via CLI
      • Get the number of ingress controller replicas:
        kubectl get IngressController default -n openshift-ingress-operator -o=jsonpath='Replicas: {.status.availableReplicas}{"\n"}'
      • Tune by increase the number of ingress controller replicas:
        kubectl patch IngressController default -n openshift-ingress-operator --type=json -p '[{ "op": "replace", "path": "/spec/replicas", "value": 5 }]'
      • Restart HAProxy
        systemctl daemon-reload
        systemctl restart haproxy
    • Or Ingress-controller via the OpenShift console
      • Select “Search” under Home.
      • Make sure the project name is openshift-ingress-operator.
      • Under Resources, select “IngressController”, then expand “IngressControllers”. Click on “default” and select YAML.
      • Search for “replicas” under spec:
      • Change replicas from default 2 to minimum of 5

    • HaProxy from OCP infra node
      • Edit /etc/haproxy/haproxy.cfg, and look for backend ingress-https
      • Change from balance source to balance roundrobin

      • Restart HAProxy
        • CLI: systemctl daemon-reload
        • CLI: system restart haproxy

Resource management

  • Regularly review and adjust resource requests and limits based on application performance. Utilize automation tools to dynamically adjust resources based on real-time usage patterns.

Quality of Service (QoS)

  • Identify critical and non-critical workloads, and assign appropriate QoS classes, ie. Guaranteed, Burstable, and Best Effort. Adjust resource allocations as needed.

Metrics and Logging

  • Set up alerts for critical workloads. Regularly review and analyze metrics and logs to identify performance risks. Automate actions based on insights.

Horizontal Pod Autoscaling (HPA)

  • Set appropriate scaling thresholds based on application performance metrics. Review and adjust HPA configurations regularly based on workload and usage patterns.

Vertical Pod Autoscaling (VPA)

  • Regularly review and adjust VPA configuration recommendations based on workload performance and usage patterns.

2. Cloud Pak Foundational Services (CPFS)

IBM Cloud Pak Foundational Services (CPFS) provide the platform and services for Cloud Paks families to build on. Cloud Pak for Business Automation (CP4BA) is no exception, which is built on top of CPFS. 

The foundational services include Platform UI (role-based UI), Common Web UI (Administration Hub), Identity Management (manage security & access), Business Team Service (manage global teams across business applications), Certificate Manager (manage certificate creation and usage), License Service (collect and measures license usage), and OpenSearch (Ingest, search, visualize, and analyze data). You may read more about CPFS from the last section on reference URL.

By optimizing IBM Cloud Pak Foundational Services, it will improve CP4BA stability, efficiency, and scalability. In this section, the specific tuning examples are applied in the large cluster from our case study. Please note tuning instructions can be varied depending on the cluster of your choice.

Note: CPFS version 4.6+ is bundled with CP4BA 24.0.0. We use the default CPFS in our case study.

Tuning Foundational services (common service)

In the OpenShift console, select “Search” under Home. Make sure the project name is where CP4BA was deployed. Under Resources, select “CommonService”, then expand “CommonServices”. Click on “common-service” and select YAML.

Or alternatively, oc edit CommonServices -o yaml through commandline.

Search for “size” under spec:

Note: spec size can be small, medium, or large. For further details, please follow: https://www.ibm.com/docs/en/cloud-paks/foundational-services/4.10?topic=options-configuring-foundational-services#hardware-profile

Tuning Zen Layer (Zen Service)

In the OpenShift console, select “Search” under Home. Make sure the project name is where CP4BA was deployed. Under Resources, select “ZenService”, then expand “ZenServices”. Click on “iaf-zen-cpdservice” and select YAML.

Or alternatively, oc edit ZenServices -o yaml through commandline.

Search for “scaleConfig” under spec:

Note: spec size can be small, medium, or large. For further details, please follow: https://www.ibm.com/docs/en/cloud-paks/foundational-services/4.10?topic=installer-hardware-requirements-recommendations-foundational-services

3. Cloud Pak for Business Automation (CP4BA)

Tuning Deployment profile size

In the OpenShift console, select “Search” under Home. Make sure the project name is where CP4BA was deployed. Under Resources, select “ICP4ACluster”, then expand “ICP4AClusters”. Click on “icp4adeploy” and select YAML.

Or alternatively, oc edit ICP4ACluster -o yaml through commandline.

Search for “sc_deployment_profile_size” under shared_configuration:

Note: profile size can be small, medium, or large. For further details, please follow: https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/24.0.0?topic=pcmppd-system-requirements

ADS Tuning

In the OpenShift console, select “Search” under Home. Make sure the project name is where CP4BA was deployed. Under Resources, select “ICP4ACluster”, then expand “ICP4AClusters”. Click on “icp4adeploy” and select YAML.

Or alternatively, oc edit ICP4ACluster -o yaml through commandline.

Search for “ads_configuration”, then set the deployment_profile_size value to large

Note: profile size can be small, medium, or large. For further details, please follow: https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/24.0.0?topic=pcmppd-system-requirements

Additional tuning details can be found in Reference section.

ODM Tuning

In the OpenShift console, select “Search” under Home. Make sure the project name is where CP4BA was deployed. Under Resources, select “ICP4ACluster”, then expand “ICP4AClusters”. Click on “icp4adeploy” and select YAML.

Or alternatively, oc edit ICP4ACluster -o yaml through commandline.

Search for “odm_configuration”, then set the deployment_profile_size value  to large

Note: profile size can be small, medium, or large. For further details, please follow: https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation/24.0.0?topic=pcmppd-system-requirements

Additional tuning details can be found in Reference section.

BAW Tuning

All BAW performance materials are updated regularly under workflow-performance GitHub.  The GitHub URLcan be found in the last section, Reference URL,  for further details. 

If you use WfPS ( Workflow Process Service), you can maximize allowed connections to PostgreSQL database by increasing max_connections from default 100 to 300. Remember to set the database to unmanaged through spec.database.managed.managementState in WfPSRuntime custom resource prior making changes to max_connections. 

 In the OpenShift console, select “Search” under Home. Make sure the project name is where CP4BA was deployed. Under Resources, select “Cluster”, then expand “Clusters”. Click on “postgres-cp4ba” and select YAML.

Or alternatively, oc edit Cluster -o yaml through commandline.

Search for “max_connections”, then set the value from default 100 to 300.

There are lots more tuning can be done on Workflow by following workflow-performance GitHub. Unlike t-shirt sizing, each would require a specific value. You will need run through numerous attempts and verifications, to fully fine tune your cluster. In this case study, I choose to leave the default values as the base for tuned performance benchmark

Key points:

OpenShift tuning is given as general recommendations to consider. Specific tuning would be varied depending on your cluster details.

IBM Cloud Pak tuning guide, in contrary, is quite straight forward with clear steps and values to implement. To follow through should not take more than an hour to complete. It is due to the t-shirt sizing (small, medium, large) simplifies the tuning effort. In return, a significant improvement in performance can be observed.

In this case study, a simplified tuning guide is introduced. More advanced tuning tips can be explored in future topics.

Bonus:

IBM automated observability and tuning solution can optimize your cluster easy.

IBM Instana offers more than traditional APM solutions. It provides automated observability with AI and the ability to democratize observability, making it accessible to anyone across DevOps, SRE, platform engineering, ITOps and development. 

    • Instana gives you 1-second granularity, which helps you quickly detect problems or transactions 
    • Additionally, you get 100% traces that allow you to fix issues easily 
    • Instana contextualizes data from all sources, including OpenTelemetry, to provide the insights needed to keep up with the pace of change
    • Instana gives you the data you want, with the context you need, to take intelligent action.

IBM Turbonomic is a software platform that helps organizations optimize the performance and cost of their IT infrastructure, including public, private and hybrid cloud environments. With Turbonomic, you can:

    • Continuously automate optimization actions in real-time without human intervention
    • Proactively deliver the most efficient compute, storage, and network resources across your stacks
    • Avoid over-provisioning resources to your cloud environment
    • Increase ROI and lower your cloud bill by using only what you need

Reference Links:

  1. IBM Cloud Pak for Business Automation Documentation https://www.ibm.com/docs/en/cloud-paks/cp-biz-automation
  2. IBM Cloud Pak foundational services https://www.ibm.com/docs/en/cloud-paks/foundational-services
  3. OpenShift Documentation https://docs.openshift.com
  4. IBM Cloud Pak foundational services 4.6 https://www.ibm.com/docs/en/cloud-paks/foundational-services/4.6?topic=about-whats-new
  5. Performance Check list of OCP for CP4BA 24.0.0 https://community.ibm.com/community/user/automation/blogs/johanne-sebaux/2024/07/12/performance-check-list-of-ocp-for-cp4ba-2400
  6. Workflow performance https://github.com/icp4a/workflow-performance
  7. CP4BA 24.0.0 ODM Tuning Guide https://community.ibm.com/community/user/automation/viewdocument/cp4ba-2400-odm-tuning-guid
  8. CP4BA 24.0.0 ADS Tuning Guide https://community.ibm.com/community/user/automation/viewdocument/automation-decision-services-2400
  9. IBM Instana https://www.ibm.com/products/instana
  10. IBM Turbonomic https://www.ibm.com/products/turbonomic

Credits

Special thanks to CP4BA performance teams!

Contributors: Kevin Trinh, Johanne Sebaux, Marvin Schaefer, Pierre Andre Paumelle

Editors: Kevin Trinh, Johanne Sebaux, Marvin Schaefer, Stephan Volz, Anthony Damiano, Maria Kovanko-Michalet, Lawrence Louie, Conor Murphy, Cristina Tecsa

Publisher: Frank Lu

0 comments
61 views

Permalink

https://community.ibm.com/community/user/blogs/frank-lu/2024/11/25/cp4ba-tuning-large-workloadds