Introduction
IBM Orchestration Pipelines is now supported on IBM Power servers. This blog helps users understand how to manage and automate their data and flow of AI assets on IBM Power by creating, deploying, and managing end-to-end data integration pipelines and AI/ML workflows to generate automated, consistent, and repeatable outcomes.
Capabilities
With the 5.2.1 release, the Orchestration Pipelines service is now fully supported on the IBM Power platform (ppc64le). This enhancement allows you to seamlessly create, modify, test, and execute end-to-end data and AI workflows directly on IBM Power by leveraging the platform’s strong performance and reliability.
This service now brings support for Orchestration Pipelines on IBM Power, expanding the range of supported architectures and enhancing cross-platform compatibility. It also enables optimization of hybrid cloud infrastructure by leveraging platform-specific benefits while maintaining the use of trusted IBM services.
Prerequisites
This blog assumes that users are already familiar with:
- Deploying and running other services such as IBM SPSS Modeler, IBM AutoAI, IBM DataStage, and Red Hat OpenShift on IBM Power
- Using the Orchestration Pipelines service to assemble and configure a pipeline
Environment
The Orchestration Pipelines service on Power has been validated in the following environments:
- IBM Software Hub version 5.2.1
- Supported platform: IBM Power (ppc64le)
- Deployment target: Red Hat OpenShift 4.x on PowerVM
Orchestration Pipelines on Power
Orchestration Pipelines can be used to create, modify, test, and run an end-to-end flow to achieve the following goals:
- Ingest and process data securely from a variety of internal and external sources.
- Achieve desired outcomes by cohesively building, executing, evaluating, and deploying models or by running scripts.
- Simplify the flow execution by creating branches and using visual tools to gather and interpret results.
You can design a pipeline by dragging nodes onto the canvas, specifying objects and parameters, then running and monitoring the pipeline. Pipelines can run activities including but not limited to:
- AutoAI experiments
- Jupyter Notebook jobs
- Data Refinery jobs
- SPSS Modeler jobs
- IBM DataStage jobs (also known as sequence jobs)
This service can also be used to automate the process of running tasks to ingest, transform, and save data using IBM DataStage. With the support for IBM Power, this unlocks several important possibilities:
- It enables a consistent data engineering experience across multiple architectures.
- With IBM Power, organizations can leverage their existing infrastructure and skills to advance AI and analytics projects.
- This enhancement enables tighter integration with other IBM Software Hub services that are already supported on Power systems.
- Traditional DataStage sequence jobs can be seamlessly migrated to Orchestration Pipelines, allowing them to be managed and visualized alongside newer, cloud-native data processes.
As part of this release, the Orchestration Pipelines service on Power is functionally equivalent to x86. There are no changes required in how you deploy or manage the service on Power. If you're already familiar with enabling IBM Software Hub services on x86, the process is identical for Power.
Installation process
To enable the Orchestration Pipelines service on a Power cluster:
- Install or upgrade to IBM Software Hub version 5.2.1 on your OpenShift cluster running on Power.
- Navigate to the IBM Software Hub web console, go to the Services/Services catalog, and enable Orchestration Pipelines.
- Follow the standard installation steps, referring to the IBM Software Hub version 5.2.x documentation and use installation flows, which are now validated for Power.
For installing Orchestration Pipelines on IBM® Software Hub version 5.2, refer to https://www.ibm.com/docs/en/software-hub/5.2.x?topic=pipelines-installing
Image and registry support
Orchestration Pipelines images are now available for the ppc64le architecture and are automatically retrieved by the Orchestration Pipelines operator during installation, eliminating the need for manual handling of architecture-specific image sources.
Architecture advantage
IBM Power is designed for data-intensive workloads. With high memory bandwidth, simultaneous multithreading (SMT), and enterprise reliability, Power offers a compelling platform for running long-lived, high-throughput extract, transform, and load (ETL) jobs.
Key advantages include:
- Better throughput for parallel ETL pipelines
- Drive workload consolidation by leveraging lower infrastructure footprint
- Seamless integration with Power-based AI and analytics stacks
Summary
With the 5.2.1 release, the Orchestration Pipelines service is now available on IBM Power (ppc64le), further enhancing IBM’s multi-architecture support. This addition enables seamless deployment and workload portability between x86 and Power platforms, simplifying operations and supporting the modernization of data and AI environments.
By running Orchestration Pipelines on IBM Power, you can:
- Leverage your existing Power infrastructure.
- Manage and automate your data and AI lifecycle in ppc64le architecture.
- Maintain architectural consistency across hybrid and multi-cloud environments.
Call to action (CTA)