Today, IBM is announcing that Red Hat OpenShift AI on IBM Z® and IBM® LinuxONE is generally available. This new offering is designed to deliver consistent and security-focused AI operations across the entire AI/ML lifecycle—from model development to deployment and monitoring. This integration represents a significant milestone in bringing enterprise-grade AI capabilities to where the world's most critical data resides.

With Red Hat OpenShift AI on IBM Z and LinuxONE, enterprises get:

• AI capabilities on the existing Red Hat OpenShift, allowing enterprises to benefit from inherent application platform capabilities such as simplified service management, monitoring and configuration management.
• An enterprise-grade AI and MLOps platform to deploy, serve and monitor AI models.
• A security-focused, reliable, and governed environment for AI developers and AI engineers to work with familiar open‑source tools.
• A cloud-like experience for AI engineers and data scientists with self-service data access and on-demand compute.
• Access to revolutionary hardware architecture – the IBM Telum® II processor paired with the IBM Spyre™ Accelerator – that enables predictive and generative AI capabilities on the mainframe.

Why does Red Hat OpenShift AI on IBM Z and LinuxONE matter?

Industry experts emphasize the importance of bringing AI closer to enterprise data.

“As AI expands beyond the training of large models to include inferencing in the enterprise, it is critical to bring AI closer to data. Red Hat OpenShift AI for IBM Z and LinuxONE achieves this by combining an open platform for developing and deploying enterprise applications with the enterprise IT infrastructure that hosts much of today’s mission‑critical data. Co‑locating AI models and applications with enterprise data on IBM Z or IBM LinuxONE can reduce latency and accelerate time to insights while improving security.”
— Matt Kimball, Vice President & Principal Analyst, Moor Insights & Strategy

For AI Developers: A Unified Development Experience

Red Hat OpenShift AI provides AI developers with an integrated platform featuring industry-standard tools and frameworks:​​

• Familiar development environment: Access select Jupyter notebooks, PyTorch, Kubeflow, Ray and other Kubernetes-native AI tools and frameworks directly on IBM Z and LinuxONE. The platform includes open-source libraries such as vLLM, KServe, Triton Inference Server for CPU and hardware accelerated AI inferencing model serving. It also supports Kueue, the job queuing system for AI/ML batch jobs on Kubernetes.
• Data proximity advantage: Run AI workloads close to enterprise data residing on IBM Z and LinuxONE, engineered to reduce costly and risky data movement. This co-location is designed to deliver low latency and high throughput.
• Streamlined lifecycle management: Leverage the dashboard provided by Red Hat OpenShift AI to create auditable AI pipelines for model deployment and model serving. Integration with DevOps pipelines is built to enable faster and more secure delivery of AI-enabled applications throughout the enterprise.​​

For Data Scientists: Security-forward, Scalable Compute at Your Fingertips

Data scientists gain access to a production-ready environment optimized for their workflows:​

• Accelerated performance: Harness the combined power of IBM Telum II's on-chip AI accelerator and Spyre's dedicated AI cores for both predictive AI and generative AI inference workloads.​
• Optimized infrastructure and data co-location:  Work directly with mainframe-resident data for advanced model serving without data movement, reducing security risks and latency. Utilize VMs and container consolidation on IBM Z and LinuxONE ​designed to reduce infrastructure cost including power consumption and reduce data center footprint.
• Responsible AI capabilities:  Support enterprise governance requirements with auditable, explainable workflows that help with model tracking, lineage, and auditability. Red Hat OpenShift AI supports TrustyAI for responsible AI development and deployment.

Hardware Acceleration for AI Workloads

The combination of Red Hat OpenShift AI, Telum and Spyre accelerators makes IBM Z and LinuxONE a preferred platform for enterprise AI:​

Telum II on-chip accelerator:  Seamlessly integrated into the processor, the Telum II AI accelerator enables consistent low-latency inference directly where transactions are processed, achieving quicker response times. This architecture eliminates the need for off-platform inference that can prioritize stringent compliance needs.​

Spyre PCIe accelerator:  Purpose-built for generative and agentic AI workloads, Spyre cards can be clustered (up to 48 cards in IBM z17) to scale AI capabilities as business needs evolve. This modular approach allows organizations to accelerate AI innovation.​

Conclusion

Red Hat OpenShift AI on IBM Z and LinuxONE brings AI capabilities to where the most critical data resides—removing the trade-off between innovation and security. For enterprises managing sensitive data on the mainframe, the value is clear: accelerate AI projects, reduce infrastructure costs, and maintain governance standards. Powered by the IBM Telum II processor and Spyre Accelerator, this platform is engineered to deliver faster response times and more efficient AI performance. The path towards security-focused, scalable enterprise AI starts now.

Watch our webinar to learn more about Red Hat OpenShift AI on IBM Z and LinuxONE and its new capabilities and features.

Learn more about Red Hat OpenShift here.