IBM Z Deep Learning Compiler / ONNX-MLIR for Linux on IBM Z and LinuxONE
Overview
Today we are excited to announce that the IBM Z Deep Learning Compiler (IBM zDLC) / ONNX-MLIR image is now available on the IBM Z and LinuxONE Container Image Registry as onnx-mlir!
In earlier blogs, we've wrote about how IBM is leveraging the open source ONNX AI format to enable a “Build and Train Anywhere, Deploy on IBM Z” strategy. IBM zDLC is a powerful tool enabling your data scientists to develop deep learning models using the tools and frameworks they are already familiar with and deploy those models on Linux on IBM Z where your mission critical data lives.
By compiling models with IBM zDLC, your production workloads gain multiple benefits including: increased throughput, reduced latency, and increased security by not transferring data off-platform or needing to install extra AI frameworks, dependencies, or packages to your IBM zSystem.
We're also excited to share that IBM zDLC enables your existing models to quickly and easily take advantage of the new IBM z16 Telum processor's Integrated Accelerator for AI. By simply setting a flag at model compile time, IBM zDLC will compile models so ONNX Operators which support acceleration are sent to the accelerator automatically. The compilation handles all the data conversion and switching between CPU and Integrated Accelerator for AI based Operators.
Models can also be compiled to use only CPU based ONNX Operators. This brings all the advantages that don't require the Integrated Accelerator for AI to your existing Linux on IBM Z environments.
Background
Earlier blogs like Leveraging ONNX Models on IBM Z and LinuxONE and IBM Z and the Open Neural Network Exchange give a more detailed overview.
ONNX, a brief recap
ONNX stands for Open Neural Network Exchange. It's an open source vendor neutral format for saving AI models that can be exported or imported into many popular AI frameworks. The .onnx model files can then be served using runtimes like ONNX Runtime or compiled into shared library files with tools like IBM zDLC / ONNX-MLIR. Some frameworks such as PyTorch have direct support for ONNX files. For other frameworks, there are open source conversion tools available. A more complete listing of frameworks and converters is available at https://onnx.ai/supported-tools.html.
Model Usage
In addition to building the s390x onnx-mlir image in ICR, we've also created a detailed tutorial on compiling models using the image at
IBM Z Deep Learning Compiler Documentation and Examples for C/C++, Java, and Python using real deep learning models from the ONNX Model Zoo. ONNX-MLIR documentation is available from the open source community at
http://onnx.ai/onnx-mlir/.
Details are in the tutorial, but the general steps are as follows:
- Build and train the model using the platform and framework of your choice.
- Convert the model to .onnx format.
- For best results, convert the model to .onnx format on the same platform originally used to build the model. Once converted to .onnx files, the models can be easily transferred between machines with different endianness. However. the same is not true for the original files from all frameworks.
- Compile the model into a .so or .jar shared library file using IBM zDLC / onnx-mlir image from IBM ICR.
- Include the shared library file into your application.
- Use the ONNX-MLIR Runtime API to interact with the compiled model within your application.
Watson Machine Learning for z/OS
This blog focuses on the onnx-mlir image now available in IBM ICR. However the same features are available to our z/OS customers as well. IBM zDLC powers the ONNX model support available in the free trial Watson Machine Learning for z/OS Online Scoring Community Edition (OSCE) and Watson Machine Learning for z/OS since version 2.3. In WMLz 2.4, the included IBM zDLC in WMLz also supports the IBM z16 Integrated Accelerator for AI.
To learn more about WMLz see
https://www.ibm.com/products/machine-learning-for-zos and
https://www.ibm.com/downloads/cas/JVPO3P58
Meeting You Where You Need Us
IBM Z is a big ecosystem that spans several distributions of Linux on IBM Z, zCX and z/OS. Regardless of where you are and your deep learning needs are, IBM zDLC / ONNX-MLIR is available to you.
Linux on IBM Z
On the IBM Z and LinuxONE Container Registry, the onnx-mlir image is freely available and best suited for Linux on IBM Z cases. It is for those who want the utmost control of their model and inferencing. The entire `onnx-mlir` CLI is directly available from the image with all fine grain controls via CLI flags. The compiled model shared object files can be included in your applications and used with the serving environments of your choice. You are in total control.
z/OS Container Extensions (zCX)
IBM zDLC is available in both the free trial Watson Machine Learning for z/OS Online Scoring Community Edition as well as the full Watson Machine Learning for z/OS. The WMLz OSCE features a GUI interface for compiling models and REST interface for model serving. As a trial, it limits the number of models available at a time but makes it easy to deploy models and enables rapid prototyping on z/OS systems that have zCX available. Since WMLz OSCE runs in zCX, the running models are zIIP eligible.
z/OS Native
IBM zDLC is also available in Watson Machine Learning for z/OS. This includes the same features of WMLz OSCE without the limitations. It also includes additional features like high availability, scalable model serving, batch support, CICS support, and doesn't require zCX. Models compiled using Watson Machine Learning for z/OS are also zIIP eligible without zCX.
Conclusion
Whether you are just starting your AI journey or already an AI veteran, IBM Z Deep Learning Compiler and ONNX-MLIR makes deploying existing deep learning models to your IBM zSystems easier than ever. For IBM z16 customers, your existing models can take advantage of the IBM Integrated Accelerator for AI with just a compile setting. With the free onnx-mlir image in ICR or trial WMLz OSCE, you can prototype new AI-based workflows on IBM zSystems quickly. When ready for production, the models can be used with open-source serving tools or production-scale ready products like Watson Machine Learning for z/OS 2.4.
If you have questions on getting started with AI on IBM Z, reach out to us at aionz@us.ibm.com