Today, we are excited to announce the release of IBM Z Deep Learning Compiler 5.1.0, which supports the new Telum II processor on IBM z17. IBM Z Deep Learning Compiler is enabling AI workloads with continued improvement to efficiency and performance on z17 for our direct users and several new IBM Products which are using IBM Z Deep Learning Compiler 5.1.0 in their latest/upcoming release.

• Watsonx-AI
• IBM SQL Data Insights Pro
• MLz
• AI Toolkit for IBM Z and LinuxOne’s IBM Z Accelerated for Triton Inference Server

The IBM Z Deep Learning Compiler 5.1.0 builds incrementally on top of the zDLC-5.0.0 version to enable a larger set of AI models for inference acceleration on IBM z17 using the Telum II processor, both on z/OS and IBM LinuxONE. 

Some salient features of this release include.

• Multiple new LLM Encoder family of models enabled for z17
• Transformer Text Encoders: BERT, RoBERTa (context/base), CamemBERT (French monolingual NER), EuroBERT (multilingual embeddings)
• Document Layout & Spatial Vision Encoders: Docling-layout-heron (RT-DETRv2), DocumentFigureClassifier-V2 (26-class figure enrichment)
• Multi-Engine / Hybrid Ensemble: IBM IOCR — with Word UNet V2, Angle Detector CLS+REG, SSOCR printed, and SSOCR eng_hw described as the five sub-models that compose the ensemble pipeline

The above new family of models are allowing customers to bring more complex and mature Operational and transactional AI use cases to the platform. This is possible due to the new model operation primitives added to the IBM Z Deep Learning Compiler 5.1.0, which enable Real-time in-transaction insights.

• Decoder Model Support on Telum II autoregressive causal language models (decoder-only). In general, any dense transformer-based causal LM that can be exported to ONNX is expected to work with the zDLC-5.1.0 on z17. Below is the list of models currently enabled but not limited to:
• OpenAI-community/gpt2       
• IBM-granite/granite-3.3-2b
• IBM -granite/granite-4.0-350M
• IBM -granite/granite-4.0-1b 
• meta-llama/Llama-3.2-1b

All the above models take advantage of the IBM z17 Telum II processor support for INT8 quantization, designed to reduce inference latency when compared to the non-quantized models. IBM Z Deep Learning Compiler 5.1.0 uses quantization support for z17, which allows smaller Decoder models to be run on the Telum processor, bringing models with billions of parameters to the inference use case for small context lengths and offline scenarios.

• New Configuration management option: A configuration file to instruct the zDLC compiler on how to compile a model where the users can provide below types of information, using a .json file.
  • Compiler Options Changes for NNPA:
    • --config-file: Now also applies to NNPA configuration (JSON file supports NNPA device placement and quantization settings with tensor-level matching).
  • New Configuration Features for NNPA:
    • JSON configuration file for device_placement and quantization: Supports per-tensor NNPA-specific settings (op placement, quantization parameters).
    • Global NNPA configuration object: New GenerateConfigFile pass to produce and consume NNPA config; cleaned-up JSON config handling across the pipeline.

IBM Z Deep Learning Compiler 5.1.0 is now available for download from the IBM Z and LinuxONE Container Image Registry. The documentation is now available at the IBM zDLC product page. New Credit Card Fraud Detection sample and LLM Code Examples with IBM Z Deep Learning Compiler(zDLC) usage have been added for users of zDLC, existing samples have been updated to use the latest open-source package levels.

For those interested in enterprise-level support for mission-critical workloads, IBM zDLC 5.1.0 is included in the AI Toolkit for IBM Z and IBM LinuxONE. IBM zDLC 5.1.0 will become available with the above-listed products in their upcoming release.

IBM zDLC uses semantic versioning. We've updated the version number to 5.1.0 to indicate that there are changes in this release, which means that at least some workflows will need modifications to work in the 5.1.0 release compared to the previous 5.0.0. The workflow changes are isolated to Python Runtimes support for Python 3.9. See " PyRuntime Support Update" for details. 

Changes in this release:

Major Feature Changes

• IBM z17 Telum II support with 19 new operators for z17 CPU & z17 NNPA
• Encoder and Decoder models families supported on NNPA z17
• Quantization support for encoder and decoder models on z17
• 2 New IBM product integration using zDLC-5.1.0
• Performance Improvements in support of encoder models on z17
• New compile-time options for both z17 CPU and z17 NNPA

Major Package Changes

• ONNX-MLIR 0.5.1.0
• ONNX 1.21.0
• IBM zDNN 1.1.2
• LLVM updates
• UBI-10 from UBI-9
• Java21 from Java 8
• Protobuf upgraded to v6.33.5
• Bazel-7.4.1
• Pybind11 was upgraded to version 2.12.0
•  Google Benchmark was upgraded to 1.8.

New Operators for z17

z17 CPU

16 New Operations Supported on z17 CPU:

New Linalg Lowering (ONNX → Linalg):

• MatMul (via new ONNXToLinalg conversion pass)
• Conv (basic 2D convolution)
• ONNXAdd
• Selective lowering via new --linalg-ops option

Optimizations/Features for CPU:

• Convolution: New optimizations for 1×1 convolutions and 2D convolutions using im2col; parallel convolutions recomposed into a single combined convolution pass.
• MatMul fusions: MatMul+Add fusion covering stacked/unstacked/bcast1/bcast23 patterns; MatMul+Mul fusion when Mul operand is a scalar constant; MatMul+Div fusion when Div's 2nd operand is a scalar constant.
• Canonicalization passes:
• Fuse consecutive Clip patterns.
• Move Relu/LeakyRelu before Split operations.
• Reorder Relu-to-MaxPool optimization pass.
• Merge nested Concat operations.
• Remove zero-size operands from ConcatOp.
• Detect and remove Slice-of-Concat patterns.
• LayerNorm/RMSNorm recomposition even when scale multiplication has multiple uses.
• New pattern: LayoutTransform → reshape-merge(squeeze) → LayoutTransform.
• Shape inference enhancements: Improved dynamic dimension analysis for Reshape (more general); shape inference extended for MatMul and Gemm (input dimension updates); special-case shape inference for onnx.Range; shape inference for RandomUniformLike; enhanced --shapeInformation option; ConstantOfShape default value fix.
• Profiling:
• Profiling data is now written into an in-memory buffer during inference, reducing I/O overhead during profiled runs.
• Instrumentation (krnl.InstrumentOp) updated with MemoryEffectsOpInterface to ensure correct interaction with optimization passes.
• Instrumentation node naming made consistent — all nodes now receive the same name during profiling as they carry in the original ONNX graph.
• Decoding: Python decoding algorithms added; support for appending decoding strategies (greedy/beam-search) into the input ONNX model; merged decoder ONNX model support in Python scripts.
• Runtime validation: `verifyInputTensors` option now enabled by default; extended to validate shape info provided via compile options; enhanced runtime checks for invalid tensor inputs.
• Instance and Group Norm: Now require shape inference before lowering.
• POW operator: Allows mixed input types (2nd input auto-cast to match 1st input type).
• Negative permute values: Now supported in layout/transpose transforms.
• Constant loading: Global constructors/destructors added for constant loading; on z/OS, switched from mmap to malloc for external constant files.
• Debug support: GDB/LLDB debug info embedded into compiled .so files; debug listing of operations that vanished between passes; IsolatePass improvements for listing def-use counts and constant scalars.
• Sub-expression elimination: New pass --onnx-cse-with-node-name for ONNX op common sub-expression removal.
• Compile info embedding: Compile options and metadata now embedded directly into the compiled .so file.
• AveragePool: New flag to force behavior consistent with PyTorch and ONNXRuntime.
• LayoutTransform: Extended to handle additional dimension collapse/expansion patterns.
• TopK: Performance-optimized implementation.
• GridSample: Performance optimization pass added on top of initial support.

Compiler Option Changes for CPU:

·        --march=native: New value to auto-detect and target the current host CPU architecture (also works on macOS).

·       --replace-op-with-its-operand: New option to replace a specified op with one of its operands (useful for debugging/ablation).

·       --printONNXBasicIR: New option to print ONNX basic IR during compilation.

·       --linalg-ops: New option for selective ONNX→Linalg lowering of specific ops.

·       --config-file: New compile flag to pass a JSON configuration file containing compile options (config file options applied first; CLI options take precedence).

·       --onnx-cse-with-node-name: New pass for sub-expression elimination scoped to ONNX node names.

·       verifyInputTensor: Now true by default (previously opt-in).

z17 NNPA

New Operations Supported on z17 NNPA:

Optimizations/ Features for NNPA:

·        Stick/Unstick decomposition: New algorithm for stick/unstick decomposition; enabled by default.

·       ZLow rewrite patterns: Remove unused zlow.stick and zlow.unstick operations; rewrite zlow.unstick → view → zlow.stick into zlow.reshape.

·       Elementwise fusion with Stick: Fuse elementwise computations directly with Stick/Unstick; normalization operations fused with stick.

·       Expand/Slice + Stick patterns: New rewrite patterns expand-slice and expand-stick.

·       fp32 data alignment: Stick/unstick fp32 data allocated at 4K page boundaries for performance.

·       Shape inference: ShapeInference now called before lowering ZHigh ops; lightweight DimAnalysis callable from ShapeHelper.

·       z17 CPU vs NNPA performance model: Added a performance model to guide op placement decisions between z17 CPU and NNPA.

·       Large tensor handling: Fixed issues in zdnnx for very large tensors.

·       Transpose-MatMul parallelism: Quantized matmul parallelized across multiple NNPAs in zdnnx.

·       NNPA Pool: Dynamic size disabling for correctness; output shape conditions for Conv on NNPA relaxed.

PyRuntime Support Update

With Python 3.9 now past end-of-life support, IBM zDLC is dropping support for PyRuntimes for these versions of Python. However, if you are using Python version 3.10 and later, the PyRuntimes can still be copied from the zDLC container image.

If you have questions on getting started with AI on IBM Z, refer to the AI on IBM Z 101, or reach out to us at aionz@us.ibm.com, and for getting started with the IBM zDLC, use the official product page.