Your Db2 for z/OS Agent just got more flexible

Most updates add features. This one gives you more control.

In the latest update to the Db2 for z/OS agent, we’ve changed how the agent connects to AI models. There’s no new feature to explore—but you now have more flexibility in where inference runs and how the agent integrates with your AI stack.

The problem we’re solving

Until now, the agent worked with a defined set of AI runtimes. That worked well—but it also meant:

• You were limited to the runtimes the agent natively supported

• Integrating alternative inference stacks required extra effort

• Adapting to new enterprise AI tooling wasn’t always straightforward

That approach works—until the ecosystem starts moving faster than your integrations.

TL/DR: Up until now, the agent used a "Custom Plug." You could only connect to specific, pre-supported AI runtimes. If you wanted to use a new tool, you had to wait for a new adapter. 

i.e, The AI ecosystem moves faster than manual integrations. Relying on "fixed" connections creates bottlenecks and vendor lock-in.

What changed: a more open way to connect to AI

We’ve introduced a new runtime option based on the OpenAI protocol, allowing the agent to connect to any endpoint that follows the OpenAI API standard.

That includes:

• AI Optimizer

• Internal enterprise AI gateways

• Other OpenAI-compatible inference services

Just as importantly:

• Existing runtimes (on-prem, cloud) continue to work as before

• This is an addition, not a replacement

TL/DR: We’ve replaced the custom plug with a Universal AI Adapter. By supporting the OpenAI Protocol, the agent can now "plug into" any AI endpoint, gateway, or orchestrator that follows this global standard.

We’ve moved to a protocol-driven architecture. By supporting the OpenAI standard, the agent can now connect to almost any AI tool or gateway instantly.

Wait, what’s an AI Optimizer? Think of the Db2 for z/OS Agent as the "Data Provider" and the LLM as the "Brain." An AI Optimizer acts as the "Smart Middleman Orchestrator" that sits between them to make the whole interaction cheaper, faster, and more reliable. 

i.e, By moving to a Universal AI Adapter (the OpenAI Protocol), the agent can now plug into these Orchestrators effortlessly.  

Why this actually matters

1. You’re no longer tied to a single AI backend

You can choose where and how inference runs based on your environment—not the agent’s limitations.

• Cost considerations

• Data residency requirements

• Internal platform standards

This is a shift from “supported integrations” → “supported protocols.”

2. You can adopt new AI platforms faster

If your organization introduces a new LLM gateway or optimization layer, the agent doesn’t need explicit support.

If it speaks the protocol, it works.

3. It reduces integration friction

No custom adapters or one-off implementations.

Configuration replaces code:

• Endpoint (LLM_BASE_URL)

• Credentials (LLM_API_KEY)

• Runtime (MODEL_RUNTIME)

4. It improves reliability where it counts

We’ve standardized timeout behavior, connection handling, and retry logic—based on performance testing—to make LLM interactions more predictable under load.

5. You get better control over model output

With MAX_COMPLETION_TOKENS, you can:

• Limit response size

• Reduce latency

• Tune behavior for specific models (like Granite variants)

What you need to do

Possibly nothing.

If your current setup works, it continues to work. No architectural rework required.

To take advantage of this:

• Update configuration (env variables + Helm charts)

• Point the agent to your preferred endpoint

• Validate deployment

Bottom Line: Operational Autonomy

This release isn't just a feature update; it’s an architectural liberation. Integrating AI with the IBM Z felt like custom-wiring a new engine every time you wanted to change fuels.

Think of this as the "Universal Plug" moment for the Db2 for z/OS Agent. By decoupling the agent from specific runtimes, we’ve shifted the power back to the Architect. You can now switch your inference backends or integrate with AIOptimizer as easily as swapping a cable—without having to rebuild the car underneath it.

You gain granular control over performance and cost, all while maintaining the uncompromising stability that is the hallmark of the IBM Z ecosystem. We’ve built the future into the agent without disturbing the present.