The Shift Has Already Begun

It starts with a question.

An enterprise architect needs to deploy AI-powered database management. The team evaluates options, reviews compliance requirements, analyzes cost models, and debates infrastructure choices.

The question was simple: Where should our AI live?

The path to the answer was not.

Now imagine a different experience.

The same question is asked. But this time, there is no forced choice between convenience and control. No compromise between quick deployment and data sovereignty. Instead, there is an AI configuration model that adapts to the organization’s requirements.

This is not just flexibility. This is Agentic AI on your terms.

With Db2 Agentic AI, organizations can choose how the service connects to and uses AI capabilities. That choice affects where data is processed, which AI models are used, how costs are managed, and who owns the operational responsibilities.

Why AI Configuration Matters

The AI stack powers intelligent capabilities such as natural language understanding, SQL generation, performance analysis, and conversational interactions with your database.

That makes AI configuration more than a setup step. It is an architectural decision.

Some organizations want a fully managed experience so they can start quickly. Others need tighter control over data processing, model selection, infrastructure, and compliance. Db2 Agentic AI supports both approaches.

Two Configuration Approaches

Db2 Agentic AI supports two distinct AI configuration approaches: IBM provided AI stack and Bring your own AI stack.

IBM Provided AI Stack: Managed Service

The IBM provided AI stack is the managed option. IBM hosts and manages the AI infrastructure, including models, knowledge base, scaling, updates, and maintenance.

This option is designed for organizations that want to use AI capabilities immediately without deploying or operating their own AI infrastructure.

Best for: Quick deployment, variable workloads, managed operations, multi-region availability, and teams that want to focus on using AI rather than managing infrastructure.

Bring Your Own AI Stack: Self-Managed

Bring your own AI stack is the self-managed option. Organizations deploy and manage their own AI infrastructure on-premises or in a preferred cloud environment.

This option is designed for organizations that need greater control over AI models, data processing, infrastructure, and compliance.

Best for: Data sovereignty, high-volume predictable workloads, air-gapped or on-premises environments, custom or fine-tuned model requirements, and teams with AI/ML infrastructure expertise.

What BYOAI Really Means

A common misunderstanding is to treat Bring your own AI stack as one single option. It is better understood as a self-managed category with multiple supported model-provider choices.

In the current documentation, BYOAI includes the following AI model options:

·       vLLM, recommended for air-gapped environments

·       watsonx.ai, IBM’s enterprise AI platform

·       AWS Bedrock, for Claude Sonnet models through AWS

·       Google Cloud Vertex AI, for Claude models through Vertex AI

You do not deploy all of these together. You choose the model option that fits your environment, then configure the supporting knowledge base and infrastructure around it.

The Four BYOAI Model Options

1. vLLM

vLLM is the recommended option for air-gapped environments or customers who need full control over local inference infrastructure. It is an open-source, high-performance LLM inference server with GPU acceleration, OpenAI-compatible APIs, and support for IBM Granite 4.0.

Best for: Air-gapped environments, GPU-backed infrastructure, strict data sovereignty requirements, and organizations that want direct control over inference infrastructure.

Implementation note: The official setup guidance includes Red Hat container-based deployment paths for AMD MI300X and NVIDIA H100 GPU environments using Podman.

2. watsonx.ai

watsonx.ai is the IBM-managed option within the BYOAI model choices. It is the simplest path for customers who want to use IBM’s enterprise AI platform without managing their own vLLM runtime or GPU infrastructure. This option is easier to set up and operate, while still giving customers access to supported IBM model capabilities.

Best for: Organizations that want IBM Granite with enterprise platform support, governance, and IBM ecosystem alignment.

3. AWS Bedrock

AWS Bedrock is the AWS-managed AI service option for accessing Claude Sonnet models within the AWS ecosystem.

Best for: Organizations already operating in AWS that want Claude Sonnet models and AWS-native integration.

4. Google Cloud Vertex AI

Google Cloud Vertex AI is the Google Cloud option for accessing Claude models. The Agentic AI service supports Vertex AI credentials and supports available Claude model parameters and features through Vertex AI.

Best for: Organizations already operating in Google Cloud that want Claude models and Google Cloud-native integration.

Configuration note: Vertex AI authentication can use either a Google access token or a Google API key. The required configuration includes Google Cloud location,Google Cloud project, model identifier, and the selected authentication credential.

What You Manage with BYOAI

With Bring your own AI stack, you manage the AI configuration components that run outside IBM's fully managed stack. After choosing one of the supported BYOAI model options, your organization is responsible for managing the knowledge base and infrastructure that support the deployment.

1. AI Model Endpoint

You choose and configure one supported AI model option: vLLM, watsonx.ai, AWS Bedrock, or Google Cloud Vertex AI. This model endpoint is what Db2 Agentic AI connects to for AI-powered capabilities such as natural language understanding, SQL generation, performance analysis, and conversational interactions.

2. Redis Knowledge Base

BYOAI uses a self-hosted Redis knowledge base with RediSearch for document indexing, vector search, semantic retrieval, and caching. This keeps knowledge-base processing under the organization’s control.

3. Self-Managed Infrastructure

The organization controls deployment location, resource allocation, networking, security controls, and scaling strategy. This can include on-premises, private cloud or hybrid environments, depending on your requirements.

Choosing Between the BYOAI Options

Testing Before Production

Before production deployment, Db2 Agentic AI provides a Test Connection feature to validate the AI configuration. This is useful because BYOAI environments can vary significantly depending on model provider, network setup, credentials, and hardware.

The validation checks four areas:

·       Connection test: Confirms the AI service endpoint is reachable and responding.

·       Authentication test: Confirms credentials, permissions, and model availability.

·       Time to first token test: Measures how quickly the first token is returned to the user.

·       Tokens per second test: Measures generation throughput and infrastructure capacity.

Video Walk-through

This video demonstrates the end-to-end setup of a BYOAI deployment, from AI provider configuration to connection validation and operational readiness.

Real-World Deployment Scenarios

Financial Services: BYOAI with vLLM

A financial services organization with strict data residency requirements may choose BYOAI with vLLM so inference can run on controlled infrastructure. This keeps AI processing aligned with internal data-handling policies while still enabling Db2 Agentic AI capabilities.

Healthcare: BYOAI with watsonx.ai

A healthcare organization that needs enterprise support and strong governance may choose BYOAI with watsonx.ai. This supports IBM Granite access while keeping the deployment aligned with enterprise controls.

Cloud-Native Enterprise: BYOAI with AWS Bedrock

An organization already standardized on AWS may choose AWS Bedrock to access Claude Sonnet models through its existing cloud ecosystem, security model, and operational practices.

Google Cloud Enterprise: BYOAI with Vertex AI

An organization already standardized on Google Cloud may choose Vertex AI to access Claude models through Google Cloud-native configuration and authentication options.

Quick Comparison

Getting Started with BYOAI

1.    Choose the AI model option. Use vLLM for on-premises control, watsonx.ai for IBM enterprise platform alignment, AWS Bedrock for AWS-based Claude access, or Google Cloud Vertex AI for Google Cloud-based Claude access.

2.    Plan the knowledge base. BYOAI uses self-hosted Redis with RediSearch for indexing, vector search, and retrieval.

3.    Prepare infrastructure. Confirm compute, GPU, networking, security, and scaling requirements.

4.    Configure Db2 Agentic AI. Point the service to the selected AI model endpoint and provide the required credentials.

5.    Validate with Test Connection. Confirm connectivity, authentication, TTFT, and TPS before production use.

6.     Refer to the below official setup docs for implementation steps.

Official Documentation and Setup Guides

·       Db2 Agentic AI service configuration

·       Getting started with AI usage and metering

·       Vertex AI support for Claude models

·       Setting up vLLM with Red Hat container on AMD

·       Setting up vLLM with Red Hat container on NVIDIA

Final Thought

The future of enterprise AI is not about forcing every organization into the same deployment model.

It is about giving teams a clear choice between a managed AI stack and a self-managed AI stack.

For organizations that need speed, the IBM provided AI stack offers a managed path. For organizations that need control, BYOAI provides supported options

across vLLM, watsonx.ai, AWS Bedrock, and Google Cloud Vertex AI.

The question is not whether AI will transform database management. It already has.

The question is: Where should your AI stack live?

About Authors

Merlin Moncy

Merlin Moncy is a Software Developer in Hybrid Data Management at IBM’s Ireland Lab, focusing on containerized Db2 offerings and automation workflows. She has experience in Python development, QA automation, CI/CD pipelines, and containerized environments, and is currently contributing to CAE team initiatives around Db2 Genius Hub and AI-powered database operations. She holds a Master’s degree in Data Analytics from the University of Galway, with a focus on machine learning, NLP, and data-driven systems. Merlin can be reached at merlin.moncy@ibm.com.

Rahul Gadamsetty

Rahul Gadamsetty is a Software Developer in the Db2 Agentic Ai team. His work primarily focuses on Agentic AI technologies, with an emphasis on MCP server development, Vertex AI integration, security guardrails, and token optimization for enterprise AI applications. He contributes to the design and development of intelligent, scalable, and secure AI driven solutions within the Db2 Agentic Ai and Genius hub MCP server. He is part of the Db2 Runtime team, the Db2 Agentic AI team, and the MCP Server development team. He holds a bachelor's degree from National Institute of Technology Calicut. He can be reached at Rahul.Gadamsetty@ibm.com.

Taniya Bagh

Taniya Bagh is a Software Developer in Hybrid Data Management at IBM's Ireland Lab, focusing on Data Virtualisation, agentic AI solutions, and intelligent data-driven systems. She has experience in Java and Python development, containerized environments, and building AI-powered workflows and automation solutions. As part of the CAE team, She contributes to initiatives focused on AI-enhanced data management and next-generation enterprise solutions. She holds a Master's degree in Data Analytics from National College of Ireland, with a focus on Machine Learning, Deep Learning models, and data-centric intelligent systems. Taniya can be reached at taniya.bagh@ibm.com.