You are building an LLM-powered application. Your data lives in a database. Your model lives on an external provider. Connecting the two should be simple. It is not.
Between your database and your model sits code you never wanted to write: HTTP clients, authentication headers, credential management, retry logic, response parsing, error handling, and data protection. As request volumes grow, you add connection pooling and concurrency management.
None of this advances your application. Every line costs you time you should be spending on the queries, the logic, and the features your application needs. It is plumbing. You are the one maintaining it.
There is a better way.
How Db2 eliminates the integration layer
Db2's LLM integration feature removes the integration layer entirely. Register a language model once as a database object. From that point, call it from Db2 SQL like any other built-in function. Db2 handles the HTTP call, authentication, response parsing, and error handling. The result comes back directly in your query.
No integration code. No middleware. No new services to deploy, monitor, or maintain.
This feature is currently available through the Db2 Early Access Program, with general availability planned for an upcoming Db2 release.
Your data and your model are now reachable from the same Db2 SQL query.
Db2's LLM integration supports two provider types:
- Watsonx.ai: IBM's enterprise AI platform, supported natively through a dedicated provider type
- OpenAI-compatible providers: any LLM provider or inference engine that implements the OpenAI API standard, covering a wide range of cloud, commercial, and open-source options
An earlier post covered the first path: registering a Watsonx.ai embedding model and building a semantic search system entirely in Db2 SQL.
Related: Db2 12.1.x EAP: Call Language Models Directly from SQL. This post covers registering a Watsonx.ai embedding model, generating embeddings with TO_EMBEDDING, and running vector similarity search, all from Db2 SQL.
This post takes the second path: calling a text generation model from Amazon Bedrock through Db2's OpenAI-compatible provider support. The same pattern has been validated with Watsonx.ai, Amazon Bedrock, OpenAI, vLLM, llama.cpp, and Ollama. This is not an endorsement of any provider; they appear here to show the range of options confirmed to work with Db2.
Calling Amazon Bedrock from Db2 SQL in three steps
This walkthrough uses Amazon Bedrock as one example from the supported provider list. The same three steps apply to any OpenAI-compatible provider: only the API key, model ID, and URL differ.
Three steps take you from an Amazon Bedrock API key to a working LLM call from Db2 SQL.
Step 1: Get an Amazon Bedrock API key
Amazon Bedrock exposes an OpenAI-compatible API, using the same /v1/chat/completions endpoint format adopted by many LLM providers. Registering a Bedrock model in Db2 follows the same pattern as any other OpenAI-compatible provider.
Generate a long-term API key from the Amazon Bedrock console and copy it immediately: it is only shown once. For detailed steps, see Generate an Amazon Bedrock API Key.
Any model available through Bedrock's OpenAI-compatible endpoint works with Db2's registration pattern. See Amazon Bedrock OpenAI-Compatible APIs for the current model catalog. As of this writing, the endpoint supports text generation models. If Bedrock adds embedding models to this endpoint, Db2 will call them the same way.
Step 2: Register the model in Db2
Register an LLM from Amazon Bedrock as an external model in Db2:
CREATE EXTERNAL MODEL BEDROCK_TEXT_GEN
PROVIDER OPENAI
KEY 'your-aws-bedrock-api-key-here'
ID 'qwen.qwen3-32b'
TYPE TEXT_GENERATION RETURNING CLOB(30000)
URL 'https://bedrock-mantle.us-east-1.api.aws/v1/chat/completions'
Three parameters to note:
PROVIDER OPENAI instructs Db2 to use the OpenAI-compatible protocol, which Bedrock supportsID is the model identifier from Bedrock's /v1/models endpointURL is the chat completions URL for Bedrock's OpenAI-compatible endpoint
Registration is a one-time step. Every subsequent call is a single SQL function call.
Step 3: Call it from Db2 SQL
Call the registered model from SQL with a prompt:
VALUES TEXT_GENERATION('Explain the causes of the 2008 financial crisis' USING BEDROCK_TEXT_GEN)
This is what happens when you run the above statement:
Db2 returns the LLM's response as the query result:
The 2008 financial crisis resulted from a convergence of several factors:
1. Subprime mortgage lending: Banks issued mortgages to borrowers with poor
credit histories, often with adjustable rates that became unaffordable
when they reset.
2. Securitization: These risky mortgages were bundled into complex financial
instruments (MBS, CDOs) and sold to investors globally, spreading the
risk throughout the financial system.
3. Excessive leverage: Financial institutions held far more debt relative to
their capital than was prudent, amplifying losses when asset values fell.
4. Rating agency failures: Credit rating agencies assigned top ratings to
mortgage-backed securities that were far riskier than advertised.
5. Regulatory gaps: Derivatives markets and shadow banking activities
operated with little oversight, allowing systemic risk to accumulate
largely unseen.
When housing prices declined, defaults cascaded through the system,
triggering a global credit freeze and the deepest recession since the 1930s.
1 record(s) selected.
A complete LLM response, returned in a Db2 SQL result set. No HTTP client. No response parsing. No credential management. Db2 handled it all.
Note: This example uses a static prompt to show the mechanics. Prompts can be constructed dynamically from data already in Db2. You can build a retrieval-augmented generation (RAG) prompt entirely within Db2: query your vector store, retrieve the most relevant document chunks, inject them as context, and call the model, all in a single query with no application code outside the database.
Cleanup
Remove the registered model when you no longer need it:
DROP EXTERNAL MODEL BEDROCK_TEXT_GEN
What becomes possible once the integration layer is gone
Your data and your model live in the same query
Enrich rows at query time: summarize customer feedback, classify support tickets, generate product descriptions. The model operates on data where it already lives, with no extraction step.
TEXT_GENERATION works anywhere a function is valid in Db2 SQL
Inside queries, views, and stored procedures. The model becomes part of your data layer, not a separate service call your application must coordinate.
Switching models requires no application code changes
Register a new model from a different provider, or switch to a different model from the same provider. Both happen in Db2 SQL. Your application code stays untouched.
The pattern holds across providers
This post showed Amazon Bedrock. The same register-call-result sequence applies to every OpenAI-compatible provider Db2 supports. The provider changes. The SQL does not. Your application does not.
Try it yourself
- Download the Db2 early preview from the Db2 Early Access Program
- Get an Amazon Bedrock API key from the AWS Console
- Follow the three steps above
References
About the Authors
Shaikh Quader
Shaikh Quader is the AI Architect and a Master Inventor for IBM Db2, where he builds capabilities such as vector search and LLM integration into the database. He has spent over two decades in software development at IBM, with the last ten years focused on AI. He holds 14 issued patents in AI and data systems and has authored 14 peer-reviewed scientific publications in AI and machine learning. He is also a Ph.D. candidate at York University, where his research explores the intersection of AI and relational databases.
Ian Finlay
Ian Finlay, an IBM Senior Technical Staff Member (STSM), brings over 38 years of experience in software development, with the last 31 years dedicated to mastering IBM Db2’s Query Compiler, Runtime, and the operational engine stack. Over the past year he has been focused on AI Enablement in the Db2 Engine, and AI Adoption in the development environment.
Ashok Kumar
Ashok Kumar is a Program Director of Engineering at IBM with over 15 years of industry experience. He leads global engineering teams within IBM's Hybrid Data Management organization and drives innovation across the Db2 product portfolio, including on-premises, cloud, and BYOC deployments. His work spans advanced database technologies, from infusing AI into the Db2 query optimizer to enabling Db2 as a vector store and building agentic AI capabilities. He leads the integration of large language models into the Db2 Intelligence Center for natural language interaction, root-cause analysis, text-to-SQL translation, and automated troubleshooting. His focus sits at the intersection of AI, databases, and enterprise data management.