Introduction
Starting with IBM Instana Observability 1.0.312, the LLM Observability architecture has been simplified.
Previously, sending Metrics required a separate service called OTel Data Collector for GenAI (ODCG), but this is no longer necessary.
What Changed:
- Old: Traces/Logs → Instana, Metrics → ODCG → Instana (separate paths)
- New: Traces/Logs/Metrics → Instana (unified path)
This article explains the new architecture configuration and the role of each component.
Architecture Changes
Previous Configuration (Using ODCG)
Traces/Logs Flow:
OpenLLMetry → Instana Backend (direct)
or
OpenLLMetry → Instana Agent → Instana Backend
Metrics Flow:
OpenLLMetry → ODCG (separate service) → Instana Backend (direct)
or
OpenLLMetry → ODCG (separate service) → Instana Agent → Instana Backend
New Configuration (ODCG Not Required)
Traces/Logs/Metrics Flow (Unified):
OpenLLMetry → Instana Backend (direct - recommended)
or
OpenLLMetry → Instana Agent → Instana Backend
Key Points of the Change:
- ODCG as a separate service is no longer needed
- Traces/Logs/Metrics now use the same path (path unification)
- OpenLLMetry remains required (instrumentation source)
- Instana Agent is optional (you can choose direct sending or via Agent)
Role of Each Component
Component Roles
Old Configuration Diagram:
New Configuration Diagram:
Three Important Facts
1. OpenLLMetry Remains Required
Role:
- Operates within GenAI applications
- Automatically detects and instruments LLM calls
- Captures token count, cost, and latency
- Generates data in OpenTelemetry format
Status: No Change - Still Required
Important: Without OpenLLMetry, GenAI operation visibility is impossible.
2. ODCG Only Handled Relay Processing
Role:
- Relay point between OpenLLMetry and Instana
- Data aggregation and forwarding
- Price information management (prices.properties)
Status: Deprecated - Removal Recommended
Why It's No Longer Needed:
- Instana Backend can now process GenAI data directly
- Price information can be managed in Instana UI
- Single endpoint receives both traces and metrics
- Additional relay point is unnecessary
3. Instana Agent Is an Optional Component
Role:
- Optional data aggregation point
- Aggregates data from multiple applications in one place
- Integration with existing Instana monitoring infrastructure
Important: ODCG functionality did not move to the Agent. The Agent was and remains an "optional component."
Essence of the Change
What Changed
| Item |
Old |
New |
| Data Path |
OpenLLMetry → ODCG → Instana |
OpenLLMetry → Instana (direct or via Agent) |
| Relay Point |
ODCG required |
ODCG not needed |
| Number of Endpoints |
2 (for traces and metrics) |
1 (unified) |
| Price Management |
ODCG's prices.properties |
Instana UI |
| Configuration Complexity |
Complex (ODCG deployment & config needed) |
Simple (environment variables only) |
What Didn't Change
| Item |
Status |
Description |
| OpenLLMetry |
Still required |
Continues as instrumentation source |
| Instrumentation Method |
No change |
Automatic LLM call detection |
| Instrumentation Code |
No change needed |
Traceloop code remains the same |
| Instana Agent |
Optional (was optional before) |
Free to choose whether to use |
| Visualization Features |
No change |
Display in Instana UI is the same |
Configuration Patterns
Pattern 1: Agentless
GenAI App
↓ [OpenLLMetry instruments] ← Required
OpenLLMetry
↓ [Direct sending]
Instana Backend
Characteristics:
- Simplest approach
- Instana Agent not required
- ⚠️ OpenLLMetry is required
Configuration Example:
export TRACELOOP_BASE_URL=export"x-instana-key=exporttrueexportfalseexport"INSTANA_PLUGIN=genai"
Pattern 2: Via Agent
GenAI App
↓ [OpenLLMetry instruments] ← Required
OpenLLMetry
↓ [Via Agent]
Instana Agent
↓
Instana Backend
Characteristics:
- Effective for aggregating multiple applications
- Leverages existing Agent infrastructure
- Enables correlation analysis with infrastructure (Important)
- ⚠️ OpenLLMetry is required
Benefits:
- Can correlate infrastructure metrics (CPU, memory, network, etc.) with GenAI metrics
- Unified understanding of application performance issues and infrastructure problems
- Easier troubleshooting
Pattern 3: Old Configuration (Deprecated)
GenAI App
↓ [OpenLLMetry instruments]
OpenLLMetry
↓ [Via ODCG]
ODCG ← Deprecated
↓
Instana Backend
Characteristics:
- Removal recommended
- Complex configuration
- High operational cost
- ODCG redeployment needed when updating prices
Summary
Core Understanding
-
OpenLLMetry = Instrumentation Source (Required, no change)
- The only means to automatically instrument GenAI operations
- Visibility is impossible without this
-
ODCG = Relay Point (Deprecated, removal recommended)
- Simply data relay and aggregation processing
- No longer needed as Instana Backend can process directly
-
Instana Agent = Optional Aggregation Point (Optional, no change)
- Was and remains an "optional" component
- ODCG functionality did not move here
-
Instana Backend = Final Storage Destination (Required, enhanced functionality)
- Can now process GenAI data directly
- Integrated price management functionality
Essence of the Change
"Simplification through removal of relay point," not "change in instrumentation method"
References
Conclusion
Instana's LLM Observability has become simpler and easier to adopt with the removal of ODCG.
Configuration Points:
- Instrument GenAI operations with OpenLLMetry
- Send directly to Instana Backend (or via Agent)
- Complete with environment variable configuration only
I hope this helps those starting LLM monitoring. If you have questions or additional information, feel free to comment!
Tags: #Instana #LLM #GenAI #Observability #OpenTelemetry #OpenLLMetry #IBM
#LLM