Global Storage Forum

Global Storage Forum

Connect, collaborate, and stay informed with insights from across Storage

 View Only

Self-Hosting Langfuse: A Practical Guide for Secure Agentic AI Systems

By Subhashini Raman posted 07/10/26 04:47 AM

  

Co-authored by: @Shruti Sullad

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------


Why We Chose to Self-Host Langfuse

When we began building agentic AI workflows, observability was not optional,it was foundational. However, our primary challenge wasn’t whether to add observability, but how to do it in a way that aligned with our early-stage constraints and long-term goals.

Starting Constraints: Cost, Licensing, and Control

At the outset, we made a deliberate decision to prioritize:

  • Cost efficiency – avoiding recurring SaaS expenses while still experimenting and iterating
  • Licensing flexibility – leveraging open-source tooling without restrictive enterprise contracts
  • Infrastructure ownership – retaining full control over our data and deployment environments

Self-hosting Langfuse naturally aligned with all three.

This wasn’t just a technical choice,it was a strategic one. At an early stage, we needed the freedom to explore, fail fast, and evolve our architecture without long-term vendor commitments.

Why Langfuse (and Not Alternatives)?

Our evaluation focused on tools that could support real-world agentic systems, not just experimentation. Langfuse stood out for several reasons:

1. Built for Production-Grade LLM Applications

Langfuse is designed as a full LLM engineering platform, supporting debugging, monitoring, and continuous improvement of live systems. 

While alternatives like Phoenix are strong in experimentation and evaluation workflows, Langfuse provides a more cohesive end-to-end lifecycle view, including prompts, deployments, and environments.

2. Unified Visibility into Agent Behavior and Cost

One of the biggest advantages was the ability to see a complete request lifecycle in one place:

  • Agent hops
  • Model calls
  • Tool invocations
  • Latency
  • Token usage

This level of detail allowed us to directly connect:

  • system behavior → performance
  • performance → cost
  • cost → business impact

Langfuse makes these relationships explicit, not inferred.

3. Integrated Prompt and Evaluation Workflow

Langfuse brings together:

  • Prompt management
  • Versioning
  • Observability
  • Evaluations

This unified approach enables both engineers and product teams to:

  • iterate on prompts
  • deploy changes
  • monitor real-world impact

..all within a single toolchain.

4. Open, Self-Hosted, and Vendor-Neutral

A key requirement for us was avoiding vendor lock-in.

Langfuse provided:

  • Fully open-source deployment
  • Self-hosting capability
  • Flexible APIs and export options
  • Alignment with OpenTelemetry (OTel) standards

The OpenTelemetry alignment was a major advantage. It allows us to:

  • Integrate Langfuse with existing observability stacks
  • Correlate LLM traces with broader system telemetry (services, APIs, infra)
  • Avoid being tied to a proprietary tracing format

This ensures that our observability layer remains interoperable and portable, which is critical in enterprise environments where multiple monitoring and tracing systems coexist.

Ultimately, this allowed us to treat observability as a neutral data layer, rather than tying it to a specific platform ecosystem.

5. Best Fit for Our Team and Stage

As a small, engineering-driven team working on multi-agent systems, our needs were:

  • Real-time production visibility
  • Simple but effective evaluation workflows
  • Minimal operational overhead

Langfuse aligned well with this stage of maturity. Importantly, it also leaves room to evolve, more experimentation-heavy tools can still be layered later if needed.

Architecture and Deployment Setup

To support our observability needs, we deployed Langfuse in a containerized setup within an OpenShift cluster, aligning with our broader platform strategy and enterprise deployment standards.

Core Deployment Components

Our deployment consists of the following key components:

  • Langfuse Web – the user interface for exploring traces, metrics, and workflows
  • Langfuse Worker – handles ingestion, processing, and background jobs
  • ClickHouse – serves as the primary analytical database for high-volume trace and event data

These components run as containers within OpenShift, allowing us to scale and manage them independently based on workload characteristics.

Supporting Managed Services

To reduce operational overhead and leverage existing enterprise infrastructure, we integrated IBM-managed services for supporting components:

  • PostgreSQL – metadata and relational storage
  • Redis – caching and queueing
  • Cloud Object Storage (COS) – long-term storage for logs and artifacts

This hybrid approach allowed us to:

  • Focus on Langfuse-specific scaling challenges
  • Avoid reinventing reliable infrastructure for stateful services

Current State: Early-Stage but Functional

As we are still in the early stages of adoption, our setup is intentionally simple:

  • A single ClickHouse replica (no HA yet)
  • Moderate resource allocation for web and worker components
  • Basic scaling tuned for current traffic patterns

At this stage, our priority is learning system behavior under real workloads, rather than over-engineering for scale upfront.

Planned Improvements

As usage grows, we are actively working on strengthening the data layer:

  • Backup and Disaster Recovery for ClickHouse
    Ensuring trace data durability and recovery capabilities

  • High Availability (HA) Setup
    Evaluating replication and distributed configurations for ClickHouse

  • Scalability Planning
    Preparing for increased ingestion rates and concurrent workloads

These improvements are being driven by observed usage patterns rather than assumptions.

What the Docs Don’t Tell You

One important reality:
    This setup was not achieved on Day 1.

We didn’t start with a perfect architecture. Instead, we iterated through:

  • Resource sizing (CPU, memory)
  • Number of pods for workers and web services
  • Handling concurrent trace ingestion
  • Balancing write throughput vs query performance

Every decision, from how many replicas to run to how much load ClickHouse can handle, was refined through trial, error, and observation in real environments.

This iterative approach helped us:

  • Avoid premature optimization
  • Understand real bottlenecks
  • Build confidence in our design choices

There is no “one-size-fits-all” deployment for Langfuse.

Your architecture will depend heavily on:

  • Traffic patterns
  • Agent complexity
  • Trace volume
  • Latency and cost sensitivity

Start simple, observe real usage, and evolve deliberately.

Trace Pipeline Architecture (OTel Integration)

One of the most important design decisions we made was introducing an OpenTelemetry (OTel) layer as an intermediary, instead of sending traces directly to Langfuse.

Why This Matters

All application traces follow this path:

[Agent / Application Layer]
            ↓
   [OpenTelemetry Collector]
            ↓
       [Langfuse]
            ↓
      [ClickHouse]

How It Works

  • Agent applications are instrumented using OpenTelemetry SDKs, which generate traces, spans, and associated metadata during workflow execution.
  • The OTel SDK exports telemetry data to a centrally deployed OpenTelemetry Collector running within our platform environment.
  • The Collector acts as an ingestion layer, handling batching, buffering, processing, and export operations before forwarding traces downstream.
  • Authentication and endpoint configuration are managed at the Collector layer, allowing application teams to remain decoupled from Langfuse-specific credentials and ingestion endpoints.
  • The Collector exports traces to Langfuse using the configured exporter.
  • Langfuse enriches, processes, and stores observability data within ClickHouse for querying and analysis.

Why We Chose This Approach

1. Vendor-Agnostic Observability Layer

By routing all telemetry through OpenTelemetry:

  • We are not tightly coupled to Langfuse ingestion APIs
  • We can redirect or duplicate traces to other backends if needed
  • Observability remains portable and ecosystem-friendly

2. Future-Proofing for Tuning and Scale

The OTel layer gives us a powerful control plane for telemetry:

  • Sampling control
    → Reduce ingestion cost by sampling traces

  • Filtering and enrichment
    → Add metadata or drop noisy signals

  • Routing flexibility
    → Send traces to multiple systems if needed

This becomes especially important as traffic grows and trace volume increases.

3. Alignment with Enterprise Observability Standards

Many enterprise systems already rely on OpenTelemetry for:

  • Distributed tracing
  • Metrics
  • Logs

By adopting OTel here, we ensure:

  • Consistency with broader platform observability
  • Easier integration with existing tooling
  • A standardized telemetry format across systems

4. Clean Separation of Concerns

This design cleanly separates responsibilities:

Closing Thoughts: Beyond Deployment

Self-hosting Langfuse gave us a strong foundation for building observability into our agentic systems but deployment was only the beginning.

Once we moved beyond setup and started using the platform in real workflows, a new set of challenges emerged:

  • Collecting traces is relatively straightforward
  • Understanding them in context is harder
  • Turning them into actionable insights is the real challenge

We also began to see how seemingly small architectural decisions, around isolation, data flow, and scaling, could directly impact the reliability and usefulness of observability data.

These are not just infrastructure concerns, they are design decisions that shape how effectively you can reason about your system.

In the next part, we’ll explore these challenges in detail and share the lessons we learned while designing observability for real-world agent workflows.
0 comments
22 views

Permalink

https://community.ibm.com/community/user/blogs/subhashini-raman/2026/06/26/self-hosting-langfuse-a-practical-guide-for-secure