IBM i Global

IBM i 

A space for professionals working with IBM’s integrated OS for Power systems to exchange ideas, ask questions, and share expertise on topics like RPG and COBOL development, application modernization, open source integration, system administration, and business continuity.


#Power


#IBMi
#Power
#Operatingsystems
#Servers
 View Only

Instana on IBM i + DB2 OpenTelemetry: full-stack observability for legacy-rich enterprise estates

  • 1.  Instana on IBM i + DB2 OpenTelemetry: full-stack observability for legacy-rich enterprise estates

    Posted 04/29/26 08:06 AM
      |   view attached

    Sharing a long-form write-up of a recent IBM Instana deployment we ran across an estate that combines IBM i (AS/400), DB2 batch, JBoss / Tagetik, SQL Server and IIS - with a focus on the OpenTelemetry pattern that brought batch DB2 jobs into distributed tracing. Three reusable lessons at the bottom. Curious to hear from anyone running similar setups.

    Title card for the blog post: Instana on IBM i + DB2 OpenTelemetry, full-stack observability for legacy-rich enterprise estates. Author Roberto Renna, CEO YoctoIT.
    The observability conversation in enterprise IT has a blind spot.
    Most of the modern literature on application performance management assumes a Kubernetes-first, microservices-first world. Distributed tracing diagrams show OpenTelemetry spans flowing between containers; alerting demos are built around HTTP latency and error rates; the case studies cite tech companies whose entire estate is a few years old.
    The reality of mid-to-large European enterprises is different. Modern web frontends and Kubernetes clusters live next to IBM i (AS/400) systems running mission-critical ERP logic that has been production-stable for fifteen years. Microsoft SQL Server transactional databases share the same datacenter as IBM Db2 backends serving batch jobs that nobody has touched since the original architects retired. JBoss application servers running CPM tools like Tagetik talk to all of them, and the integrations are where the failure modes hide.
    Most APM tools struggle the moment you have an AS/400 in the picture, or batch jobs that don't have HTTP endpoints. The result is observability silos: one tool for the modern stack, another for the IBM i environment, a third for infrastructure metrics, and zero correlation between them. Mean time to detect anomalies stays high because no single team sees the full picture.
    We recently deployed IBM Instana Observability across exactly this kind of estate - Windows, Unix, IIS, SQL Server, JBoss / Tagetik, IBM i, custom Java applications, and DB2 batch - and configured a specific extension that uses OpenTelemetry to bring batch DB2 jobs into the same distributed tracing view as the web layer.
    This post is what we learned. Three lessons that I think are reusable for any partner navigating the same kind of legacy-rich enterprise estate.

    The customer estate

    The customer is a mid-to-large European enterprise running a representative hybrid stack - modern web frontends and Java applications coexisting with mission-critical legacy systems. Specifically:

    • Web layer: Microsoft IIS with 13 publicly-facing or internal sites.
    • Application layer: JBoss application server hosting CPM workloads (Tagetik / Wolters Kluwer CCH Tagetik 5.3.28.413) plus several custom Java applications.
    • Database layer: Microsoft SQL Server as the transactional database for modern apps. IBM Db2 as the backend for legacy batch processes.
    • Core systems: IBM i (AS/400) running the gestionale ERP - V7R5M0 with NVMe storage, structured ASP layout, hundreds of active jobs.
    • Hosts: A mix of Windows and Unix application servers. Custom Java services and scheduled batch jobs running across them.

    Posture pre-progetto: monitoring siloed across three different tools, four different consoles, zero correlation between them. MTTR for cross-layer issues was high because no single team had the full picture. Batch jobs running against DB2 were essentially invisible - only their start/end timestamps and error logs were captured, with no insight into what they actually did between those two points.

    Why the obvious tools don't fit

    Before settling on Instana we evaluated the alternatives, and I want to be honest about why each was ruled out - none of these tools is bad, they just don't fit this profile of estate.

    Cloud-native APM platforms like Datadog, New Relic and Dynatrace - all excellent in modern Kubernetes and microservices contexts - have very limited or no native support for IBM i. OpenTelemetry instrumentation of batch DB2 workloads is not out-of-the-box on any of them; you can do it with custom engineering, but you lose the opinionated tracing UX that makes those platforms attractive in the first place. Pricing also tends to scale opaquely on host count plus telemetry volume, which makes long-term TCO unpredictable on legacy-heavy estates.

    Pure infrastructure monitoring like Check_MK, Nagios or Zabbix is the historical answer for legacy-rich estates. It gives you good host metrics - CPU, memory, disk, processes - but no application-level tracing, no distributed correlation, no per-query database visibility. You learn that something is broken, not why it broke.

    Best-of-breed mix - running an APM for modern apps, an infrastructure tool for legacy, and a database monitoring tool separately - is what most organizations end up with by accident. The problem isn't the monitoring per layer; it's the lack of correlation across layers. Three teams, three on-call rotations, three sets of dashboards, and no single source of truth when an incident spans the boundary.

    Instana wins not because it's the only APM in the market, but because two specific gaps mattered for this estate: the IBM i bridge agent that brings AS/400 metrics into the same backend as everything else, and the OpenTelemetry Collector pattern that makes batch DB2 jobs traceable end-to-end without rewiring the network. Both of those existed in Instana, ready to use.

    The architecture

    Layered architecture diagram showing IBM Instana backend at the top, with web layer (IIS), application layer (JBoss/Tagetik), database layer (SQL Server, IBM Db2), and physical hosts (Windows, Unix, IBM i, Lenovo) below. Custom Zone
    Figure 1 - Full-stack observability architecture: IBM Instana + IBM i bridge + OpenTelemetry Collector
    The deployed architecture is layered in the conventional way (hosts → database → applications → web edge), but every layer reports into a single Instana backend with one-second granularity. The pieces that make the architecture distinctive are three.
    First, the Instana Agent. A lightweight agent installed on every target host. Auto-discovery brings up sensors for IIS, SQL Server, JBoss and the underlying OS without manual configuration - a real time-saver on Day 1. The agent forwards metrics, logs and traces to the backend, and on most platforms self-updates (a setting we keep enabled).
    Second, the IBM i bridge. This is the unique piece. IBM i isn't a host where you can drop a modern Java agent and expect auto-discovery to work; it requires a dedicated bridge component installed on the AS/400 itself, with credentials configured, and a set of explicit metrics published from the OS, the storage subsystem (NVMe + ASP layout), the database (operations, plan cache, commit/rollback rates) and the active jobs. Once it's in, the IBM i side of the estate becomes a first-class citizen of the observability backend - visible, alertable, comparable to the modern stack. Without this bridge, IBM i monitoring stays in its own silo.
    Third, the OpenTelemetry Collector for batch DB2. This is the pattern that solved our most painful blind spot. We installed an OpenTelemetry Collector on the same host that already runs the Instana Agent - same machine that launches the batch jobs. The Collector receives traces emitted from the batch processes via OTLP, and forwards them to the Instana Agent, which acts as a native bridge to the backend. No new ports opened to the network, no new firewall rules, no extra infrastructure. The result is that every scheduled job execution becomes a distributed trace in the Instana UI - duration, sequence, every DB2 query that was issued, status of each step.
    Six-step flow diagram: Job Scheduler launches a Batch Process which emits OTLP spans via OpenTelemetry SDK; spans are received by the OpenTelemetry Collector co-located with the Instana Agent on the same host; the Agent forwards them to the Instana Backend for distributed trace storage and visualization.
    Figure 2 - OpenTelemetry batch tracing flow: from job scheduler to Instana backend without network changes
    On top of all this, we created a Custom Zone called "Rainbow" that groups the customer's hosts into a single logical perimeter. The dashboard for that zone is the customer's daily operational view; alert routing is scoped to it; access controls are applied at the zone level. Custom Zones cost nothing extra in licensing terms but they are the operational difference between a multi-tenant deployment that scales cleanly and one that spirals into alert fatigue and access-control confusion after the third customer is onboarded.
    Three custom zones (Rainbow, Aurora, Cascade) representing three customer perimeters within a single Instana backend, each containing different host types (AS/400, Power11 LPAR, AIX) and applications. The bottom band shows: single Instana backend, separate dashboards/alerts/access per zone, zero cross-tenant noise.
    Figure 3 - Multi-tenant segregation with Custom Zones: same backend, separate operational views

    Three lessons from the field

    1. The IBM i bridge requires upfront planning - it's not auto-discovery

    Unlike the IIS or SQL Server sensors that come up automatically the moment the Instana Agent is installed on a Windows or Unix host, the IBM i bridge is a separate component installed directly on the AS/400. You have to plan for it: install the bridge, configure the credentials securely (use a vault, not config files), open the right ports between the AS/400 and the Agent, and decide what to monitor at the OS level versus at the database level. Once the configuration is right, the bridge becomes invisible - but if you treat it as an afterthought, you'll discover gaps weeks into operations. Set it up early in the project, not late.

    2. OpenTelemetry tracing for batch DB2 is the unlock

    Without OpenTelemetry, batch monitoring is structurally limited: you see job duration, exit codes and log files. With it, you see the full call chain - which tables were hit, which queries took time, which dependent calls cascaded into anomalies. The instrumentation is non-trivial: it requires the OpenTelemetry SDK in the batch process and a configured Collector, plus discipline in deciding which spans are worth emitting. But the visibility unlock is generational. This is the pattern that, more than any other, paid back the entire deployment within the first quarter - both in terms of developer productivity (debugging long batch jobs that used to require log archaeology) and in terms of MTTR when something went wrong. If your estate has batch jobs critical to the business, this is where I would invest engineering time first.

    3. Custom Zones make multi-tenant deployments operationally clean

    On the technical side this is a small thing. On the operational side it's the difference between an Instana deployment that scales gracefully across customers and one that becomes an unmanageable alert-fatigue and access-control nightmare. Create a Custom Zone per tenant from Day 1. Scope alert routing to it. Constrain dashboards to it. The customer gets a clean operational view of their own perimeter; your support team isn't paged for someone else's incident; access permissions are simple to reason about. This is one of those choices that costs nothing if you make it on Day 1 and costs weeks of refactoring if you postpone it.

    Why this pattern is reusable

    The architecture described above isn't specific to a single industry or customer. It works for any Italian or EMEA mid-to-large enterprise where IBM i still runs mission-critical workloads and modern Java applications coexist with legacy batch jobs. We see this profile most often in banking, manufacturing, distribution and pharmaceutical companies - environments with twenty-year ERP estates that have been progressively wrapped with modern web layers but never fully replaced.

    It also works particularly well for managed-services providers who run multi-customer Instana deployments. The Custom Zones, OpenTelemetry pattern and IBM i bridge approach scale linearly across tenants without requiring per-customer rewiring of the backend.

    If you're navigating the same territory, two communities are good places to discuss the details: the IBM Instana Community for the product-side conversations on agent configuration and OTLP patterns, and the IBM i Community for the AS/400-side considerations on bridge integration and PTF compatibility. The intersection of the two - observability for legacy-rich estates - is underdiscussed and there's a lot of practitioner wisdom waiting to be shared.

    Closing

    Modern observability shouldn't end at the boundary of modern technology. The estates that pay your salary are usually the ones the marketing slides forgot - the AS/400 running the ERP that books every order, the DB2 batch that runs every night and that nobody understands fully anymore, the IIS frontend that's been working since 2014. Bringing them into the same distributed-tracing view as your Kubernetes cluster isn't glamorous, but it's where most of the operational value of observability actually lives.

    Happy to swap notes - and to be challenged on any of these views - with anyone working on similar terrain.

    #IBMChampions #IBMInstana #IBMi #IBMDb2 #OpenTelemetry #Observability #AIOps



    ------------------------------
    Roberto Renna
    ------------------------------

    Attachment(s)

    pdf
    Instana_IBMi_DB2.pdf   502 KB 1 version