TL;DR: IBM watsonx.data integration supports data observability across your entire data pipeline ecosystem - watsonx.data integration, Airflow, Databricks, AWS Glue, from a single unified interface.

The Multi-Platform Monitoring Challenge

If you're a data engineer in 2026, you're probably managing a heterogeneous technology stack. Maybe you orchestrate with Apache Airflow, and transform data an integration style with watsonx.data integration (batch and streaming). Each tool has its own monitoring dashboard, its own alerting system, and its own way of telling you when things go wrong. Sound familiar?

Here's what this fragmentation costs you:

• Context-Switching Overhead: You're constantly jumping between the Airflow UI, DataStage Operations Console, and StreamSets Control Hub. Each context switch takes time and mental energy.

• Manual Failure Correlation: When a DataStage job fails at 3 AM, you have to manually trace back through Airflow logs to figure out if an upstream DAG issue caused it. By the time you've connected the dots, your SLA is already breached.

• Alert Fatigue: You've configured alerts separately in each tool. Now you're getting notifications from email, Slack, PagerDuty, and Teams - often for the same underlying issue - which one do you respond to first?

Introducing Off-Platform Data Observability

Today, we're excited to announce that IBM watsonx.data integration now supports off-platform asset data observability, with built in support for the OpenLineage industry format. The tools push information through the Open Lineage to watsonx.data integration.  This leads to the ability to monitor anything that works with open lineage. This means you can now monitor your Airflow DAGs, AWS glue, Databricks and others alongside your DataStage jobs - all from a single, unified interface.

What You Get

• Single, Unified Interface: See the health of your entire data pipeline ecosystem in one dashboard—no more context-switching required.

• Consistent Alerting: Configure your alert rules once in watsonx.data integration. They work across all asset types. One notification channel, one alert history, one place to acknowledge and resolve issues.

• Extensible By Default: Adapt your observability investment as your stack evolves, with built-in support for OpenLineage - the open industry standard.

Use this example as a reference to build and send your own OpenLineage event payload to the Data Observability API.

curl -k -X POST \
"https://{url}/data-obs/v1/lineage" \
-H 'Content-Type: application/json' \
-H "Authorization: Basic $AUTH" \
-d '{
"eventType": "COMPLETE",
"eventTime": "2026-07-16T07:24:00Z",
"inputs": [],
"job": {
"facets": {
"IBMworkspace": {
"_producer": "https://some.producer.com/version/1.0",
"_schemaURL": "https://openlineage.io/spec/facets/1-0-0/IBMWorkspaceFacet.json",
"project_id": "project_id"
},
"jobType": {
"integration": "my-engine",
"processingType": "batch"
},
"externalLink": {
"_producer": "https://some.producer.com/version/1.0",
"_schemaURL": "https://openlineage.io/spec/facets/1-0-0/ExternalJobLinkFacet.json",
"url": "https://link.to.your.job.com"
}
},
"namespace": "your-job-namespace",
"name": "your-job-name"
},
"outputs": [],
"run": {
"facets": {
"tags": {
"_producer": "https://some.producer.com/version/1.0",
"_schemaURL": "https://raw.githubusercontent.com/OpenLineage/OpenLineage/main/spec/
OpenLineage.json#/definitions/BaseFacet",
"runName": "test_run_name"
},
"metrics": [
{
"metricName": "rows_processed",
"timestamp": "2026-07-16T07:24:00Z",
"metricValue": 4763.99,
"_producer": "https://some.producer.com/version/1.0",
"_schemaURL": "https://raw.githubusercontent.com/OpenLineageOpenLineage/main/spec/
OpenLineage.json#/definitions/BaseFacet"
},
{
"metricName": "execution_time_ms",
"timestamp": "2026-07-16T07:24:00Z",
"metricValue": 2024.9,
"_producer": "https://some.producer.com/version/1.0",
"_schemaURL": "https://raw.githubusercontent.com/OpenLineageOpenLineage/main/spec/
OpenLineage.json#/definitions/BaseFacet"
}
],
"externalLink": {
"_producer": "https://some.producer.com/version/1.0",
"_schemaURL": "https://openlineage.io/spec/facets/1-0-0/ExternalRunLinkFacet.json",
"url": "https://link.to.your.run.com"
}
},
"runId": "10dd62e6-f4f0-4b72-94c3-be43fa90f3b3"
},
"producer": "https://custom.api",
"schemaURL": "https://openlineage.io/spec/1-0-5/OpenLineage.json#/definitions/RunEvent"
}'

See how to set up an Airflow DAG SLA Alert https://www.youtube.com/watch?v=XGf6tAC8GmY 

If you haven’t tried IBM watsonx.data integration yet, get started with a free trial by clicking here.