As Node.js applications scale, controlling the volume of tracing data becomes essential for both performance and cost management. Starting from version 4.9.0 of the Node.js tracer, IBM Instana introduces advanced endpoint filtering, which allows you to selectively ignore specific traces or calls, not just by method, but also by endpoint, particularly for Kafka.

This new capability provides greater precision in trace data collection, helping you focus observability on high-value operations while minimizing noise.

What is new: Advanced filtering with Kafka

Previously, trace filtering in Instana was limited to method-level exclusions, for example get in Redis or query in DynamoDB. See the previous blog post for details.

With the latest update:

• Kafka filtering supports both method-based and endpoint-based exclusion.

• You can precisely control which Kafka operations and topics should be traced or ignored.

Why ignore some endpoints?

“Not all operations are equally important. Filtering out noise keeps your monitoring clean and actionable.”

Ignoring some endpoints is particularly useful when:

• Operations are high-frequency but low-risk. For example, cache lookups and metadata fetches.

• You want to reduce ingestion volume and storage costs.

• You need to improve application performance by minimizing tracing overhead.

• You aim to declutter dashboards for better observability focus.

How Kafka filtering Works

Filtering is based on three parameters:

Filtering Rules

• If a span is ignored, all downstream spans are also ignored.

• You can use * wildcard to ignore all methods or endpoints.

• Endpoint values (for example, Kafka topic names) must be consistent across services.

• Always verify method and endpoint names in the Instana UI to make sure correct filtering.
  
  

Impacts and considerations

• Missed spans: Instana does not record or show Excluded operations. For example, any trace data related to those operations, such as Kafka send or consume, you will not see any related data from the trace view.

• Trace continuity impact: If Instana filters a Kafka span that is part of a longer trace, it may create gaps in the trace view. These gaps can potentially disrupt the continuity of service interactions. This is especially important in distributed systems, where intermediate spans are vital for understanding the complete service flow.

• Suppression of downstream calls: When Instana ignores a span, all downstream calls linked to it are also ignored by default. For example, if a Kafka operation (send to a specific topic) is excluded, any subsequent consume operations or other downstream traces will not appear in the trace, even if they are critical for understanding the overall service interaction.

Configuration methods

You can configure ignoring endpoints in three ways, depending on your setup:

1. Environment variables

This method is ideal for dynamic or containerized environments where you prefer to configure your application without modifying source code.

Use a YAML configuration file:

Set the INSTANA_IGNORE_ENDPOINTS_PATH environment variable to point Instana to a YAML configuration file. This approach supports detailed filtering by both method and endpoint.

INSTANA_IGNORE_ENDPOINTS_PATH=/absolute/path/to/config.yaml

Example config.yaml:

tracing:
  ignore-endpoints:
    kafka:
      - methods: ["consume"]
        endpoints: ["topic1", "topic2"]
      - methods: ["send"]
        endpoints: ["topic3"]

This configuration tells Instana to ignore:

• consume operations for topic1 and topic2

• send operations for topic3

Use simple method-only filtering:

Set the INSTANA_IGNORE_ENDPOINTS environment variable, If you need a quicker, more concise setup.

INSTANA_IGNORE_ENDPOINTS=kafka:send

This configuration tells Instana to ignore:

• All send operations in Kafka

2. In-code configuration

This method is for developers who prefer managing configurations programmatically can define ignored endpoints within the application itself.

Filtering by method only:

require('@instana/collector')({
  tracing: {
    ignoreEndpoints: {
      kafka: ['send']
    }
  }
});

This code snippet configures the Instana collector to ignore all send operations for Kafka.

Filtering by method and endpoint:

require('@instana/collector')({
  tracing: {
    ignoreEndpoints: {
      kafka: [
        { methods: ['consume'], endpoints: ['topic1', 'topic2'] },
        { methods: ['send'], endpoints: ['topic3'] }
      ]
    }
  }
});

This configuration provides the same granularity as the YAML configuration, but within the codebase, offering better visibility and version control for developers.

3. Agent configuration

This method is used for centralized management, especially in environments with many microservices, you can define ignored endpoints within the Instana agent’s configuration.yaml. This method allows you to apply consistent ignore rules across all services monitored by the agent, without touching individual application code or environment settings.

Example configuration:

com.instana.plugin.generic.trace:
  tracing:
    ignore-endpoints:
      kafka:
        - methods: ["consume"]
          endpoints: ["topic1", "topic2"]
        - methods: ["send"]
          endpoints: ["topic3"]
      redis:
        - methods: ["get"]
      dynamodb:
        - methods: ["query", "scan"]

With this configuration:

• Kafka tracing is disabled for consume on topic1 and topic2, and for send on topic3.

• Redis get and DynamoDB query or scan operations are also excluded.

To Apply:

1. Save the configuration.yaml.

2. Restart the Instana agent.

Trace view: Before and after Kafka endpoint filtering

The following examples illustrate how filtering out specific Kafka operations, such as send and consume for selected topics impacts the trace view in the Instana UI.

Before filtering:

Without filtering, Instana traces all Kafka activity. This includes both send and consume operations.

After filtering:

After you apply advanced endpoint filtering (for example ignoring send to test-topic), Instana excludes the send operation and all related downstream calls. This results in a cleaner trace view. Unnecessary spans are completely filtered out, reduces noise and improves trace clarity.

Limitations

• Supported technologies: Redis, DynamoDB, and Kafka.

• Advanced filtering (method + endpoint): Supported only for Kafka.

• Tracer availability: This feature is available on a limited set of tracers, such as Node.js and Java tracers.

Conclusion

IBM Instana’s advanced Kafka trace filtering gives you detailed control over observability data to trace. This feature enables you to:

• Reduce system overhead

• Lower trace ingestion costs

• Focus monitoring on what truly matters

When used correctly, this feature helps you enhances the clarity, efficiency, and effectiveness of your observability strategy.

Further reading

• Instana Node.js Tracing Documentation

• Configuration Guide: Ignoring Endpoints

• @instana/collector on npm

• Blog: introduction to the ignoring endpoints feature

#Tracing