Skip to content

[processor/filter] Standardize configuration on OTTL #18642

Open
@TylerHelmuth

Description

Is your feature request related to a problem? Please describe.

With #16369, OTTL was added to the filterprocessor. This allows us to standardize the way the filterprocessor configures conditions for when data should be dropped.

For spans, the existing non-ottl configuration options are:

  • matching by service name, span name, attributes, scopes, resource attributes, and span kind.
  • Both strict and regex is supported.
  • When checking attributes, converting attribute values from bool, double, and int to string is supported

For metric, the existing non-ottl configuration options are:

  • matching by metric name, resource attributes
  • Both strict and regex is supported.
  • matching by datapoint attributes via the expr expression engine.
  • When checking attributes, converting attribute values from bool, double, and int to string is supported.

For logs, the existing non-ottl configuration options are:

  • matching by resource attributes, attributes, severity text, body, and severity number.
  • Both strict and regex is supported.
  • When checking attributes, converting attribute values from bool, double, and int to string is supported.
  • When matching severity number, a minimum log severity can be used and any values equal to or greater than the severity match. You can also specify if an undefined severity number can be match.

With the completion of #16413, OTTL can now handle each of these use cases, and more, in a uniform library.

In the past we only had what was in internal/filter and as different components needed different condition features those packages grew. As users wanted access to different fields in the telemetry payload the signal-specific filters would change to reflect those needs. The filterprocessor grew with it, providing important value but with increased maintenance burden and configuration complexity. Reading through the filterprocessor's readme highlights the difficulty in maintaining and understanding the different features and fields available.

By unifying on OTTL we'll have a solution that has access to all fields on all signals. Users no longer need to worry about whether or not a field for their signal is available to use and maintainers no longer need to worry about adding more fields to filter on in the future (OTLP changes excluded). Due to OTTL's functions, adding more features to enable complex conditions is simpler as the functions encapsulate the logic and can be added without modifying the underlying libraries or configuration. On top of its field access and functions, OTTL's grammar also provides more robust conditions, allowing users to use inequalities, nil, and arithmetic.

Steps to completion

Metadata

Assignees

Labels

enhancementNew feature or requestnever staleIssues marked with this label will be never staled and automatically removedpkg/ottlpriority:p2Mediumprocessor/filterFilter processorroadmappingIssue describes several feature requests for a topic

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions