Skip to content

Latest commit

 

History

History

flink

README.md

OpenLineage Flink

The OpenLineage Flink integration uses JVM instrumentation to emit OpenLineage metadata. See more docs here.

Installation

Requires running in application mode with setting execution.attached: true. If execution.attached is false, we don't receive proper information about job completion.

Maven:

<dependency>
    <groupId>io.openlineage</groupId>
    <artifactId>openlineage-flink</artifactId>
    <version>1.26.0</version>
</dependency>

or Gradle:

implementation 'io.openlineage:openlineage-flink:0.16.0'

Usage

An instance of OpenLineageFlinkJobListener need to be created and registered as jobListener.

This can be achieved by:

StreamExecutionEnvironment env = ...

JobListener jobListener = OpenLineageFlinkJobListener.builder()
    .executionEnvironment(env)
    .jobNamespace(jobNamespace)
    .jobName(jobName)
    .build();

env.registerJobListener(jobListener);

Alternatively, you can pass in job name and namespace via Flink configuration instead of specifying it in the builder

execution.job-listener.openlineage.job-name : "custom job name"
execution.job-listener.openlineage.namespace : "custom job namespace"

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2025 contributors to the OpenLineage project