Skip to content

Expose the pipeline hop graph on lineage events, fixes #9004 - #9042

Merged
mattcasters merged 1 commit into
apache:mainfrom
bamaer:9004
Oct 21, 2026
Merged

mattcasters merged 1 commit into
apache:mainfrom
bamaer:9004

Conversation

@bamaer

@bamaer bamaer commented Oct 20, 2026

Copy link
Copy Markdown
Contributor
  • add the PIPELINE_GRAPH event kind with PipelineGraphLineagePayload (enabled hops, in pipeline order)
  • emit it once per pipeline run, next to STARTED, with the same context
  • document that sinks skip event kinds they do not handle
  • the OpenLineage sink ignores the new kind, so its output does not change yet

Please add a meaningful description for your change here


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
  • If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
  • Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

- add the PIPELINE_GRAPH event kind with PipelineGraphLineagePayload (enabled hops, in pipeline order)
- emit it once per pipeline run, next to STARTED, with the same context
- document that sinks skip event kinds they do not handle
- the OpenLineage sink ignores the new kind, so its output does not change yet
@mattcasters
mattcasters merged commit 0c7da4d into apache:main Oct 21, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants