[bamaer]

• call the PipelineCompleted extension point before releasing waiters, and release them in a finally
• register the TransformFinished extension point ahead of the pipeline's completion detection
• fire TransformBeforeStart/TransformFinished for the SingleThreaded pipeline type
• flush the lineage hub in HopRun before System.exit

Please add a meaningful description for your change here

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Run mvn clean install apache-rat:check to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
• If you have a group of commits related to the same change, please squash your commits into one and force push your branch using git rebase -i.
• Mention the appropriate issue in your description (for example: addresses #123), if applicable.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004
you have to acknowledge this by using the following check-box.

• I hereby declare this contribution to be licensed under the Apache License Version 2.0, January 2004
• In any other case, please file an Apache Individual Contributor License Agreement.

[mattcasters]

[bug] This waiter calls waitUntilFinished() before startThreads() creates pipelineWaitUntilFinishedBlockingQueue. That method returns immediately while the queue is still null, so the waiter appends "waiter" and counts the latch down before PipelineCompleted runs. startThreads() invokes PipelineStartThreads and other extension points before allocating the queue, so the new thread often wins that window and the assertion sees [waiter, extension-point] even though the production order is correct. The 400ms wait inside the extension point does not cover this case: the waiter has already recorded itself.

Suggestion: Do not call waitUntilFinished() until the queue exists. It is assigned before setRunning(true), and an empty pipeline signals only after that. Spin while !line.isRunning() && !line.isFinished(), then wait. A late waiter still observes the object already on the queue, so the expected order stays deterministic without depending on the null-queue return.

[mattcasters]

[suggestion] Releasing the waiter here happens before a PipelineCompleted failure is visible on the result. The exception leaves fireExecutionFinishedListeners() only after this finally, including releaseVfsNamespace(), and the transform-finished listener increments the transform error count only after that. HopRun reads getResult().getNrErrors() as soon as waitUntilFinished() returns, so a throwing extension point can still produce exit code 0. ExtensionPointMap does not wrap runtime exceptions, so those take the same path.

Suggestion: Record the failure on the pipeline errors count before pipelineCompleted(). getResult() already adds that field. Drop the extra setErrors in the transform listener for this same failure, or the run is counted twice. Catch Exception, not only HopException.

[mattcasters]

[suggestion] LineageHub.shutdown() runs only when hopRun.run() returns. The ExecutionException and other catch paths call System.exit without draining. TransformBeforeStart can now throw from startThreads() after an earlier transform thread has already emitted, and the OpenLineage sink drains its own dispatcher only from shutdown().

Suggestion: Call shutdown() on those failure paths immediately before System.exit as well. A finally attached to this try does not run once System.exit has been entered from a catch. The call is already a no-op when the hub never started.