Skip to content

[Java] Honor GOOGLE_CLOUD_UNIVERSE_DOMAIN for the Dataflow endpoint - #40672

Open
shunping wants to merge 1 commit into
apache:masterfrom
shunping:tpc-2
Open

shunping wants to merge 1 commit into
apache:masterfrom
shunping:tpc-2

Conversation

@shunping

@shunping shunping commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #40569, Builds upon #40670 (needs universe domain support from googleapis/google-api-java-client#2435)

Problem

The Java Dataflow runner always sends requests to https://dataflow.googleapis.com/, even when GOOGLE_CLOUD_UNIVERSE_DOMAIN points to another universe.
apiRootUrl defaults to https://dataflow.googleapis.com/, and DataflowTransport always calls Dataflow.Builder#setRootUrl. In google-api-client, calling setRootUrl (even with the default URL) marks the endpoint as user-configured, so the library never builds the https://dataflow.<universe>/ endpoint.

Changes

  • apiRootUrl no longer has a default.
  • DataflowTransport sets the root URL and service path only when apiRootUrl or dataflowEndpoint is set. Otherwise the client library resolves the endpoint, honoring GOOGLE_CLOUD_UNIVERSE_DOMAIN and falling back to the public URL.
dataflowEndpoint apiRootUrl Endpoint
unset unset resolved by client library (universe-aware)
relative path unset universe-aware root + path
relative path / unset set apiRootUrl (+ path)
absolute URL any dataflowEndpoint

Breaking change

DataflowPipelineDebugOptions.getApiRootUrl() now returns null by default instead of https://dataflow.googleapis.com/.

@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (76f6e6d) to head (a6f8095).
⚠️ Report is 247 commits behind head on master.

Additional details and impacted files
@@              Coverage Diff              @@
##             master   #40672       +/-   ##
=============================================
- Coverage     56.16%        0   -56.17%     
=============================================
  Files          1124        0     -1124     
  Lines        178336        0   -178336     
  Branches       1489        0     -1489     
=============================================
- Hits         100170        0   -100170     
+ Misses        75645        0    -75645     
+ Partials       2521        0     -2521     
Flag Coverage Δ
go ?
java ?
python ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@shunping
shunping marked this pull request as ready for review October 20, 2026 02:41
@shunping
shunping marked this pull request as draft October 20, 2026 02:41
github-actions[bot]

This comment was marked as outdated.

apiRootUrl defaulted to https://dataflow.googleapis.com/, so
DataflowTransport always called setRootUrl, which stopped
google-api-client from honoring GOOGLE_CLOUD_UNIVERSE_DOMAIN.

apiRootUrl now has no default, and the root URL and service path are set
only when apiRootUrl or dataflowEndpoint is given.
@shunping

Copy link
Copy Markdown
Collaborator Author

/gemini review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📋 Review Summary

This PR updates the Java Dataflow runner to defer Dataflow API root URL resolution to the Google API client library when neither --apiRootUrl nor an absolute --dataflowEndpoint is explicitly configured, enabling GOOGLE_CLOUD_UNIVERSE_DOMAIN support. The implementation, nullability annotations, documentation updates, and unit test coverage in DataflowTransportTest are clean and well-structured.

🔍 General Feedback

  • Removing @Default.String(Dataflow.DEFAULT_ROOT_URL) from DataflowPipelineDebugOptions#getApiRootUrl() and conditionally calling builder.setRootUrl(...) in DataflowTransport accurately avoids setting isUserConfiguredEndpoint = true in AbstractGoogleClient.Builder, preserving universe domain resolution.
  • The subtle behavior change for callers inspecting getApiRootUrl() (now returning null when unset) is appropriately annotated with @Nullable and documented in CHANGES.md.

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

@shunping
shunping marked this pull request as ready for review October 20, 2026 13:59
@github-actions

Copy link
Copy Markdown
Contributor

Assigning reviewers:

R: @chamikaramj added as fallback since no labels match configuration
Backup reviewer: damccorm (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant