[shunping]

Summary

On Dataflow, GcsUtil V2 (java-storage) was up to 36% slower than V1 (gcsio) for workloads with thousands of small files. This PR makes three changes to GcsUtilV2 that bring it within noise of V1:

1. Use the shared HTTP transport. V2 clients now use the process-wide Transport.getTransport(), the same one V1 uses. Before, each client built its own NetHttpTransport. With --gcsPerformanceMetrics, V2 builds a client on every open and create, so every file got a separate connection pool, and most ranged GETs paid a fresh TLS handshake.
2. Start upload sessions in the background. java-storage's Storage.writer() blocks while it starts the resumable upload session. create() now does that on a background thread, and the channel waits for it at the first write() or at close(). gcsio does the same for V1.
3. Raise the default rewrite concurrency from 32 to 128. java-storage can't batch rewrites, so copy and move run files in parallel. A call never uses more threads than it has files, so only copies and moves of more than 32 files are affected. --gcsMaxConcurrentRewrites still overrides the default.

Performance

Setup: ParquetIOLT on Dataflow, f10_s1k (40 GB), 3 × e2-standard-2, legacy worker, one run per arm, so treat differences under about ±15% as noise. Each row shows the effect of adding that change on top of the previous ones. V1 served as a control and stayed within ±7%.

Change	Metric	Before → after (V2)	V2 vs V1 gap
Shared HTTP transport	Read, 4,096 shards	1,372 → 1,187 s	+14% → +0.3%
	Glob, 4 pages	2,255 → 2,216 ms	+365 → +185 ms
Background upload sessions	Time per create, 4,096 shards	85 → 2.2 ms (summed 348 → 9 s)	V1: 1.3 ms
	Write, 4,096 shards	455 → 406 s	+20% → +9%
Rewrite concurrency 128	Rename, 4,096 shards (summed)	36.3 → 13.0 s	3.9× → 1.5× V1
	Write, 4,096 shards	406 → 371 s	+9% → +4.5%

All three together, compared with master:

Shards	Write gap (V2 vs V1)	Read gap (V2 vs V1)
4,096	+15% → +4.5%	+14% → parity
16,384	+36% → +3.9% (create 1,537 → 14 s summed; rename 77.7 → 33.6 s)	not yet measured

Bytes and GCS request counts are unchanged; only connection setup, session-start latency and parallelism changed.

Behavior changes

• GcsUtilV2.create() no longer reports errors from starting the upload session. They surface at the first write() or at close(), as an IOException (e.g. AccessDeniedException), the same as V1.
• The GcsInsert API-request metric is recorded when the session start actually succeeds or fails, not when create() returns.
• The default for --gcsMaxConcurrentRewrites is now 128 (it was 32), but the actual number of threads is MIN(total_number_of_files, maxCurrrentRewrites).

Notes

• Rename is still about 1.5× V1 per file, because V1 batches its rewrites. Batching V2's destination lookups and source deletes with StorageBatch is a possible follow-up.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 0.00%. Comparing base (76f6e6d) to head (cf3aa1d).
⚠️ Report is 242 commits behind head on master.

Additional details and impacted files

@@              Coverage Diff              @@
##             master   #40677       +/-   ##
=============================================
- Coverage     56.16%        0   -56.17%     
=============================================
  Files          1124        0     -1124     
  Lines        178336        0   -178336     
  Branches       1489        0     -1489     
=============================================
- Hits         100170        0   -100170     
+ Misses        75645        0    -75645     
+ Partials       2521        0     -2521     

Flag	Coverage Δ
go	?
java	?
python	?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Assigning reviewers:

R: @chamikaramj for label java.
Backup reviewer: kennknowles (if the main reviewer is unresponsive, you can escalate to them by tagging their username)

This pull request likely touches a core component ("core" label). Please review with scrutiny.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[github-actions]

📋 Review Summary

This PR effectively closes the small-file performance gap between GcsUtilV2 and GcsUtilV1 by sharing the process-wide HTTP transport (Transport::getTransport), opening resumable upload sessions asynchronously on a background thread pool, and increasing default rewrite concurrency from 32 to 128. The changes are clean, well-documented, and thoroughly tested.

🔍 General Feedback

• The asynchronous WriteChannel initialization in GcsWritableByteChannel properly handles exception translation, thread interruption, and metric recording (ServiceCallMetric and MetricsContainer binding).
• Sharing Transport::getTransport across Storage instances resolves connection pool fragmentation when --gcsPerformanceMetrics creates per-operation scoped clients.
• Unit tests in GcsUtilV2Test cover background execution, error propagation on write/close, and precondition/chunk-size verification.

---

💡 Tip: Comment /gemini review on this pull request to trigger a new Gemini code review.

[shunping]

Also adding @damccorm since that's some more changes I mentioned offline to close the performance gaps between GcsUtil V2 and V1.

[damccorm]

Nice investigation and benchmark results! Left two minor comments on GcsWritableByteChannel.

[damccorm]

Thanks!