Repository navigation
Conversation
️✅ There are no secrets present in this pull request anymore.If these secrets were true positive and are still valid, we highly recommend you to revoke them. 🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request. |
…) credentials An operator-seeded BigQuery connection can now carry an external_account config. Studio fetches the subject token from credential_source.url, exchanges it at Google STS for an access token, and repeats both before the token expires, so per-person access no longer needs a key per person or an outside job rewriting short-lived access_tokens. - url sources only, over https unless the host is this machine; the fetch runs through the egress guard and never follows redirects - STS at https://sts.googleapis.com/v1/token only; jwt and id_token subject types; no service-account impersonation - refused on user-created connections, since the server would fetch a URL the user named Closes libredb#2278 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
45303a1 to
d593ccc
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
@cevheri thanks for approving the workflow runs. All checks are green now and the branch has no conflicts. Whenever you have time, I'd appreciate a review. The change is limited to the BigQuery provider: it accepts |
|
We want this feature but are not merging it in this form: BigQuery and Firestore carry duplicate Google credential code (D294 in docs/BACKLOG.md), and adding external_account to one copy widens that split. We are first moving Google auth into src/lib/db/auth/google in PRs we will link here; once they land, could you rebuild this change on top of it? The main points: external_account lives in the shared module and BigQuery opts in through its policy; it is accepted on a managed connection the operator's seed file or the governance control plane writes, never on one pasted in the browser or from a literal-mode seed file, with no server opt-in, so it works in governance mode as #2278 asks; subject and STS requests use module-built lanes that refuse link-local addresses and protocol upgrades; repeated or transport-owned header names and workforce pool audiences are refused; fields that change what is sent are refused, not ignored; an STS answer without expires_in is refused. We will post the full checklist once those PRs are merged. When the rebuild is ready, please test it against Google again, as you did here, and share the result. Thank you for the PR and the live test. |
|
The shared Google auth is now on main (#2299, #2302, #2305, #2306), so this can be rebuilt on top of it. The checklist:
Then the live test against Google, please. |
|
Thanks for landing the shared Google auth and for the checklist. It is clear, and I agree with the direction: the mechanism in the shared module, BigQuery opting in through its policy, and external_account accepted only on a server-written connection. I will rebuild this on top of main following the eight points, and run the live test against Google again when it is ready. |
Description
Adds
external_account(Google Workload Identity Federation) credentials to the BigQuery provider, as proposed in #2278. An operator-seeded connection can carry a standardexternal_accountJSON. Studio then gets a federated access token itself (subject token → Google STS) and renews it before it expires. This lets a deployment give each person their own BigQuery identity, with grants per table, without keys per person or a sidecar that rewrites short-livedaccess_tokens.Type of Change
Related Issue
Closes #2278
Changes Made
auth.ts: parsesexternal_account. It fetches the subject token fromcredential_source.url(textorjsonformat, optional headers) and exchanges it athttps://sts.googleapis.com/v1/token(grant_type=token-exchange, BigQuery scope). Caching and refresh are shared with the other credential types: the token is renewed 60 s before expiry, and the subject token is fetched again each time.transport.ts: a second single-socket lane, pinned to the subject token's origin, with TLS checked against that host. It uses the same egress guard (DB_HTTP_BLOCK_PRIVATE_HOSTS), follows no redirects, and is closed together with the rest of the transport.index.ts:external_accountis accepted only for operator-seeded (managed) connections. On a user-created connection the server would fetch whatever URL the user typed, so it is refused before anything is sent.service_account_impersonation_urlfile,executableand environment sourcestoken_urlsjwtandid_tokenhttpto anything other than this machinedocs/providers/bigquery.md: a new "External account" section with an example and the rules above. It also removesexternal_accountfrom the list of refused types.service_accountkey.Testing
Automated tests:
tests/unit/db/bigquery/external-account.test.ts, which is new, plus additions to the transport, provider and token-source tests.tests/integration/db/bigquery-provider.test.ts:SELECT 1with the federated bearerChecks:
format,lint,typecheck,knip,security:checkandbuildall pass.Full suite: green, except one unrelated editor test, "WITH list of 1,000,000 CTEs", which hit its 5 s limit once under full parallel load and passes on its own.
Live against Google (a real project, a real workload identity pool, and an OIDC provider with an uploaded JWKS):
roles/bigquery.dataViewergrant toprincipal://…/subject/test-personwas given on one table only.SELECT * FROM <dataset>.ordersreturned rows, and the bigger<dataset>.salariestable →Access Denied.Not covered live:
httpfrom the same machine, which is the loopback exception. Thehttpspath to a remote host is covered by the unit and integration tests only.Test Environment
Screenshots
The granted table returns rows, and the table without a grant is refused by Google:
Checklist
src/lib/db/providers/, I updated the matchingdocs/providers/documentation andtests/integration/db/tests in the same PR (provider triad)service_accountkey.Additional Notes
The sidebar's "Nothing to show" in the screenshots is expected. The test identity was granted one table and nothing at the dataset level, so it can't list datasets. That is the per-table setup this feature is meant for.
🤖 Generated with Claude Code