Skip to content

AIRGAP (VM test needed): restrict tnctl access tokens, tnctl realm client, shape gateway denials - #101

Open
tekn0ir wants to merge 5 commits into
teknoir-localfrom
airgap/access-restrictions-gateway
Open

tekn0ir wants to merge 5 commits into
teknoir-localfrom
airgap/access-restrictions-gateway

Conversation

@tekn0ir

@tekn0ir tekn0ir commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Not verified in a VM yet. There is no live airgapped cluster; this needs one VM run (fresh install, update from the current release, idempotent re-run, then the second sync below) on the dev host, only when Anders asks for it. Merge only after that run. The offline checks below all pass.

Five commits, ports of the staging changes (gitops #90 access template, #97 checksum, #91 gateway) adapted to this branch, plus the release pins.

What it does

  1. A3 backstage: restrict the Keycloak externalAccess principal with accessRestrictions. Backstage service principals skip the permission policy; a tnctl (teknoir-cli) access token would reach every plugin of backstage-api, including the kubernetes proxy, which runs with backstage-api's cluster-wide ServiceAccount (checked on staging: DeviceList readable with a developer's token). Template renders config.externalAccess.accessRestrictions into the jwks entry; default is the P0 list catalog.entity.read + plugin device (the target [{plugin: device}] once tnctl uses the user-scoped fleet routes is documented in the values comment). A plugin listed twice fails at template time (Backstage refuses it at start). The rendered config was run through Backstage's real readAccessRestrictionsFromConfig (backend-defaults 0.18.0): catalog -> catalog.entity.read, device.
  2. Checksum backstage: roll backstage-api when the app-config changes. Backstage reads its config only at start; without this the restriction stays inactive until a manual restart. checksum/app-config on the backstage-api pod template only (the frontend does not mount the app-config).
  3. K3 auth: add the tnctl client teknoir-cli to realm teknoir. The realm file gets the public client teknoir-cli (device flow only; no password grant, no browser code flow; audience mapper teknoir-cli), copied from staging's master.yaml. oauth2Proxy.oidc.cliClientId stays "" by default. Reason: an extra JWT issuer makes oauth2-proxy run OIDC discovery at start. On a fresh install the realm does not exist until the PostSync import, so oauth2-proxy would crash-loop, ArgoCD would never reach PostSync, and the realm would never be imported (deadlock). The template already had {{- with .Values.oauth2Proxy.oidc.cliClientId }} here, so no template change; this commit adds its tests. The auth README documents the second sync: release the auth chart with cliClientId: teknoir-cli (the Applications take no Helm overrides), after the first install has converged and with the Backstage restriction already rolled out. Without it the exposure is not reachable: oauth2-proxy rejects teknoir-cli tokens at the gateway.
  4. G5 auth: shape denied requests at the gateway. Port of auth: 302 login redirect for browsers only, 401 challenge for API callers (gateway denials) #91. Browser navigations denied by the edge authorizer still get the login redirect (302, empty body, Content-Length: 0, no-store); every other denied request (fetch/XHR/WebSocket/tnctl) gets 401 + WWW-Authenticate: Bearer realm="teknoir" + a small JSON body; a backend 401/403 after ext_authz passed the request is delivered unchanged (marker filter envoy.filters.http.lua.authz_passed, INSERT_AFTER ext_authz). This branch has no portNumber: 443 scoping (as before), and the new patches use exactly the ext_authz sub-filter match of the existing Lua patch (unit-tested). Contract in charts/auth/README.md.
  5. release: auth 0.0.9, backstage 0.0.84, app-of-apps 0.0.5. A chart version is released once (Harbor keeps the first push), and tests/render.sh enforces the pins. Drop this commit if you batch the versions with other changes.

What does NOT change (proof)

Parsed render of charts/auth (33 documents before and after G5): only the EnvoyFilter differs; the ext_authz patch, the workload selector and all five vhost/route exemption patches are JSON-identical, one patch (the marker) is added (7 -> 8). oauth2-proxy args are unchanged by this PR (cliClientId stays empty).

Test results

  • helm unittest charts/auth charts/backstage: 5 suites, 24 tests pass (new: extra issuer, realm client, gateway filters, accessRestrictions, checksum).
  • tests/render.sh (render, pins, crds, apps, project, credentials, determinism for all 14 charts): 0 failed checks, 10 warnings (all existing mutable-tag/Harbor-default WARNs).
  • helm template | kubeconform -strict -ignore-missing-schemas: auth (with keycloakx) 24 valid / 0 invalid / 13 skipped; backstage 16 valid / 0 invalid / 7 skipped.
  • charts/auth/tests/realm-docker-test.sh against Keycloak 26.5.3 and keycloak-config-cli 6.5.1-26.5.5 (docker, cached images): 22 passed, 0 failed, including the new R11: teknoir-cli refuses the password grant and the browser code flow; a real device-flow login as platform-admin (device authorization, consent, polling) returns an access token with aud containing teknoir-cli, azp: teknoir-cli, and the realm teknoir issuer. All earlier checks (R1-R10) still pass: the extra client breaks nothing in the import.
  • Offline Envoy harness charts/auth/tests/gateway/run.sh (istio/proxyv2:1.29.2, Envoy 1.37.2) on the rendered EnvoyFilter: 771 checks, 771 pass. On the old template the same harness fails 197 of 771. Because this chart needs _helpers.tpl and the harness matrix hard-codes the cookie name _teknoir_oauth2_proxy_v2 and domain teknoir.test, the EnvoyFilter was rendered with --set domain=teknoir.test --set oauth2Proxy.cookie.name=_teknoir_oauth2_proxy_v2 and passed as RENDERED=<file> (the harness itself is byte-identical to staging; the exact command is in the README). The real Istio filter order was not exercised (no cluster).
  • istioctl analyze --use-kube=false on the rendered EnvoyFilter: no issues.

VM run to do (when Anders asks)

  1. Fresh install of this version (teknoir-airgap, VM rig in infra/airgap/test/vm): auth Synced, keycloak-config Job completed, realm teknoir has client teknoir-cli; oauth2-proxy args have no --extra-jwt-issuers; backstage-api running; browser login at https://teknoir.airgapped works (trust the rig's CA); denied API call -> 401 JSON, browser navigation -> 302.
  2. Update from the currently released versions (auth 0.0.8, backstage 0.0.83, app-of-apps 0.0.4): realm import adds the client, backstage-api restarts once (Recreate), EnvoyFilter updates live.
  3. Idempotent re-run: no diff, no pod restarts.
  4. Second sync: set cliClientId: teknoir-cli as the auth default in a test release; oauth2-proxy rolls and becomes Ready (OIDC discovery with the install's CA); tnctl login --domain teknoir.airgapped --realm teknoir works; with its token GET /api/kubernetes/clusters and /api/search/query are 403, tnctl devices works (catalog read + device); catalog writes are 403.

Rollback

Revert the release commit and the commit you want to undo, publish new chart versions (a version is released once), pin them. Turning tnctl off again without a release is not possible, because the Applications take no value overrides; the safe state is the default (cliClientId: "").

🤖 Generated with Claude Code

tekn0ir and others added 5 commits October 19, 2026 14:48
…Restrictions

Backstage service principals skip the permission policy. A tnctl (teknoir-cli)
access token would therefore reach every plugin of backstage-api, including the
kubernetes proxy, which runs with backstage-api's cluster-wide ServiceAccount.
On the airgapped install this is only reachable once oauth2Proxy.oidc.cliClientId
is set, but the restriction must be in place before that.

Render config.externalAccess.accessRestrictions into the jwks entry. Default is
the P0 list (catalog.entity.read plus the device plugin); the target once tnctl
uses the user-scoped fleet routes is [{plugin: device}]. A plugin listed twice
is refused at template time because Backstage refuses it at start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnotation)

Backstage reads its config only at start, so the accessRestrictions of the
previous commit would stay inactive until backstage-api was restarted by hand.
Add a checksum/app-config annotation to the backstage-api pod template (same
fix as staging, but only on the API: the frontend does not mount the app-config).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stays off)

Realm teknoir gets the public client teknoir-cli (OAuth device flow only, no
password grant, no browser code flow, audience mapper teknoir-cli), copied from
staging. oauth2Proxy.oidc.cliClientId stays "" by default: an extra JWT issuer
makes oauth2-proxy run OIDC discovery at start, so on a fresh install it would
crash-loop before the PostSync realm import, and ArgoCD would never reach
PostSync. The README documents the second sync that turns it on.

Tests: helm unittest for the extra issuer (default off, domain/realm/client id,
trailing slash) and the realm client; realm-docker-test.sh R11 runs a real
device-flow login against Keycloak 26.5.3 and checks aud teknoir-cli and the
realm teknoir issuer. backstage .helmignore excludes tests/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… APIs)

Port of the staging change (gitops #91) to the airgapped EnvoyFilter. The
allow decision is untouched: the ext_authz patch, the workload selector and
every per-vhost/per-route exemption are JSON-identical to before. Only the
denial Lua changes, and a marker Lua filter (INSERT_AFTER ext_authz) is added so
a backend 401/403 passes unchanged.

A browser navigation that the edge authorizer denied is redirected to login;
every other denied request (fetch, WebSocket, tnctl) gets 401 with
WWW-Authenticate and a small JSON body. The new patch uses the same listener
scoping as this branch's other patches (no portNumber 443).

Adds the helm unittest suite, the offline Envoy harness (charts/auth/tests/gateway)
and the contract in charts/auth/README.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A chart version is released once (Harbor keeps the first push), so the changes
of the previous commits ship as new versions: bump the two charts, pin them in
app-of-apps, bump app-of-apps. Drop this commit if the versions are batched with
other changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant