Repository navigation
fix(security): authorize APIs and harden allocation and uploads - #104
Conversation
Comprehensive review 2026-10-13, findings C-1/C-2/C-3/H-4/M-5 (also found independently as N1/N2/N5): the identity management API was authentication-only. Every route under /v3 (except /auth/*) honored any valid token, so a plain member could: - reset any user's password — including the admin's — via PATCH /users/:id (which, unlike ChangePassword, never asks for the original) [C-1/N1, account takeover] - grant itself the admin role via PUT /projects/:id/users/:uid/roles/:rid and revoke anyone else's [C-2/N2, privilege escalation] - dump every tenant's recoverable EC2/S3 secret via GET /credentials, and mint/overwrite/delete credentials for other users [C-3] - mint unrestricted application credentials for any user, receiving the plaintext secret in the 201 [H-4/N5] - enumerate users, projects, roles, assignments, groups, domains, services, endpoints [M-5] Fix: the same self-or-admin check the EC2-credentials API and ChangePassword already use, plus an admin-only variant, applied across the surface (new internal/keystone/authz.go helpers): - users: List/Create admin-only; Get/GetProjects/GetGroups self-or-admin; PATCH self-or-admin, and password/enabled stay admin-only even for self (ChangePassword keeps its original-password contract) - role assignment: Assign/Unassign/List/Check admin-only, with existence checks for project/user/role (the old code 204'd assignments referencing nonexistent rows) - projects, groups, roles, role assignments, domains, services, endpoints: admin-only management; catalog reads admin-only - generic credentials: List self-scoped for members (admin sees all), Create self-or-admin, Get/Update/Delete owner-or-admin - application credentials: all four user-scoped routes self-or-admin; the id-only lookup 404s for foreign owners (no existence leak) Convention kept (cloud-storage#515, TestIdentityAuthzPrecedence): gated routes with path ids moved from the uuid-validating 'val' group to 'mgmt' and validate ids in-handler AFTER the authz check, so 403 precedes the verbatim 404. routes_wiring_test and identity_notfound_test updated to the new wiring (admin caller, same 404 shapes); new identity_authz_gates_test pins 19 member denials, the self-service allowances, and the unchanged admin path. Unit suite (incl. DB-backed): PASS. Contract suite: no regressions (the 5 failures in this environment are the docker-exec-dependent tests that cannot pass without docker, unchanged from baseline). Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
…thz, upload caps Comprehensive review 2026-10-13 (H-1, M-1, M-2, M-3, N-3): H-1 — upload authorization was visibility-based: the gate was `visibility = 'public' OR project_id = caller`, so any tenant that could SEE a public image could overwrite its bytes while queued — image poisoning of an operator-staged base that every tenant boots. Now the same owner-or-admin rule delete already uses (imageDeleteAuthz): 403 for a visible-but-foreign image, 404 for an invisible one, admin override. Pinned by TestUploadImageDataOwnership_DB. N-3 — the recorded checksum was MD5 of the upload stream, but normalizing backends (local, local,rbd, local,s3) rewrite the bytes (qcow2 → raw base): MD5(raw replica) != MD5(original qcow2), so EnsureLocalBase's integrity check failed on every node without the local base and boot-from-image errored — the exact multi-node case local,s3 exists for. New ImageStore.StoredChecksum hashes the stored object, and UploadImageData records it whenever the backend normalizes (stream hash stays correct for as-is backends: s3, stub). Pinned by TestStoredChecksumMatchesStoredBytesNotUploadBytes (real qemu-img, real qcow2). M-1 — glance tasks leaked cross-tenant input/result/message and echoed internal errors: ListTasks/GetTask are now owner-scoped (admin sees all), GetTask 404s for foreign tasks, and no debug_error / err.Error() in response bodies. M-2 — cache management (prefetch/evict/clear) is admin-only (upstream admin_api): any member could flush the deployment-wide cache with one request. Pinned by TestImageCacheManagementAdminOnly_DB. M-3 — unbounded bodies: PUT /images/:id/file and POST stage now wrap the request in http.MaxBytesReader with a 10 GiB cap (upstream Glance's image_size_cap default). Unit suite (incl. DB-backed): PASS. Contract suite: unchanged (only the 5 docker-exec-dependent tests fail, same as baseline). Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Comprehensive review 2026-10-13 (H-2, N-6, N-8, N-9, N-10, L-3, L-4): H-2 — the flavor catalog was member-writable: create/delete flavor, extra-specs CRUD, and tenant access grants (FlavorAction) all honored any authenticated caller. The catalog is a shared deployment-wide resource; upstream gates all of it admin_api. Now requireAdmin on all seven handlers. A member could resize the world's flavors out from under every tenant, or grant itself another project's private flavor. L-4 — flavor inputs get upper sanity bounds (1 TiB RAM, 4096 vCPUs, 1 PiB disk) instead of only >= 0: a typo (MB vs GB) used to schedule absurd values that surfaced only at boot time. N-9 — AttachVolume was check-then-act across two statements: the loser of a concurrent double-attach surfaced as a 500 from the UNIQUE constraint, and a failure between the attachment INSERT and the volume UPDATE left the row inconsistent. Now one transaction with the volume row locked (FOR UPDATE), mirroring cinder's attach CAS; the concurrent double-attach answers 409, the state refusal stays 400. N-10 — DetachVolume had the mirror problem: DELETE then UPDATE as two statements, so a failure after the DELETE stranded the volume in-use with no attachment row — unfixable through the API. Now one transaction; both statements or neither. N-6 — a multi-network boot that failed on a later network leaked the ports already allocated for the earlier ones. The boot goroutine now releases what it allocated on every failure path after the first successful allocation (new NeutronService.ReleaseInstancePorts: deletes the instance's compute:nova ports, their TAP/FDB plumbing and DHCP reservations). Pinned by TestFailedBootReleasesAllocatedPorts / TestSuccessfulBootKeepsItsPorts. N-8 — DeleteServer used a blanket DELETE of every port with the server's device_id, destroying ports the TENANT created and attached (os-interface with port_id) along with the boot-allocated ones. Upstream semantics now: nova-owned ports are deleted with the server, tenant-created ports are detached (device cleared, parked DOWN) and survive for their owner. Pinned by TestDeleteServerDetachesTenantPortsButDeletesItsOwn. L-3 — CreateKeypair raced check-then-insert into duplicate-key 500s; the UNIQUE(user_id, name) violation now maps to the same 409 as the pre-check. Imported public keys are also bounded (64 KiB) before being parsed and stored (M-3). Unit suite (incl. DB-backed): PASS. Contract suite: unchanged (only the 5 docker-exec-dependent tests fail, same as baseline). Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
…e; constant-time transfer keys Comprehensive review 2026-10-13 (H-3, L-1): H-3 — volume types, QoS specs and manage/unmanage were authentication-only. Any member could create volume types every tenant then booted from, rewrite extra specs, add its project to a private type (VolumeTypeAction), define QoS specs, and import host backings into its own quota (os-volume-manage addresses operator storage). All admin-only now, matching upstream's volume_extension:* and volume_admin_actions:* policies; reads of the type catalog stay open since tenants need them to pick a type at create. Pinned by TestOperatorSurfacesAdminOnly; the concurrent managed-import quota test drives the route as an administrator now (quota enforcement itself unchanged). L-1 — the volume-transfer auth key was compared with a plain string equality, which short-circuits on the first differing byte; a transfer id is cheap to enumerate, so the key could be recovered byte-by-byte from response timing. Now crypto/subtle. ConstantTimeCompare. Unit suite (incl. DB-backed): PASS. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
…ate, working network_data.json Comprehensive review 2026-10-13 (N-4, M-4, N-7): N-4 — port IP allocation was a purely random draw from the subnet, so two ports in one subnet landed on the same address at birthday- problem rates: two MACs claiming one IP, dnsmasq handing it to one of them, and a guest that could not reliably reach anything. New Service.allocateFreeIP draws random candidates until one is free of every port's fixed_ips AND the subnet's gateway, used by both CreatePort and AllocatePortForInstance; a full subnet is an honest refusal instead of a duplicate. Adjacent defect found and fixed while in here: allocateIP's candidate range [10, maxOffset) goes negative for subnets with fewer than ~12 addresses, and crypto/rand.Int PANICS on a non-positive bound — tiny subnets now fall back to the whole usable range. Pinned by TestAllocateFreeIP* (4 tests). M-4 — CreateNetwork accepted shared=true from anyone, letting a member open their network to every project in the deployment (and every project's guests to it). Now gated exactly like router:external on create: only true is refused to a member, so clients echoing the default keep working (upstream's explicit-set rule). UpdateNetwork does not accept shared at all, so no gap there. Pinned by TestNonAdminCannotCreateSharedNetwork. N-7 — the metadata service's network_data.json selected p.fixed_ip, a column that does not exist (ports store fixed_ips as a JSONB array), so the query failed and every guest fetching it got a 500 — cloud-init fell back to DHCP or configured nothing. Rewritten to resolve each port's addresses through the JSONB and their own subnets: one link per port (not per address, and not fanned out per subnet-on-network as the old join would have), subnet-resolved netmask/gateway/DNS, and a link even for an address-less port. Pinned by TestNetworkDataJSONServesPortAddresses. Race-enabled unit suite (incl. DB-backed): PASS. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Findings from the review of this PR's own diff, all fixed: HIGH — the N-8 tenant-port fix was unreachable through the real flow: os-interface stamped device_owner='compute:nova' on EVERY port it attached, including a tenant's pre-existing port_id, so every attached port looked nova-owned and server delete destroyed it anyway. The stamp now keys on which path ran: a port Nova creates (net_id) is nova's and deleted with the server; a tenant's port keeps its own owner and is detached. Mirrors upstream deallocate_for_instance, which skips ports it did not create. Pinned end-to-end by TestDeleteServerDetachesTenantPortsButDeletesItsOwn (attach via os-interface, then delete) and the new net_id companion TestDeleteServerDeletesInterfaceAttachCreatedPort. MEDIUM — os-snapshot-manage missed by the H-3 sweep: admin-gated like os-volume-manage (upstream volume_admin_actions:snapshot_manage). MEDIUM — volume types with a NULL description vanished from GET /v3/types (scan into a plain string failed and the row was silently dropped). COALESCE'd; pinned by TestListVolumeTypesServesNullDescription. The authz test is also idempotent now (unique names + cleanup; it would fail a second run against the shared test database). MEDIUM — os-interface's net_id path wrote the NETWORK id into its fixed_ips subnet_id, so every consumer that joins on the subnet — network_data.json (N-7), DHCP, the N-4 collision check — silently missed those ports; and it ran its own next-free IP allocator (allocateNextIP), a second allocator over the same subnet that could hand out an address another path held. It now resolves the real subnet id and allocates through neutron.AllocateFreeIP (exported for exactly this); allocateNextIP is deleted. The port_id path also reported a hardcoded placeholder IP in its response — it now parses the port's actual first fixed IP. MEDIUM — glance cache LIST was ungated while the rest of the cache surface was admin-only. Gated (upstream cache_list). LOW — nova double-attach now answers 400 like upstream, not 409. LOW — over-cap image uploads answer 413 (upstream image_size_cap semantics), not 503/400; new common.NewRequestEntityTooLargeError. LOW — network_data.json port ordering gets a deterministic tiebreaker (created_at, id). LOW — the boot test no longer writes under /var/lib: the node-local state root is redirected via paths.SetBase(t.TempDir()). Unit suite (incl. DB-backed): PASS. Contract suite: unchanged (only the 5 docker-exec-dependent tests fail, same as baseline). Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Round 2 of the review loop (0 High, 1 Medium, 4 Low — all fixed):
MEDIUM (F1) — the port_id path of os-interface scanned the ports row's
nullable device_id into a plain string. Every free port has NULL
there, so the scan FAILED on every successful attach; the error was
dropped and the attach worked only because pgx fills earlier columns
before the failed one — and any real database error on that lookup
surfaced as a bogus 200 with an empty net_id and MAC. The lookup now
reads COALESCE(device_id::text, '') and handles its errors like the
instance lookup above it. Pinned by
TestAttachInterfacePortIDReportsRealState (attach a device_id-NULL
port, assert the response carries the port's real net_id, MAC and
fixed_ips).
LOW (F2) — the attach response fabricated one {"ip_address": ""} for
a port with no addresses and omitted subnet ids. It now reports the
port's actual fixed_ips entries (or an empty list). Pinned by
TestAttachInterfacePortIDReportsRealState and
TestAttachInterfacePortIDWithoutAddressesReportsNone.
LOW (F3) — an explicit fixed_ip on the net_id path was stored
unvalidated: not checked against the subnet, not against the
addresses other ports hold (the explicit side of the N-4 collision
rule), and arbitrary text reached the JSONB build (a quote 500'd on
the cast). Now: parse + in-subnet (400), availability via the new
exported neutron.IPAvailable (409, upstream's 'IP address already
allocated'). AllocateFreeIP and IPAvailable share one
usedSubnetAddresses helper. Pinned by
TestAttachInterfaceExplicitFixedIPIsValidated.
LOW (F4) — uploadImageLocal applied the 5-second metadata-op timeout
to the body copy, so an over-cap upload hit the deadline (503) long
before the 10 GiB MaxBytesReader cap could answer 413. The copy now
runs under the caller's context only (cancelled on client
disconnect), matching the S3 path's connect-only timeout policy
(FR-014). maxImageUploadBytes is a var now, and
TestOverCapUploadAnswers413 pins the 413 end-to-end (image left
queued, not stuck saving).
LOW (F5) — the boot test's paths.SetBase restore could run while the
boot goroutine was still reading the paths global (LIFO cleanup ran
the restore before the service Shutdown). The redirect is now
registered before newBootService so Shutdown stops the goroutine
first.
Unit suite (incl. DB-backed): PASS. Contract suite: unchanged (only
the 5 docker-exec-dependent tests fail, same as baseline).
Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Two LOW pre-existing findings surfaced by round 3 of the review loop, in the same file and class as the round-2 fixes: L-1 — ListInterfaceAttachments skipped any attachment whose port has no addresses, so an attachment created a moment earlier was invisible in the list — disagreeing with the attach response, which (since F2) reports it with an empty fixed_ips. The list now emits every attachment, defaulting fixed_ips to [] (JSON null in the column included). Pinned by TestListInterfacesIncludesAddresslessAttachments. L-2 — DetachInterface had the dropped-error NULL scan F1 fixed in the attach path: a detached port (device_id NULL) failed the scan, the error was dropped, and the 'not attached' 409 was right only by accident — a real database error also 409'd instead of 500-ing. Mirrors the F1 fix: COALESCE(device_id::text, ''), errors handled. Round 3's verdict on the PR's own commits was CLEAN; these two were pre-existing on main. Unit suite (incl. DB-backed): PASS. Contract suite: unchanged (only the 5 docker-exec-dependent tests fail, same as baseline). Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
…ace list Round 4's only observation: the list test pinned the attachment's presence but not the [] serialization it must agree with the attach response on. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Review-fix loop on this PR — converged CLEAN ✅This PR went through a review-fix loop after the initial implementation. Full trend:
Round 1 — the notable one
Round 2 — fix-on-fix verification
Rounds 3–4Round 3 verified Every fix is pinned by a DB-backed regression test; the reviewer verified each pin fails RED against the pre-fix code. Unit suite (incl. |
The CI unit-test runner does not install qemu-img; the test's premise (a boot that reaches ACTIVE) requires overlay staging, which needs it. Same guard as pkg/storage's requireQemuImg — the failure-path N-6 tests, which expect ERROR anyway, still run everywhere. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
2942942 to
fa43d45
Compare
…th upstream
Review findings (comprehensive review of the open senolcolak PRs):
- HIGH: usedSubnetAddresses scanned ports.fixed_ips with a bare
jsonb_array_elements. A port on a subnet-less network persists JSON-scalar
null, and one such row made every allocation fail deployment-wide
(SQLSTATE 22023) with a misleading 409. Guard the SRF with jsonb_typeof and
write [] instead of null; legacy rows are tolerated by the readers.
- MEDIUM: metadata network_data.json used COALESCE, which does not fire for
JSON null; same guard. A guest on such a port got a 500.
- LOW: the same unguarded scan in the nova ?ip= server filter.
- LOW: gate /os-hypervisors, /os-hypervisors/detail, /os-hypervisors/stats,
/os-services and /os-console-auth-tokens/{token} with requireAdmin;
statistics leaked a real deployment-wide running-VM count to members.
- MEDIUM: allocateIP now mirrors upstream generate_pools() for /30, /31,
/32 and IPv6 pools instead of overflowing int64 on IPv6; small pools
allocate, only exhaustion fails.
Tests pin the DoS and the metadata 500 (each fails on the unfixed query), the
admin gates, and tiny/IPv6 allocation.
Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Shared CI unblocker for the open PRs (independent of this PR's content): - govulncheck now reports GO-2026-6809 in crypto/x509 (reachable through the HTTP server), fixed in Go 1.27.3. Main pins 1.27.2 everywhere, so every PR's Runtime Vulnerabilities and SBOM job fails. Pin 1.27.3 in go.mod, tools/go.mod, CI workflows, the builder image, the test container and the single-node deploy script (version + sha256). - The unpinned Helm install now pulls Helm 4, whose 'helm plugin install' rejects the helm-unittest source without --verify=false, failing the Helm Charts job with 'plugin source does not support verification'. Install Helm 3.22.1 and helm-unittest 1.2.1 in an isolated plugin dir, matching PR #107, so the gate is version-stable. Signed-off-by: Uwe Schwaeke <uwe.schwaeke@sap.com>
Comprehensive security review — findings fixedReviewed against Found 4 (1 High, 1 Medium, 2 Low) — all fixed in High — tenant-triggerable deployment-wide IP-allocation DoS
Medium —
|
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Signed-off-by: Senol Colak <senol.colak@sap.com>
Authenticated members could change other users' credentials, grant administrative roles, modify deployment-wide resources and overwrite public image data. This PR adds the missing authorization checks and fixes the allocation, upload and cleanup failures found while reviewing those paths.
Changes
The follow-up review report records the ten additional findings, reproductions, fixes and validation limits. Earlier review context is in #101.
Validation
make lint: zero issues. Linux binary build passed.docker exec lightstack-postgresenvironment; the Cinder fixture also panics while parsing its missing output. This local contract run is not a complete pass.Real libvirt guests, host networking and Ceph RGW were not exercised locally. The existing CI jobs provide the repository's integration gates. JWT token revocation, rate limiting and the broader shutdown changes tracked by the earlier review remain separate work.