lalalune · Sep 23, 2026 · Sep 22, 2026 · Sep 22, 2026 · Sep 22, 2026 · Sep 22, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- dstack-util: the measured `gpu-attestation` event now exposes aggregate `dbgstat` and `secboot` values from the NVIDIA-signed claims. `dbgstat` is `enabled` if any attested GPU reports enabled, while `secboot` is true only if every attested GPU reports secure boot; the event schema version is bumped to 3.
 - sdk: every SDK ships two clients, one per guest-agent surface, and nothing else. `DstackClientV0` speaks the frozen v0.5.11 API on the unversioned paths -- every method a v0.5.x SDK had, including `verify()` (which calls the server `Verify` RPC) and `emit_event()` (which surfaces the agent's removal error verbatim). `DstackClientV1` speaks `dstack.guest.v1` at `/v1` and has exactly its six methods: `issue_cert`, `get_key`, `attest`, `attest_gpu`, `info`, `version`. They are transport mirrors, not a compatibility layer: neither translates a call to the other, and each one's method set is its surface's method set. **v1 derives different key material than v0 for the same inputs** -- see `docs/guest-api-v1.md` for the migration. `TappdClient` is untouched.
 
   The blockchain adapters (Ethereum/viem, Solana) remain **v0-only** and stay typed against the v0 `GetKeyResponse`. v1 has no chain-related surface, deliberately: it returns key material, and what an application builds from those bytes is its own business rather than something the SDKs model.
@@ -94,6 +95,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - os: the Yocto guest-OS backend (`os/yocto/`) is deprecated in favor of mkosi (`os/mkosi/`), which is now the default and recommended backend. `os/build.sh` defaults to `--backend mkosi`, and `make os-image` / `make os-repro-check` build with mkosi. The Yocto builds move to `make os-image-yocto` / `make os-repro-check-yocto`; `make os-image-mkosi` / `make os-repro-check-mkosi` remain as aliases. Every Yocto entrypoint prints a deprecation warning, and the backend is kept only to rebuild existing Yocto images. The mkosi build still reads patches, units and scripts from `os/yocto/`, so the directory stays until those files move
 
 ### Removed
+- verifier: the `debug` request field and the `acpi_tables` / `rtmr_debug` response fields. The per-event RTMR diff never had the events it diffed, so it reported every expected digest as missing. Requests that still send `debug` are accepted and the field is ignored.
 - sdk: `TlsKeyOptions.path` in the JavaScript SDK. `GetTlsKeyArgs` has no such field and `getTlsKey` never read it, so a caller who set it was silently ignored. Breaking at the type level only, and only for code whose value was already being discarded. `deriveKey`'s `path` is a real, deprecated Tappd-era parameter and stays; the Python, Rust and Go v0 TLS-key options never carried one
 - guest-agent: the `EmitEvent` RPC no longer records anything -- runtime RTMR3 events are system-owned in 0.6.0, so an app can no longer extend the measurement chain. The method itself stays on the unversioned path and always fails with an error naming the removal, rather than being deleted outright: a deleted method answers HTTP 404 `Service not found: EmitEvent`, which tells a 0.5.x caller nothing about why its events stopped being recorded, while the kept stub fails with a message naming the removal and pointing at `report_data`. **Breaking:** any app extending RTMR3 at runtime must stop; bind app data through `report_data` instead, which is what most callers wanted anyway
 - gateway: `core.debug.insecure_skip_attestation`. It turned off both checks that make a gateway cluster a trust boundary: the node stopped asking its guest agent for its own app id, and every WaveKV sync and push, plus `ensure_from_gateway`, stopped checking the peer's. With it set, anything that could reach the sync routes could insert entries that replicated to every gateway in the cluster. It existed only because the integration suites could not run without it; they now run against a guest agent simulator and verify quotes through the production path, so nothing sets it. `docs/security/security-model.md` argues dstack's development switches are acceptable because they are visible in attestation measurements or public contract state -- that argument cannot cover the switch deciding whether attestation happens at all, which is why this one is deleted rather than documented. There is no replacement. **Breaking:** the config struct does not use `deny_unknown_fields`, so a leftover line is ignored rather than rejected, and what follows depends on why it was set. On a TDX host with a guest agent the gateway starts normally and peers that cannot present a verifiable app id stop being accepted. On a host with no guest agent -- the usual reason to have set it -- the gateway no longer starts at all, failing with `Failed to get app info`, because `my_app_id` is what every peer check compares against and a node that cannot learn its own identity must not come up without one. Remove the line and make sure every node can attest

diff --git a/docs/attestation-tdx.md b/docs/attestation-tdx.md
@@ -43,7 +43,8 @@ For a GPU launch, any `init-script-hash` events are followed by
 evaluation, `gpu-attestation`. The `gpu-policy-hash` payload is
 `SHA-256(JCS(requirements.gpu_policy))`, using `{}` when the field is omitted.
 The `gpu-attestation` payload is JSON containing the verified device count,
-CC/DevTools state, and `evidence_sha256`.
+CC/DevTools state, aggregate signed-claim `dbgstat` and `secboot`, and
+`evidence_sha256`.
 
 The guest-agent returns the complete `nvattest` record captured during boot from `/v1/Attest`, when the request sets `include_boottime_gpu_evidence`, so a verifier can fetch the quote and the GPU evidence in one round trip. `AttestResponse.boottime_gpu_evidence` is a list of `GpuEvidenceBundle` (`{vendor, format, evidence}`); the boot record is the bundle whose `vendor` is `nvidia` and whose `format` is `nvidia-nvattest-boottime-json-v1`, and its `evidence` is hex-encoded bytes that decode to the exact UTF-8 `nvattest` output. It is not trustworthy by itself. (`/v1/AttestGpu` runs a *fresh* attestation against a caller nonce and returns bundles tagged `nvidia-nvattest-collect-evidence-json-v1`, a deliberately distinct format that a boot-record verifier does not appraise; its result is not bound to the TD and must not be used as remote evidence; only the boot-time record below is.) After verifying the TDX quote and replaying the event log to RTMR3, hash the *decoded* bundle bytes — `SHA-256(hex_decode(bundle.evidence))`, never the JSON string as returned nor a re-serialized form — and require the result to equal the `gpu-attestation` event's `evidence_sha256`. See [GPU Security for AI Workloads](./security/security-model.md#gpu-security-for-ai-workloads) for the event schema, ordering, Rego example, and platform differences.
 

diff --git a/docs/auth-simple-operations.md b/docs/auth-simple-operations.md
@@ -227,6 +227,8 @@ The Gateway is a special app that routes traffic to other apps. Once deployed, a
 
 The `gatewayAppId` is returned in boot responses and used by KMS for key derivation.
 
+To trust several gateways, join their App IDs with commas (`"0xaaa...,0xbbb..."`). CVMs accept a gateway whose App ID is contained in this value.
+
 ---
 
 ## KMS Onboarding (Multi-Node)

diff --git a/docs/aws-attested-instance-security-evaluation.md b/docs/aws-attested-instance-security-evaluation.md
@@ -35,7 +35,7 @@ NitroTPM path meets them today.
 | P1 | Verifiable platform root of trust | TDX/SNP/Nitro quote verification against vendor root; debug rejected; TCB surfaced | NitroTPM Attestation Documents are verified against the AWS Nitro Attestation PKI, including document timestamp sanity. AWS exposes no TDX/SNP-style TCB advisory field, so a verified attestation is normalized to `tcbStatus = "UpToDate"` and passes the standard authorization gate unchanged. |
 | P2 | Reproducible or independently computable base image measurement | meta-dstack rebuild plus `dstack-mr` computes `MRTD`/`RTMR0-2` | The unified `os/build.sh` flow emits the AWS image archive with `sha256sum.txt`, `digest.txt`, and `measurement.aws.cbor`; its output directory also contains the reference-PCR side-car `aws-pcrs.json`. `os_image_hash = sha256(sha256sum.txt)` is the same identity used on all platforms; the verifier recomputes it from the downloaded image directory (`dstack/verifier/src/verification.rs`). The hardening audit script `os/yocto/tools/aws/audit-aws-ec2-image-hardening.sh` checks the image for operator mutation channels. |
 | P3 | Boot command line and root filesystem integrity are measured | `RTMR1/2`, rootfs hash, dm-verity, measured initrd/cmdline | The UKI commits kernel, initrd, and embedded cmdline into `PCR4`; the rootfs is dm-verity-protected. `VmConfig.aws_measurement` is required and must bind `boot_pcr_digest = sha256(PCR4||PCR7||PCR12)` to the attested PCRs, so `PCR12` (external cmdline) is always part of the bound digest — a missing-PCR12 bypass is not expressible. Enforced in guest quote generation (`dstack/dstack-attest/src/attestation.rs`), `verify_os_image_hash_for_aws_nitro_tpm` (`dstack/verifier/src/verification.rs`), and the KMS pipeline via the same verifier check. |
-| P4 | Runtime application identity is cryptographically bound | RTMR3 `compose-hash`, `app-id`, `instance-id`, `key-provider`; event log replay | SHA384 `PCR14` event-log replay is the authoritative binding (RTMR3 analogue; non-resettable). Launch events: `system-preparing`, `app-id`, `compose-hash`, zero or more ordered `init-script-hash` events, `instance-id`, `boot-mr-done`, `key-provider`, `storage-fs`, `system-ready` (`dstack/dstack-util/src/system_setup.rs`). GPU launches also include `gpu-policy-hash` and `gpu-attestation` before `instance-id`. `dstack-attest`, `dstack-verifier`, and KMS reject missing/mismatched PCR14 and bad replay. Optionally, the guest extends the raw `MrConfig` V2 `config_id` into `PCR8` once (`PCR8 = sha384(0^48 || config_id)`) so a lightweight third-party verifier can check compose hash + key provider without event-log replay; dstack's own verifier and KMS do not check PCR8. |
+| P4 | Runtime application identity is cryptographically bound | RTMR3 `compose-hash`, `app-id`, `instance-id`, `key-provider`; event log replay | SHA384 `PCR14` event-log replay is the authoritative binding (RTMR3 analogue; non-resettable). Launch events: `system-preparing`, `app-id`, `compose-hash`, zero or more ordered `init-script-hash` events, `instance-id`, `boot-mr-done`, `key-provider`, `storage-fs`, `storage-encrypted`, `system-ready` (`dstack/dstack-util/src/system_setup.rs`). GPU launches also include `gpu-policy-hash` and `gpu-attestation` before `instance-id`. `dstack-attest`, `dstack-verifier`, and KMS reject missing/mismatched PCR14 and bad replay. Optionally, the guest extends the raw `MrConfig` V2 `config_id` into `PCR8` once (`PCR8 = sha384(0^48 || config_id)`) so a lightweight third-party verifier can check compose hash + key provider without event-log replay; dstack's own verifier and KMS do not check PCR8. |
 | P5 | Challenge/liveness and caller key binding | `report_data` challenge or RA-TLS public key hash in quote | RA-TLS binds `report_data` to the TLS certificate public key. KMS key release is bound to the live RA-TLS handshake; external `/verify` callers supply and check their own `report_data` challenge. The low-level NitroTPM document verifier also rejects stale or far-future document timestamps. |
 | P6 | Secret release only to attested code | dstack KMS verifies attestation, checks auth policy, derives per-app keys | dstack KMS verifies the NitroTPM attestation, runs the same `verify_os_image_hash_for_aws_nitro_tpm` binding check as the verifier, builds `BootInfo` from verified boot PCRs plus PCR14 launch events, and checks auth policy before deriving app keys (`dstack/kms/src/main_service.rs`). AWS NitroTPM key release is gated behind the opt-in `aws_nitro_tpm_key_release` flag (default false in `kms.toml`). |
 | P7 | Key-release policy is not controlled by the untrusted account admin | KMS runs inside TEE; policy from auth API/contracts; KMS identity measured | Satisfied with dstack KMS or another verifiable secret authority outside the untrusted AWS account admin's control. A NitroTPM-backed dstack KMS keeps root material out of account-admin snapshots and clones. The policy backend (auth-simple in a trusted control plane, or on-chain `DstackKms`/`DstackApp`) must be outside the workload account admin's control. Same-account AWS KMS fails this property if the admin can change key policy, create grants, or call secret-bearing operations through a policy they control. |

diff --git a/docs/onchain-governance.md b/docs/onchain-governance.md
@@ -98,6 +98,8 @@ GATEWAY_APP_ID=<app-id> \
 
 Output: `Set gateway app ID: <app-id>`
 
+To trust more than one gateway (for example during a gateway migration), set the value to several app IDs joined by commas, e.g. `<app-id-1>,<app-id-2>`. A CVM accepts a gateway whose certificate carries any app ID contained in this string.
+
 Add the gateway's compose hash to the whitelist. To compute the compose hash:
 
 ```bash
@@ -155,7 +157,7 @@ The central governance contract that manages OS image whitelisting, app registra
 |----------|-------------|
 | `addOsImageHash(bytes32)` | Whitelist an OS image hash |
 | `removeOsImageHash(bytes32)` | Remove an OS image from whitelist |
-| `setGatewayAppId(string)` | Set the trusted Gateway app ID |
+| `setGatewayAppId(string)` | Set the trusted Gateway app ID(s) |
 | `registerApp(address)` | Register an app contract |
 | `deployAndRegisterApp(...)` | Deploy and register app in one transaction |
 | `isAppAllowed(AppBootInfo)` | Check if an app is allowed to boot |

diff --git a/docs/security/security-model.md b/docs/security/security-model.md
@@ -118,7 +118,7 @@ For every NVML-enumerated GPU, dstack calls `Device::is_cc_enabled()` and `Devic
 
 ### Dual Attestation
 
-GPU workloads require verification of both hardware components. The CPU TEE quote verifies the CVM and its measured guest code. NVIDIA-signed evidence, checked against NVIDIA RIMs and certificate status by `nvattest`, verifies the GPU appraisal. After the optional policy and ready-state operations succeed, dstack emits a `gpu-attestation` launch event before `system-ready`. Its versioned payload records the number of appraised devices, asserted CC/DevTools state, and SHA-256 of the complete nvattest JSON output (claims and detached EAT). On TDX, both `gpu-policy-hash` and `gpu-attestation` are append-only RTMR3 events; they are never derived from application-controlled `report_data`.
+GPU workloads require verification of both hardware components. The CPU TEE quote verifies the CVM and its measured guest code. NVIDIA-signed evidence, checked against NVIDIA RIMs and certificate status by `nvattest`, verifies the GPU appraisal. After the optional policy and ready-state operations succeed, dstack emits a `gpu-attestation` launch event before `system-ready`. Its versioned payload records the number of appraised devices, asserted CC/DevTools state, aggregate signed-claim debug and secure-boot status, and SHA-256 of the complete nvattest JSON output (claims and detached EAT). On TDX, both `gpu-policy-hash` and `gpu-attestation` are append-only RTMR3 events; they are never derived from application-controlled `report_data`.
 
 For a successful TDX GPU launch, the GPU-relevant RTMR3 event order is:
 
@@ -135,15 +135,23 @@ boot-mr-done
 
 ```json
 {
-  "version": 2,
+  "version": 3,
   "provider": "nvidia",
   "devices": 1,
   "cc_mode": "on",
   "devtools": false,
+  "dbgstat": "disabled",
+  "secboot": true,
   "evidence_sha256": "<sha256-of-complete-nvattest-json>"
 }
 ```
 
+`devtools` is the aggregate NVML state and is true if any device has DevTools
+mode enabled. `dbgstat` and `secboot` summarize the NVIDIA-signed claims:
+`dbgstat` is `enabled` if any device reports enabled, and `secboot` is true only
+if every device reports secure boot. The complete per-device claims remain in
+the boot-time evidence committed by `evidence_sha256`.
+
 `/v1/Attest` returns that complete boot-time `nvattest` record when the request sets `include_boottime_gpu_evidence`; it runs no new attestation. `AttestResponse.boottime_gpu_evidence` is a list of `GpuEvidenceBundle`, each carrying `vendor`, `format`, and `evidence`. The boot record is the bundle whose `vendor` is `nvidia` and whose `format` is `nvidia-nvattest-boottime-json-v1`; its `evidence` is hex-encoded bytes that decode to the exact UTF-8 nvattest output as the agent read it from disk. The other format, `nvidia-nvattest-collect-evidence-json-v1`, comes from `/v1/AttestGpu` and is fresh evidence against a caller nonce; the two are deliberately distinct because a verifier for one does not appraise the other. To bind the API result to TDX evidence: verify the quote, replay the event log to the quote's RTMR3, require exactly one pre-`system-ready` `gpu-attestation` event, decode its JSON payload, select the boot-time bundle by `format`, and compare `evidence_sha256` with `SHA-256(hex_decode(bundle.evidence))`. Hash the decoded bytes, not the JSON string as returned and not a re-serialized form: parsing and re-serializing changes the digest and breaks the comparison. Only after this comparison should the verifier inspect the returned claims. This exact-byte comparison includes any whitespace or trailing newline in the decoded record.
 
 A verifier must replay the measured event log, require exactly one `gpu-policy-hash` event immediately after `compose-hash`, and compare its 32-byte payload with the expected policy digest (`SHA-256(JCS({}))` for the omitted/default policy). When MrConfigV3 includes `gpu_policy_hash`, it must match the same digest. When GPU protection is required, the verifier must also require exactly one pre-`system-ready` `gpu-attestation` event with `devices > 0` and, when applicable, the expected deployment count. The raw `attestation.out` file is not trusted by itself; if it is supplied for inspection, its digest must match the `gpu-attestation` event.

diff --git a/docs/tutorials/attestation-verification.md b/docs/tutorials/attestation-verification.md
@@ -486,12 +486,13 @@ These are the standard events you'll see in the log:
 | `compose-hash` | SHA-256 of docker compose config | Should match `tcb_info.compose_hash` |
 | `init-script-hash` | SHA-256 of one init script; repeated in configured order (maximum 5) | Should match the independently approved script bytes |
 | `gpu-policy-hash` | SHA-256 of the JCS-canonicalized GPU policy (default `{}`) | Should match the expected `requirements.gpu_policy` digest |
-| `gpu-attestation` | Verified GPU state and digest of the boot-time `nvattest` JSON | Required for an attested GPU launch; verify as described below |
+| `gpu-attestation` | Verified GPU state, aggregate `dbgstat`/`secboot`, and digest of the boot-time `nvattest` JSON | Required for an attested GPU launch; verify as described below |
 | `instance-id` | Unique instance identifier | Should match `instance_id` from response |
 | `boot-mr-done` | Boot measurements complete | Marker event |
 | `os-image-hash` | Guest OS image hash | Should match `tcb_info.os_image_hash` |
 | `key-provider` | Key provider type | e.g., `kms` |
 | `storage-fs` | Storage filesystem type | Storage configuration |
+| `storage-encrypted` | `1` when the data disk is LUKS-encrypted, `0` when it is not | `0` means the host can read the application's data at rest |
 | `system-ready` | System ready marker | Always present at end |
 
 For a successful GPU launch, the relevant order is `compose-hash`, any

diff --git a/dstack/dstack-mr/cli/src/main.rs b/dstack/dstack-mr/cli/src/main.rs
@@ -325,7 +325,7 @@ fn run_diagnose(config: &DiagnoseConfig) -> Result<()> {
     let initrd = image_dir.join(&image_info.initrd).display().to_string();
     let cmdline = dstack_mr::tdx::measured_kernel_cmdline(&image_info.cmdline);
 
-    // Same resolution order as the verifier (see verifier::compute_measurement_details):
+    // Same resolution order as the verifier (see verifier::compute_measurements):
     // explicit vm_config.ovmf_variant > image_info.ovmf_variant > legacy default.
     let ovmf_variant = vm
         .ovmf_variant
-Original file line number
+Diff line change
@@ Expand Up @@
     The `gatewayAppId` is returned in boot responses and used by KMS for key derivation.
+    To trust several gateways, join their App IDs with commas (`"0xaaa...,0xbbb..."`). CVMs accept a gateway whose App ID is contained in this value.
     ---
     ## KMS Onboarding (Multi-Node)
@@ Expand Down @@