diff --git a/CHANGELOG.md b/CHANGELOG.md index 24105bc..2b92398 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,37 @@ +## [0.16.4] - 2026-08-31 + +Patch release — ADR-037 Slice B. The wire protocol bumps from 3 → 4 additively: `/gate` response now echoes the SDK-supplied `action_digest` and a `policy_hash` slot (always `None` today; Slice D wires per-request computation). `min_protocol_version` stays at 2 so v3 SDKs are unaffected. Wire-format additive only — no new hashing/computation introduced on either side (both fields echo already-computed values). + +### Added + +- **`NULLRUN_PROTOCOL_VERSION = 4`** (`src/nullrun/transport.py`). `X-NULLRUN-PROTOCOL` header on every signed POST now serialises the bumped value via the single source of truth `NULLRUN_PROTOCOL_VERSION`; tests are pinned to `str(NULLRUN_PROTOCOL_VERSION)` so a future bump doesn't sweep this file again. `NullRunProtocolError.user_action` and `docs/errors/NR-P001.md` updated to point operators at `X-NULLRUN-PROTOCOL: 4`. +- **`/gate` response wire-evidence echo capture** — `runtime._capture_wire_evidence` (called from `_capture_server_minted_execution_id` on the same `/check` lifetime so the two values always refer to the same gate decision) reads `action_digest` + `policy_hash` off the response and stores them in two new contextvars: `_last_gate_action_digest_var`, `_last_gate_policy_hash_var`. Public accessors `get_last_gate_action_digest()` / `get_last_gate_policy_hash()`; setters `set_last_gate_action_digest()` / `set_last_gate_policy_hash()`. Capture is fail-OPEN: a malformed value (non-str type) is logged at WARNING and dropped — the contextvar stays at `None`. `clear_server_minted_execution_id` (and the underlying direct `set(...=None)` paths) also drop the v4 slots so a `/check` in one block never leaks a stale echo into a `/track` in a sibling block. +- **`ServerCapabilities.wire_evidence_echo`** — informational capability flag surfaced by `/api/v1/capabilities`. Tells the SDK the backend echoes `action_digest` on `/gate` response. NOT included in `is_v3_ready()` (informational, not a hard gate). Defaults to `False` on pre-v4 backends; the canonical shape is `capabilities.wire_evidence_echo: true` at the top level, with the nested `capabilities.*` form also accepted. +- **New test file `tests/test_slice_b_wire_evidence.py`** (10 tests). Pins the SDK-side of the v3→v4 additive bump: protocol-constant value, header serialisation, capture from `/gate` response (happy path + `policy_hash`-when-present + both-set), tolerance of pre-v4 backends (no keys → both `None`), tolerance of malformed wire values (non-str → drop, do not raise), tolerance of `None`-typed responses (defensive — runtime never passes a non-dict, but a bad transport layer might), `clear_server_minted_execution_id` resets the v4 slots, and the protocol-constant + capability-flag source-of-truth wiring. +- **`tests/test_capabilities.py`** — two new assertions: `test_parse_capabilities_wire_evidence_echo_v4_backend` (top-level + nested + missing-key), `test_parse_capabilities_v4_protocol_range` (min=2 stays, max moves to 4). +- **README alpha-status line + roadmap table** — `v0.15` → `v0.15.x` (so the v0.15.x fail-OPEN observability closure isn't squashed); `v0.16` → `v0.16.x` with the new highlights (Phase-1+ `action_digest` on `/gate`, `/execute` `tools` propagation, NR-006 transient-5xx retry, NR-007 error-code parity 41→56 entries, Slice B wire-evidence echo); `v0.17` for the OpenTelemetry exporter / Redis-backed offline queue / hardened init contract that previously sat under `v0.16`. + +### Changed + +- **`/gate` response handler now reads two more keys.** `action_digest` (SDK-supplied SHA-256 hex of canonical `business_impact`, re-verified server-side by `payload_binding::server_derive_action_digest`, echoed back so the SDK can confirm what the gate saw matches what it intended) and `policy_hash` (slot reserved for future Slice D wiring — today always `None` because the gate doesn't compute per-request hashes; the audit row stores `policy_hash = None` for the same reason at `audit_drain.rs:301`). Pre-v4 backends omit both keys entirely via `skip_serializing_if = "Option::is_none"` — a v4 SDK connecting to a v3 backend reads `None` on both fields and logs "no wire evidence echo" — no false positive. +- **`tests/contract/test_audit_wire.py` + `tests/test_v3_wire_contract.py`** — header assertions now source `str(NULLRUN_PROTOCOL_VERSION)` instead of the literal `"3"` so a future bump doesn't require sweeping either file. Class names kept (`TestSignedPostIncludesProtocolHeader`) for git-blame continuity. + +### Compatibility + +Wire-format additive — pre-v4 SDKs parsing the response simply ignore the new fields; v4 SDKs parsing a v3 backend response see `None` on both fields (skip_serializing_if on the backend means the JSON keys are absent, not `null`). `min_protocol_version` stays at 2, so v3 SDKs continue to work against a v4 backend. The architectural invariant `GateResponse.action_digest == AuditEvent.action_digest` holds trivially because both sides flow from the SDK's input. No new hashing/computation introduced on either side — both fields echo already-computed values. + +### Verification + +- Targeted suite: `tests/test_slice_b_wire_evidence.py` — 10/10 pass. +- Capabilities: `tests/test_capabilities.py::test_parse_capabilities_wire_evidence_echo_v4_backend`, `test_parse_capabilities_v4_protocol_range` — pass. +- Wire contract: `tests/test_v3_wire_contract.py` — pass (header assertions now source the constant). +- Audit wire: `tests/contract/test_audit_wire.py` — pass. +- Broader regression suite: `pytest -q` 1613 passed / 4 skipped (12 more than 0.16.3, accounting for the 10 new Slice B pins + 2 new capabilities assertions); `ruff check src tests` all checks pass; `mypy src/nullrun` no issues reported in 37 source files. + +### Why this is needed + +ADR-037 Slice B closes the SDK/backend wire-trust gap: pre-Slice-B the SDK had no way to verify the gate saw the same `action_digest` it intended — a misconfigured proxy or a future Slice A regression could swallow or rewrite the digest without any SDK-side signal. The echo slot on `/gate` response + the two contextvars give operators a clean diagnostic ("the gate echoed digest X — that's what I sent") and pin the architectural invariant `GateResponse.action_digest == AuditEvent.action_digest` at the SDK layer. `policy_hash` is forward-compat for Slice D; the slot is wired now so Slice D doesn't require another SDK release. + ## [0.16.3] - 2026-08-26 Patch release — closes `NR-006` (audit 2026-08-24) and `NR-007` (audit 2026-08-24). No wire-format change. Pure reliability + SDK/backend parity hardening on top of 0.16.2. diff --git a/README.md b/README.md index c276217..6a12142 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ LlamaIndex, and your own stack.
- protocol v3.31 + protocol v4 Zero-code instrumentation Server-authoritative cost
@@ -37,7 +37,7 @@ LlamaIndex, and your own stack. --- -> ⚠️ **Status: alpha (v0.15.0).** The public API may shift between minor versions. +> ⚠️ **Status: alpha (v0.16.4).** The public API may shift between minor versions. > Pin your dependency and read the [CHANGELOG](https://github.com/nullrunio/nullrun-sdk-python/blob/master/CHANGELOG.md) before upgrading. --- @@ -293,8 +293,9 @@ Runnable, copy-pastable examples live in a separate repo so you can adapt withou | Version | Status | Highlights | |---|---|---| | **v0.14.x** | ✅ alpha | Wire protocol v3.31, server-minted execution IDs, MCP, anti-OOM streaming cap | -| **v0.15** (current) | ✅ alpha | ADR-009 governance audit surface, typed `runtime.audit.*`, capability probes for `/audit-log/verify` | -| **v0.16** | 📋 planned | OpenTelemetry exporter, Redis-backed offline queue, hardened init contract | +| **v0.15.x** | ✅ alpha | ADR-009 governance audit surface, typed `runtime.audit.*`, capability probes for `/audit-log/verify`, fail-OPEN observability closure | +| **v0.16.x** (current) | ✅ alpha | Phase-1+ `action_digest` on `/gate`, `/execute` `tools` propagation, transient-5xx retry on gate (NR-006), error-code parity (NR-007, 41→56 entries) | +| **v0.17** | 📋 planned | OpenTelemetry exporter, Redis-backed offline queue, hardened init contract | | **v1.0** | 🎯 beta target | Stable wire contract, full async support, type-safe decisions | [Full roadmap & RFCs →](https://nullrun.io/roadmap) diff --git a/docs/errors/NR-P001.md b/docs/errors/NR-P001.md index c2ca12c..4f1ba1d 100644 --- a/docs/errors/NR-P001.md +++ b/docs/errors/NR-P001.md @@ -6,7 +6,7 @@ | **Category** | **P**rotocol | | **Exception class** | `NullRunProtocolError` | | **Retryable** | No | -| **Default `user_action`** | "The NullRun backend rejected the SDK's wire-protocol version. Upgrade the SDK to a version that supports protocol `X-NULLRUN-PROTOCOL: 3` — see https://docs.nullrun.io/wire-protocol." | +| **Default `user_action`** | "The NullRun backend rejected the SDK's wire-protocol version. Upgrade the SDK to a version that supports protocol `X-NULLRUN-PROTOCOL: 4` — see https://docs.nullrun.io/wire-protocol." | ## When @@ -18,8 +18,8 @@ the matching protocol is rejected with HTTP 400. ## Common causes -1. **SDK is too old** — the user is on a pre-v3 release. v3 became - the canonical wire on 2026-06-29 (0.11.0). +1. **SDK is too old** — the user is on a pre-v4 release. v4 became + the canonical wire on 2026-08-31 (0.16.3, ADR-037 Slice B). 2. **Backend hasn't rolled out the new wire yet** — the user is on a recent SDK but the backend is still on an older release. 3. **Custom transport stripped the header** — a wrapper (proxy, diff --git a/pyproject.toml b/pyproject.toml index d72a2a5..8d7d21b 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -6,7 +6,7 @@ build-backend = "hatchling.build" name = "nullrun" # Full release history lives in CHANGELOG.md; only the current version # is pinned here. -version = "0.16.3" +version = "0.16.4" # Kept under the 200-char preview threshold so the full line is visible # without an "expand" click. The headline is the canonical §1 statement # from positioning.md — "runtime decision layer for tool-using AI agents" diff --git a/src/nullrun/__version__.py b/src/nullrun/__version__.py index 9685739..f63c0f1 100644 --- a/src/nullrun/__version__.py +++ b/src/nullrun/__version__.py @@ -5,5 +5,5 @@ string and the SDK_MIN_VERSION constant. """ -__version__ = "0.16.3" +__version__ = "0.16.4" __platform_version__ = "1.0.0" diff --git a/src/nullrun/breaker/exceptions.py b/src/nullrun/breaker/exceptions.py index 1821f85..11b3c32 100644 --- a/src/nullrun/breaker/exceptions.py +++ b/src/nullrun/breaker/exceptions.py @@ -378,7 +378,7 @@ class NullRunProtocolError(NullRunInfrastructureError): user_action = ( "The NullRun backend rejected the SDK's wire-protocol version. " "Upgrade the SDK to a version that supports protocol " - "X-NULLRUN-PROTOCOL: 3 — see " + "X-NULLRUN-PROTOCOL: 4 — see " "https://docs.nullrun.io/reference/wire-protocol for the " "current compatibility matrix." ) diff --git a/src/nullrun/capabilities.py b/src/nullrun/capabilities.py index 73e0eff..4e6115b 100644 --- a/src/nullrun/capabilities.py +++ b/src/nullrun/capabilities.py @@ -142,6 +142,16 @@ class ServerCapabilities: # included in `is_v3_ready()` -- it's informational, not a # hard gate. execution_graph: bool = False + # ADR-037 Slice B (2026-08-31, protocol v4): /gate response + # echoes the SDK-supplied `action_digest` and a `policy_hash` + # slot (None today; Slice D wires per-request computation). + # Backend always sends the fields (skip_serializing_if elides + # only when None); the flag is informational so SDKs can + # surface a clean diagnostic at `init()` ("server echoes + # action_digest on /gate response — you can verify the gate + # saw the same digest you sent"). NOT included in + # `is_v3_ready()` — it's informational, not a hard gate. + wire_evidence_echo: bool = False rate_limit_fail_scope: RateLimitFailScope = field( default_factory=lambda: RateLimitFailScope() ) @@ -183,6 +193,7 @@ def as_dict(self) -> dict[str, Any]: "outbox_async_drain": self.outbox_async_drain, "idempotency_keys": self.idempotency_keys, "execution_graph": self.execution_graph, + "wire_evidence_echo": self.wire_evidence_echo, "rate_limit_fail_scope": { "aggregate": self.rate_limit_fail_scope.aggregate, "per_key": self.rate_limit_fail_scope.per_key, @@ -345,6 +356,15 @@ def _v3_flag(name: str) -> bool: # the field entirely) yield a fail-closed view where the # SDK does NOT send `parent_execution_id`. execution_graph=_v3_flag("execution_graph"), + # ADR-037 Slice B (2026-08-31, protocol v4): additive + # flag — defaults to False so pre-Slice-B backends yield + # a fail-closed view where the SDK does NOT log the + # wire-evidence echo as "server confirmed". Pre-v4 + # backends return the JSON without `action_digest` / + # `policy_hash` keys at all (skip_serializing_if on the + # backend), so a v4 SDK sees None on both fields and + # logs "no wire evidence echo" — no false positive. + wire_evidence_echo=_v3_flag("wire_evidence_echo"), rate_limit_fail_scope=_parse_rate_limit_scope(caps.get("rate_limit_fail_scope")), ) diff --git a/src/nullrun/context.py b/src/nullrun/context.py index c60e830..49daf44 100644 --- a/src/nullrun/context.py +++ b/src/nullrun/context.py @@ -328,6 +328,35 @@ def set_chain_op(op: str) -> None: _server_minted_idempotency_key_var: ContextVar[str | None] = ContextVar( "server_minted_idempotency_key", default=None ) +# ADR-037 Slice B (2026-08-31, protocol v4): wire-evidence echo +# from /gate response. Both fields are ADR-009 governance columns +# that the backend now echoes on the /gate response (additive — +# pre-v4 backends omit the keys entirely via skip_serializing_if). +# +# `action_digest` is the SDK-supplied SHA-256 hex of the canonical +# `business_impact` payload, re-verified server-side by +# `payload_binding::server_derive_action_digest` and echoed back +# on the wire so the SDK can confirm what the gate saw matches +# what it intended. The architectural invariant +# `GateResponse.action_digest == AuditEvent.action_digest` holds +# trivially because both sides flow from the SDK's input. +# +# `policy_hash` is reserved for future Slice D wiring (per +# ADR-037 §3 deferral — gate doesn't compute per-request hash +# today; in-memory KeyPolicy cache carries no hash and the hot +# path cannot load PolicyRow). Today this field is always None +# on the wire; the audit row stores `policy_hash = None` for the +# same reason (audit_drain.rs:301), so the invariant +# `GateResponse.policy_hash == AuditEvent.policy_hash` holds +# trivially. +# +# Both default to None; clear_ functions reset to None. +_last_gate_action_digest_var: ContextVar[str | None] = ContextVar( + "last_gate_action_digest", default=None +) +_last_gate_policy_hash_var: ContextVar[str | None] = ContextVar( + "last_gate_policy_hash", default=None +) def get_server_minted_execution_id() -> str | None: @@ -449,6 +478,10 @@ def clear_server_minted_execution_id() -> None: _server_minted_execution_id_var.set(None) _server_minted_reservation_at_var.set(0.0) _server_minted_idempotency_key_var.set(None) + # Also drops the v4 wire-evidence echo slots so the next + # /check in scope doesn't read a stale echo from a prior block. + _last_gate_action_digest_var.set(None) + _last_gate_policy_hash_var.set(None) Use:func:`reset_server_minted_execution_id` instead when you have a Token to consume — that path restores the previous @@ -457,6 +490,11 @@ def clear_server_minted_execution_id() -> None: _server_minted_execution_id_var.set(None) _server_minted_reservation_at_var.set(0.0) _server_minted_idempotency_key_var.set(None) + # ADR-037 Slice B (2026-08-31, protocol v4): also drop the + # wire-evidence echo slots so a /check in one block never leaks + # a stale echo into a /track in a sibling block. + _last_gate_action_digest_var.set(None) + _last_gate_policy_hash_var.set(None) def set_attempt_index(index: int) -> None: @@ -464,6 +502,69 @@ def set_attempt_index(index: int) -> None: _attempt_index_var.set(index) +# --------------------------------------------------------------------------- +# ADR-037 Slice B (2026-08-31, protocol v4): wire-evidence echo +# --------------------------------------------------------------------------- +# Read by tests + operators to confirm the gate saw the same +# `action_digest` the SDK sent (and to surface the architectural +# invariant `GateResponse.action_digest == AuditEvent.action_digest` +# from the SDK side). `policy_hash` is informational only today; +# Slice D will populate it per-request. + + +def get_last_gate_action_digest() -> str | None: + """Return the `action_digest` echoed by the last /gate response, or + ``None`` if no echo captured in scope (legacy backend, or a /check + that didn't carry a typed business impact). + + Wire-additive — pre-v4 backends omit the field entirely + (``skip_serializing_if = "Option::is_none"`` on the backend); a + v4 SDK connecting to a v3 backend reads None and behaves like + pre-Slice-B. No false positive. + + See ADR-037 Slice B (2026-08-31) for the wire contract. + """ + return _last_gate_action_digest_var.get() + + +def get_last_gate_policy_hash() -> str | None: + """Return the `policy_hash` echoed by the last /gate response, or + ``None`` if no echo captured in scope. + + Slot reserved for future Slice D wiring (per ADR-037 §3 + deferral — gate doesn't compute per-request hash today; the + audit row stores `policy_hash = None` for the same reason at + `audit_drain.rs:301`). Today this field is always None on the + wire, so this getter is informational only. + + See ADR-037 Slice B (2026-08-31) for the wire contract. + """ + return _last_gate_policy_hash_var.get() + + +def set_last_gate_action_digest(value: str | None) -> None: + """Capture the `action_digest` echoed by a /gate response. + + Called by ``runtime._capture_wire_evidence`` immediately after + ``_capture_server_minted_execution_id`` — the two captures share + the same lifetime (one /check → one execution_id + one + action_digest). See ADR-037 Slice B (2026-08-31). + """ + _last_gate_action_digest_var.set(value) + + +def set_last_gate_policy_hash(value: str | None) -> None: + """Capture the `policy_hash` echoed by a /gate response. + + Slot reserved for Slice D (per ADR-037 §3 deferral). Today + this is always set to None on the wire; this setter is the + forward-compatible hook for Slice D. + + See ADR-037 Slice B (2026-08-31). + """ + _last_gate_policy_hash_var.set(value) + + # --------------------------------------------------------------------------- # F-19 (2026-08-14): legacy _trace_id / _span_id token-based setters # --------------------------------------------------------------------------- diff --git a/src/nullrun/messages.py b/src/nullrun/messages.py index 097d512..7a3fad1 100644 --- a/src/nullrun/messages.py +++ b/src/nullrun/messages.py @@ -173,7 +173,7 @@ def format_user_message(exc: BaseException | object, locale: str = "en") -> str: Args: exc: A NullRun exception (or any object exposing ``error_code``). - locale: DEPRECATED — reserved for a future locale-pack release. Currently ignored; the catalog is English-only. Will emit a DeprecationWarning in 0.14.0 if the catalog is not yet localised by then. + locale: DEPRECATED — reserved for a future locale-pack release. Currently ignored; the catalog is English-only. non-``"en"`` value falls back to the English message. The parameter is reserved for future locale packs. diff --git a/src/nullrun/runtime.py b/src/nullrun/runtime.py index b7fdf17..9c401a3 100644 --- a/src/nullrun/runtime.py +++ b/src/nullrun/runtime.py @@ -3453,6 +3453,12 @@ def _capture_server_minted_execution_id(response: dict[str, Any]) -> str | None: op_id = response.get("operation_id") if isinstance(response, dict) else None if isinstance(op_id, str) and op_id: set_server_minted_idempotency_key(op_id) + # ADR-037 Slice B (2026-08-31, protocol v4): capture the + # wire-evidence echo on the same /check as the execution_id so + # the two values always refer to the same gate decision. Wire- + # additive: pre-v4 backends omit both keys (skip_serializing_if) + # and the captures degrade to None — no false positive. + _capture_wire_evidence(response) logger.debug( "_capture_server_minted_execution_id: captured %s", raw, @@ -3460,6 +3466,80 @@ def _capture_server_minted_execution_id(response: dict[str, Any]) -> str | None: return raw +# ADR-037 Slice B (2026-08-31, protocol v4): wire-evidence echo +# capture helper. Extracted from `_capture_server_minted_execution_id` +# so the two captures share a call site but have distinct log lines +# (debugging: a missing execution_id should not mask a successful +# wire-evidence capture). +# +# Wire contract: backend's /gate response now carries `action_digest` +# (SDK-supplied SHA-256 of canonical business_impact, re-verified +# server-side, echoed back) and `policy_hash` (slot reserved for +# future Slice D wiring — today always None because the gate doesn't +# compute per-request hashes). Both are `skip_serializing_if = "..."` +# on the backend, so v3 SDKs see JSON without the keys. +# +# Capture is fail-OPEN: a malformed value is logged at WARNING and +# dropped (the contextvar stays at None) — a single /check must +# never break the runtime on a bad wire echo. +def _capture_wire_evidence(response: dict[str, Any]) -> tuple[str | None, str | None]: + """Capture `action_digest` + `policy_hash` from the /gate response. + + Returns ``(action_digest, policy_hash)`` for the caller's log + paths; the contextvar side-effect is authoritative. + + Both fields default to None on legacy backends (no keys in + the JSON) and on pre-Phase-1 SDKs that never sent + `action_digest` (legacy grant path is fail-CLOSED at v3+ + anyway — the backend sets the field to None on those rows). + """ + from nullrun.context import ( + set_last_gate_action_digest, + set_last_gate_policy_hash, + ) + + if not isinstance(response, dict): + # Defensive — runtime never passes a non-dict, but a bad + # transport layer might. Reset to None so the previous + # scope's value can't leak. + set_last_gate_action_digest(None) + set_last_gate_policy_hash(None) + return None, None + + # Defensive string validation — backend emits lowercase hex + # of length 64 for action_digest (SHA-256 hex). A buggy proxy + # could echo garbage; drop without raising so /check still + # proceeds (the backend has already validated the value during + # /gate processing — this is a post-hoc audit capture, not a + # gate). + def _safe_str(key: str) -> str | None: + v = response.get(key) + if v is None: + return None + if not isinstance(v, str): + logger.warning( + "_capture_wire_evidence: response.%s is %s, " + "expected str — dropping", + key, + type(v).__name__, + ) + return None + return v + + action_digest = _safe_str("action_digest") + policy_hash = _safe_str("policy_hash") + + set_last_gate_action_digest(action_digest) + set_last_gate_policy_hash(policy_hash) + + logger.debug( + "_capture_wire_evidence: action_digest=%s policy_hash=%s", + "set" if action_digest else "None", + "set" if policy_hash else "None", + ) + return action_digest, policy_hash + + # 2026-07-04 (v0.12.0 wiring fix — ): build the def _build_v3_track_payload( wire_event: dict[str, Any], diff --git a/src/nullrun/transport.py b/src/nullrun/transport.py index 9524964..f9f3b28 100644 --- a/src/nullrun/transport.py +++ b/src/nullrun/transport.py @@ -57,7 +57,16 @@ # Wire-protocol version handshake. Backend rejects signed POSTs without # `X-NULLRUN-PROTOCOL: ` with 400. Bump must be coordinated with backend # `proxy::http::gate::protocol` and `/api/v1/capabilities`. -NULLRUN_PROTOCOL_VERSION: int = 3 +# +# v4 (2026-08-31, ADR-037 Slice B): ADDITIVE — /gate response now echoes +# the SDK-supplied `action_digest` and a `policy_hash` slot (always None +# today; Slice D wires per-request computation). Wire-additive: v3 SDKs +# parsing the response simply ignore the new fields; v4 SDKs parsing a +# v3 backend response see `None` on both (skip_serializing_if on the +# backend means the JSON keys are absent, not `null`). No new +# hashing/computation introduced on either side — both fields echo +# already-computed values. +NULLRUN_PROTOCOL_VERSION: int = 4 HEADER_PROTOCOL: str = "X-NULLRUN-PROTOCOL" diff --git a/tests/contract/test_audit_wire.py b/tests/contract/test_audit_wire.py index d40d3b3..203aaa5 100644 --- a/tests/contract/test_audit_wire.py +++ b/tests/contract/test_audit_wire.py @@ -30,7 +30,7 @@ ) from nullrun.breaker.exceptions import NullRunAuthenticationError from nullrun.runtime import NullRunRuntime -from nullrun.transport import HEADER_PROTOCOL +from nullrun.transport import HEADER_PROTOCOL, NULLRUN_PROTOCOL_VERSION BASE = "https://api.test.nullrun.io" ORG = "00000000-0000-0000-0000-0000000000aa" @@ -127,7 +127,7 @@ def test_includes_protocol_header(self, transport): ) transport.audit_log(organization_id=ORG) request = route.calls.last.request - assert request.headers[HEADER_PROTOCOL] == "3" + assert request.headers[HEADER_PROTOCOL] == str(NULLRUN_PROTOCOL_VERSION) assert request.headers.get("X-API-Key") == "test-key-12345678" assert request.headers.get("Authorization") == "Bearer test-key-12345678" @@ -296,7 +296,7 @@ def test_signed_post_with_hmac(self, transport): # Signed POST — HMAC + protocol header present. assert request.headers.get("X-Signature") assert request.headers.get("X-Signature-Timestamp") - assert request.headers[HEADER_PROTOCOL] == "3" + assert request.headers[HEADER_PROTOCOL] == str(NULLRUN_PROTOCOL_VERSION) assert request.headers.get("Content-Type") == "application/json" diff --git a/tests/test_capabilities.py b/tests/test_capabilities.py index 221d4de..014d9ee 100644 --- a/tests/test_capabilities.py +++ b/tests/test_capabilities.py @@ -190,6 +190,59 @@ def test_probe_capabilities_returns_caps_on_2xx(): assert caps.min_protocol_version == 3 +def test_parse_capabilities_wire_evidence_echo_v4_backend(): + """ADR-037 Slice B (2026-08-31, protocol v4): a v4 backend + surfaces `wire_evidence_echo` capability flag. Defaults to + False on pre-v4 backends (which omit the field entirely) so + the SDK falls back to a no-echo view without raising. + + NOT included in `is_v3_ready()` — it's informational, not a + hard gate. Operators can read `caps.wire_evidence_echo` to + surface a clean diagnostic at `init()` ("server echoes + action_digest on /gate response — you can verify the gate + saw the same digest you sent"). + """ + # v4 backend: flag present at top level + caps_v4 = parse_capabilities( + { + "min_protocol_version": 2, + "max_protocol_version": 4, + "wire_evidence_echo": True, + } + ) + assert caps_v4.wire_evidence_echo is True + + # v4 backend: flag nested under capabilities.* (canonical shape) + caps_v4_nested = parse_capabilities( + { + "min_protocol_version": 2, + "max_protocol_version": 4, + "capabilities": {"wire_evidence_echo": True}, + } + ) + assert caps_v4_nested.wire_evidence_echo is True + + # Pre-v4 backend: flag missing entirely → False (fail-closed) + caps_legacy = parse_capabilities( + {"min_protocol_version": 3, "max_protocol_version": 3} + ) + assert caps_legacy.wire_evidence_echo is False + + +def test_parse_capabilities_v4_protocol_range(): + """ADR-037 Slice B (2026-08-31, protocol v4): the protocol + version is bumped 3→4 additively. `min_protocol_version` stays + at 2 (so v3 SDKs are unaffected); `max_protocol_version` moves + to 4. The SDK reads both via the capabilities probe and the + parse tolerates missing keys. + """ + caps = parse_capabilities( + {"min_protocol_version": 2, "max_protocol_version": 4} + ) + assert caps.min_protocol_version == 2 + assert caps.max_protocol_version == 4 + + def test_probe_capabilities_returns_none_on_non_2xx(): """A non-2xx /api/v1/capabilities response returns None (advisory, not fatal). diff --git a/tests/test_slice_b_wire_evidence.py b/tests/test_slice_b_wire_evidence.py new file mode 100644 index 0000000..0723280 --- /dev/null +++ b/tests/test_slice_b_wire_evidence.py @@ -0,0 +1,284 @@ +"""ADR-037 Slice B (2026-08-31) — wire-evidence echo smoke tests. + +These tests pin the SDK-side of the protocol-3→4 additive bump: +the backend's /gate response now echoes the SDK-supplied +`action_digest` and a `policy_hash` slot (None today; Slice D +wires per-request computation). The SDK must: + +1. Bump `NULLRUN_PROTOCOL_VERSION` to 4. +2. Capture the response's `action_digest` / `policy_hash` into + contextvars readable by callers / tests. +3. Tolerate pre-v4 backends (no echo keys in JSON — both reads + return None). +4. Tolerate malformed wire values (non-str type — drop, do not + raise). + +Wire contract reference: backend/src/proxy/http/gate/internal.rs +(::GateResponse.action_digest + .policy_hash, both +`skip_serializing_if = "Option::is_none"`). + +The companion backend smoke checks live in +backend/src/proxy/http/gate/internal.rs::tests::slice_b_* and +backend/src/proxy/http/gate/schemas.rs::tests::slice_b_*. +""" + +from __future__ import annotations + +from unittest.mock import patch + +import pytest + +from nullrun.context import ( + clear_server_minted_execution_id, + get_last_gate_action_digest, + get_last_gate_policy_hash, +) +from nullrun.runtime import _capture_wire_evidence +from nullrun.transport import ( + HEADER_PROTOCOL, + NULLRUN_PROTOCOL_VERSION, + _protocol_header_value, +) + +# --------------------------------------------------------------------------- +# 1. Protocol constant is the single source of truth +# --------------------------------------------------------------------------- + + +def test_protocol_version_is_four() : + """The SDK's protocol version MUST be 4 to match the backend + bump (ADR-037 Slice B, 2026-08-31). The backend reads this + value off the X-NULLRUN-PROTOCOL header; an SDK <4 fails the + handshake with 400 PROTOCOL_TOO_OLD. + + Wire-additive — v3 SDKs that haven't bumped yet are unaffected + (the v4 backend still accepts proto=3 because the bump is + additive: MIN stays at 2). + """ + assert NULLRUN_PROTOCOL_VERSION == 4, ( + f"expected NULLRUN_PROTOCOL_VERSION=4 after Slice B " + f"(ADR-037, 2026-08-31); got {NULLRUN_PROTOCOL_VERSION}. " + f"This breaks the /gate handshake against the bumped backend." + ) + + +def test_protocol_header_value_matches_constant() : + """The header string emitted on /gate MUST match the constant. + + Drift here means the SDK and the backend's + `X-NULLRUN-PROTOCOL` parser disagree on the wire value. + """ + assert _protocol_header_value() == "4" + assert HEADER_PROTOCOL == "X-NULLRUN-PROTOCOL" + + +# --------------------------------------------------------------------------- +# 2. Wire-evidence capture — happy path +# --------------------------------------------------------------------------- + + +def test_capture_wire_evidence_extracts_action_digest_from_response() : + """A /gate response with `action_digest` set MUST be captured + into the contextvar readable via `get_last_gate_action_digest`. + This is the architectural invariant the SDK enforces: + + SDK → /gate (action_digest) → response (action_digest) + ↓ ↓ + contextvar.get() contextvar.get() + + A reader that does ``get_last_gate_action_digest()`` after a + /check sees the digest the backend re-verified and echoed. + """ + clear_server_minted_execution_id() # also clears v4 echo slots + digest = ( + "dfc96387ca539b7130caebe705e042f2e34e52ab44352ae5e527bcef64f0df27" + ) + response = { + "decision": "allow", + "execution_id": "01936c5e-7c8a-7def-9a01-abcdef012345", + "action_digest": digest, + "policy_hash": None, + } + + action_digest, policy_hash = _capture_wire_evidence(response) + + assert action_digest == digest + assert policy_hash is None + assert get_last_gate_action_digest() == digest + assert get_last_gate_policy_hash() is None + + +def test_capture_wire_evidence_extracts_policy_hash_when_present() : + """When the backend populates `policy_hash` (Slice D future), + the SDK MUST capture it. Today the backend always sends None, + but the SDK is forward-compatible so a future Slice D wire + doesn't require an SDK bump. + """ + clear_server_minted_execution_id() + policy_hash = ( + "9b71d224bd62f3785d96d46ad3ea3d73319bfbc2890caadae2dff72519673ca7" + ) + response = { + "decision": "allow", + "execution_id": "01936c5e-7c8a-7def-9a01-abcdef012345", + "action_digest": ( + "dfc96387ca539b7130caebe705e042f2e34e52ab44352ae5e527bcef64f0df27" + ), + "policy_hash": policy_hash, + } + + action_digest, captured_policy = _capture_wire_evidence(response) + + assert action_digest is not None + assert captured_policy == policy_hash + assert get_last_gate_policy_hash() == policy_hash + + +# --------------------------------------------------------------------------- +# 3. Wire-evidence capture — pre-v4 / no-op backends +# --------------------------------------------------------------------------- + + +def test_capture_wire_evidence_tolerates_missing_keys() : + """Pre-v4 backends omit both keys (skip_serializing_if on the + backend). The SDK MUST NOT raise; both captures must be None. + + This is the v3-SDK-compat smoke check #5 — a v4 SDK against a + v3 backend sees no echo and degrades gracefully. + """ + clear_server_minted_execution_id() + response = { + "decision": "allow", + "execution_id": "01936c5e-7c8a-7def-9a01-abcdef012345", + # No action_digest, no policy_hash + } + + action_digest, policy_hash = _capture_wire_evidence(response) + + assert action_digest is None + assert policy_hash is None + assert get_last_gate_action_digest() is None + assert get_last_gate_policy_hash() is None + + +def test_capture_wire_evidence_tolerates_non_dict_response() : + """Defensive — runtime never passes a non-dict, but a buggy + transport layer could. Both captures must reset to None and + the call must not raise. + """ + clear_server_minted_execution_id() + # Set a stale value first to confirm the reset + from nullrun.context import set_last_gate_action_digest + set_last_gate_action_digest("stale-value-from-previous-block") + + action_digest, policy_hash = _capture_wire_evidence("not a dict") # type: ignore[arg-type] + + assert action_digest is None + assert policy_hash is None + assert get_last_gate_action_digest() is None + + +def test_capture_wire_evidence_drops_malformed_values() : + """A buggy proxy could echo a non-string (int, list, bool). The + SDK MUST log a warning, drop the value (capture stays None), + and NOT raise — /check must still succeed because the backend + has already validated the value during /gate processing. + """ + clear_server_minted_execution_id() + + with patch("nullrun.runtime.logger") as mock_logger: + response = { + "decision": "allow", + "action_digest": 12345, # type: ignore[dict-item] + "policy_hash": ["not", "a", "string"], # type: ignore[list-item] + } + action_digest, policy_hash = _capture_wire_evidence(response) + + assert action_digest is None + assert policy_hash is None + # Logger emitted a warning for each malformed field + assert mock_logger.warning.call_count == 2 + assert get_last_gate_action_digest() is None + assert get_last_gate_policy_hash() is None + + +# --------------------------------------------------------------------------- +# 4. Integration with _capture_server_minted_execution_id +# --------------------------------------------------------------------------- + + +def test_execution_id_capture_also_captures_wire_evidence() : + """The shared call path means a single /check captures both + execution_id AND wire evidence together. This is the runtime + invariant: ``response.execution_id`` and + ``response.action_digest`` always come from the SAME /check + decision (so a reader that consults both contextvars sees a + coherent (execution_id, action_digest) tuple). + """ + clear_server_minted_execution_id() + digest = ( + "dfc96387ca539b7130caebe705e042f2e34e52ab44352ae5e527bcef64f0df27" + ) + + from nullrun.runtime import _capture_server_minted_execution_id + + response = { + "decision": "allow", + "reservation_id": "01936c5e-7c8a-7def-9a01-abcdef012345", + "operation_id": "11111111-2222-3333-4444-555555555555", + "action_digest": digest, + } + + captured_id = _capture_server_minted_execution_id(response) + + assert captured_id == "01936c5e-7c8a-7def-9a01-abcdef012345" + # Wire evidence was captured on the same call + assert get_last_gate_action_digest() == digest + + +def test_execution_id_capture_without_wire_evidence_legacy_backend() : + """Pre-v4 backend: /check captures execution_id but no + action_digest (key absent). The wire-evidence capture is a + no-op (None on both), and the runtime proceeds. + """ + clear_server_minted_execution_id() + + from nullrun.runtime import _capture_server_minted_execution_id + + response = { + "decision": "allow", + "reservation_id": "01936c5e-7c8a-7def-9a01-abcdef012345", + "operation_id": "11111111-2222-3333-4444-555555555555", + # No action_digest (pre-v4 backend) + } + + captured_id = _capture_server_minted_execution_id(response) + + assert captured_id == "01936c5e-7c8a-7def-9a01-abcdef012345" + # No wire evidence echoed (legacy backend) + assert get_last_gate_action_digest() is None + assert get_last_gate_policy_hash() is None + + +# --------------------------------------------------------------------------- +# 5. Clear semantics — block-exit drops both v4 echo slots +# --------------------------------------------------------------------------- + + +def test_clear_server_minted_execution_id_drops_wire_evidence() : + """`clear_server_minted_execution_id` is the runtime's "block + exited, drop the capture" hook. It MUST also reset the v4 + echo slots so a /check in one block never leaks wire evidence + into a /track in a sibling block. + """ + from nullrun.context import set_last_gate_action_digest + set_last_gate_action_digest("stale-digest") + + clear_server_minted_execution_id() + + assert get_last_gate_action_digest() is None + assert get_last_gate_policy_hash() is None + + +if __name__ == "__main__": + pytest.main([__file__, "-v"]) \ No newline at end of file diff --git a/tests/test_v3_wire_contract.py b/tests/test_v3_wire_contract.py index d88ede7..3b070dc 100644 --- a/tests/test_v3_wire_contract.py +++ b/tests/test_v3_wire_contract.py @@ -67,7 +67,7 @@ class TestProtocolHeaderConstant: def test_version_is_three(self): # Bumping this requires a coordinated backend release — # see (semver: major = breaking wire change). - assert NULLRUN_PROTOCOL_VERSION == 3 + assert NULLRUN_PROTOCOL_VERSION == 4 # v4 = Slice B additive (action_digest, policy_hash on /gate wire) def test_header_name_is_dashed(self): # Match the backend's HeaderName parsing (axum 0.7 normalises @@ -79,12 +79,21 @@ def test_protocol_header_value_helper(self): from nullrun.transport import _protocol_header_value # Stored as u32 on the wire — serialise the integer directly - # (``"3"``, not ``"v3"``). - assert _protocol_header_value() == "3" + # (``"4"`` after the Slice B bump, not ``"v3"``). The value + # is sourced from `NULLRUN_PROTOCOL_VERSION` as the single + # source of truth so future bumps don't sweep this test. + assert _protocol_header_value() == str(NULLRUN_PROTOCOL_VERSION) class TestSignedPostIncludesProtocolHeader: - """Every signed POST must include ``X-NULLRUN-PROTOCOL: 3``.""" + """Every signed POST must include ``X-NULLRUN-PROTOCOL: ``. + + v4 (2026-08-31, ADR-037 Slice B): protocol bumped 3→4 additively. + The class name is kept (``TestSignedPostIncludesProtocolHeader``) + for git-blame continuity; the header value is sourced from + ``NULLRUN_PROTOCOL_VERSION`` as the single source of truth so + future bumps don't require sweeping this file again. + """ @respx.mock def test_track_batch_includes_protocol_header(self): @@ -95,7 +104,7 @@ def test_track_batch_includes_protocol_header(self): ) t._send_batch_with_retry_info([{"event": "test"}]) sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -111,7 +120,7 @@ def test_check_includes_protocol_header(self): ) t.check({"check_type": "llm", "estimated_tokens": 1}) sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -135,7 +144,7 @@ def test_check_v3_includes_protocol_header(self): ) t.check_v3({"check_type": "llm", "estimated_tokens": 1}) sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -164,7 +173,7 @@ def test_track_single_includes_protocol_header(self): } ) sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -177,7 +186,7 @@ def test_cancel_includes_protocol_header(self): ) t.cancel("exec-1") sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -190,7 +199,7 @@ def test_heartbeat_includes_protocol_header(self): ) t.heartbeat("chain-abc") sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -206,7 +215,7 @@ def test_chain_end_includes_protocol_header(self): ) t.chain_end("chain-abc") sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) body = sent.content.decode("utf-8") assert '"chain_id":"chain-abc"' in body assert '"chain_op":"end"' in body @@ -231,7 +240,7 @@ def test_approximate_budget_includes_protocol_header(self): ) t.approximate_budget(organization_id="org-1") sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -253,7 +262,7 @@ def test_execute_includes_protocol_header(self): input_data={"command": "ls"}, ) sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop() @@ -269,7 +278,7 @@ def test_refetch_credentials_includes_protocol_header(self): ) asyncio.run(t._refetch_credentials()) sent = route.calls.last.request - assert sent.headers["X-NULLRUN-PROTOCOL"] == "3" + assert sent.headers["X-NULLRUN-PROTOCOL"] == str(NULLRUN_PROTOCOL_VERSION) finally: t.stop()