Skip to content

Add opaque relying-service client_reference to proof bundles and lifecycle lookup - #52

Open
BitcoinErrorLog wants to merge 6 commits into
pubky:masterfrom
BitcoinErrorLog:feat/client-reference
Open

BitcoinErrorLog wants to merge 6 commits into
pubky:masterfrom
BitcoinErrorLog:feat/client-reference

Conversation

@BitcoinErrorLog

Copy link
Copy Markdown

Add opaque relying-service client_reference to proof bundles and lifecycle lookup

Motivation

{ pubky_lock_resource, criterion_id, reader_public_key } binds a verification
task to a content lock and its terms, but not to one instance of a relying
service's business object. A relying service that sells access per order cannot
distinguish "the buyer completed this exact bundle for this order" from "the
buyer completed an identical bundle (same lock, same reader, same terms) for an
earlier order" — the old completed task could be registered against a new order
for the same item. The relying service needs a value it mints itself, per
payment, that Locks stores verbatim and echoes back so the service can require
exact equality at completion.

Changes

  • locks-core: SubmittedProofBundle gains an optional
    client_reference: Option<ClientReference>
    (#[serde(default, skip_serializing_if = "Option::is_none")],
    deny_unknown_fields kept). ClientReference is a newtype over String
    validated on construction and deserialize: 1..=64 bytes of UTF-8 (bytes, not
    chars), no control characters, and no interpretation of any kind (no
    trimming, no case folding). Bundles without the field behave exactly as
    before.
  • locks-service: the reference rides inside the existing
    submitted_proof_bundle JSONB column, and the exact persisted replay
    comparison in SubmitProofBundleUseCase compares the whole stored bundle,
    so a replay under the same { creator, bundle_id } with a different
    client_reference (present vs absent, or a different value) returns the
    existing 409 task_state_conflict while an identical replay stays
    idempotent. Tests pin both behaviors at the use-case level. Task updates are
    now lifecycle-only at the repository boundary: update_verification_task
    writes only status, started_at, completed_at, and failure_message in
    both the Postgres and in-memory repositories and never writes the stored
    bundle or task identity, so the stored client_reference is immutable after
    insert by construction, not by caller discipline. Adversarial memory and
    Postgres tests submit updates whose bundles carry a different
    client_reference (and different proofs) and assert the stored bundle is
    unchanged. VerificationTaskLifecycleView gains pubky_lock_resource,
    criterion_ids, reader_public_key, and client_reference echoed from the
    stored bundle.
  • locks-server: VerificationTaskLifecycleHttpResponse gains the same four
    fields; viewer-access contract fixtures updated; route tests assert the echo
    and the unchanged secret exclusions (no task_id, no raw proof payloads, no
    invoice data, no bearer credentials).
  • locks-sdk: VerificationTaskLifecycleResponse (still
    deny_unknown_fields) gains the four fields, all read as optional
    (Option / #[serde(default)]), so an upgraded SDK parses lifecycle
    responses from both old and new servers; parsing tests cover both shapes.
    The JS/WASM bindings delegate lifecycle validation to this Rust parser and
    do not mirror the struct, so they inherit the same tolerance (covered by a
    binding test).
  • locks-e2e: the in-process submit → completion → lookup retrieval flow now
    submits with a client_reference and asserts it plus the three binding
    fields on both the submission and lookup responses.
  • docs: API.md (request field + lookup response shape), DOMAIN_MODEL.md
    (aggregate invariants), SDK.md (response field list + rollout order), and
    new ADR 0021.

Contract impact

  • POST /proof-bundles: the request field is optional and backwards
    compatible; bundles without it behave exactly as today. client_reference
    participates in the replay identity: a changed reference under the same
    handle is 409 task_state_conflict, an identical replay stays idempotent.
  • The lifecycle lookup response (POST /verification-task-lookups, and the
    same view from submission and dev completion) gains four fields:
    pubky_lock_resource, criterion_ids, reader_public_key,
    client_reference.

Rollout order and compatibility

The new SDK reads the four new fields as optional, so an upgraded SDK works
against both the old server (fields absent) and the new server (fields
present). The only safe rollout order is:

  1. Release the SDK with the binding fields read as optional.
  2. Consumers (Rust and JS/WASM, which share the same Rust parser) upgrade to
    that SDK release.
  3. Deploy the Lock Server that returns the four new fields.

There is no safe server-first order: SDK releases from before this change use
deny_unknown_fields parsers — the repository's pre-existing parser posture,
consistent with CONTRIBUTING's statement that maintainers may change APIs
without backwards compatibility — so they reject the new server's lifecycle
responses on submit, lookup, and dev completion. That break is inherent to
already-published strict SDK releases and cannot be shimmed from the server
side; consumers pinned to such a release must upgrade before the server
deploys. This order and the strict-SDK impact are documented in
docs/SDK.md and ADR 0021.

criterion_ids note

The ask names criterion_id, but a submitted bundle may carry several proofs,
one per criterion. The response therefore returns criterion_ids: all of the
submitted proofs' criterion_id values, in submission order.

Persistence impact

No migration: the reference is stored inside the existing
submitted_proof_bundle JSONB column. Immutability is enforced at the
repository boundary: task updates are lifecycle-only and cannot rewrite the
stored bundle in either repository implementation, proven by adversarial tests
that submit updates carrying a different client_reference and assert the
stored value is unchanged. Replay-conflict semantics are unchanged because the
existing comparison already covers the whole stored bundle.

Security notes

  • The value is inert: bounded to 1..=64 bytes of UTF-8 without control
    characters, never trimmed, folded, or interpreted by Locks.
  • It is not a secret and grants nothing by itself; the possession boundary is
    unchanged ({ creator, bundle_id } handle), and lookup responses keep the
    existing exclusions (no bearer secrets, invoice data, or raw proof payloads —
    asserted in tests).
  • No change to auth, credential issuance, Paykit invoice handling, or worker
    completion logic beyond what the lookup view reads.

Verification

Baseline (a9d52b8, before editing): fmt clean; clippy clean; 328 passed
(workspace minus e2e/service/sdk-wasm); 183 passed / 31 skipped
(locks-service lib, postgres skipped); 31 passed (locks-service postgres);
git diff --check clean.

Final (this branch):

cargo fmt --check
# exit 0 (no output)
cargo clippy --workspace --all-targets --all-features -- -D warnings
# Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.76s
cargo nextest run --workspace --exclude locks-e2e --exclude locks-service --exclude locks-sdk-wasm
# Summary [   0.545s] 342 tests run: 342 passed, 0 skipped
cargo nextest run -p locks-service --lib -- --skip infrastructure::postgres
# Summary [   0.302s] 186 tests run: 186 passed, 32 skipped
cargo nextest run -p locks-service --lib infrastructure::postgres   # TEST_DATABASE_URL set
# Summary [   1.266s] 32 tests run: 32 passed, 186 skipped
git diff --check
# exit 0 (no output)

Additionally: cargo nextest run -p locks-sdk-wasm — 56 passed;
cargo nextest run -p locks-e2e --test retrieval_access_http — 1 passed (the
extended submit → lookup flow; other e2e targets require Docker/live services
and were not run).

SubmittedProofBundle gains an optional client_reference field, a newtype
over String validated on construction and deserialize: 1..=64 bytes, no
control characters, no trimming or case folding. Locks persists the
value with the bundle and never interprets it; relying services use it
to bind a verification task to exactly one instance. Bundles without
the field behave exactly as before.
The exact persisted replay comparison in SubmitProofBundleUseCase
compares the whole stored submitted_proof_bundle, so a replay under the
same {creator, bundle_id} handle with a different client_reference
(present vs absent, or a different value) already returns the existing
409 task_state_conflict mapping; an identical replay stays idempotent.
Add use-case tests pinning both behaviors, and memory/postgres store
tests proving the stored reference never changes across status
transitions.
The handle-based lifecycle view now returns pubky_lock_resource, the
submitted proofs' criterion_ids in submission order, reader_public_key,
and client_reference alongside the existing status fields, across the
locks-service view, the locks-server HTTP response, and the locks-sdk
typed parser. The possession boundary stays {creator, bundle_id} and
the response still excludes task_id, raw proof payloads, invoice data,
and bearer secrets. Viewer-access contract fixtures are updated; the
JS/WASM bindings delegate lifecycle validation to the Rust sdk parser
and only their test JSON gains the new fields.
API.md documents the optional request field on POST /proof-bundles and
the four echoed binding fields on the lifecycle lookup response.
DOMAIN_MODEL.md records the client_reference invariants on the
SubmittedProofBundle aggregate, SDK.md lists the lifecycle response
fields and the lockstep upgrade requirement, and ADR 0021 records the
decision and its boundaries.
The in-process submit -> completion -> lookup flow now submits a bundle
carrying a client_reference and asserts the reference plus
pubky_lock_resource, criterion_ids, and reader_public_key on both the
submission and lookup lifecycle responses, with no proof payloads or
task ids leaking into either.
@dzdidi

dzdidi commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

looking into it

@dzdidi

dzdidi commented Sep 17, 2026

Copy link
Copy Markdown
Collaborator

@BitcoinErrorLog bundle id should be unique and never be reused

@BitcoinErrorLog

Copy link
Copy Markdown
Author

Agreed — and the PR keeps that: bundle ids stay viewer-generated, unique, never reused, and an exact persisted replay of the same {creator, bundle_id} is still idempotent (a different client_reference under the same handle is a 409 task_state_conflict, same as any other changed field).

The gap this closes is one level up. Uniqueness of the bundle id proves "this is one task"; it doesn't prove "this task was paid for this order". Concrete case from our integration test: buyer completes a task for lock L (own unique bundle id B1, completed), buys the same item again a week later, and registers B1 against the new order. Everything the relying service can read from the lookup — {pubky_lock_resource, criterion_id, reader_public_key} — matches the new order too, because it's the same lock, same reader, same terms. Nothing in the task says which purchase it belongs to, so the old completed task confirms a new order that was never paid.

Blocking reuse on our side isn't enough either: we can refuse to register a bundle id twice, but the buyer never registered B1 against the first order in the failing schedule — the first order can be paid through another rail, or expire — so the first registration of B1 is against the wrong order. What fixes it is a value the relying service mints itself before the buyer submits, carried inertly by the task, and echoed on lookup so we can require exact equality at completion: that's client_reference. Locks doesn't interpret it, and an absent echo is treated by the relying service as a mismatch (added to SDK.md/API.md in the last commit).

Happy to rename or reshape anything (e.g. criterion_idscriterion_id if you'd rather guarantee one), and to split docs from code if that helps review.

@dzdidi dzdidi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head e571d6d23fce84944e5abab9b162280ad3e7440e; no concrete code findings. Independent same-digest pass agreed.

Recommended human verdict: needs follow-up. PR currently conflicts with base. Exact-head RustSec also fails on rustls 0.23.43 (fixed versions start at 0.23.45). Other GitHub checks pass. Local cargo test --workspace --all-targets passed non-Postgres coverage; explicit Postgres tests could not run because TEST_DATABASE_URL is unset.

@dzdidi

dzdidi commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

@BitcoinErrorLog ignore my comment (it was "autogenerated") I'll take over this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants