Skip to content

fix(api): cache-control for the PL-2 monitoring responses, and a guard that can see them (BACKLOG #1185) - #874

Open
wshallwshall wants to merge 3 commits into
mainfrom
claude/backlog-1185-no-store-classification-guard
Open

fix(api): cache-control for the PL-2 monitoring responses, and a guard that can see them (BACKLOG #1185)#874
wshallwshall wants to merge 3 commits into
mainfrom
claude/backlog-1185-no-store-classification-guard

Conversation

@wshallwshall

Copy link
Copy Markdown
Collaborator

BACKLOG #1185, ASVS 14.2.2. Three monitoring routes returned PL-2-classified free text with no cache directive, and the coverage guard was structurally unable to see them. Both are fixed. The item stays OPEN -- it closes by a vault scorecard re-score, which no builder can do. The status banner is untouched.

The old guard was blind, measured with a positive control in the same run

Re-measured at 2b8bccb43. The item's line numbers had moved AGAIN: the tuple is at messagefoundry/api/app.py:355, not the item's :336 or the re-score's :346.

route gate old _is_phi_read under a prefix wire Cache-Control
GET /events monitoring:read False no none
GET /connections/{name}/events monitoring:read False no none
GET /alerts/active monitoring:diagnose False no none
GET /messages (control) require_phi_read True yes no-store

Re-running the old guard's own assertion over that app returned uncovered == []. It passed, green, with three PL-2 responses shipping uncovered. connection_event.reason and alert_instance.reason are both rated PL-2 in docs/PHI.md section 2.

And the new one is not blind

The predicate now selects on what a response carries, not on what gates it. A route is sensitive when its response model, walked recursively, projects a store column docs/PHI.md section 2 rates PL-1/PL-2/PL-3. The rating is read out of section 2, reusing the at-rest inventory guard's own row parser, so this file cannot drift from the document it measures against -- and test_the_two_phi_md_parsers_agree fails if the two readings of that table ever disagree about a level.

Three mutations, all red, each with a distinct failure set:

mutation what it simulates result
empty _NO_STORE_ROUTE_PATHS the fix reverted 4 red, naming all seven paths
delete the ConnectionEventInfo.reason binding a NEW unclassified PL-2 field 4 red, incl. the deny-by-default hinge
parse the wrong section of PHI.md a broken instrument 6 red -- no vacuous green

The planted-route control is permanent in the suite: it registers a route returning ConnectionEventInfo at an uncovered path with no PHI gate at all, then asserts the old predicate misses it and the new one fires.

The reviewed part, and why it exists

_RESPONSE_FIELD_COLUMN binds 27 response fields to the store column each projects, or to None with a stated reason. It is needed because a bare column NAME is ambiguous across tables: detail is PL-2 as message_events.detail and PL-4 as audit_log.detail. Matching on the name alone selected 34 uncovered routes, almost all of them SimpleMessage.detail -- a literal operation-result string.

An unresolved collision FAILS. That is the hinge: a new response field whose name collides with a classified column reds the module until somebody classifies it, whatever permission gates its route.

None is a statement about provenance, not sensitivity. It says no section 2 row rates the value; it does not assert the value is harmless.

Engine change

_NO_STORE_ROUTE_PATHS holds route path templates, read off request.scope["route"] during call_next, because /connections/{name}/events cannot be a prefix without blanketing the whole /connections dashboard. The prefix set stays, and the reason is not style: GET /messages/{message_id}/attachments/{attachment_id} streams raw bytes and declares no response_model, so no model walk can ever see it. The gate-shaped predicate is therefore kept as a second disjunct.

The four alert-mutation POSTs are covered too -- same AlertInstanceInfo, same PL-2 reason -- which is part of the wider surface the 2026-08-20 research named.

The prefix test runs first in the condition and short-circuits the PHI-read surface before the scope lookup. A review measured the added arm at 40-65 ns on a miss, against an ASGI response cycle of tens of microseconds, and it allocates nothing.

OPEN QUESTION for the owner: the docstring conflict, not resolved here

GET /events and GET /alerts/active both say "metadata only, no PHI" while docs/PHI.md rates their reason column PL-2. One of the two is wrong and the ASVS cell rests on which. The 2026-08-20 research adds a third artifact carrying the same claim, messagefoundry_webconsole/routes/monitoring.py:22-23.

This is built as though PL-2 is correct, because PL-2 is the shipped classification. docs/PHI.md was not edited -- downgrading the rating to settle the conflict is the move the item disqualifies by name, since it edits the document the verb measures against.

Second unresolved ruling: whether "sensitive data" in the 14.2.2 verb tracks the PHI permission or the PHI classification. This guard makes the classification-shaped reading buildable and asserts nothing about the other.

Named, not built

  • The second disjunct is untouched. _state_cache and _reference_cache still evict on abandonment plus age, never on use, so they hold PHI-derived values for the store object's lifetime. Whether a decrypted cache with process-lifetime residency can satisfy "purged after use" at all, when keeping hot keys resident is what the design is for, remains open.
  • The register's altitude. Review raised that _RESPONSE_FIELD_COLUMN is engine knowledge living in a test file, and that messagefoundry/config/retention_classification.py is the precedent for carrying a PL rating in shipped code with a doc-drift test beside it. Moving it is what would let _NO_STORE_ROUTE_PATHS be derived at app construction rather than hand-typed -- the deny-by-default inversion this item's own research already proposes. Left as a separate change; it alters middleware behaviour and deserves its own review.
  • Two adjacent free-text fields, registered as projecting no store column and flagged in place: ConnectionRow.error and ConnectionMetadata.error carry a connector start-failure string (ADR 0031) whose persisted twin connection_event.reason IS rated PL-2. Whether the live string deserves the same rating is a classification ruling, not a build.

Checks

Run locally, against this branch:

  • ruff check messagefoundry tests -- clean. ruff format --check on both changed files -- clean.
  • mypy messagefoundry strict -- Success: no issues found in 267 source files.
  • pytest on test_no_store_phi_coverage (21), test_phi_at_rest_inventory, test_security_doc_drift, test_api_security_header_floor, test_api_alerts -- 112 passed.
  • Earlier passes also green: test_api, test_webconsole_absent, test_api_request_timeout, test_ui_hardening, test_ui_csp_canary (90 passed, 3 skipped); test_backlog_status_check, test_ledger_check, test_backlog_citation_check (90 passed + 1 flake, below).
  • scripts/docs/backlog_status_check.py -- OK, 664 backlog items, each declaring exactly one status. #1185 parses with an empty closed-banner list and a live open banner.

Not run: the full suite. The box is contended and a full local run was measured at 4 percent in 18 minutes. Please read the hosted-only legs -- SQL Server, Postgres, windows-service-smoke, and the tooling job.

One flake, recorded: test_backlog_citation_check.py::test_the_citation_regex_sees_every_ledger_LINK_the_link_checker_sees failed once with git ls-files exit 128, then passed on a single rerun. The same command succeeds standalone from this worktree; a peer session holding a git lock is the likely cause. One rerun only, per the rerun budget.

Guard runtime went 36.5s to 6.6s: the module now shares one engine and one app, since every test reads the app and the only one that mutates a route table builds its own.

wshallwshall added 3 commits September 4, 2026 18:10
… text (BACKLOG #1185)

GET /events, GET /connections/{name}/events and GET /alerts/active return
connection_event.reason and alert_instance.reason, both rated PL-2 in
docs/PHI.md section 2. All three are gated on monitoring:read or
monitoring:diagnose rather than on a PHI permission, so the no-store control --
keyed to the PHI-read route families -- never reached them. Measured at
2b8bccb: all three served with no Cache-Control header at all, against
/messages serving no-store in the same run.

_NO_STORE_ROUTE_PATHS carries the classified stragglers outside every PHI-read
family. It holds route path TEMPLATES rather than URL prefixes because
/connections/{name}/events cannot be written as a prefix without blanketing the
whole /connections dashboard, most of which returns no classified column; the
middleware reads the template off request.scope["route"], which the router
populates during call_next. The prefix set stays: GET
/messages/{message_id}/attachments/{attachment_id} streams raw bytes and
declares no response_model, so no model walk can ever see it.

The four alert-mutation replies are included -- they return the same
AlertInstanceInfo, PL-2 reason and all.

The prefix test runs first in the condition and short-circuits the whole
PHI-read surface before the scope lookup is reached.
…ion gate (BACKLOG #1185)

The guard selected a route as sensitive by its permission gate -- require_phi_read,
an explicit PHI-hop charge, or a GET/HEAD require_step_up. That predicate is
structurally unable to see a route returning a classified column under a
DIFFERENT permission, so it could never have caught the three monitoring routes
the previous commit covers, however many paths were added to the constant.

Measured before the fix: _is_phi_read returned False for all three while the
wire carried no Cache-Control, and the guard's own assertion came back
uncovered == [] and PASSED. A green from a check that cannot see the failure
class is worth nothing, which is this item's whole subject.

The predicate now selects on what the response CARRIES. A route is sensitive
when its response model, walked recursively, projects a store column docs/PHI.md
rates PL-1/PL-2/PL-3. The rating is READ OUT OF docs/PHI.md section 2 rather
than restated, reusing the inventory guard's own row parser, so this file cannot
disagree with the document it measures against; test_the_two_phi_md_parsers_agree
fails if the two readings of that table ever diverge on a level.

_RESPONSE_FIELD_COLUMN is the one reviewed part -- 27 field-to-column bindings,
needed because a bare column NAME is ambiguous across tables (detail is PL-2 as
message_events.detail, PL-4 as audit_log.detail; matching on the name alone
selected 34 routes, almost all SimpleMessage.detail). An unresolved collision
FAILS: that is the deny-by-default hinge, and it is how the next classified
monitoring route gets classified instead of shipping uncovered.

The gate-shaped arm is kept as a second disjunct -- it reaches
GET /messages/{message_id}/attachments/{attachment_id}, which declares no
response_model at all.

Three mutations, all red, each with a distinct failure set: emptying the covered
set reds 4 (naming all seven paths); deleting the ConnectionEventInfo.reason
binding, which simulates a NEW unclassified PL-2 field, reds 4 including the
hinge; pointing the parse at the wrong section reds 6, so a broken instrument
cannot yield a vacuous green.

Two rulings deliberately NOT made, both recorded on the item: the route
docstrings say "metadata only, no PHI" against PHI.md's PL-2, and whether
"sensitive data" tracks the PHI permission or the classification. Built as
though PL-2 is correct, because PL-2 is what ships.

One engine and one app for the module (function scope cost about 2s of
Engine.create per test): 36.5s to 6.6s.
…did not rule

Progress note only. The status banner is untouched -- #1185 closes by a vault
scorecard re-score, which no builder can do, so shipped code beside an open item
is the correct outcome.

Records: the re-measured line numbers (the tuple had moved again, to :355, not
the item's :336 or the re-score's :346); the blindness proof with its positive
control; what shipped; the three mutations; the two rulings left open for the
owner; and what was named but not built -- the decrypted-cache disjunct, the
register's altitude (it is engine knowledge living in a test file), and the two
adjacent free-text connector fields whose rating is a classification call.
@wshallwshall wshallwshall added the reviewed A reviewer has read this. Removed automatically when new commits arrive. label Sep 4, 2026
@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Disclosure, per the Manager playbook section 6: the reviewed label on this pull request was applied by the Manager seat that commissioned the diff, not by an independent reader. It records that a step happened, and it is not evidence that a second party read the scope.

The owner has since disabled the reviewer role and its tag requirement. Removing the required status context is a branch-protection change and belongs to the Lander, not to this seat.

Known and named rather than left to be found: the full suite was not run on a contended box, so the SQL Server, Postgres, windows-service-smoke and tooling legs need reading here. The docstring-versus-classification conflict is deliberately unresolved and PHI.md was not edited.

@wshallwshall

Copy link
Copy Markdown
Collaborator Author

Correction to my disclosure above: it said the required status context still exists on main mechanically. That was true when written and is false now. Re-read live at 23:39Z, printing the full set rather than grepping for a zero: 13 required contexts, and a reviewer has read this is not among them. The set moved while several seats were measuring it.

The disclosure's load-bearing half stands unchanged: the reviewed label here was applied by the Manager seat that commissioned the diff and is not an independent read. It is now also not load-bearing for the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

reviewed A reviewer has read this. Removed automatically when new commits arrive.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant