Skip to content

chore: share the escaping helpers and collapse the ai-gateway twins - #604

Merged
chubes4 merged 1 commit into
mainfrom
chore/duplicate-audit
Sep 17, 2026
Merged

chubes4 merged 1 commit into
mainfrom
chore/duplicate-audit

Conversation

@chubes4

@chubes4 chubes4 commented Sep 17, 2026

Copy link
Copy Markdown
Member

The sibling finding to #603. That audit asked which mechanisms have no consumer; this one asks which have two bodies.

The drift that had already happened

Escaping was written three times. Two of the JSON escapers were identical:

value=${value//\\/\\\\}
value=${value//\"/\\\"}
value=${value//$'\n'/\\n}

The third, in lib/agent-state-ownership.sh, was not:

printf '%s' "$1" | sed -e 's/\\/\\\\/g' -e 's/"/\\"/g'

No newline handling. A value containing a newline emits a literal newline inside a JSON string literal, which is invalid JSON — and that copy's output is the machine-read {"status":"root_repair_required",...} record. Nothing exercised the newline case, which is exactly why the divergence survived: there was no test, and nobody knew there were three implementations to compare.

XML text escaping was triplicated across the kimaki bridge and both launchd services — still identical, waiting to do the same thing to a plist.

This is the shape #601 predicted. Short helpers feel harmless to re-type, so they get copied instead of shared, and then one copy gets a fix the others never see.

What changed

json_escape and xml_escape now live in lib/common.sh, one implementation each, with six local copies deleted and every call site rewritten. Escaping is precisely the code that must exist once: short enough to feel trivial, consequential enough that a divergence is a correctness bug rather than a style difference.

xml_escape deliberately handles & < > and not quotes — text nodes need those three, and &quot; inside a plist <string> is six literal characters, not a quote. That reasoning is now written down once instead of implied three times.

setup_ai_gateway and upgrade_ai_gateway had byte-identical bodies. Two names for one behaviour reads as "these differ somehow" to anyone who hasn't diffed them. Now one ai_gateway_apply, called from both.

The check

tests/duplicate-mechanism.sh, wired into CI, comparing normalised bodies (comments and blank lines stripped, lines trimmed) with a four-line floor so trivial accessors stay out of it.

Exact matches only. Near-duplicates are a judgement call, and a checker that makes judgement calls gets argued with rather than fixed. ALLOWED exists for deliberate pairs and is empty.

compared 576 production function bodies (>=4 lines) across 51 files

no duplicated mechanisms

Verification

tests/escaping-helpers.sh — 14 assertions covering both helpers, including JSON round-trips through python3 -c json.loads, the backslash-before-quote ordering that every JSON escaper gets wrong eventually, and the previously-divergent consumer now emitting parseable JSON.

The test earns its place: swapping the pre-consolidation escaper back in makes exactly the right three fail.

--- with the pre-consolidation divergent escaper:
FAIL escapes a newline
FAIL a newline survives a JSON round-trip
FAIL agent-state path list parses as JSON
11 passed, 3 failed

--- with the shared helper:
14 passed, 0 failed

Full suite on Linux: 73 pass, 4 fail — the same 4 that fail on a pristine main clone on the same host, for environmental reasons (/tmp world-writable, wp present so it resolves absolute). datamachine-worker is one of them and I edited that file, so I diffed its failure against main's: byte-identical, and unrelated to escaping.

tests/ci-coverage.sh passes with all three new jobs wired. bash -n clean across every .sh. Both hygiene checks from #603 still report clean.


AI assistance: audited and implemented by Claude Opus 4.5 via Claude Code, driven by Chris. The model wrote a normalised-body similarity audit to choose the target, found the JSON escaper drift, consolidated the copies, and verified the new test fails against the old implementation before trusting it.

The sibling finding to #603. That audit asked which mechanisms have no
consumer; this one asks which have two bodies.

Escaping was written three times. Two JSON escapers handled backslash, quote
and newline; lib/agent-state-ownership.sh's handled only backslash and quote,
so a value containing a newline emitted a literal newline inside a JSON string
literal -- invalid JSON, in the one copy whose output is machine-read. Nothing
exercised the newline case, which is why the drift survived. XML text escaping
was triplicated across the kimaki bridge and both launchd services, still
identical, waiting to do the same thing.

json_escape and xml_escape now live in lib/common.sh with one implementation
each. Escaping is exactly the code that must exist once: short enough to feel
harmless to re-type, consequential enough that a divergence is a correctness
bug rather than a style difference.

setup_ai_gateway and upgrade_ai_gateway had identical bodies. Two names for one
behaviour reads as 'these differ somehow' to anyone who has not diffed them.
One ai_gateway_apply, called from both.

tests/duplicate-mechanism.sh makes the rule enforceable, comparing normalised
bodies and reporting only exact matches -- near-duplicates are a judgement call,
and a checker that makes judgement calls gets argued with rather than fixed.

tests/escaping-helpers.sh covers the behaviour, including the newline case that
was missing. Verified to fail against the pre-consolidation escaper and pass
against the shared one.

Refs #601
@chubes4
chubes4 merged commit 5788948 into main Sep 17, 2026
74 checks passed
@chubes4
chubes4 deleted the chore/duplicate-audit branch September 17, 2026 03:02
chubes4 added a commit that referenced this pull request Sep 17, 2026
…605)

Block-level duplication, which the function-level audits in #603 and #604 could
not see: the plist document frame was copy-pasted into six renderers across
three bridges and two services.

The copies had already disagreed. Both services ran their values through
xml_escape; all three bridges interpolated them raw. A SITE_PATH, service home,
log directory, model name, or bot token containing an ampersand rendered a plist
that is not well-formed XML, which launchd refuses to load -- so the agent never
starts, with no failing test anywhere to say why. Verified against main: a path
with '&' in it produces a ParseError at the WorkingDirectory line.

The snapshot fixtures could not catch this because they use tidy values like
/var/www/site. A golden file locks in whatever was rendered the day it was
written, including a bug.

Only the frame is shared. The bodies genuinely differ -- the worker schedules
with StartInterval where the bridges use KeepAlive, and the WordPress service
renders no EnvironmentVariables -- so they stay as readable heredocs. A single
renderer taking seven parameters and three optional blocks to absorb that
variation would be harder to read than the duplication it removed.

All 8 committed snapshots remain byte-identical, so this changes nothing for
values that were already safe.

tests/plist-rendering.sh renders every plist with a value carrying & < > and
asserts the result parses as XML and still contains the value. It fails against
main and passes here.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant