fix(jq): make join linear in the number of elements

[chaliy]

What changed

jq's join is linear in the number of elements again. It was quadratic: every
element copied the whole accumulated result, so join over 300k elements cost
68 s in an unoptimized build and now costs 14 s, scaling linearly.

Observable behaviour is unchanged. 14 cases captured from real jq 1.7 — including
the wording and operand order of both error messages — are pinned by a new
join_follows_jq test.

Why

Nightly run #247 (AddressSanitizer) failed on
memory_growth_security_tests::jq::ordinary_workloads_still_fit, which hit its
600 s budget. That test exists to catch exactly this, and it had been passing
only because 600 s is generous on an uninstrumented build: 68 s fits, the same
work under ASAN's instrumentation does not. The red nightly was a true
positive, not a slow-runner artifact.

The copy happens in Val::add, which reuses the left operand's buffer through
BytesMut::from(Bytes) — zero-copy only while that Bytes is unique. jq's own
fold shape, (if . == null then "" else . + $x end) + $elem, keeps the
accumulator reachable while the right operand is evaluated, so the inner add
sees a shared buffer and copies it.

Measured at 100k → 300k iterations, where linear is 3.0x:

filter body	100k	300k	ratio
. + "x"	0.28 s	0.83 s	2.9x
. + ("," + "x")	0.48 s	1.79 s	3.7x
. + "," + "x"	0.91 s	5.20 s	5.7x
. + "x" + "y"	1.02 s	7.49 s	7.4x

The rule that falls out: the accumulator appears once, as the left operand of
a single add, with the right operand fully parenthesized. join now coerces
the elements first and folds with . + ($x + $s), slicing off the separator the
fold prepends.

Before / After

jq -n '[range(N)] | map(tostring) | join(",") | length', unoptimized build:

N	before	after
50k	5.35 s	2.46 s
100k	11.79 s	4.59 s
300k	68.28 s	14.44 s

Ratio 100k → 300k: 5.3x before (quadratic) vs 3.15x after (linear, 3.0x
is perfect). Isolating the stages confirmed join was the only superlinear one —
[range(300000)] is 57 ms, map(tostring) 0.98 s, and reduce . + "x" 0.96 s
were all already linear.

Under the nightly ASAN configuration the failing test now finishes in 157 s
against its 600 s budget (3.8x headroom); see the measurement comment below.
The jq module of memory_growth_security_tests drops from 80 s to 31 s; the whole
bashkit integration binary runs in 22.5 s.

Differential check against the jq 1.7 binary — all 14 identical, before and
after:

[1, null, "a", true] | join("-")      =>  "1--a-true"
[null] | join(",")                    =>  ""
[] | join(",")                        =>  ""
{} | join(",")                        =>  ""
[false] | join(",")                   =>  "false"
["a"] | join(1)                       =>  "a"            # separator unused, no error
["a","b","c"] | join("")              =>  "abc"
["a","b"] | join("<>")                =>  "a<>b"
["é","ü"] | join("—")                 =>  "é—ü"
{"a":"x","b":"y"} | join("-")         =>  "x-y"
[[1]] | join(",")                     =>  string ("") and array ([1]) cannot be added
["a","b"] | join(1)                   =>  string ("a") and number (1) cannot be added
[1,2] | join(2)                       =>  string ("1") and number (2) cannot be added
"abc" | join(",")                     =>  Cannot iterate over string ("abc")

Risk

• Low
• join is defined in the compat prelude, so @csv, @tsv and @sh go through
  it too; all 343 jq unit tests, the jq spec cases and all 1488 integration tests
  pass. The fold now materializes the coerced elements as an array first, which
  charges roughly N * size_of(Val) extra against
  max_live_intermediate_bytes (~7 MB at 300k, against a 32 MB default) — a very
  large join can therefore reach the value-size cap marginally sooner, with the
  normal value size limit (N bytes) exceeded diagnostic.
• Validated: cargo fmt --check, cargo clippy --all-targets -D warnings,
  343 jq unit tests, 24 jq no_leak tests, 1488 integration tests,
  check_okf.py, plus the ASAN run of the previously failing test.

Checklist

• Tests added or updated
• Backward compatibility considered

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Preview URL	Updated (UTC)
✅ Deployment successful! View logs	bashkit	25409e5	Commit Preview URL Branch Preview URL	Oct 08 2026, 09:35 AM

[chaliy]

Reproduced the nightly job locally to confirm the headroom rather than infer it from the ~15x figure in knowledge/operations/testing.md.

Same configuration as the ASAN job — -Z build-std, RUSTFLAGS=-Z sanitizer=address, ASAN_OPTIONS=detect_stack_use_after_return=1:detect_leaks=0, --test-threads=1:

test memory_growth_security_tests::jq::ordinary_workloads_still_fit ... ok
test result: ok. 1 passed; 0 failed; ... finished in 156.87s

All 7 jq tests in that module pass under ASAN, 235.6 s for the module.

157 s against a 600 s budget — 3.8x headroom. That also pins the instrumentation multiplier for this workload at ~10.5x, which reconciles with nightly #247: the quadratic join cost 75 s natively, so ~790 s under ASAN, past the budget. The same multiplier says a re-introduced quadratic would still land near 790 s and still be caught, so the budget keeps discriminating in both directions — which is the property that matters, since raising it to fit a slow linear run would also let a quadratic one through.

	native (unoptimized)	under ASAN	vs 600 s budget
before (quadratic join)	75 s	~790 s (inferred)	over — nightly #247
after (linear join)	~15 s	157 s (measured)	3.8x under

---

Generated by Claude Code