Skip to content

chore(deps): bump the harness bench tools, keep postgres on Zero's major - #64

Open
cjber wants to merge 1 commit into
mainfrom
cb/pr57-postgres-v4
Open

cjber wants to merge 1 commit into
mainfrom
cb/pr57-postgres-v4

Conversation

@cjber

@cjber cjber commented Nov 16, 2026 •

Copy link
Copy Markdown

Summary

Takes the bench tool bumps from #57 (@codspeed/tinybench-plugin 6.0.1 to 6.1.0, tinybench 6.2.2 to 6.2.3) and leaves postgres on 3.4.9 instead of 4.0.0.

@rocicorp/zero 1.11.4 depends on postgres 3.4.7. The harness hands its own postgres.js client to Zero's adapter (zeroPostgresJS(schema, postgres(url))), so with 4.0.0 the oracle ran on a client Zero does not ship with. v4 lets the server type every parameter, and eight wire tests diverged on #57 (test_pushes_match_upstream and the matching test_sql.py cases): an array written to a text column came back as '{{"false' from the oracle instead of 'false'. The CodSpeed regression on query.simple.b1 (-18.27%, flagged as different hardware) came from the same run.

  • tests/test_pin.py: new check that each harness's postgres is on the major @rocicorp/zero depends on.
  • .github/dependabot.yml: skip postgres major bumps, which move with Zero, like @rocicorp/zero itself.

Evidence

Supersedes #57.

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

Bumps @codspeed/tinybench-plugin 6.0.1 to 6.1.0 and tinybench 6.2.2 to
6.2.3 from dependabot PR #57, without its postgres 3.4.9 to 4.0.0 bump.

@rocicorp/zero 1.11.4 depends on postgres 3.4.7. The harness hands its
own postgres.js client to Zero's adapter, so with 4.0.0 the oracle ran
a client Zero does not ship with: v4 types parameters differently, and
eight wire tests in test_mutate.py and test_sql.py diverged (an array
written to a text column came back as '{{"false' instead of 'false').

tests/test_pin.py now checks that each harness's postgres is on the
major @rocicorp/zero depends on, and dependabot skips postgres major
bumps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T14:30:15.202024Z c3f2683 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatednpm/​tinybench@​6.2.2 ⏵ 6.2.310010010094100
Updatednpm/​@​codspeed/​tinybench-plugin@​6.0.1 ⏵ 6.1.0951009896 +1100

View full report

@codspeed

codspeed Bot commented Nov 16, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 13.14%

2 warnings

⚠️ 19 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 19 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

⚡ 19 improved benchmarks
✅ 27 untouched benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
⚡ test_cpu[mutate_cpu.put.n100] 23.4 ms 19.9 ms +17.75%
⚡ test_cpu[query_json.simple.b100] 16.2 ms 14 ms +15.47%
⚡ test_cpu[z2s.complex] 1.3 ms 1.1 ms +15.03%
⚡ test_cpu[query.complex.b100] 49.6 ms 43.2 ms +14.96%
⚡ test_cpu[mutate_cpu.crud.n100] 55.6 ms 48.3 ms +14.94%
⚡ test_cpu[map.simple] 108.3 µs 94.3 µs +14.79%
⚡ test_cpu[query_json.complex.b100] 58.3 ms 50.8 ms +14.73%
⚡ test_cpu[z2s.related] 609.2 µs 531.9 µs +14.53%
⚡ test_cpu[query.simple.b100] 11.1 ms 9.7 ms +14.31%
⚡ test_cpu[parse.transform.q1000] 78.2 ms 69.3 ms +12.82%
⚡ test_cpu[build.simple] 94.3 µs 84.1 µs +12.07%
⚡ test_cpu[map.complex] 272.2 µs 243.1 µs +11.96%
⚡ test_cpu[mutate_cpu.crud.n1] 1,053.6 µs 943.7 µs +11.64%
⚡ test_cpu[build.complex] 510.8 µs 458.9 µs +11.31%
⚡ test_cpu[query.simple.b1] 345.9 µs 311.2 µs +11.15%
⚡ test_cpu[query_json.simple.b1] 438 µs 394.9 µs +10.9%
⚡ test_cpu[query_json.complex.b1] 876.1 µs 791.1 µs +10.74%
⚡ test_cpu[parse_json.transform.q1000] 55.3 ms 50 ms +10.49%
⚡ test_cpu[parse_json.push.m1000] 44.3 ms 40.2 ms +10.37%

Tip

Curious why performance improved? Comment @codspeedbot explain why performance improved on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cb/pr57-postgres-v4 (c3f2683) with main (1e7f6a6)

Open in CodSpeed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant