Repository navigation
Replace top-level metadata keys on SQLite merges - #1533
Merged
Merged
Conversation
bgentry
force-pushed
the
sqlite-shallow-metadata-merge
branch
2 times, most recently
from
October 24, 2026 21:33
796bf38 to
73d32f0
Compare
brandur
approved these changes
Oct 24, 2026
brandur
left a comment
Contributor
There was a problem hiding this comment.
Ah wow, okay that seems like a bad one. Nice find!
Metadata updates replace whole top-level values on Postgres. SQLite's JSON Merge Patch instead merges nested objects and deletes null values, leaving stale resumable cursors that can rerun completed steps. Restore top-level replacement in every affected port and preserve Java's transactional checkpoint fallback. Keep JSON nulls and numeric value tokens intact. Use fixed SQL so large updates avoid expression-depth limits. Share one SQL template across Go's updates and both completion paths. Apply supplied values with `jsonb_set` without reconstructing existing metadata or decoding its keys. Keep the template cache enabled and materialize the update list once for Turso. Preserve malformed stored metadata and Postgres behavior for non-object stored values. Cover cursor retries, large updates, literal keys, empty objects, nulls, and numeric precision in shared driver and port regression coverage.
Keep shared features and fixes in one pull request across the Go reference implementation and every affected port. Require equivalent coverage and documentation of intentional behavior differences.
bgentry
force-pushed
the
sqlite-shallow-metadata-merge
branch
from
October 24, 2026 21:42
73d32f0 to
398a149
Compare
bgentry
marked this pull request as ready for review
October 24, 2026 21:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SQLite metadata updates currently use JSON Merge Patch, which recursively merges nested objects and deletes keys set to JSON null. River's intended behavior is to preserve unrelated top-level keys and replace each supplied value completely, as Postgres's
metadata || updatesdoes. This difference can leave a stale resumable cursor on SQLite and cause a completed step to run again on a later retry.All resumable cursors live inside the same top-level key,
river:resumable_cursor. Each cell below shows the full metadata object: Postgres replaces that outer key's entire value, while the existing SQLite merge retains nested entries or interprets null as deletion.{"river:resumable_cursor":{"c":1}}{"river:resumable_cursor":{"d":7}}{"river:resumable_cursor":{"d":7}}{"river:resumable_cursor":{"c":1,"d":7}}{"river:resumable_cursor":{"c":1}}{"river:resumable_cursor":{"d":null}}{"river:resumable_cursor":{"d":null}}{"river:resumable_cursor":{"c":1}}{"river:resumable_cursor":{"c":1}}{"river:resumable_cursor":null}{"river:resumable_cursor":null}{}For example,
csaves a cursor and fails on attempt 1. On attempt 2,ccompletes, so the worker removes its cursor from the map; thendsaves a cursor and fails. The update replaces the cursor map with{"d":7}. A recursive merge retains the oldcentry, makingclook partly done on attempt 3 and causing it to run again.Go uses one shared
metadata_merge.sqlexpression to apply each supplied top-level value withjsonb_set. The update list is materialized once, avoiding expression-depth limits and repeated work on Turso. The three generated queries insert this fixed expression through the existing template mechanism because sqlc cannot resolve its CTEs. Fixed SQL and parameter names keep the template replacer's cache enabled.Java, JavaScript, Rust, and Ruby use fixed SQLite queries that rebuild metadata from retained keys and replacement values. Values are read with
->to preserve JSON types and numeric precision.The change covers Go, Java, JavaScript, Rust, and Java's conformance helper. Ruby already replaces top-level values; its SQLite merge now also handles large updates without nesting one setter per key. Java's transactional checkpoint fallback keeps its precedence over stored metadata, with attempt updates taking precedence over both. Go keeps malformed stored metadata intact and uses top-level JSON paths without reconstructing existing key names.
Regression coverage includes completed cursor steps, running and stale completions, large updates, literal key names, empty objects, nulls, booleans, and numeric precision. Changelog entries cover every affected language. A separate commit records the cross-language parity expectations in
AGENTS.md.