Repository navigation
Add opt-in BreadthFirst traversal mode - #3
Merged
Merged
Conversation
A multi-hop step deduplicates on (node, depth), so a node reachable at several depths is expanded at each. BreadthFirst compiles the step to one CTE per hop level holding nodes first reached at that level, so each node is expanded once. Results are identical. Each level costs a fixed ~20us, so which mode wins depends on how much the walk revisits nodes, which the query cannot predict. It is opt-in: 10 hops on a dense 500-node graph drops from 9.5ms to 2.0ms, and on a 100-node chain rises from 40us to 243us. It requires minHops of 1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C
justintout
added a commit
that referenced
this pull request
Sep 14, 2026
…er injection (#2) * Add large-graph traversal and file-backed write benchmarks Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Rework traversal indexes and query compilation Every traversal step used idx_edges_type, scanning all edges of a type per expanded node, and the final SELECT scanned the nodes table. On a 10k-node graph a 6-hop query took 2.5s; it now takes 2.5ms. Migration 2 replaces the edge indexes with covering (source_id, type, target_id) and (target_id, type, source_id) indexes, rebuilds node_labels as WITHOUT ROWID, and indexes nodes.name. The compiler emits one CTE of node IDs per step, joins nodes once for the final set, forces join order with CROSS JOIN, and splits Both into two indexed SELECTs. LIMIT/OFFSET are bound, and emitted only when set, so pagination does not grow the per-connection statement cache. Fixes WhereRel on intermediate multi-hop steps being ignored and Offset without Limit producing invalid SQL. Count now ignores Limit/Offset for traversals, matching simple matches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Use RETURNING for node and edge inserts and updates Drops the follow-up SELECT that read back IDs and timestamps. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Open connections in WAL mode File-backed autocommit CreateNode drops from ~380us to ~76us, and readers no longer block on the writer. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Update README for new query compilation and benchmarks Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Implement Return projection and close filter SQL injection Return was accepted but ignored. It now selects only the named columns and extracts property paths in SQL, skipping the full JSON decode. On a 6-hop traversal reaching ~700 nodes with 10-key properties, returning only name drops from 6.3ms to 1.8ms and 63k to 2k allocations. Where and WhereRel wrote the field name into SQL unchecked; they now accept only node columns. JSON paths in WhereJSON, WhereRelJSON, and Return are bound parameters instead of interpolated literals, which measures the same per execution. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Add comparison benchmarks against GraphQLite A separate module loads the same generated graphs into go-sqlite-graph, GraphQLite (Cypher via its SQLite extension), and go-sqlite-graph's own SQL on C SQLite, and fails if any engine disagrees on a result size. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Document Return, filter columns, and comparison benchmarks Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Remove RETURNING and in-transaction savepoints from writes Both forced SQLite to write the statement sub-journal to a temp file on large transactions: RETURNING buffers its rows in a temp table, and a savepoint journals every page it touches. Loading 100k nodes and 500k edges in one Tx drops from 18.2s to 5.3s. Timestamps are now generated in Go and bound, in the schema default's format. Savepoints are still used for multi-statement writes outside a transaction; inside a Tx, a failed method may leave partial writes, and the caller rolls back. CreateNode -32%, CreateEdge -35%, BulkInsertTx/batch=1000 -62%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Build query SQL without fmt and test traversals against a reference fmt allocated for each formatted value while compiling. Writing strings directly makes simple matches ~12% faster and single-hop traversals 10-18% faster; heavy traversals are unchanged. The new test compares every traversal compilation path, in all three directions, against walks computed in Go on a random graph with cycles. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Update README benchmarks for write and compile changes Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C * Add BreadthFirst traversal mode (#3) A multi-hop step deduplicates on (node, depth), so a node reachable at several depths is expanded at each. BreadthFirst compiles the step to one CTE per hop level holding nodes first reached at that level, so each node is expanded once. Results are identical. Each level costs a fixed ~20us, so which mode wins depends on how much the walk revisits nodes, which the query cannot predict. It is opt-in: 10 hops on a dense 500-node graph drops from 9.5ms to 2.0ms, and on a 100-node chain rises from 40us to 243us. It requires minHops of 1. Claude-Session: https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C Co-authored-by: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #2.
Summary
Adds
Query.BreadthFirst(), which makes the most recentRelatedstep expand each reached node once. Results are identical to the default mode.Why opt-in
By default, a multi-hop step is a recursive CTE that deduplicates on (node, depth). A node reachable at several depths gets expanded once per depth.
BreadthFirstcompiles the step to one CTE per hop level, holding only nodes first reached at that level.Each level costs a fixed ~20µs for SQLite's temporary tables. Which mode wins depends on how often the walk revisits nodes. That's a property of the data near the start node, and the query shape can't predict it. Choosing automatically would mean running a probe query on every deep traversal.
Medians of 6 runs, Apple M3 Max:
Details
minHops: requiresminHops == 1; otherwiseRunreturns an error. A largerminHopsneeds nodes reachable at an exact depth, and a visited set discards those.maxHops == 1: compiles as a normal single hop.Test plan
go vet ./...andgo test ./...pass.TestTraversalMatchesReferenceruns everyminHops == 1case in both modes, for all directions and 1–6 hops, against walks computed in Go on a random graph with cycles and self-loops.TestWhereRelIntermediateStepcoversBreadthFirstfollowed byWhereReland another step.BreadthFirstwithminHopsof 2 returns an error.🤖 Generated with Claude Code
https://claude.ai/code/session_01YB9owLYxU6NsNuVk2NVc7C