Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
122 changes: 122 additions & 0 deletions .openspec/adr/0036-pragma-synchronous-fsync-policy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
# 0036: `PRAGMA synchronous` fsync-skip policy, and why `SynchronousMode` lives in `header.rs`

Date: 2026-08-29

## Context

#645: `PRAGMA synchronous` was silently ignored — every commit
unconditionally did the fsyncs that only match stock SQLite's `FULL`
(the default), with no way to opt into `NORMAL`/`OFF`. Implementing it
for real means deciding, precisely, which of `Pager`'s three
commit-time fsyncs each level skips:

- The rollback-journal fsync (`flush_locked`, before writing any dirty
page into the main file) — the safety ordering `recover_hot_journal`
depends on to replay a hot journal after a crash mid-write.
- The rollback main-file fsync (`flush_locked`, after writing every
dirty page) — the second half of the two-fsync rollback-journal
commit protocol.
- The WAL frame fsync (`flush_wal_locked`, before publishing the new
`mxFrame`) — WAL mode's own single commit-time fsync.

Stock SQLite's documentation gives a clear, if slightly asymmetric,
answer: `NORMAL` still fsyncs "at the most critical moments" in
rollback-journal mode, but in WAL mode it only syncs at checkpoint
boundaries, not on every commit. `OFF` never fsyncs at all. Chasing
every historical nuance of `getSafetyLevel`'s masking of arbitrary
out-of-range integer values, `EXTRA`, and the `ON` boolean alias was
explicitly out of scope (#645's acceptance criteria only names
`FULL`/`NORMAL`/`OFF`).

A second, unrelated question came up during implementation: `vdbe/`
may never `use crate::pager` directly (spec 001-architecture
Requirement 1, "VDBE does not know file format", enforced by
`tests/unit/layer_isolation.rs`). `synchronous`'s state is exactly the
kind of thing `vdbe/pragma.rs` needs to name directly (to convert
between the wire-level `i32` opcode operand and a real enum, and to
report the current value back as a query result) — the same situation
`JournalMode` was already in, and already solved by living in
`src/header.rs` rather than `src/pager.rs`.

## Decision

**Fsync policy** (`Pager::flush_locked`/`flush_wal_locked`):

| Level | Journal fsync (rollback) | Main-file fsync (rollback) | WAL frame fsync |
|----------|:---:|:---:|:---:|
| `Full` | yes | yes | yes |
| `Normal` | yes | no | no |
| `Off` | no | no | no |

`Normal` keeps the rollback-journal fsync because it's what makes
`recover_hot_journal` safe at all — without it, a crash between the
journal write and the main-file write could leave a journal on disk
whose own bytes never made it to a stable state, defeating the
recovery it exists to enable. It drops the main-file fsync (the thing
`FULL` adds on top) and the WAL frame fsync, matching stock SQLite's
documented "still consistent, less durable" `NORMAL` semantics for
both journal modes.

Two other fsync call sites are deliberately left ungated:
`Pager::set_journal_mode`'s own page-1 write-back (a mode switch, not
part of any user transaction) and `checkpoint::checkpoint_passive`'s
post-backfill fsync (checkpoints sync the main file under `NORMAL`
too, per stock SQLite's docs — gating it would need threading
`synchronous` through a free function that doesn't otherwise touch
`Pager` state, for a case `#645`'s acceptance criteria doesn't ask
for).

**Where `SynchronousMode` lives**: `src/header.rs`, next to
`JournalMode`, even though — unlike `JournalMode` — it has no on-disk
representation at all (stock SQLite never persists `synchronous`; a
fresh connection always starts at `Full`). `header.rs` is the
established "vdbe-safe vocabulary" module for exactly this situation:
an enum `vdbe/pragma.rs` must reference by name (to build/read it) but
that `src/pager.rs` itself would otherwise be the natural home for.

## Alternatives rejected

- **Cache the fsync policy as three precomputed booleans** (`sync_journal`,
`sync_main_file`, `sync_wal_frame`) instead of a `SynchronousMode`
enum matched at each call site. Rejected: the enum is the thing
`PRAGMA synchronous` (query form) needs to report back verbatim
(`0`/`1`/`2`), and three booleans would just be a less legible
re-encoding of the same three-level table above, computed twice.
- **Also gate `checkpoint_passive`'s fsync and the mode-switch page-1
write-back** on `synchronous`. Deferred: neither is a per-commit hot
path, `#645`'s acceptance criteria only asks about "the correct
fsync call pattern" for commits, and stock SQLite's own checkpoint
fsync isn't skipped by `NORMAL` anyway — only `OFF` would change
anything there, a narrower case not worth the extra plumbing yet.
- **A general per-connection settings struct** (rather than fields
directly on `Pager`, mirroring `journal_mode`) to hold `synchronous`
and future PRAGMA-set state. Rejected as premature: there is
currently exactly one other stateful PRAGMA (`journal_mode`), it
already lives as a bare `Pager` field, and introducing a wrapper
struct for two fields is speculative until a third stateful PRAGMA
actually arrives.
- **`Pager::set_synchronous` re-checking for a pending transaction**
(mirroring `set_journal_mode`'s `PendingTransaction` guard). Rejected:
stock SQLite explicitly allows changing `synchronous` mid-transaction
(it only affects fsync behavior at the *next* commit, unlike
`journal_mode`, which needs a clean transaction boundary to safely
rewrite page 1's version bytes and swap journal implementations).

## Consequences

- `Pager::synchronous`/`Pager::set_synchronous` are the only two new
public `Pager` methods; no existing call site changes shape.
- `vdbe/pragma.rs::synchronous` is the first pragma executor to both
set state (mirroring `set_journal_mode`) and emit a result row
(mirroring `integrity_check`) from the same opcode, keyed off a
sentinel `P1` value (`SYNCHRONOUS_QUERY = -1`) rather than a second
opcode — kept as one opcode since the two forms share the same
"resolve the writer, if any" prologue.
- A future PRAGMA that also needs bidirectional get/set (unlike
`journal_mode`'s still-write-only shape) has a second precedent to
follow beyond `integrity_check`'s read-only one.
- `src/vfs/memory.rs`'s `MemoryVfs` gained a `sync_calls()` counter
(shared across clones and every file handle opened from it) purely
so `src/pager.rs`'s new fsync-gating tests can assert *whether* a
commit fsynced — an in-memory backend has no other way to observe
that.
1 change: 1 addition & 0 deletions .openspec/adr/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,4 @@ Specs record what the system must do; ADRs record **why it is shaped this way**
| [0033](0033-constant-propagation-and-or-to-in-extend-fast-paths-in-place.md) | Constant propagation and OR-to-IN extend existing equality fast paths in place; only genuine range seeks wait for a new opcode | 2026-08-28 |
| [0034](0034-index-range-seeks.md) | Real-index range seeks for `BETWEEN`/`IN`/`LIKE`-prefix (`SeekIndexGE`/`IdxCompareGT`) | 2026-08-28 |
| [0035](0035-wal-resume-hint-cache-supersedes-0026.md) | `Pager`-cached WAL resume hint supersedes ADR-0026's per-flush rescan | 2026-08-29 |
| [0036](0036-pragma-synchronous-fsync-policy.md) | `PRAGMA synchronous` fsync-skip policy, and why `SynchronousMode` lives in `header.rs` | 2026-08-29 |
30 changes: 20 additions & 10 deletions .openspec/grammar/sqlite.ebnf
Original file line number Diff line number Diff line change
Expand Up @@ -345,26 +345,36 @@ drop-view-stmt ::= "DROP" "VIEW" [ "IF" "EXISTS" ] view-name ;
view-name ::= identifier ; (* V6 *)

(* ===================== PRAGMA journal_mode / integrity_check / quick_check
* (V6/V7 carve-outs) =====================
* / synchronous (V6/V7 carve-outs) =====================
* [parse.y:1715 cmd ::= PRAGMA nm dbnm EQ nmnum, :1717 cmd ::= PRAGMA nm
* dbnm] -- `nmnum` (parse.y:1723) is where `DELETE` is folded in as a
* bare keyword value alongside a plain `nm` (identifier) value like
* `WAL`, which is why the `journal_mode` value alternation below needs
* its own two literal cases rather than one `identifier` production.
* dbnm] -- `nmnum` (parse.y:1723) is where `DELETE`/`FULL` are folded in
* as bare keyword values alongside a plain `nm` (identifier) value like
* `WAL`/`OFF`/`NORMAL`, which is why the `journal_mode`/`synchronous`
* value alternations below need their own literal cases for those two
* rather than one `identifier` production.
* `integrity_check`/`quick_check` (#540, #541) are the bare
* `PRAGMA nm dbnm` form (no `EQ`) with no result-set arguments --
* stock SQLite's `PRAGMA integrity_check(N)` (bounding the number of
* errors reported) and any `dbnm` (schema-qualified) prefix are out of
* this carve-out's scope. General PRAGMA support -- every other pragma
* name, and journal_mode's other stock-SQLite values (MEMORY/OFF/
* TRUNCATE/PERSIST) -- stays deferred to V7 (see Future blocks below);
* #388/#540/#541 carve out only these cases.
* this carve-out's scope. `synchronous` (#645) is the first pragma here
* that's genuinely bidirectional: `PRAGMA synchronous` (no `EQ`, same
* :1717 production as `integrity_check`) queries the connection's
* current level instead of changing it -- `journal_mode`'s own bare
* query form stays unimplemented (`Unsupported`). General PRAGMA
* support -- every other pragma name, journal_mode's other
* stock-SQLite values (MEMORY/OFF/TRUNCATE/PERSIST), and synchronous's
* `EXTRA`/`ON`/boolean aliases and out-of-0-2-range integers (with
* their own legacy masking quirks) -- stays deferred to V7 (see Future
* blocks below); #388/#540/#541/#645 carve out only these cases.
*)

pragma-stmt ::= "PRAGMA" "journal_mode" "=" ( "WAL" | "DELETE" )
| "PRAGMA" ( "integrity_check" | "quick_check" ) ;
| "PRAGMA" ( "integrity_check" | "quick_check" )
| "PRAGMA" "synchronous" [ "=" ( "OFF" | "NORMAL" | "FULL"
| "0" | "1" | "2" ) ] ;
(* V6 [parse.y:1715 cmd] -- #388 *)
(* V7 [parse.y:1717 cmd] -- #540, #541 *)
(* V7 [parse.y:1715,1717 cmd] -- #645 *)

(* ===================== ANALYZE (V7 carve-out) =====================
* [parse.y:1875 cmd ::= ANALYZE, :1876 cmd ::= ANALYZE nm dbnm] --
Expand Down
4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,10 @@ path = "tests/unit/transaction_parser.rs"
name = "unit_pragma_parser"
path = "tests/unit/pragma_parser.rs"

[[test]]
name = "unit_pragma_synchronous_repl"
path = "tests/unit/pragma_synchronous_repl.rs"

[[test]]
name = "unit_introspection_pragmas"
path = "tests/unit/introspection_pragmas.rs"
Expand Down
19 changes: 18 additions & 1 deletion src/bin/sqlite-rs/repl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,24 @@ fn run_one_statement(
}
};
match execute_transaction_step(&program, Rc::clone(pager), header, state.autocommit) {
Ok((_, ac)) => state.autocommit = ac,
Ok((rows, ac)) => {
state.autocommit = ac;
// #645: a non-`SELECT` statement can still emit result rows
// (e.g. `PRAGMA synchronous`'s bare query form, or
// `PRAGMA integrity_check`) — print them the same way the
// `SELECT` branch above does, rather than silently dropping
// them as this branch did before. No column names to derive
// here (this path has no `Select` AST to read them from),
// so `.headers on` renders a blank header line for these.
if !rows.is_empty() {
let mut stdout = io::BufWriter::new(io::stdout().lock());
if let Err(e) = print_rows(&mut stdout, state.mode, state.headers, &[], &rows) {
eprintln!("Error: {e}");
return;
}
stdout.flush().ok();
}
}
Err(e) => eprintln!("Error: {e}"),
}
}
Expand Down
72 changes: 65 additions & 7 deletions src/codegen/pragma.rs
Original file line number Diff line number Diff line change
@@ -1,20 +1,25 @@
// Copyright 2026 Schuberg Philis
// SPDX-License-Identifier: Apache-2.0
//! `Pragma` AST -> `Program` compilation: `journal_mode` (#388) and
//! `integrity_check`/`quick_check` (#540, #541). Mirrors
//! `src/codegen/transaction.rs`'s shape: one control opcode per pragma,
//! operands carrying whatever the executor needs.
//! `Pragma` AST -> `Program` compilation: `journal_mode` (#388),
//! `integrity_check`/`quick_check` (#540, #541), and `synchronous`
//! (#645). Mirrors `src/codegen/transaction.rs`'s shape: one control
//! opcode per pragma, operands carrying whatever the executor needs.

use crate::codegen::Emitter;
use crate::parser::ast::{Pragma, PragmaJournalMode};
use crate::vdbe::{Instruction, Opcode, Program, JOURNAL_MODE_DELETE, JOURNAL_MODE_WAL};
use crate::parser::ast::{Pragma, PragmaJournalMode, PragmaSynchronous};
use crate::vdbe::{
Instruction, Opcode, Program, JOURNAL_MODE_DELETE, JOURNAL_MODE_WAL, SYNCHRONOUS_FULL,
SYNCHRONOUS_NORMAL, SYNCHRONOUS_OFF, SYNCHRONOUS_QUERY,
};

/// Compiles a `PRAGMA` statement into an `Init -> <op> -> Halt` program.
/// `journal_mode` emits `SetJournalMode` (`P1` carries the target mode,
/// no result rows); `integrity_check`/`quick_check` emit
/// `IntegrityCheck` (`P1` = 1 for the `quick_check` reduced pass, 0 for
/// the full `integrity_check`), which produces a result set of `TEXT`
/// rows.
/// rows; `synchronous` emits `Synchronous` (`P1` carries the target
/// level, or `SYNCHRONOUS_QUERY` for the bare query form, which
/// produces a single `INTEGER` result row instead of a side effect).
pub fn compile_pragma(pragma: &Pragma) -> Program {
let mut em = Emitter::new();
let init_addr = em.emit(Instruction::new(Opcode::Init, 0, 0, 0));
Expand All @@ -38,6 +43,15 @@ pub fn compile_pragma(pragma: &Pragma) -> Program {
0,
));
}
Pragma::Synchronous { level, .. } => {
let p1 = match level {
None => SYNCHRONOUS_QUERY,
Some(PragmaSynchronous::Off) => SYNCHRONOUS_OFF,
Some(PragmaSynchronous::Normal) => SYNCHRONOUS_NORMAL,
Some(PragmaSynchronous::Full) => SYNCHRONOUS_FULL,
};
em.emit(Instruction::new(Opcode::Synchronous, p1, 0, 0));
}
}
em.emit(Instruction::new(Opcode::Halt, 0, 0, 0));
em.finish()
Expand Down Expand Up @@ -108,4 +122,48 @@ mod tests {
);
assert_eq!(program.instructions[1].p1, 1);
}

#[test]
fn synchronous_query_compiles_p1_sentinel() {
let pragma = match parse_pragma("PRAGMA synchronous") {
ParseOutcome::Accepted(p) => p,
other => panic!("expected Accepted, got {other:?}"),
};
let program = compile_pragma(&pragma);
assert_eq!(
opcodes(&program),
vec![Opcode::Init, Opcode::Synchronous, Opcode::Halt]
);
assert_eq!(program.instructions[1].p1, SYNCHRONOUS_QUERY);
}

#[test]
fn synchronous_off_compiles_p1_zero() {
let pragma = match parse_pragma("PRAGMA synchronous = OFF") {
ParseOutcome::Accepted(p) => p,
other => panic!("expected Accepted, got {other:?}"),
};
let program = compile_pragma(&pragma);
assert_eq!(program.instructions[1].p1, SYNCHRONOUS_OFF);
}

#[test]
fn synchronous_normal_compiles_p1_one() {
let pragma = match parse_pragma("PRAGMA synchronous = NORMAL") {
ParseOutcome::Accepted(p) => p,
other => panic!("expected Accepted, got {other:?}"),
};
let program = compile_pragma(&pragma);
assert_eq!(program.instructions[1].p1, SYNCHRONOUS_NORMAL);
}

#[test]
fn synchronous_full_via_integer_compiles_p1_two() {
let pragma = match parse_pragma("PRAGMA synchronous = 2") {
ParseOutcome::Accepted(p) => p,
other => panic!("expected Accepted, got {other:?}"),
};
let program = compile_pragma(&pragma);
assert_eq!(program.instructions[1].p1, SYNCHRONOUS_FULL);
}
}
24 changes: 24 additions & 0 deletions src/header.rs
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,30 @@ pub enum JournalMode {
Wal,
}

/// `PRAGMA synchronous` (#645): how aggressively `Pager::flush` fsyncs
/// on commit. Unlike [`JournalMode`], this is never read from or
/// written to the header bytes — stock SQLite keeps it purely as
/// per-connection state, defaulting to `Full` on every fresh
/// connection, and so does `Pager`. Lives here (rather than
/// `src/pager.rs`, its otherwise-natural home) only so `vdbe/pragma.rs`
/// can name it without importing `crate::pager` directly, which spec
/// 001-architecture Requirement 1 ("VDBE does not know file format")
/// forbids — see ADR-0036.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub enum SynchronousMode {
/// No fsyncs at all on commit — fastest, least crash-safe.
Off = 0,
/// Skips the fsync(s) stock `FULL` performs that aren't needed for
/// basic crash *consistency* (as opposed to guaranteeing durability
/// of the most recent commits after a power loss).
Normal = 1,
/// fsyncs at every point needed to guarantee a commit survives a
/// crash or power loss — this pager's behavior before #645, and
/// still the default.
#[default]
Full = 2,
}

/// The parsed 100-byte SQLite database header.
///
/// See the page-1 trap note in the module doc: this struct describes only
Expand Down
Loading
Loading