Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 17 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,23 @@

## Unreleased

Future changes will be listed here.
### v0.5.0 (prepared; not yet published)

- Add normative optional `openstatspec-database-io-v1`, explicitly selected by
capability field `database_io_policy`. Reads, validation and export perform
no database writes, including audit, temporary DDL and failure paths; export
loss diagnostics are returned while safe file publication remains required.
- Permit default Dolt writes through packaged exact tested 2.2.2/2.2.3 policy
backed by adapter release CI, with no runtime user declaration/evidence files.
External declarations and their unchanged validator remain an optional
advanced override. Identity, limits, atomicity, security and branch/HEAD
checks remain mandatory; unknown or untested versions fail closed.
- Preserve unselected adapters' old requirements, SAV 1.0 source semantics,
catalog/data/metadata schema and fixtures, and all canonical plan, frontend
and SQL binding contracts. This pre-1.0 minor release adds an independently
selected policy; it does not silently revise an existing profile.
- Keep the separately released `openstatspec-specification` companion package
at `0.1.0`. No adapter conformance or package release is asserted here.

## v0.4.1 - 2026-08-21

Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,19 @@ MySQL 8.4.x/9.7.x, MariaDB 11.4.x/11.8.x/12.3.x and PostgreSQL 17.x/18.x
series while requiring exact patch-version CI evidence. Microsoft SQL Server
remains an explicitly unsupported [future dialect](docs/mssql-dialect-roadmap.md).

## Optional database I/O execution policy

Prepared for specification `v0.5.0`, the normative optional
[Database I/O Execution Policy v1](docs/database-io-policy-v1.md) is explicitly
selected with `database_io_policy: "openstatspec-database-io-v1"`.
It requires database-read-only reads, validation and exports (including errors
and loss diagnostics), and permits default Dolt writes on packaged exact tested
2.2.2/2.2.3 policy backed by adapter release CI, without user evidence files.
External declarations remain an optional advanced override. Unselected adapters
retain existing requirements; SAV 1.0 semantics, schemas, fixtures and canonical
Plan/Frontend/Binding contracts are unchanged. Publication and downstream
adapter support remain separately gated.

## Core contract

- One source dataset maps to exactly one dedicated SQL data table.
Expand Down
21 changes: 21 additions & 0 deletions RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,27 @@ Package tags use the `package-vX.Y.Z` convention; specification tags use
`vX.Y.Z`. A specification tag MUST NOT publish the package, and a package tag
MUST match the version declared in `pyproject.toml`.

## Prepared v0.5.0 scope

This release prepares the normative optional
[Database I/O Execution Policy v1](docs/database-io-policy-v1.md). It is not
published until the gates below complete. Release notes MUST identify explicit
`database_io_policy` selection, database-read-only export with returned loss
diagnostics, and packaged exact tested Dolt 2.2.2/2.2.3 write policy with optional
external evidence. Existing unselected-profile requirements, SAV 1.0 semantics
and fixtures, schemas, and canonical Plan/Frontend/Binding artifacts are unchanged.
This is a pre-1.0 minor optional addition under [VERSIONING](VERSIONING.md).

Run `python -m pytest` and `python tools/validate_repository.py` at the selected
commit. Adapter package release CI supplies real exact-version write and
read-only/failure-path evidence under the new policy; neither fabricated
repository declarations nor user-supplied evidence files are release substitutes.
Python `v0.8.0` and PHP `v0.7.0` are downstream adapter release targets, not
conformance claims made by this specification. Their pins and publication are
managed separately. The companion `openstatspec-specification` distribution
remains `0.1.0`: no package API, schema or validator contract change requires a
package release for this documentation policy addition.

## Before preparing a release

- Choose an exact reviewed commit on the appropriate branch and record its
Expand Down
5 changes: 5 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,11 @@ follows and does not block specification publication.
remain downstream and are not implied by this release.
8. [x] Published documentation patch release `v0.4.1` to align the frontend
roadmap with the stable Frontend 0.3 profile; no normative contract changed.
9. [x] Prepare the optional [Database I/O Execution Policy v1](docs/database-io-policy-v1.md)
and `v0.5.0` release notes without changing existing source or plan contracts.
10. [ ] Publish `v0.5.0` after exact-commit and tag-context CI and the release
checklist; downstream Python `v0.8.0` and PHP `v0.7.0` pins, policy evidence
and package releases remain separately managed.

The pending in-place service matrix and adapter conformance work in sections 6
and 7 are downstream implementation gates. They do not block the `v0.4.0`
Expand Down
9 changes: 9 additions & 0 deletions VERSIONING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,15 @@ OpenStatSpec uses semantic versioning for tagged specification releases.
- A minor release may add backward-compatible optional metadata, a new independent source or SQL profile, or new fixtures that do not invalidate an implementation's existing declared conformance scope.
- A patch release may clarify wording, correct examples, or fix fixture and schema defects without changing normative meaning. A schema-defect correction may be backported to a published release series only when it restores the already-stated normative contract; it must not introduce new required behavior, broaden the accepted contract, or silently revise canonical artifacts.

The optional [Database I/O Execution Policy v1](docs/database-io-policy-v1.md)
is an independent execution contract selected by `database_io_policy`, not a
revision of SAV 1.0 or any canonical plan, frontend, binding or catalog schema.
Its explicitly scoped overrides apply only when selected; otherwise existing
requirements remain unchanged. Adding it in pre-1.0 `v0.5.0` is a minor optional
addition, not a patch clarification. No blanket profile freeze or version bump
is required. Future incompatible changes to this policy require a new policy
identifier; they MUST NOT silently change v1 selection.

Optional workflow profiles are versioned and claimed independently from the
core source profile. Adding or revising an optional profile does not authorize
changes to core datasets. A breaking workflow-profile change increments that
Expand Down
146 changes: 146 additions & 0 deletions docs/database-io-policy-v1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Optional Database I/O Execution Policy v1

Status: normative optional addition prepared for OpenStatSpec `v0.5.0`;
publication remains subject to [RELEASING](../RELEASING.md).
Terms **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative.

## Explicit selection and precedence

An adapter selects this policy by publishing this exact field in its
machine-readable capability declaration for the declared execution scope:

```json
{"database_io_policy": "openstatspec-database-io-v1"}
```

Support alone is not selection. The declaration MUST identify the selected
policy, supported directions and SQL profiles, exact adapter version, and
immutable specification commit, status and release provenance under
[VERSIONING](../VERSIONING.md). An absent or null field selects no override;
an unknown identifier MUST NOT be treated as this policy. Adapters not selecting this policy retain every existing requirement.

Only two execution requirements are overridden when selected:

1. Database-read-only operations, including export, return diagnostics instead
of persisting operation or fidelity records, even where
[SAV/ZSAV 1.0](spss-sav-zsav-profile-1.0.md) or an implementation profile
requires export records or persistence of loss opt-in.
2. Dolt writes use an exact-version tested policy shipped with the adapter
package by default, instead of requiring a runtime external declaration
matching the adapter/specification bindings in the
[Dolt profile](../sql/dialect-profiles.md#dolt-profile).

This is not a reinterpretation of SAV/ZSAV 1.0 or a new source, plan, frontend,
SQL binding, or catalog schema version. All other requirements remain in force.
The source/data/metadata schema, catalog identity, one-dataset/one-wide-table
mapping, dictionary preservation, case/variable order, values and missingness
are unchanged. Existing Plan/Frontend/Binding identifiers, canonical bytes and
hashes MUST NOT be rewritten or supplemented with this capability field.
The [SAV 1.0 fixtures](../conformance/spss-sav-zsav-1.0.json) retain their
semantic expectations, including the import-only failed-preflight audit case.
Claims MUST name both the source profile and this policy rather than imply
unmodified SAV 1.0 export-record behavior.

## Database-read-only operations

All read, inspection, standalone validation and export operations, including
all their setup, validation, error and cleanup paths, MUST NOT issue database writes.
This prohibition includes data or metadata updates, operation/fidelity audit
inserts, catalog initialization, identity bootstrapping, installation or
migration, adapter-created temporary tables, staging objects, and Dolt
working-set or history changes. A rolled-back write is still a prohibited write.
Reads MUST NOT create a missing database or SQLite file. Engine-internal query
execution state is not adapter-created database mutation.

The adapter MUST retain positive server identity and version checks, supported
driver/transport checks, exclusive catalog binding, catalog identity and schema
validation, source-to-physical mapping validation, and data/type/order checks.
MySQL wire compatibility MUST NOT establish Dolt or InnoDB identity. Missing,
foreign, ambiguous or unsupported catalogs and invalid data fail without
repair or migration; any installation or migration requires a separate explicit
write operation. Read-only access MUST NOT depend on passing an import/write
conformance declaration gate, but unsupported server identities, versions or
drivers still fail closed.

Exports MUST read the normative catalog and preserve all declared SAV/ZSAV
writer capability and fidelity checks. Known unsupported semantics MUST fail
before creating an output artifact unless the caller explicitly supplies
operation-scoped `allow_loss` for each identified loss. Identified losses MUST
be reported to the caller using the adapter's documented diagnostic format and
stable loss codes. Success and failure diagnostics travel in the operation
result or error, not database persistence; existing database audit rows MUST
NOT be altered. This policy does not introduce a new diagnostic schema. `allow_loss` never grants future consent or full fidelity for a lost
semantic.

Database-read-only does not mean filesystem-read-only. Export MUST preserve
safe file publication: validate before output creation, write only to an
operation-owned temporary file, and publish the completed file atomically
only after successful writing and required checks. Normal failure MUST leave
any pre-existing destination unchanged and remove operation-owned partial output.
If filesystem restoration or cleanup itself fails, the adapter MUST report that
failure without claiming clean completion and preserve any recoverable prior
file content; it MUST NOT delete a surviving backup just to hide the failure.
Such failures still MUST NOT produce database writes. Destination authorization,
overwrite rules, path safety and writer errors MUST NOT be bypassed.

## Packaged Dolt write policy

The default policy permits only exact Dolt product versions 2.2.2 and 2.2.3
that the installed adapter release actually tested. An adapter MAY ship a
narrower tested subset; it MUST NOT infer support for other 2.2.x patches from
a range or from MySQL compatibility. Unknown or untested versions MUST fail before mutation.
On the same connection, the adapter MUST trim and case-fold `@@version_comment`
and require `Dolt`, obtain a non-empty `DOLT_VERSION()`, and match that exact
product version to its selected tested policy.

The package MUST ship its exact supported/tested version list, effective limit
policy and atomicity/cleanup behavior. Its published capabilities MUST identify
the exact adapter release and specification commit and distinguish theoretical,
observed and adapter-policy limits with units and sources. Successful release CI
for that exact adapter release and specification commit is the evidence: it
MUST exercise each supported exact product version with reproducible service
identity and the applicable Dolt product-identity, limit/boundary, binary64/text,
DDL-atomicity and fault/cleanup cases from the existing dialect profile.
Publishing this specification does not assert those downstream tests passed.

Default writes MUST NOT require user-supplied declaration or evidence files,
or invoke the external-file validator as a prerequisite. Packaged policy need
not use the external JSON schema or evidence-file layout; release CI supplies
the real evidence instead. It is not a fabricated concrete declaration: the
symbolic baseline remains pending and MUST NOT be relabelled tested or used as
measured limits.

An adapter MAY offer an explicitly selected external declaration as an advanced
override. If offered, it MUST use the unchanged
[external schema and validator contract](../sql/dolt-adapter-declarations/README.md),
including exactly one matching adapter/version/specification/product binding,
real resolving evidence and path/digest validation. Invalid, ambiguous or
untested overrides fail closed, never silently fall back to the package.
An override does not bypass the [server family policy](../sql/server-version-policy.md)
or any safety gate. The external format remains optional evidence, not a
runtime requirement for default package users; evidence MUST NOT be manufactured.

Only declaration delivery/selection and evidence format change for writes.
Effective limit preflight, driver/security checks, verified database/catalog
ownership, atomicity and audit-safe cleanup remain mandatory. Import audits
remain required under SAV 1.0. Transformation writes retain their selected
[Binding 0.1](transformation-plan-sql-binding-0.1.md) or
[Binding 0.2](transformation-plan-sql-binding-0.2.md) contract, including actor,
expected branch and HEAD, clean working-set preflight and completion checks,
same dataset/table identity, compact audit and native transaction boundary.
Non-atomic create-target apply remains rejected. No automatic Dolt commit,
branch switch, merge, reset, tag or persistent recovery/copy artifact is allowed.

## Adapter verification

In addition to the unchanged source fixtures and write conformance cases,
release CI for each claimed SQL profile MUST demonstrate successful export,
rejected and accepted loss, invalid identity/catalog/data, and writer/publication
failure with zero attempted database writes (including temporary DDL), unchanged
data/metadata/audit and, for Dolt, unchanged branch, HEAD, status and diff.
Read-only database credentials or engine read-only mode SHOULD be used where
available, alongside checks that catch attempted writes hidden by rollback.
Tests MUST verify destination preservation and temporary-file cleanup on failure.
Dolt package tests MUST cover supported exact versions without external files,
unknown-version rejection before mutation, and strict validation of an external
override if that optional path is offered.
5 changes: 5 additions & 0 deletions docs/implementation-profiles/python-pyspssio-0.5.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ profiles. Its capability boundary is engine-specific: a caller must use this
declaration, not a generic claim of “SPSS support”, to determine whether an
operation can produce a full semantic round trip.

If the adapter explicitly selects [Database I/O Execution Policy v1](../database-io-policy-v1.md),
export records and loss diagnostics required below are returned, not persisted
in the database. That execution override does not relax the writer's fidelity
boundary; without selection, the existing requirements remain unchanged.

## Declared directions and preserved semantics

The pinned pyspssio fork is the required, sole engine for unencrypted SAV and ZSAV.
Expand Down
5 changes: 5 additions & 0 deletions docs/spss-sav-zsav-profile-1.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ At minimum, the declaration MUST expose:
- an identifier limit with an explicit value, unit (bytes or characters), discovery source, and character repertoire or encoding policy; and
- the active server version and configuration sources used to derive effective limits.

The explicitly selected [Database I/O Execution Policy v1](database-io-policy-v1.md)
overrides export operation/fidelity persistence with returned diagnostics and
zero database writes. It does not change this profile's source semantics or
fixtures; adapters not selecting it retain all 1.0 requirements below.

## Source-faithful relational contract

For one imported SPSS dataset, an importer MUST create exactly one dedicated physical wide SQL data table. Every source case MUST map to one row, every source variable MUST map to one physical column, and no source variable may be transposed, split, stored in EAV/JSON, dropped, or coerced into a different logical shape.
Expand Down
6 changes: 6 additions & 0 deletions sql/dialect-profiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,12 @@ or incompatible storage configuration rather than claiming atomic import.

## Dolt profile

The explicitly selected [Database I/O Execution Policy v1](../docs/database-io-policy-v1.md)
permits packaged exact-version tested Dolt write support instead of mandatory
runtime external declarations, and prohibits database writes for reads,
validation and export. Only its stated overrides take precedence; without
selection, all requirements below remain unchanged.

Dolt is an independent profile. Its MySQL wire compatibility is a transport and
SQL-syntax property only; it MUST NOT cause an adapter to select the
MySQL/MariaDB/InnoDB profile or inherit an InnoDB column, row, LOB, DDL, or
Expand Down
7 changes: 7 additions & 0 deletions sql/dolt-adapter-declarations/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Concrete Dolt adapter declarations

With explicit [Database I/O Execution Policy v1](../../docs/database-io-policy-v1.md)
selection, packaged tested policy is the default Dolt write gate and this
external format is an optional advanced override. Its schema, validator and
real-evidence requirements remain unchanged; the runtime declaration gate
below still applies to adapters not selecting that policy and to external
overrides selected under it.

This directory is the only repository path for concrete, import-capable Dolt
adapter declarations. Each `*.json` file MUST be a full Dolt profile object
conforming to `../dolt-adapter-declaration-schema.json`; it is not a partial
Expand Down
8 changes: 8 additions & 0 deletions sql/profile-capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,14 @@ In particular, a MySQL URL or driver does not authorize a
identity requirements decide the product. Ambiguity fails closed before any
catalog or dataset mutation and leaves zero database mutation.

## Optional database I/O policy

An adapter explicitly selects [Database I/O Execution Policy v1](../docs/database-io-policy-v1.md)
with `database_io_policy: "openstatspec-database-io-v1"` in its capability
declaration. Only that selection overrides export database persistence and
mandatory external Dolt declarations; absence retains existing requirements.
The policy preserves identity, driver, catalog, data and effective-limit checks.

## Dolt declaration package

The checked-in Dolt baseline is a symbolic template and is not an
Expand Down
7 changes: 7 additions & 0 deletions sql/server-version-policy.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ and [PostgreSQL](https://github.com/docker-library/official-images/blob/master/l

## Dolt family policy and SQLite

The explicitly selected [Database I/O Execution Policy v1](../docs/database-io-policy-v1.md)
restricts default packaged Dolt writes to the adapter release's tested subset
of exact 2.2.2/2.2.3, not the whole family range. It replaces mandatory runtime
external declarations with package release CI evidence; all other identity,
limit and safety gates remain. Without selection, the existing policy below
is unchanged.

Dolt remains an independent, essential profile. Its conservative 2.2.x claim
starts at the previously validated 2.2.2 floor and ends before 2.3.0; runtime
preflight MUST reject versions outside `>=2.2.2,<2.3.0` before mutation. Exact
Expand Down
Loading