diff --git a/CHANGELOG.md b/CHANGELOG.md index ff53fa4..32d8097 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,23 @@ ## Unreleased -Future changes will be listed here. +### v0.5.0 (prepared; not yet published) + +- Add normative optional `openstatspec-database-io-v1`, explicitly selected by + capability field `database_io_policy`. Reads, validation and export perform + no database writes, including audit, temporary DDL and failure paths; export + loss diagnostics are returned while safe file publication remains required. +- Permit default Dolt writes through packaged exact tested 2.2.2/2.2.3 policy + backed by adapter release CI, with no runtime user declaration/evidence files. + External declarations and their unchanged validator remain an optional + advanced override. Identity, limits, atomicity, security and branch/HEAD + checks remain mandatory; unknown or untested versions fail closed. +- Preserve unselected adapters' old requirements, SAV 1.0 source semantics, + catalog/data/metadata schema and fixtures, and all canonical plan, frontend + and SQL binding contracts. This pre-1.0 minor release adds an independently + selected policy; it does not silently revise an existing profile. +- Keep the separately released `openstatspec-specification` companion package + at `0.1.0`. No adapter conformance or package release is asserted here. ## v0.4.1 - 2026-08-21 diff --git a/README.md b/README.md index 0edc9e1..e18a60a 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,19 @@ MySQL 8.4.x/9.7.x, MariaDB 11.4.x/11.8.x/12.3.x and PostgreSQL 17.x/18.x series while requiring exact patch-version CI evidence. Microsoft SQL Server remains an explicitly unsupported [future dialect](docs/mssql-dialect-roadmap.md). +## Optional database I/O execution policy + +Prepared for specification `v0.5.0`, the normative optional +[Database I/O Execution Policy v1](docs/database-io-policy-v1.md) is explicitly +selected with `database_io_policy: "openstatspec-database-io-v1"`. +It requires database-read-only reads, validation and exports (including errors +and loss diagnostics), and permits default Dolt writes on packaged exact tested +2.2.2/2.2.3 policy backed by adapter release CI, without user evidence files. +External declarations remain an optional advanced override. Unselected adapters +retain existing requirements; SAV 1.0 semantics, schemas, fixtures and canonical +Plan/Frontend/Binding contracts are unchanged. Publication and downstream +adapter support remain separately gated. + ## Core contract - One source dataset maps to exactly one dedicated SQL data table. diff --git a/RELEASING.md b/RELEASING.md index 95df955..3e6fd6d 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -12,6 +12,27 @@ Package tags use the `package-vX.Y.Z` convention; specification tags use `vX.Y.Z`. A specification tag MUST NOT publish the package, and a package tag MUST match the version declared in `pyproject.toml`. +## Prepared v0.5.0 scope + +This release prepares the normative optional +[Database I/O Execution Policy v1](docs/database-io-policy-v1.md). It is not +published until the gates below complete. Release notes MUST identify explicit +`database_io_policy` selection, database-read-only export with returned loss +diagnostics, and packaged exact tested Dolt 2.2.2/2.2.3 write policy with optional +external evidence. Existing unselected-profile requirements, SAV 1.0 semantics +and fixtures, schemas, and canonical Plan/Frontend/Binding artifacts are unchanged. +This is a pre-1.0 minor optional addition under [VERSIONING](VERSIONING.md). + +Run `python -m pytest` and `python tools/validate_repository.py` at the selected +commit. Adapter package release CI supplies real exact-version write and +read-only/failure-path evidence under the new policy; neither fabricated +repository declarations nor user-supplied evidence files are release substitutes. +Python `v0.8.0` and PHP `v0.7.0` are downstream adapter release targets, not +conformance claims made by this specification. Their pins and publication are +managed separately. The companion `openstatspec-specification` distribution +remains `0.1.0`: no package API, schema or validator contract change requires a +package release for this documentation policy addition. + ## Before preparing a release - Choose an exact reviewed commit on the appropriate branch and record its diff --git a/ROADMAP.md b/ROADMAP.md index f68a0a4..4460d1a 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -129,6 +129,11 @@ follows and does not block specification publication. remain downstream and are not implied by this release. 8. [x] Published documentation patch release `v0.4.1` to align the frontend roadmap with the stable Frontend 0.3 profile; no normative contract changed. +9. [x] Prepare the optional [Database I/O Execution Policy v1](docs/database-io-policy-v1.md) + and `v0.5.0` release notes without changing existing source or plan contracts. +10. [ ] Publish `v0.5.0` after exact-commit and tag-context CI and the release + checklist; downstream Python `v0.8.0` and PHP `v0.7.0` pins, policy evidence + and package releases remain separately managed. The pending in-place service matrix and adapter conformance work in sections 6 and 7 are downstream implementation gates. They do not block the `v0.4.0` diff --git a/VERSIONING.md b/VERSIONING.md index aff342f..913b591 100644 --- a/VERSIONING.md +++ b/VERSIONING.md @@ -6,6 +6,15 @@ OpenStatSpec uses semantic versioning for tagged specification releases. - A minor release may add backward-compatible optional metadata, a new independent source or SQL profile, or new fixtures that do not invalidate an implementation's existing declared conformance scope. - A patch release may clarify wording, correct examples, or fix fixture and schema defects without changing normative meaning. A schema-defect correction may be backported to a published release series only when it restores the already-stated normative contract; it must not introduce new required behavior, broaden the accepted contract, or silently revise canonical artifacts. +The optional [Database I/O Execution Policy v1](docs/database-io-policy-v1.md) +is an independent execution contract selected by `database_io_policy`, not a +revision of SAV 1.0 or any canonical plan, frontend, binding or catalog schema. +Its explicitly scoped overrides apply only when selected; otherwise existing +requirements remain unchanged. Adding it in pre-1.0 `v0.5.0` is a minor optional +addition, not a patch clarification. No blanket profile freeze or version bump +is required. Future incompatible changes to this policy require a new policy +identifier; they MUST NOT silently change v1 selection. + Optional workflow profiles are versioned and claimed independently from the core source profile. Adding or revising an optional profile does not authorize changes to core datasets. A breaking workflow-profile change increments that diff --git a/docs/database-io-policy-v1.md b/docs/database-io-policy-v1.md new file mode 100644 index 0000000..6b7229e --- /dev/null +++ b/docs/database-io-policy-v1.md @@ -0,0 +1,146 @@ +# Optional Database I/O Execution Policy v1 + +Status: normative optional addition prepared for OpenStatSpec `v0.5.0`; +publication remains subject to [RELEASING](../RELEASING.md). +Terms **MUST**, **MUST NOT**, **SHOULD**, and **MAY** are normative. + +## Explicit selection and precedence + +An adapter selects this policy by publishing this exact field in its +machine-readable capability declaration for the declared execution scope: + +```json +{"database_io_policy": "openstatspec-database-io-v1"} +``` + +Support alone is not selection. The declaration MUST identify the selected +policy, supported directions and SQL profiles, exact adapter version, and +immutable specification commit, status and release provenance under +[VERSIONING](../VERSIONING.md). An absent or null field selects no override; +an unknown identifier MUST NOT be treated as this policy. Adapters not selecting this policy retain every existing requirement. + +Only two execution requirements are overridden when selected: + +1. Database-read-only operations, including export, return diagnostics instead + of persisting operation or fidelity records, even where + [SAV/ZSAV 1.0](spss-sav-zsav-profile-1.0.md) or an implementation profile + requires export records or persistence of loss opt-in. +2. Dolt writes use an exact-version tested policy shipped with the adapter + package by default, instead of requiring a runtime external declaration + matching the adapter/specification bindings in the + [Dolt profile](../sql/dialect-profiles.md#dolt-profile). + +This is not a reinterpretation of SAV/ZSAV 1.0 or a new source, plan, frontend, +SQL binding, or catalog schema version. All other requirements remain in force. +The source/data/metadata schema, catalog identity, one-dataset/one-wide-table +mapping, dictionary preservation, case/variable order, values and missingness +are unchanged. Existing Plan/Frontend/Binding identifiers, canonical bytes and +hashes MUST NOT be rewritten or supplemented with this capability field. +The [SAV 1.0 fixtures](../conformance/spss-sav-zsav-1.0.json) retain their +semantic expectations, including the import-only failed-preflight audit case. +Claims MUST name both the source profile and this policy rather than imply +unmodified SAV 1.0 export-record behavior. + +## Database-read-only operations + +All read, inspection, standalone validation and export operations, including +all their setup, validation, error and cleanup paths, MUST NOT issue database writes. +This prohibition includes data or metadata updates, operation/fidelity audit +inserts, catalog initialization, identity bootstrapping, installation or +migration, adapter-created temporary tables, staging objects, and Dolt +working-set or history changes. A rolled-back write is still a prohibited write. +Reads MUST NOT create a missing database or SQLite file. Engine-internal query +execution state is not adapter-created database mutation. + +The adapter MUST retain positive server identity and version checks, supported +driver/transport checks, exclusive catalog binding, catalog identity and schema +validation, source-to-physical mapping validation, and data/type/order checks. +MySQL wire compatibility MUST NOT establish Dolt or InnoDB identity. Missing, +foreign, ambiguous or unsupported catalogs and invalid data fail without +repair or migration; any installation or migration requires a separate explicit +write operation. Read-only access MUST NOT depend on passing an import/write +conformance declaration gate, but unsupported server identities, versions or +drivers still fail closed. + +Exports MUST read the normative catalog and preserve all declared SAV/ZSAV +writer capability and fidelity checks. Known unsupported semantics MUST fail +before creating an output artifact unless the caller explicitly supplies +operation-scoped `allow_loss` for each identified loss. Identified losses MUST +be reported to the caller using the adapter's documented diagnostic format and +stable loss codes. Success and failure diagnostics travel in the operation +result or error, not database persistence; existing database audit rows MUST +NOT be altered. This policy does not introduce a new diagnostic schema. `allow_loss` never grants future consent or full fidelity for a lost +semantic. + +Database-read-only does not mean filesystem-read-only. Export MUST preserve +safe file publication: validate before output creation, write only to an +operation-owned temporary file, and publish the completed file atomically +only after successful writing and required checks. Normal failure MUST leave +any pre-existing destination unchanged and remove operation-owned partial output. +If filesystem restoration or cleanup itself fails, the adapter MUST report that +failure without claiming clean completion and preserve any recoverable prior +file content; it MUST NOT delete a surviving backup just to hide the failure. +Such failures still MUST NOT produce database writes. Destination authorization, +overwrite rules, path safety and writer errors MUST NOT be bypassed. + +## Packaged Dolt write policy + +The default policy permits only exact Dolt product versions 2.2.2 and 2.2.3 +that the installed adapter release actually tested. An adapter MAY ship a +narrower tested subset; it MUST NOT infer support for other 2.2.x patches from +a range or from MySQL compatibility. Unknown or untested versions MUST fail before mutation. +On the same connection, the adapter MUST trim and case-fold `@@version_comment` +and require `Dolt`, obtain a non-empty `DOLT_VERSION()`, and match that exact +product version to its selected tested policy. + +The package MUST ship its exact supported/tested version list, effective limit +policy and atomicity/cleanup behavior. Its published capabilities MUST identify +the exact adapter release and specification commit and distinguish theoretical, +observed and adapter-policy limits with units and sources. Successful release CI +for that exact adapter release and specification commit is the evidence: it +MUST exercise each supported exact product version with reproducible service +identity and the applicable Dolt product-identity, limit/boundary, binary64/text, +DDL-atomicity and fault/cleanup cases from the existing dialect profile. +Publishing this specification does not assert those downstream tests passed. + +Default writes MUST NOT require user-supplied declaration or evidence files, +or invoke the external-file validator as a prerequisite. Packaged policy need +not use the external JSON schema or evidence-file layout; release CI supplies +the real evidence instead. It is not a fabricated concrete declaration: the +symbolic baseline remains pending and MUST NOT be relabelled tested or used as +measured limits. + +An adapter MAY offer an explicitly selected external declaration as an advanced +override. If offered, it MUST use the unchanged +[external schema and validator contract](../sql/dolt-adapter-declarations/README.md), +including exactly one matching adapter/version/specification/product binding, +real resolving evidence and path/digest validation. Invalid, ambiguous or +untested overrides fail closed, never silently fall back to the package. +An override does not bypass the [server family policy](../sql/server-version-policy.md) +or any safety gate. The external format remains optional evidence, not a +runtime requirement for default package users; evidence MUST NOT be manufactured. + +Only declaration delivery/selection and evidence format change for writes. +Effective limit preflight, driver/security checks, verified database/catalog +ownership, atomicity and audit-safe cleanup remain mandatory. Import audits +remain required under SAV 1.0. Transformation writes retain their selected +[Binding 0.1](transformation-plan-sql-binding-0.1.md) or +[Binding 0.2](transformation-plan-sql-binding-0.2.md) contract, including actor, +expected branch and HEAD, clean working-set preflight and completion checks, +same dataset/table identity, compact audit and native transaction boundary. +Non-atomic create-target apply remains rejected. No automatic Dolt commit, +branch switch, merge, reset, tag or persistent recovery/copy artifact is allowed. + +## Adapter verification + +In addition to the unchanged source fixtures and write conformance cases, +release CI for each claimed SQL profile MUST demonstrate successful export, +rejected and accepted loss, invalid identity/catalog/data, and writer/publication +failure with zero attempted database writes (including temporary DDL), unchanged +data/metadata/audit and, for Dolt, unchanged branch, HEAD, status and diff. +Read-only database credentials or engine read-only mode SHOULD be used where +available, alongside checks that catch attempted writes hidden by rollback. +Tests MUST verify destination preservation and temporary-file cleanup on failure. +Dolt package tests MUST cover supported exact versions without external files, +unknown-version rejection before mutation, and strict validation of an external +override if that optional path is offered. diff --git a/docs/implementation-profiles/python-pyspssio-0.5.1.md b/docs/implementation-profiles/python-pyspssio-0.5.1.md index a513575..80c7dee 100644 --- a/docs/implementation-profiles/python-pyspssio-0.5.1.md +++ b/docs/implementation-profiles/python-pyspssio-0.5.1.md @@ -13,6 +13,11 @@ profiles. Its capability boundary is engine-specific: a caller must use this declaration, not a generic claim of “SPSS support”, to determine whether an operation can produce a full semantic round trip. +If the adapter explicitly selects [Database I/O Execution Policy v1](../database-io-policy-v1.md), +export records and loss diagnostics required below are returned, not persisted +in the database. That execution override does not relax the writer's fidelity +boundary; without selection, the existing requirements remain unchanged. + ## Declared directions and preserved semantics The pinned pyspssio fork is the required, sole engine for unencrypted SAV and ZSAV. diff --git a/docs/spss-sav-zsav-profile-1.0.md b/docs/spss-sav-zsav-profile-1.0.md index e5babe4..4b5f07c 100644 --- a/docs/spss-sav-zsav-profile-1.0.md +++ b/docs/spss-sav-zsav-profile-1.0.md @@ -18,6 +18,11 @@ At minimum, the declaration MUST expose: - an identifier limit with an explicit value, unit (bytes or characters), discovery source, and character repertoire or encoding policy; and - the active server version and configuration sources used to derive effective limits. +The explicitly selected [Database I/O Execution Policy v1](database-io-policy-v1.md) +overrides export operation/fidelity persistence with returned diagnostics and +zero database writes. It does not change this profile's source semantics or +fixtures; adapters not selecting it retain all 1.0 requirements below. + ## Source-faithful relational contract For one imported SPSS dataset, an importer MUST create exactly one dedicated physical wide SQL data table. Every source case MUST map to one row, every source variable MUST map to one physical column, and no source variable may be transposed, split, stored in EAV/JSON, dropped, or coerced into a different logical shape. diff --git a/sql/dialect-profiles.md b/sql/dialect-profiles.md index 0032bc3..e74c888 100644 --- a/sql/dialect-profiles.md +++ b/sql/dialect-profiles.md @@ -109,6 +109,12 @@ or incompatible storage configuration rather than claiming atomic import. ## Dolt profile +The explicitly selected [Database I/O Execution Policy v1](../docs/database-io-policy-v1.md) +permits packaged exact-version tested Dolt write support instead of mandatory +runtime external declarations, and prohibits database writes for reads, +validation and export. Only its stated overrides take precedence; without +selection, all requirements below remain unchanged. + Dolt is an independent profile. Its MySQL wire compatibility is a transport and SQL-syntax property only; it MUST NOT cause an adapter to select the MySQL/MariaDB/InnoDB profile or inherit an InnoDB column, row, LOB, DDL, or diff --git a/sql/dolt-adapter-declarations/README.md b/sql/dolt-adapter-declarations/README.md index dae126d..93e86bd 100644 --- a/sql/dolt-adapter-declarations/README.md +++ b/sql/dolt-adapter-declarations/README.md @@ -1,5 +1,12 @@ # Concrete Dolt adapter declarations +With explicit [Database I/O Execution Policy v1](../../docs/database-io-policy-v1.md) +selection, packaged tested policy is the default Dolt write gate and this +external format is an optional advanced override. Its schema, validator and +real-evidence requirements remain unchanged; the runtime declaration gate +below still applies to adapters not selecting that policy and to external +overrides selected under it. + This directory is the only repository path for concrete, import-capable Dolt adapter declarations. Each `*.json` file MUST be a full Dolt profile object conforming to `../dolt-adapter-declaration-schema.json`; it is not a partial diff --git a/sql/profile-capabilities.md b/sql/profile-capabilities.md index 6062413..6be3c74 100644 --- a/sql/profile-capabilities.md +++ b/sql/profile-capabilities.md @@ -44,6 +44,14 @@ In particular, a MySQL URL or driver does not authorize a identity requirements decide the product. Ambiguity fails closed before any catalog or dataset mutation and leaves zero database mutation. +## Optional database I/O policy + +An adapter explicitly selects [Database I/O Execution Policy v1](../docs/database-io-policy-v1.md) +with `database_io_policy: "openstatspec-database-io-v1"` in its capability +declaration. Only that selection overrides export database persistence and +mandatory external Dolt declarations; absence retains existing requirements. +The policy preserves identity, driver, catalog, data and effective-limit checks. + ## Dolt declaration package The checked-in Dolt baseline is a symbolic template and is not an diff --git a/sql/server-version-policy.md b/sql/server-version-policy.md index d974f5f..7822fc0 100644 --- a/sql/server-version-policy.md +++ b/sql/server-version-policy.md @@ -48,6 +48,13 @@ and [PostgreSQL](https://github.com/docker-library/official-images/blob/master/l ## Dolt family policy and SQLite +The explicitly selected [Database I/O Execution Policy v1](../docs/database-io-policy-v1.md) +restricts default packaged Dolt writes to the adapter release's tested subset +of exact 2.2.2/2.2.3, not the whole family range. It replaces mandatory runtime +external declarations with package release CI evidence; all other identity, +limit and safety gates remain. Without selection, the existing policy below +is unchanged. + Dolt remains an independent, essential profile. Its conservative 2.2.x claim starts at the previously validated 2.2.2 floor and ends before 2.3.0; runtime preflight MUST reject versions outside `>=2.2.2,<2.3.0` before mutation. Exact diff --git a/tools/validate_repository.py b/tools/validate_repository.py index 9f9e7cc..b933e5a 100644 --- a/tools/validate_repository.py +++ b/tools/validate_repository.py @@ -141,6 +141,25 @@ def validate_repository_controls() -> None: ): require(f"CREATE TABLE {table} (" in schema, f"Schema table is missing: {table}") + io_policy = (ROOT / "docs/database-io-policy-v1.md").read_text(encoding="utf-8") + require_phrases( + io_policy, + ( + '"database_io_policy": "openstatspec-database-io-v1"', + "Adapters not selecting this policy retain every existing requirement.", + "MUST NOT issue database writes", + "temporary tables", + "`allow_loss`", + "2.2.2 and 2.2.3", + "release CI", + "Unknown or untested versions MUST fail before mutation.", + "user-supplied declaration or evidence files", + "expected branch and HEAD", + "conformance/spss-sav-zsav-1.0.json", + ), + "Optional database I/O policy", + ) + validate_release_metadata(ROOT)