Package transformation frontends for Python v0.4.0 (#9)

Separate the language-neutral canonical transformation core and in-place executor from the SPSS frontend, preserve compatibility APIs, add generic plan execution and CLI workflows, document the architecture, and prepare v0.4.0.

Author: TonisOrmisson

CHANGELOG.md

@@ -2,6 +2,37 @@
 
 All notable changes to this reference implementation are documented here.
 
+## 0.4.0 — 2026-07-31
+
+### Added
+
+- Added a public canonical transformation-plan in-place API accepting either
+  typed plan objects or strictly validated mappings.
+- Added apply-plan and install-in-place-schema CLI workflows alongside the
+  compatible apply-spss command.
+- Added explicit audit provenance distinguishing canonical plan documents from
+  SPSS syntax sources.
+- Added a transformation manual covering the canonical core, frontend boundary,
+  execution invariants, CLI/API workflows, and extension rules.
+
+### Changed
+
+- Moved the SPSS parser, binder, compiler, and convenience execution adapter to
+  the dedicated openstatspec.frontends.spss package.
+- Kept openstatspec.transform language-neutral with canonical plan, schema, and
+  live-schema validation modules while preserving previous SPSS import paths.
+- Reserved empty Stata and SAS frontend directories without exposing parsers,
+  capabilities, CLI choices, or support claims.
+- Strengthened pre-mutation target-type checks and target-scoped physical table
+  identity guards. The executor continues to modify the same dataset and table
+  without creating OpenStatSpec rollback, copy, snapshot, or history layers.
+
+### Specification basis
+
+- Conformance and release validation continue to use OpenStatSpec specification
+  release v0.2.0 at exact commit
+  79339ec3d8f8aa81789b7e85f6b8afa6f1374e50.
+
 ## 0.3.0 — 2026-07-31
 
 ### Added

README.md

@@ -26,6 +26,10 @@ import.
   capabilities, and loss reports.
 - `openstatspec.sql`: database connection and wide-table/catalog operations.
 - `openstatspec.spss`: SAV/ZSAV adapter boundary.
+- `openstatspec.transform`: canonical plans, frontend-neutral schema concepts,
+  and plan validation.
+- `openstatspec.frontends.spss`: the SPSS-like syntax frontend and convenience
+  execution adapter.
 
 ## Intended workflow
 
@@ -63,43 +67,17 @@ implemented capability boundary.
 
 ## SPSS-like transformation frontend
 
-The transformation frontend accepts `RECODE`, `VARIABLE LABELS`, and
-`VALUE LABELS`, lowers them to a canonical OpenStatSpec Transformation Plan,
-and mutates the same logical dataset and physical wide table. SQLite,
-PostgreSQL, MySQL, MariaDB, and Dolt connections are allowed. It creates no
-derived dataset, copied table, snapshot, or OpenStatSpec rollback/version layer.
-
-Install the compact operation-audit relation separately, then apply syntax:
-
-```python
-from openstatspec import (
-    apply_spss_in_place,
-    install_in_place_transformation_schema,
-)
-
-database_url = "postgresql+psycopg://user:password@host/database"
-install_in_place_transformation_schema(database_url=database_url)
-result = apply_spss_in_place(
-    database_url=database_url,
-    dataset_id="...",
-    actor="agent@example.org",
-    source_text="""
-      RECODE age (18 THRU 34 = 1) (35 THRU 64 = 2) INTO age_group.
-      VARIABLE LABELS age_group 'Age group'.
-      VALUE LABELS age_group 1 '18-34' 2 '35-64'.
-    """,
-)
-```
-
-Existing-target recodes and metadata mutations use direct DML. SQLite and
-PostgreSQL may also add a target column in the same native transaction. MySQL,
-MariaDB, and Dolt reject such schema-changing plans before mutation because
-their implicit-commit DDL could otherwise leave a partial apply. On Dolt, the
-caller additionally supplies expected branch and HEAD identities, and the
-working set must be clean. The transformer never calls `DOLT_COMMIT`.
-
-See [in-place transformations](docs/in-place-transformation.md) for the full
-execution boundary and CLI form.
+The SPSS-like frontend lowers supported `RECODE`, `VARIABLE LABELS`, and
+`VALUE LABELS` syntax into a language-neutral canonical plan. The in-place
+path applies it to the same logical dataset, physical wide table, and metadata
+catalog without a derived dataset, copied table, snapshot, or separate
+rollback/history layer. Dolt remains the sole versioning layer for Dolt-backed
+edits, and the transformer never calls `DOLT_COMMIT`.
+
+See the [dataset transformations manual](docs/transformations.md) for schema
+installation, Python and CLI surfaces, database invariants, audit provenance,
+package layout, and extension guidance. Stata and SAS are unimplemented
+placeholders.
 
 ## Current support status
 

docs/in-place-transformation.md

@@ -1,41 +1,9 @@
-# In-place SPSS-like transformation
+# In-place transformations
 
-`openstatspec.apply_spss_in_place` is the public execution path for the
-SPSS-like frontend. It accepts `RECODE`, `VARIABLE LABELS`, and `VALUE LABELS`,
-binds them to the existing core dataset, and applies the canonical plan to that
-same SQL wide table and metadata catalog.
+This page has moved to the comprehensive
+[dataset transformations manual](transformations.md).
 
-The caller supplies a supported SQL URL, the existing normative `dataset_id`,
-and a non-empty actor identity. For Dolt, the caller also supplies the expected
-active branch and current `HEAD` hash.
-
-Install the compact audit relation once with
-`openstatspec.install_in_place_transformation_schema(database_url=...)` before
-the first apply. Apply never creates schema-management objects itself.
-
-The adapter uses the engine's ordinary transaction behavior. On Dolt it first
-checks branch and HEAD and requires `dolt_status` to be empty. Existing-target
-recodes are one direct `UPDATE`. SQLite and PostgreSQL can add a new numeric
-`INTO` target to the same table. MySQL, MariaDB, and Dolt require that target
-column and variable metadata to exist before apply because their DDL can commit
-independently of the following data and metadata changes. Label commands
-update/replace the same dataset's normative and compatibility metadata rows.
-
-One compact `transformation_apply` row records operation identity, canonical
-plan/source hashes, actor, status, timestamps, and the observed Dolt branch and
-HEAD. It contains no row values and points to no copied table.
-
-The adapter does not create `derived_dataset` rows, persistent output tables,
-full-table copies, staging datasets, snapshots, rollback tables, retirement
-records, or a recovery/version catalog. It does not call `DOLT_COMMIT`, change
-branches, merge, reset, or tag. After success, the caller reviews `dolt diff`
-and independently decides whether to commit or restore the working set.
-
-The local SQLite tests exercise the public mutation path and assert that
-dataset/table counts and identities do not change. Live PostgreSQL/MySQL/MariaDB and
-exact-version Dolt service evidence remains required before release execution
-claims for those engines.
-
-```text
-openstatspec apply-spss --database-url mysql+pymysql://user:password@host/database --dataset-id ... --actor agent@example.org --expected-branch feature/recode --expected-head ... --syntax-file transform.sps
-```
+The manual covers the canonical plan, SPSS-like frontend, executor, audit
+schema, Python and CLI surfaces, database and Dolt invariants, provenance,
+package layout, and future frontend boundary. Stata and SAS remain
+unimplemented placeholders.

docs/release-readiness.md

@@ -1,4 +1,4 @@
-# 0.3.0 release readiness
+# 0.4.0 release readiness
 
 This page records the expected release contract, not a publication event.
 Creating a version tag remains a separate maintainer action.
@@ -56,6 +56,36 @@ otherwise the strict export policy reports and blocks the loss. A caller
 that supplies consent for an available loss code receives the
 machine-readable loss report with the export result.
 
+## Transformation release gates
+
+The canonical transformation core and SPSS syntax frontend are separate public
+surfaces. A release must run the specification-owned canonical-plan and SPSS
+frontend conformance fixtures, then exercise both the generic plan apply API
+and the SPSS compatibility apply path.
+
+The gate must prove that:
+
+- a TransformationPlan object and its strict JSON mapping produce the same
+  plan hash and in-place result;
+- top-level SPSS compiler imports and legacy openstatspec.transform re-exports
+  still load from an installed wheel;
+- install-in-place-schema, apply-plan, and apply-spss execute their documented
+  CLI workflows;
+- invalid or unsupported plans fail before the first data or metadata mutation;
+- successful applies retain the same dataset ID, physical schema/table
+  identity, dataset count, and persistent physical data-table count;
+- audit rows distinguish canonical plans from SPSS syntax, preserve correct
+  source/plan hashes and frontend contract, and contain no copied data;
+- no OpenStatSpec rollback, snapshot, staging, copy, derived-dataset, or
+  parallel history artifacts are created; and
+- Dolt checks expected branch, HEAD, and a clean working set without committing
+  or changing HEAD; other supported SQL connections remain allowed.
+
+The built wheel must contain the generic openstatspec.transform modules and the
+implemented openstatspec.frontends.spss package. Stata and SAS remain empty
+source-tree placeholders and must expose no compiler, apply API, CLI choice,
+capability claim, or implied support.
+
 ## Maintainer checks before tagging
 
 1. Publish the pinned `openstatspec-pyspssio==0.5.1.post2` engine distribution