Skip to content

Commit 051e12f

Browse files
TonisOrmissonTonisOrmisson
committed
Package transformation frontends
1 parent fc0ab96 commit 051e12f

25 files changed

Lines changed: 1184 additions & 204 deletions

‎CHANGELOG.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,37 @@
22

33
All notable changes to this reference implementation are documented here.
44

5+
## 0.4.0 — 2026-07-31
6+
7+
### Added
8+
9+
- Added a public canonical transformation-plan in-place API accepting either
10+
typed plan objects or strictly validated mappings.
11+
- Added apply-plan and install-in-place-schema CLI workflows alongside the
12+
compatible apply-spss command.
13+
- Added explicit audit provenance distinguishing canonical plan documents from
14+
SPSS syntax sources.
15+
- Added a transformation manual covering the canonical core, frontend boundary,
16+
execution invariants, CLI/API workflows, and extension rules.
17+
18+
### Changed
19+
20+
- Moved the SPSS parser, binder, compiler, and convenience execution adapter to
21+
the dedicated openstatspec.frontends.spss package.
22+
- Kept openstatspec.transform language-neutral with canonical plan, schema, and
23+
live-schema validation modules while preserving previous SPSS import paths.
24+
- Reserved empty Stata and SAS frontend directories without exposing parsers,
25+
capabilities, CLI choices, or support claims.
26+
- Strengthened pre-mutation target-type checks and target-scoped physical table
27+
identity guards. The executor continues to modify the same dataset and table
28+
without creating OpenStatSpec rollback, copy, snapshot, or history layers.
29+
30+
### Specification basis
31+
32+
- Conformance and release validation continue to use OpenStatSpec specification
33+
release v0.2.0 at exact commit
34+
79339ec3d8f8aa81789b7e85f6b8afa6f1374e50.
35+
536
## 0.3.0 — 2026-07-31
637

738
### Added

‎README.md‎

Lines changed: 15 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,10 @@ import.
2626
capabilities, and loss reports.
2727
- `openstatspec.sql`: database connection and wide-table/catalog operations.
2828
- `openstatspec.spss`: SAV/ZSAV adapter boundary.
29+
- `openstatspec.transform`: canonical plans, frontend-neutral schema concepts,
30+
and plan validation.
31+
- `openstatspec.frontends.spss`: the SPSS-like syntax frontend and convenience
32+
execution adapter.
2933

3034
## Intended workflow
3135

@@ -63,43 +67,17 @@ implemented capability boundary.
6367

6468
## SPSS-like transformation frontend
6569

66-
The transformation frontend accepts `RECODE`, `VARIABLE LABELS`, and
67-
`VALUE LABELS`, lowers them to a canonical OpenStatSpec Transformation Plan,
68-
and mutates the same logical dataset and physical wide table. SQLite,
69-
PostgreSQL, MySQL, MariaDB, and Dolt connections are allowed. It creates no
70-
derived dataset, copied table, snapshot, or OpenStatSpec rollback/version layer.
71-
72-
Install the compact operation-audit relation separately, then apply syntax:
73-
74-
```python
75-
from openstatspec import (
76-
apply_spss_in_place,
77-
install_in_place_transformation_schema,
78-
)
79-
80-
database_url = "postgresql+psycopg://user:password@host/database"
81-
install_in_place_transformation_schema(database_url=database_url)
82-
result = apply_spss_in_place(
83-
database_url=database_url,
84-
dataset_id="...",
85-
actor="agent@example.org",
86-
source_text="""
87-
RECODE age (18 THRU 34 = 1) (35 THRU 64 = 2) INTO age_group.
88-
VARIABLE LABELS age_group 'Age group'.
89-
VALUE LABELS age_group 1 '18-34' 2 '35-64'.
90-
""",
91-
)
92-
```
93-
94-
Existing-target recodes and metadata mutations use direct DML. SQLite and
95-
PostgreSQL may also add a target column in the same native transaction. MySQL,
96-
MariaDB, and Dolt reject such schema-changing plans before mutation because
97-
their implicit-commit DDL could otherwise leave a partial apply. On Dolt, the
98-
caller additionally supplies expected branch and HEAD identities, and the
99-
working set must be clean. The transformer never calls `DOLT_COMMIT`.
100-
101-
See [in-place transformations](docs/in-place-transformation.md) for the full
102-
execution boundary and CLI form.
70+
The SPSS-like frontend lowers supported `RECODE`, `VARIABLE LABELS`, and
71+
`VALUE LABELS` syntax into a language-neutral canonical plan. The in-place
72+
path applies it to the same logical dataset, physical wide table, and metadata
73+
catalog without a derived dataset, copied table, snapshot, or separate
74+
rollback/history layer. Dolt remains the sole versioning layer for Dolt-backed
75+
edits, and the transformer never calls `DOLT_COMMIT`.
76+
77+
See the [dataset transformations manual](docs/transformations.md) for schema
78+
installation, Python and CLI surfaces, database invariants, audit provenance,
79+
package layout, and extension guidance. Stata and SAS are unimplemented
80+
placeholders.
10381

10482
## Current support status
10583

‎docs/in-place-transformation.md‎

Lines changed: 7 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,9 @@
1-
# In-place SPSS-like transformation
1+
# In-place transformations
22

3-
`openstatspec.apply_spss_in_place` is the public execution path for the
4-
SPSS-like frontend. It accepts `RECODE`, `VARIABLE LABELS`, and `VALUE LABELS`,
5-
binds them to the existing core dataset, and applies the canonical plan to that
6-
same SQL wide table and metadata catalog.
3+
This page has moved to the comprehensive
4+
[dataset transformations manual](transformations.md).
75

8-
The caller supplies a supported SQL URL, the existing normative `dataset_id`,
9-
and a non-empty actor identity. For Dolt, the caller also supplies the expected
10-
active branch and current `HEAD` hash.
11-
12-
Install the compact audit relation once with
13-
`openstatspec.install_in_place_transformation_schema(database_url=...)` before
14-
the first apply. Apply never creates schema-management objects itself.
15-
16-
The adapter uses the engine's ordinary transaction behavior. On Dolt it first
17-
checks branch and HEAD and requires `dolt_status` to be empty. Existing-target
18-
recodes are one direct `UPDATE`. SQLite and PostgreSQL can add a new numeric
19-
`INTO` target to the same table. MySQL, MariaDB, and Dolt require that target
20-
column and variable metadata to exist before apply because their DDL can commit
21-
independently of the following data and metadata changes. Label commands
22-
update/replace the same dataset's normative and compatibility metadata rows.
23-
24-
One compact `transformation_apply` row records operation identity, canonical
25-
plan/source hashes, actor, status, timestamps, and the observed Dolt branch and
26-
HEAD. It contains no row values and points to no copied table.
27-
28-
The adapter does not create `derived_dataset` rows, persistent output tables,
29-
full-table copies, staging datasets, snapshots, rollback tables, retirement
30-
records, or a recovery/version catalog. It does not call `DOLT_COMMIT`, change
31-
branches, merge, reset, or tag. After success, the caller reviews `dolt diff`
32-
and independently decides whether to commit or restore the working set.
33-
34-
The local SQLite tests exercise the public mutation path and assert that
35-
dataset/table counts and identities do not change. Live PostgreSQL/MySQL/MariaDB and
36-
exact-version Dolt service evidence remains required before release execution
37-
claims for those engines.
38-
39-
```text
40-
openstatspec apply-spss --database-url mysql+pymysql://user:password@host/database --dataset-id ... --actor agent@example.org --expected-branch feature/recode --expected-head ... --syntax-file transform.sps
41-
```
6+
The manual covers the canonical plan, SPSS-like frontend, executor, audit
7+
schema, Python and CLI surfaces, database and Dolt invariants, provenance,
8+
package layout, and future frontend boundary. Stata and SAS remain
9+
unimplemented placeholders.

‎docs/release-readiness.md‎

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# 0.3.0 release readiness
1+
# 0.4.0 release readiness
22

33
This page records the expected release contract, not a publication event.
44
Creating a version tag remains a separate maintainer action.
@@ -56,6 +56,36 @@ otherwise the strict export policy reports and blocks the loss. A caller
5656
that supplies consent for an available loss code receives the
5757
machine-readable loss report with the export result.
5858

59+
## Transformation release gates
60+
61+
The canonical transformation core and SPSS syntax frontend are separate public
62+
surfaces. A release must run the specification-owned canonical-plan and SPSS
63+
frontend conformance fixtures, then exercise both the generic plan apply API
64+
and the SPSS compatibility apply path.
65+
66+
The gate must prove that:
67+
68+
- a TransformationPlan object and its strict JSON mapping produce the same
69+
plan hash and in-place result;
70+
- top-level SPSS compiler imports and legacy openstatspec.transform re-exports
71+
still load from an installed wheel;
72+
- install-in-place-schema, apply-plan, and apply-spss execute their documented
73+
CLI workflows;
74+
- invalid or unsupported plans fail before the first data or metadata mutation;
75+
- successful applies retain the same dataset ID, physical schema/table
76+
identity, dataset count, and persistent physical data-table count;
77+
- audit rows distinguish canonical plans from SPSS syntax, preserve correct
78+
source/plan hashes and frontend contract, and contain no copied data;
79+
- no OpenStatSpec rollback, snapshot, staging, copy, derived-dataset, or
80+
parallel history artifacts are created; and
81+
- Dolt checks expected branch, HEAD, and a clean working set without committing
82+
or changing HEAD; other supported SQL connections remain allowed.
83+
84+
The built wheel must contain the generic openstatspec.transform modules and the
85+
implemented openstatspec.frontends.spss package. Stata and SAS remain empty
86+
source-tree placeholders and must expose no compiler, apply API, CLI choice,
87+
capability claim, or implied support.
88+
5989
## Maintainer checks before tagging
6090

6191
1. Publish the pinned `openstatspec-pyspssio==0.5.1.post2` engine distribution

0 commit comments

Comments
 (0)