Merge pull request #10 from OpenStatSpec/agent/conditional-transforms-v0.5

Add conditional SPSS transformations for Python v0.5

Author: TonisOrmisson

.github/workflows/ci.yml

@@ -20,7 +20,7 @@ jobs:
         uses: actions/checkout@v7
         with: &specification-checkout
           repository: OpenStatSpec/specification
-          ref: 79339ec3d8f8aa81789b7e85f6b8afa6f1374e50
+          ref: e49252c00890aed76dcaabc5d1ab5121b45929db
           path: openstatspec-specification
       - name: Checkout required SPSS engine
         uses: actions/checkout@v7

CHANGELOG.md

@@ -2,6 +2,40 @@
 
 All notable changes to this reference implementation are documented here.
 
+## Unreleased
+
+Planned adapter release: `0.5.0`, after lifecycle integration and final specification conformance.
+
+### Added
+
+- Added bounded typed expressions with variable references, numeric literals,
+  parentheses, `=`, `<`, `<=`, `>`, `>=`, `AND`, and `OR`. String comparison
+  and v0.2 string assignment fail closed pending exact portable semantics.
+- Added sequential SPSS-like `COMPUTE` and `IF` assignment operations plus
+  `FORMATS`, `VARIABLE LEVEL`, and `EXECUTE`.
+- Added atomic numeric target creation on SQLite and PostgreSQL. MySQL,
+  MariaDB, and Dolt fail closed on `target_mode=create`; their targets must be
+  provisioned physically and in the catalog by a separate versioned stage
+  before this executor applies assignment and metadata operations.
+- Added synthetic exact-program, catalog, failure-boundary, and pre-existing
+  target regression coverage.
+
+### Changed
+
+- Bumped the canonical transformation-plan and SPSS frontend contracts to
+  `v0.2`; canonical JSON and hashes include every sequential operation.
+- In-place apply now records variable label, value labels, `F` print/write
+  format, and measurement level in both normative and compatibility catalogs.
+- Dolt still requires an exact branch, exact HEAD, and clean working set;
+  successful apply leaves an inspectable diff and never calls `DOLT_COMMIT`.
+
+### Specification basis
+
+- Release validation is pinned to the untagged OpenStatSpec specification
+  release candidate at immutable commit
+  `e49252c00890aed76dcaabc5d1ab5121b45929db`. Its
+  `specification_release` remains null until that commit receives a stable tag.
+
 ## 0.4.0 — 2026-07-31
 
 ### Added

README.md

@@ -67,12 +67,19 @@ implemented capability boundary.
 
 ## SPSS-like transformation frontend
 
-The SPSS-like frontend lowers supported `RECODE`, `VARIABLE LABELS`, and
-`VALUE LABELS` syntax into a language-neutral canonical plan. The in-place
-path applies it to the same logical dataset, physical wide table, and metadata
-catalog without a derived dataset, copied table, snapshot, or separate
-rollback/history layer. Dolt remains the sole versioning layer for Dolt-backed
-edits, and the transformer never calls `DOLT_COMMIT`.
+The bounded SPSS-like frontend lowers `RECODE`, sequential `COMPUTE` and `IF`,
+`VARIABLE LABELS`, `VALUE LABELS`, numeric `FORMATS`, `VARIABLE LEVEL`, and
+`EXECUTE` into a language-neutral typed canonical plan. Conditions support
+parentheses, variable and numeric literal operands, comparisons, `AND`, and
+`OR`. String comparison and v0.2 string assignment currently fail closed.
+The in-place path applies the plan to the same logical dataset, physical wide
+table, and normative/compatibility metadata without a derived dataset, copied
+table, snapshot, or hidden history layer. Numeric targets may be created
+atomically on SQLite and PostgreSQL. MySQL, MariaDB, and Dolt fail closed on
+`target_mode=create` and require a separately provisioned physical and catalog
+target before assignment. Dolt requires the caller's exact clean branch/HEAD,
+leaves success as an inspectable working-set diff, and never calls
+`DOLT_COMMIT`.
 
 See the [dataset transformations manual](docs/transformations.md) for schema
 installation, Python and CLI surfaces, database invariants, audit provenance,

docs/release-readiness.md

@@ -1,4 +1,4 @@
-# 0.4.0 release readiness
+# 0.5.0 release readiness
 
 This page records the expected release contract, not a publication event.
 Creating a version tag remains a separate maintainer action.
@@ -67,6 +67,17 @@ The gate must prove that:
 
 - a TransformationPlan object and its strict JSON mapping produce the same
   plan hash and in-place result;
+- the exact bounded `COMPUTE`/`IF` program compiles to all seven ordered
+  operations without dropping `FORMATS`, `VARIABLE LEVEL`, or `EXECUTE`;
+- boolean data results match the equivalent expression and the target's label,
+  0/1 value labels, `F1.0` print/write format, and nominal level exist in both
+  normative and compatibility catalogs;
+- injected schema, data, catalog, and audit failures leave no partial apply;
+- compensation tracks only newly created targets and never drops or rewrites a
+  pre-existing target;
+- MySQL, MariaDB, and Dolt reject create-target plans before mutation; their
+  service evidence covers assignment to a separately provisioned physical and
+  catalog target without schema DDL;
 - top-level SPSS compiler imports and legacy openstatspec.transform re-exports
   still load from an installed wheel;
 - install-in-place-schema, apply-plan, and apply-spss execute their documented
@@ -78,8 +89,11 @@ The gate must prove that:
   source/plan hashes and frontend contract, and contain no copied data;
 - no OpenStatSpec rollback, snapshot, staging, copy, derived-dataset, or
   parallel history artifacts are created; and
-- Dolt checks expected branch, HEAD, and a clean working set without committing
-  or changing HEAD; other supported SQL connections remain allowed.
+- Dolt checks expected branch, HEAD, and a clean working set; success changes
+  neither HEAD nor branch and never commits or resets; state is rechecked after
+  the dataset lock and success must leave an inspectable working-set diff;
+  other supported SQL connections remain allowed; and
+- string comparisons and v0.2 string assignments fail closed until exact
 
 The built wheel must contain the generic openstatspec.transform modules and the
 implemented openstatspec.frontends.spss package. Stata and SAS remain empty
@@ -98,9 +112,12 @@ capability claim, or implied support.
 4. Build with `python -m build` and install the generated wheel in a clean
    environment.
 5. Confirm `openstatspec capabilities` reflects the intended support boundary.
-6. Confirm the release tag matches the package version and that CI, release
-   fixtures, and capabilities use OpenStatSpec specification release `v0.2.0`
-   at exact commit `79339ec3d8f8aa81789b7e85f6b8afa6f1374e50`.
+6. Confirm CI, release fixtures, and capabilities use the untagged OpenStatSpec
+   specification release candidate at exact commit
+   `e49252c00890aed76dcaabc5d1ab5121b45929db`, publish
+   `specification_status=release_candidate`, and leave
+   `specification_release` null. If that exact commit receives a stable tag
+   before this package is tagged, update the identity and re-run every gate.
 7. Review this document, the README, and CHANGELOG for accurate scope.
 
 The tag-triggered release workflow repeats the non-service test suite, builds

docs/transformations.md

@@ -4,8 +4,13 @@ OpenStatSpec separates transformation syntax, canonical meaning, and database
 mutation. This lets multiple language frontends produce the same plan without
 coupling the executor to any one language.
 
-The implemented frontend accepts a small SPSS-like subset: `RECODE`,
-`VARIABLE LABELS`, and `VALUE LABELS`. Stata and SAS are not implemented.
+The implemented bounded SPSS-like frontend accepts `RECODE`, sequential
+`COMPUTE` and `IF`, `VARIABLE LABELS`, `VALUE LABELS`, numeric `FORMATS`,
+`VARIABLE LEVEL`, and `EXECUTE`. Predicates support typed variable/literal
+operands, parentheses, numeric comparisons, `AND`, and `OR`. String comparison
+and v0.2 string assignment fail closed until exact profile-independent
+collation and explicit-width semantics are available; arbitrary SPSS, Python,
+and SQL expressions are rejected. Stata and SAS are not implemented.
 
 ## Architecture
 
@@ -106,9 +111,13 @@ result = openstatspec.apply_spss_in_place(
     dataset_id="responses",
     actor="agent@example.org",
     source_text="""
-      RECODE age (18 THRU 34 = 1) (35 THRU 64 = 2).
-      VARIABLE LABELS age 'Age group'.
-      VALUE LABELS age 1 '18-34' 2 '35-64'.
+      COMPUTE target = 0.
+      IF (source_a = 1 AND source_b = 1) target = 1.
+      VARIABLE LABELS target 'Example label'.
+      VALUE LABELS target 0 'No' 1 'Yes'.
+      FORMATS target (F1.0).
+      VARIABLE LEVEL target (NOMINAL).
+      EXECUTE.
     """,
 )
 ```
@@ -130,20 +139,25 @@ On Dolt, also pass `--expected-branch` and `--expected-head`.
 Every successful apply preserves the logical `dataset_id` and physical
 schema/table identity. It creates no derived dataset, output table, full-table
 copy, staging table, snapshot, rollback artifact, or recovery/history layer.
-Existing-target recodes use direct `UPDATE`; label operations mutate existing
-catalog rows.
-
-SQLite and PostgreSQL may add a numeric target where native transactions make
-the complete operation atomic. MySQL, MariaDB, and Dolt reject target-creating
-plans before the first mutation because implicit-commit DDL could leave a
-partial apply. Their target column and metadata must already exist.
-
-Dolt is the sole history, diff, branch, and rollback layer for Dolt-backed
-datasets. Before mutation, the executor verifies the expected branch and
-`HEAD` and requires clean `dolt_status`. Success changes the same working set
-without changing `HEAD`. OpenStatSpec does not call `DOLT_COMMIT`, switch
-branches, merge, reset, tag, or create a hidden recovery commit. The caller
-reviews `dolt diff` and separately decides whether to commit or restore.
+Assignments and recodes use ordered `UPDATE` statements; later operations see
+earlier results. Label, value-label, format, and measurement-level operations
+update both the normative and compatibility catalogs.
+
+A numeric create target is supported atomically on SQLite and PostgreSQL.
+MySQL, MariaDB, and Dolt reject `target_mode=create` before mutation. On those
+profiles a separate versioned stage must first provision the nullable numeric
+physical column and both catalog representations; the transformation executor
+then sees a pre-existing target and performs no schema DDL.
+The public operation reports success only after physical data, both metadata
+representations, and the compact audit row are mutually complete.
+
+Before Dolt mutation, the executor verifies the expected branch and `HEAD` and
+requires clean `dolt_status`. Success changes the same working set without
+changing `HEAD`; OpenStatSpec does not call `DOLT_COMMIT`, `DOLT_RESET`, switch
+branches, merge, tag, or create a hidden recovery commit. It rechecks branch,
+HEAD, and a clean working set after locking the dataset and immediately before
+mutation. The caller reviews a successful `dolt diff` and
+separately decides whether to commit or restore.
 
 ## Audit and provenance
 

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "openstatspec"
-version = "0.4.0"
+version = "0.5.0"
 description = "Reference adapter for the OpenStatSpec relational contract"
 readme = "README.md"
 requires-python = ">=3.11"

src/openstatspec/__init__.py

@@ -14,17 +14,24 @@
 from .sql.workflow import TransformationError
 from .frontends.spss import SpssFrontendCompilation, compile_spss_syntax
 from .transform import (
+    AssignOperation, BooleanExpression, ComparisonExpression,
+    ConditionalAssignOperation, ExecuteOperation, Operand, PredicateExpression,
     RecodeMatch, RecodeOperation, RecodeResult, RecodeRule,
-    ReplaceValueLabelsOperation, SetVariableLabelOperation,
+    ReplaceValueLabelsOperation, SetFormatOperation,
+    SetMeasurementLevelOperation, SetVariableLabelOperation,
     TransformationFrontendError, TransformationPlan, TypedValue, ValueLabel,
     VariableDefinition, VariableSchema, transformation_plan_from_dict,
 )
 
 __all__ = [
+    "AssignOperation", "BooleanExpression", "ComparisonExpression",
+    "ConditionalAssignOperation", "ExecuteOperation", "Operand",
+    "PredicateExpression",
     "CapabilityDeclaration", "LossReport", "SpssFrontendCompilation",
     "TransformationError", "TransformationFrontendError",
     "RecodeMatch", "RecodeOperation", "RecodeResult", "RecodeRule",
-    "ReplaceValueLabelsOperation", "SetVariableLabelOperation",
+    "ReplaceValueLabelsOperation", "SetFormatOperation",
+    "SetMeasurementLevelOperation", "SetVariableLabelOperation",
     "TransformationPlan", "TypedValue", "ValueLabel",
     "VariableDefinition", "VariableSchema", "transformation_plan_from_dict",
     "UnsupportedOperationError", "capabilities", "capability_matrix",

src/openstatspec/api.py

@@ -28,7 +28,8 @@
 )
 from .transform import TransformationPlan
 from .sql.capabilities import (
-    SPECIFICATION_COMMIT, SPECIFICATION_RELEASE, active_connection, catalog_binding,
+    SPECIFICATION_COMMIT, SPECIFICATION_RELEASE, SPECIFICATION_STATUS,
+    active_connection, catalog_binding,
 )
 
 
@@ -44,7 +45,7 @@ def capability_matrix(database_url: str | None = None) -> Mapping[str, Any]:
     """
     declaration = {
         "specification": "OpenStatSpec",
-        "specification_status": "released",
+        "specification_status": SPECIFICATION_STATUS,
         "specification_release": SPECIFICATION_RELEASE,
         "specification_commit": SPECIFICATION_COMMIT,
         "profile": "SPSS SAV/ZSAV 1.0",
@@ -145,7 +146,10 @@ def apply_spss_in_place(
     actor: str, expected_branch: str | None = None,
     expected_head: str | None = None,
 ) -> Mapping[str, Any]:
-    """Apply supported SPSS-like syntax to the same SQL dataset/table."""
+    """Apply bounded sequential SPSS syntax to one SQL dataset/table.
+
+    Supports typed COMPUTE/IF predicates and dictionary metadata operations.
+    """
     return result(_apply_spss_in_place(
         database_url=str(database_url),
         dataset_id=dataset_id,

src/openstatspec/cli.py

@@ -79,7 +79,7 @@ def main(argv: Sequence[str] | None = None) -> int:
 
     apply_spss = commands.add_parser(
         "apply-spss",
-        help="compile supported SPSS syntax and apply it in-place",
+        help="compile bounded sequential SPSS syntax and apply it in-place",
     )
     apply_spss.add_argument("--database-url", required=True)
     apply_spss.add_argument("--dataset-id", required=True)

src/openstatspec/frontends/spss/__init__.py

@@ -11,7 +11,7 @@
 )
 
 
-SPSS_FRONTEND_CONTRACT = "openstatspec-spss-syntax-frontend-v0.1"
+SPSS_FRONTEND_CONTRACT = "openstatspec-spss-syntax-frontend-v0.2"
 
 __all__ = [
     "SPSS_FRONTEND_CONTRACT",