Test MariaDB SAV round trips

Author: TonisOrmisson

.github/workflows/ci.yml

@@ -60,9 +60,22 @@ jobs:
         options: >-
           --health-cmd "mysqladmin ping -h localhost -uroot -proot"
           --health-interval 10s --health-timeout 5s --health-retries 10
+      mariadb:
+        image: mariadb:11.4
+        ports:
+          - 3307:3306
+        env:
+          MARIADB_DATABASE: openstatspec
+          MARIADB_USER: openstatspec
+          MARIADB_PASSWORD: openstatspec
+          MARIADB_ROOT_PASSWORD: root
+        options: >-
+          --health-cmd "mariadb-admin ping -h localhost -uroot -proot"
+          --health-interval 10s --health-timeout 5s --health-retries 10
     env:
       OPENSTATSPEC_POSTGRES_URL: postgresql+psycopg://openstatspec:openstatspec@localhost:5432/openstatspec
       OPENSTATSPEC_MYSQL_URL: mysql+pymysql://openstatspec:openstatspec@localhost:3306/openstatspec
+      OPENSTATSPEC_MARIADB_URL: mysql+pymysql://openstatspec:openstatspec@localhost:3307/openstatspec
     steps:
       - uses: actions/checkout@v7
       - uses: actions/setup-python@v7

README.md

@@ -41,8 +41,8 @@ openstatspec export --database-url postgresql+psycopg://... --dataset-id respons
 
 The adapter supports unencrypted SAV and ZSAV import and SAV/ZSAV export for
 the semantics exposed by its SPSS engine. SQLite is the local reference path.
-PostgreSQL and MySQL/MariaDB are also covered by service-backed CI conformance
-checks. Use these explicit SQLAlchemy URLs:
+PostgreSQL, MySQL, and MariaDB are each covered by separate service-backed CI
+conformance checks. Use these explicit SQLAlchemy URLs:
 
 - SQLite: `sqlite:///dataset.sqlite`
 - PostgreSQL: `postgresql+psycopg://user:password@host/database`

docs/release-readiness.md

@@ -17,9 +17,8 @@ validate that representation and export it back to SAV or ZSAV.
 | PostgreSQL | `postgresql+psycopg://…` | GitHub Actions PostgreSQL service |
 | MySQL/MariaDB | `mysql+pymysql://…` | GitHub Actions MySQL service |
 
-The MySQL service test establishes the shared MySQL/MariaDB profile contract.
-It is not a claim that every MariaDB release or configuration has separately
-been tested.
+Separate MySQL 8.4 and MariaDB 11.4 services test the shared MySQL/MariaDB
+profile contract. This does not claim coverage for every server configuration.
 
 ## Export-loss policy
 
@@ -36,8 +35,8 @@ machine-readable loss report with the export result.
 ## Maintainer checks before tagging
 
 1. Run `python -m pytest`.
-2. Run the service-backed profile checks with configured PostgreSQL and MySQL
-   URLs, or confirm the GitHub Actions SQL-services job is green.
+2. Run the service-backed profile checks with configured PostgreSQL, MySQL, and
+   MariaDB URLs, or confirm the GitHub Actions SQL-services job is green.
 3. Build with `python -m build` and install the generated wheel in a clean
    environment.
 4. Confirm `openstatspec capabilities` reflects the intended support boundary.

docs/sav-profile.md

@@ -15,11 +15,11 @@ kept as metadata and is restored to SAV on export.
 
 ## Verified SQL profiles
 
-SQLite has a local reference fixture. PostgreSQL and MySQL/MariaDB have
-service-backed conformance checks in GitHub Actions that import, validate, and
-export the same supported fixture. The MySQL service test verifies the shared
-MySQL/MariaDB profile contract; it does not claim separately tested coverage
-for every MariaDB release or server configuration. An import that exceeds a
+SQLite has a local reference fixture. PostgreSQL, MySQL, and MariaDB have
+separate service-backed conformance checks in GitHub Actions that import,
+validate, and export the same supported fixture. MySQL 8.4 and MariaDB 11.4
+exercise the shared profile contract; this does not claim coverage for every
+server configuration. An import that exceeds a
 target's strict single-table column limit fails before it creates a dataset.
 
 ## Explicit current boundaries

src/openstatspec/spss/sav.py

@@ -101,7 +101,19 @@ def export_sav_dataset(*, database_url: str, dataset_id: str, destination: str |
     dataset, variables, rows = read_wide_dataset(database_url=database_url, dataset_id=dataset_id)
     import pandas as pd
     frame = pd.DataFrame(
-        [{item["source_name"]: row[item["physical_name"]] for item in variables} for row in rows],
+        [
+            {
+                item["source_name"]: (
+                    None
+                    if row[item["physical_name"]] is None
+                    else float(row[item["physical_name"]])
+                )
+                if item["storage_kind"] == "numeric"
+                else row[item["physical_name"]]
+                for item in variables
+            }
+            for row in rows
+        ],
         columns=[item["source_name"] for item in variables],
     )
     labels = {item["source_name"]: item["label"] for item in variables if item["label"]}

tests/test_sql_services.py

@@ -1,4 +1,4 @@
-"""Real-service conformance checks for PostgreSQL and MySQL/MariaDB profiles."""
+"""Real-service conformance checks for PostgreSQL, MySQL, and MariaDB profiles."""
 
 import os
 
@@ -31,7 +31,11 @@ def source_sav(tmp_path):
 
 @pytest.mark.parametrize(
     ("environment_name", "dataset_id"),
-    [("OPENSTATSPEC_POSTGRES_URL", "profile_pg"), ("OPENSTATSPEC_MYSQL_URL", "profile_mysql")],
+    [
+        ("OPENSTATSPEC_POSTGRES_URL", "profile_pg"),
+        ("OPENSTATSPEC_MYSQL_URL", "profile_mysql"),
+        ("OPENSTATSPEC_MARIADB_URL", "profile_mariadb"),
+    ],
 )
 def test_live_profile_import_validate_and_export(environment_name, dataset_id, source_sav, tmp_path):
     database_url = os.environ.get(environment_name)