From fd3f6a93b9c7f2d7d8056298a02d3f9b362e9b3c Mon Sep 17 00:00:00 2001 From: Jeremy Date: Tue, 13 Oct 2026 14:29:38 +0000 Subject: [PATCH] docs: plan generic scientific projects and agent operations --- .../execution-ledger.md | 23 ++ .../generic-scientific-projects/issues.md | 14 + .../plans/generic-scientific-projects/plan.md | 276 ++++++++++++++++++ .../reports/design-evidence.md | 55 ++++ 4 files changed, 368 insertions(+) create mode 100644 docs/plans/generic-scientific-projects/execution-ledger.md create mode 100644 docs/plans/generic-scientific-projects/issues.md create mode 100644 docs/plans/generic-scientific-projects/plan.md create mode 100644 docs/plans/generic-scientific-projects/reports/design-evidence.md diff --git a/docs/plans/generic-scientific-projects/execution-ledger.md b/docs/plans/generic-scientific-projects/execution-ledger.md new file mode 100644 index 00000000..df0a172e --- /dev/null +++ b/docs/plans/generic-scientific-projects/execution-ledger.md @@ -0,0 +1,23 @@ +# Execution ledger + +## 2026-10-13 — planning-only follow-on + +Parent assignment: prepare and present a standards-compliant generic project/scientific data/optional producer/external recipe/agent interface plan; no runtime redesign, push, PR or merge. This entry records preparation, not implementation admission. + +Inspected the accepted dataset-centered workspace plan, dataset-workflows A8 and Rheon-v2 readiness plan/audit. Reused the preceding pinned-source lab and agent-interface review. Read the adopted standards template and relevant architecture/verification requirements. No runtime, schema, source repository or published branch changes were made. + +Prepared one follow-on plan with P1 project inspection as its sole next slice, seven staged UI milestones, explicit compatibility and shared actor/lifecycle owners, original-script scientific acceptance and separate environment gates. Added source evidence and issue dispositions. Historical plan states remain unchanged. + +Verification results and independent review disposition are appended after completion below. Product acceptance remains pending; no unit/integration/UI or scientific runtime tests are claimed by this documentation work. + +### Planning verification + +- Actual adopted `standards_verifier.checks.plan_contract.validate_plan`: PASS after supplying the full standards package path and making the state/identity probe explicit on its required line. The initial incomplete PYTHONPATH import failed, and the first successful validator invocation found that structural omission; both were corrected, not waived. +- Template and validator match the adopted `dcc56f26` revision in the available read-only standards checkout. +- Local Markdown links and code fences: PASS; documentation scope/whitespace checks recorded in final handoff. +- Independent `/root/mesh_review` review: no presentation blocker; refined physical array layout/custody and CPU-versus-GPU protocol qualification as requested. See [design evidence](reports/design-evidence.md). +- No product tests were appropriate for this documentation-only change. Every G1–G8 product gate remains pending. No commit, push, PR, merge or deployment occurred. + +## 2026-10-13 — publication and implementation admission + +User approved the presented plan and staged implementation (Sentinel_d7dcd8c1780481919cb7f76d7475632e). Operation: continue `docs/plans/generic-scientific-projects/plan.md`. Publish plan first, then P1 on a separate branch; no merge/deploy. Remote PR inventory still shows PR38–43 as open drafts; refreshed main remains the baseline pending inspection. Preserve their history and qualify this stack independently. diff --git a/docs/plans/generic-scientific-projects/issues.md b/docs/plans/generic-scientific-projects/issues.md new file mode 100644 index 00000000..04e23981 --- /dev/null +++ b/docs/plans/generic-scientific-projects/issues.md @@ -0,0 +1,14 @@ +# Issues and dispositions + +| ID | Severity / evidence | Objective relationship and owner | Disposition / required verification | Revisit trigger | +| --- | --- | --- | --- | --- | +| GP1 | High: producer-specific imports/UI in current core | Generic boundary; adapter/integration owner | P2/P4 retain legacy readers and move new generation to optional adapter; prove no-adapter generic startup and exact old release custody | Before any further producer-specific core change | +| GP2 | High: HTTP caller can claim `human_reviewed`; no actor authentication | Agent authority; access owner | P3 shared actor/approval boundary across all privileged transports; P6 parity/spoofing tests | Before agent mutation or external execution enablement | +| GP3 | High: existing process assumptions cover closed trainers, not arbitrary native descendants | Execution correctness; Runs owner | P3 actual native lifecycle probe and kernel-observed cancellation/restart gate; refuse unsupported lifecycle | Before lab generation admission | +| GP4 | Medium: existing volume is uint8; lab fields are floating staggered sequences | Scientific fidelity; data owner | P2 actual capture inventory and lossless cold reader; refuse lossy reuse | Before final schema limits/centering decisions | +| GP5 | High acceptance dependency: compatible runtime/native binary/captures not qualified here | Real scientific acceptance; lab owner | Inspect availability, then separately admitted real CPU evaluation and training; no fixture substitution | Before P2 positive capture gate and P4/P5 real execution | +| GP6 | Separate existing blocker: legacy actual-v2 frames unavailable locally through recorded transfer | Legacy-v2 qualification; prior plan owner | Keep prior gate open; no bypass or substitute data; this plan preserves compatibility | Supported original-byte retrieval succeeds | +| GP7 | Separate outstanding objective: positive actual model-assisted image/text path | Overall Tuldok goal; dataset-workflows owner | Preserve A8 as open; do not close it through scientific workflow acceptance | Actual supported inference runtime/model available | +| GP8 | Medium: authentication changes existing callers and historical attribution | Compatibility; access/integration owners | Inventory all mutation paths; explicit client migration, legacy-attribution labels and complete backup/restore | P3 contract review | + +No issue is marked resolved by writing this plan. GP6/GP7 remain with their existing plans; they do not require recreating those workstreams here. diff --git a/docs/plans/generic-scientific-projects/plan.md b/docs/plans/generic-scientific-projects/plan.md new file mode 100644 index 00000000..5ca5f860 --- /dev/null +++ b/docs/plans/generic-scientific-projects/plan.md @@ -0,0 +1,276 @@ +# Plan: Generic scientific projects and agent operations + +**Plan status:** `Planned` + +**Current phase:** Plan publication authorized; P1 implementation follows on a separate feature branch. + +**Next slice:** P1 — open a generic project and inspect referenced evidence without executing code. + +**Acceptance status:** `pending` + +**Execution ledger:** [execution-ledger.md](execution-ledger.md) + +**Issues:** [issues.md](issues.md) + +## Objective + +Open an existing scientific repository as a Tuldok project, acquire lossless data through an optional producer adapter, edit a reproducible generation/training/evaluation graph, and inspect durable results through compact Data, Pipeline and Runs workspaces. An agent can perform the same domain operations through explicit, permissioned interfaces. Rheon is one optional producer; Tuldok's project, data and execution contracts contain no required Rheon semantics. + +Use `MrScripty/rheon-surrogate-lab` as the concrete complete acceptance workflow. Wrap its proven generation, modal pressure fitting and autonomous rollout evaluation scripts before considering algorithm replacement. This plan does not promise unrestricted scientific software execution or arbitrary 3D model support. + +The user approved publishing this plan and staged implementation on 2026-10-13. Commit/push and draft PR creation are authorized; merge and deployment are not. Start P1 after plan publication and progress to later stages only after their prerequisite gates qualify. All product milestone gates remain pending. + +## Roadmap continuation + +- [Dataset-centered workspaces](../dataset-centered-workspaces/plan.md) remains the owner of the accepted bounded Data/Pipeline/Runs foundation. Reuse its PostgreSQL catalog, local artifacts, revisions, releases, family splits and sole Runs supervisor. Its general-3D-training exclusion is extended by this new, concrete external-recipe scope; its historical acceptance is unchanged. +- [Dataset workflows](../dataset-workflows/plan.md) still owns A8 positive real-model image/text candidate qualification. This plan neither closes nor replaces it. ASR and broader annotation work remain separate consumer decisions. +- [Rheon v2 consumer](../rheon-v2-consumer/plan.md) owns retained v1/v2 compatibility and the outstanding actual packet gate. Its imports are compatibility inputs, not the generic scientific format. Do not add further producer-specific core branches to implement this plan. +- Reviewed baseline: main `c522bf6b5eaf94d89811578c9e73aecfdba4ef9e`; PR38–43 are unmerged work, PR43 head `51bec9ea89f03357473aca0d40f2fa8251598a25`. Main executes three CPU classification recipes. Drafts add spatial integrity/export recipes, not 3D training. Reconcile branch/PR state again before implementation; no draft feature is assumed deployed. +- Lab source reviewed: `e8a90bfef6b2955c502008ccc62443e29dd9f2ec`. Its current model fits 32,767 modal gains from 96 reference frames and evaluates independent 32³, 120-frame trajectories. Historical CNN/warm-start experiments are not the current acceptance target. [Source inventory and qualification](reports/design-evidence.md) records the relevant owners. + +## Objective Acceptance + +| ID | Observable criterion | Kind | Environment | Mode | Status | Evidence | +| --- | --- | --- | --- | --- | --- | --- | +| G1 | Open/reopen a local project, inspect available/missing artifacts and runtime requirements, navigate all three workspaces without executing or modifying repository code | user-workflow | representative | automated | pending | P1 browser and subprocess/filesystem observation | +| G2 | Scientific fields preserve exact float bytes, dimensions, staggered placement, units, time and trajectory identity through import, freeze, download and cold read | contract | representative | automated | pending | P2 independent NumPy reader and actual lab samples | +| G3 | An optional Rheon adapter exposes bounded generation controls and produces real generic scientific data; removing it leaves generic project/data use intact | integration | required-real | automated | pending | P4 actual native generator plus no-adapter startup | +| G4 | Saved graph executes existing lab fitting and autonomous evaluation scripts; verified checkpoint, full-field comparisons and physics diagnostics survive restart with complete provenance | system | required-real | automated | pending | P5 lab verifier, numerical field comparison and new execution receipts | +| G5 | Agent and UI clients use the same operations; stale requests, forged human review, unauthorized execution, path escape and approval reuse fail before side effects | contract | representative | automated | pending | P3/P6 public-interface denial and parity tests | +| G6 | Admission is idempotent; cancel, crash and restart never duplicate work, publish partial success or silently leave descendants running | system | representative | automated | pending | P3 process-tree, storage-failure and lost-acknowledgement tests | +| G7 | Existing imports/releases/runs retain identities and bytes, including Rheon v1/v2; explicit migration and complete PostgreSQL/artifact restore preserve old and new records | integration | representative | automated | pending | P2/P7 retained-history and restore workflow | +| G8 | Desktop/narrow/keyboard UI supports open → generate → review/freeze → edit graph → train/evaluate → inspect results without explanatory clutter or false qualification labels | user-workflow | required-real | either | pending | P7 reviewed screenshots and actual-run browser receipt | + +## Scope + +### In Scope + +Generic local project manifests and opening; lossless structured-grid field sequences; optional producer scripts with parameter forms; externally implemented train/evaluate recipes; domain API discovery and an agent transport over existing owners; actor permissions, provenance, cancellation, compatibility and end-to-end qualification. + +### Out Of Scope + +Distributed scheduling, remote hosting, plugin marketplace, arbitrary shell graph nodes, automatic package/model installation, algorithm replacement, arbitrary pickle loading in the server, new physics, general mesh solvers, ASR, model-quality guarantees, or reproducing historical GPU speedups on different hardware. No existing owner work, historical outputs or source repositories are rewritten. This publication slice edits planning documents only; subsequent feature branches implement the admitted milestones. + +## Constraints And Assumptions + +### Constraints + +- Adopted standards remain Coding-Standards `dcc56f26e884ade260770beceba2501d3746200d`; use its template and architecture/contract/verification policies. No Standards Engine certificate is claimed. +- PostgreSQL owns mutable metadata and durable admission; ordinary local files own large immutable payloads. Project files describe references, not database credentials, approvals or mutable run truth. +- Maintain runnable UI checkpoints. Keep the current plain JavaScript/Python application unless a concrete observed limitation requires replanning. Do not introduce a second graph engine or scheduler. +- New major implementation milestones use separate feature branches and independent review under the parent task's publication rules, after this planning-only restriction is lifted. Existing PR32/33 and PR38–43 remain separately owned. + +### Assumptions + +- Lab scripts can run behind a declared process protocol without algorithm changes. Execution owner must first run a bounded real subprocess/cancellation probe; incompatible descendant/runtime behavior stops P3/P4 expansion. +- A structured-grid contract covers this lab; unstructured/adaptive grids are unsupported initially. Data owner must compare actual named field arrays and coordinate conventions before fixing the schema. +- The recorded CPU evaluation path offers a useful qualification target. It still requires the recorded compatible PyTorch runtime, native Rheon build and input artifacts; source availability alone proves none of these. + +## Binding Decisions + +| Decision | Owner | Evidence | Supersedes | +| --- | --- | --- | --- | +| Project manifest references data, recipe and environment identities; those owners remain independent | Project owner | Separate revision and execution lifetimes | Treating `--data` as an arbitrary project opener | +| Generic scientific field sequences use exact typed arrays; no uint8 conversion of float scientific data | Data owner | Current volume format cannot represent lab arrays losslessly | Reusing bounded uint8 volumes as training tensors | +| Producer controls, physical validation and translation live in optional adapter packages | Adapter owner | User's agnostic boundary; current hard-coded Rheon paths | Extending producer-specific core parsing/UI | +| Existing Runs admits all external work through one lifecycle; recipe definitions name reviewed executables, never arbitrary shell strings | Runs owner | Existing queue, idempotency and publication guarantees | A second script scheduler or bypass execution endpoint | +| Lab scripts own algorithms and scientific acceptance; Tuldok owns custody, process lifecycle and presentation | Lab/integration owners | Existing modal fitting and rollout verification | Reimplementation of model/physics inside Tuldok | +| Permission enforcement is common to HTTP, UI and agent calls | Access owner | Current review field lacks actor authentication | Treating validation tokens or claimed `human_reviewed` as authorization | + +## Proposed contracts + +### Project opening + +Propose `tuldok.project.json` schema 1: stable project identifier, display name, dataset/release references, recipe-definition references, runtime requirements and relative artifact/source locations. Each referenced definition has its own version and identity; the project schema does not version models, arrays and permissions together. Store user-local root bindings and credentials outside the portable manifest. Existing repositories can use an external manifest without adding files to their checkout. + +Opening resolves bounded paths beneath explicitly selected roots, rejects traversal/symlink escape, and records a manifest snapshot in project context. It performs no imports of project Python, executable discovery hooks, dependency installation or code execution. Missing data/runtime is visible and nonfatal to inspection. Existing catalog data can be linked explicitly; opening a second project must not silently reassign dataset ownership. Imported historical receipts are labelled external evidence, never Tuldok-completed runs. + +### Scientific field/sequence data + +Propose `scientific_field_sequence_v1`: a bounded JSON manifest plus named non-object NPY arrays, initially one uncompressed NPY member per named field per frame, with bounded streaming reads rather than a second chunked-storage format. Confirm this layout against actual captures before implementation; an unsuitable layout triggers replanning. Require explicit dtype/endianness, shape, axis order, physical units, grid origin/spacing/orientation, sample centering (cell or each face axis), time coordinates, boundary declarations where supplied, field role and byte hash. Initially support float32/float64 and only additional numeric types demonstrated necessary by actual lab samples; forbid object arrays and silent precision changes. Define finite-value policy per field and refuse undeclared nonfinite values. + +Preserve original NPZ or other producer files unchanged as source provenance. Conversion into NPY preserves exact dtype and element bytes in declared order, not the original container bytes; each new member has its own hash. Already conforming NPY members remain byte-identical. Views may downsample with a visible sampling indicator. A generic array/geometry reader validates structure; the producer adapter owns physics, acceptable control ranges and solver diagnostics. Family IDs bind whole trajectories and initial-condition/source relationships; time samples cannot silently become independent train/test records. Bounds cover headers, dimensions, total bytes, frame count and decompression (if later admitted); determine concrete limits from actual samples before P2 implementation. Avoid whole-trajectory JSON/base64 bodies. + +An adapter can retain a source bundle as opaque provenance. Conversion creates a new generic asset with explicit source links and conversion identity; it never rewrites the old asset, release digest or human-review history. Keep old readers for existing frozen releases. Move new producer registration outside core; separately test retained legacy compatibility and generic startup with no optional adapter installed. + +### Producer and execution definitions + +An explicitly enabled local adapter supplies declarative parameter definitions, input/output type references, execution entrypoint identity and runtime requirements. The UI renders compact forms from supported scalar/enum/range/file-reference fields; unknown form constructs fail visibly. No mandatory custom JavaScript or arbitrary executable UI extensions. + +The common recipe process protocol declares a trusted executable plus argument vector, source/runtime hashes, exact inputs, bounded resource/time requests, output roles and structured progress/result events. No shell interpolation or execution on opening/validation. Generation, training and evaluation are stages in the existing graph/Run lifecycle; do not implement all possible DAG semantics before this actual three-stage use case. Freeze definition versions and dependency/source manifests, not just wrapper hashes. Inputs are read-only and outputs go to private run staging; validate output manifests and bytes before atomic publication. + +External subprocess isolation is not a security sandbox. Only explicitly trusted local code executes. Record filesystem/network permissions, permitted environment variables and resource limits; omit secrets from manifests/events. Reject unsupported lifecycle requirements. Before admitting lab work, demonstrate ownership and cessation of its Python and Rust descendants, including broken pipes and restart ambiguity. If current process groups cannot contain its descendants, choose and qualify a bounded OS containment mechanism before claiming G6; never infer native cessation from transport cancellation. + +### Agent operations and approval + +Keep domain owners canonical. Publish versioned schemas and operation discovery for project inspect/open, dataset query/import, review proposals, release preview/freeze, graph create/save/validate, run submit/status/cancel/retry, evidence download and test exposure. Generate transport projections from those contracts where practical; do not duplicate validation in MCP or UI. Expose structured errors, capability limitations, revision conflicts and durable idempotency receipts. A thin local MCP transport is the first agent convenience surface after shared HTTP/domain semantics are qualified; no separate agent database or queue. + +Introduce authenticated actor identities and scoped local credentials before new agent mutations/external execution. Distinguish inspect, edit/propose, execute, approve-review, expose-holdout and adapter-enable permissions. Agent credentials cannot assert a human actor or set `human_reviewed`. Human approval binds exact record/source revisions; execution approval binds project, graph/recipe/source/input/runtime identities, requested resources and permissions. Changed material inputs invalidate approval. Reconciliation of an already admitted request remains idempotent, not a new approval demand. Cancellation can remain an explicitly scoped operational permission. + +Enforce at the service boundary reached by every transport, not only at MCP. Audit actor, action, target revision/digest, approval reference and result without secrets. Bootstrap a local owner session explicitly; local loopback/same-origin alone is not identity. Preserve historical review entries as legacy attribution without inventing reviewer identities. Credential issuance/revocation and approvals are local authority, not portable project content. Before release, assess existing HTTP callers and migrate them explicitly rather than silently breaking or leaving an unprotected bypass. + +## Design Observations And Remaining Evidence + +| Assumption or claim | Decision affected | Owner | Executable procedure | Required environment | Status/evidence | +| --- | --- | --- | --- | --- | --- | +| Generic project inspection requires no lab runtime | P1 independence | Project owner | Open external manifest with missing runtime/data; observe no children or source writes | Local app/PostgreSQL/browser | Source-supported; product test pending | +| Full lab arrays fit the proposed contract | P2 schema and limits | Data owner | Read representative actual NPZ captures in a separate process; inventory dtype/shape/centering/time and round-trip exact arrays | Actual lab capture bytes + NumPy | Missing bulk captures may block positive acceptance | +| Native process can be controlled by Runs | P3 execution boundary | Runs owner | Start bounded native probe, cancel during request/output, close pipe, kill supervisor, reconcile descendants | Linux, compatible Rust/Rheon binary | Not executed in planning | +| CPU reproduction is available | P5 acceptance runtime | Lab owner | Inspect installed interpreter/package versions; load hash-verified checkpoint in isolated worker; execute documented CPU control | PyTorch runtime + native build + required inputs | Not qualified; do not auto-install | +| Prior performance numbers transfer | No design depends on this | Qualification owner | Separate matched-hardware benchmark with recorded protocol | Recorded GPU/CPU hardware | Not claimed | + +## Evidence And Oracle Plan + +| Claim | Domain | Deciding oracle | Independent authority | Unsupported domain | Intended negative failure | +| --- | --- | --- | --- | --- | --- | +| G2/G7 | Byte and geometry custody | Cold NumPy load plus original byte/array/coordinate comparisons; retained frozen archive hashes | NumPy reader and original lab arrays | Arbitrary tensor codecs/unstructured grids | Dtype, centering, shape/hash mismatch refuses before publication | +| G4 | Scientific behavior | Existing lab verification script plus independently recomputed full-field errors and recorded thresholds | Lab frozen protocol/native reference implementation | New physics or general model quality | Corrupt checkpoint, wrong seed/split or failed divergence/residual gate yields failed evaluation | +| G5 | Actor authority | Public HTTP and MCP requests using distinct owner/agent credentials | Permission contract and real persisted audit rows | Remote multi-tenant security | Forged identity or stale approval returns denial with no write/spawn | +| G6 | Lifecycle | OS-observed process tree and cold catalog reopen after faults | Kernel process identity plus durable receipts | Uncontained daemons/remote workers | Ambiguous cessation stays unknown and fences execution | +| G8 | User workflow | Actual browser actions and downloaded output read outside Tuldok | Real lab output + existing browser harness | Mocks as real-model acceptance | Missing dependencies remain explicit blockers | + +The lab verifier expects its own frozen schemas and paths. Keep lab-native artifacts alongside generic projections and run its unchanged verifier on those originals; adapters must not rewrite historical hashes to make checks pass. New runs freeze new source/runtime identities and label historically exposed test seeds as exposed. Hash equality proves custody, not scientific validity or source authenticity. + +## Systemic Finding Audit + +- Invariant family and canonical owner: producer-specific knowledge belongs to optional adapters; authorization belongs to shared service admission. +- Bounded authority, representation, and reachable consumer population: sequence import/batch routes, validators, immutable metadata, release reader, graph recipe selection, controls UI; review/save/release/run endpoints and future agent transport. +- Expansion facts: scientific data and external subprocesses introduce new public/persisted contracts and permission requirements, requiring explicit owners rather than additional Rheon cases. +- Consumer dispositions: retain old readers/identities; new acquisition uses adapter conversion; generic views consume generic summaries; all privileged transports converge on shared authorization; legacy clients receive explicit migration handling. +- Deletion, consolidation, smaller-Interface, stronger-proof, and evidence-replacement alternatives: reuse existing owners and queue; remove hard-coded new producer UI paths; prefer declarative forms and a bounded executable protocol over a plugin framework, duplicate verifier or general shell graph. +- Evidence-backed stopping condition: every reachable listed consumer has compatibility or migration tests, a second producer-neutral fixture proves core independence, and no generic owner imports the optional Rheon package. The fixture proves separation only, not a second real producer qualification. +- Repaired-composition comparison: preserves Data/Pipeline/Runs while moving producer semantics out and adding only the project, scientific-data, executable-contract and access responsibilities required by the clarified goal. + +## Simplicity And Ownership Review + +**Applicability:** `applicable` + +- Independent concepts and dimensions: project references organize work; scientific assets describe values/coordinates; adapters create/translate producer data; recipes declare executable operations; Runs owns when/process/results; access policy owns who may act. +- State, identity, value, time, policy, and mechanism: Keep mutable project bindings, immutable scientific values, execution identities and actor grants with their separate owners; the references below define their composition. + - Canonical authority scope and referenced authorities: project references dataset/release/recipe/runtime owners; it cannot grant review, rewrite artifacts or own scheduler state. + - Version roles and owned promises: independent project schema, data schema, recipe definitions and adapter/source versions; approval binds material identities without becoming a format version. + - Supported compatibility overlaps and consumer matrix: existing classifiers and old Rheon/mesh/point/volume readers remain; new science reader and optional adapter coexist; UI/HTTP/MCP project common domain results. + - Material identity-invalidation effects: source/parameters/runtime/input changes create new execution identity; approval refresh required; cosmetic layout changes follow existing execution-digest rules; old results remain immutable. +- Caller and composition-root knowledge: callers supply domain IDs, expected revisions and typed parameters; they do not calculate trajectory families, infer physics or manage PIDs. Composition root registers explicitly trusted adapter/recipe definitions. +- Representative change paths and forced owners: new Rheon control changes adapter/form schema/tests; new generic field representation changes data reader/views/contracts; new lab algorithm changes lab recipe source identity; permission change updates one access owner and transport parity tests. +- Stable Interfaces versus hidden knowledge: bounded manifests, artifact references, typed events and domain errors cross boundaries; raw SQL, private paths, provider command syntax and mutable subprocess handles do not. +- Independent evolution, testing, failure, and replacement: absent adapter/runtime leaves project/data inspection usable; adapter failures cannot publish successful assets; data conversion and original scientific verification remain independently testable. +- Necessary complexity and containment: preserve floating-point geometry and process/approval identity because they affect real outcomes; keep physics in lab, persistence in existing owners, and permissions in one boundary. +- Deletion and cumulative machinery result: deleting project owner spreads reference resolution into callers; deleting data contract loses scientific meaning; deleting process contract spreads lifecycle assumptions; deleting access owner leaves bypasses. A second scheduler, arbitrary UI-plugin system, duplicated model trainer and agent-specific database are unnecessary and excluded. + +## Milestones + +All statuses below describe proposed work under the planning-only boundary. Before starting each milestone, confirm its exact write set against current source, route relevant language/security/persistence standards, and record semantic changes to this plan. + +### P1: Open and inspect a project + +**Goal:** Runnable compact project selector/header and existing three-workspace navigation, with honest external evidence and missing-dependency states. + +**Allowed write set:** proposed `projects.py`, `docs/contracts/projects.md`, `tests/test_projects.py`, `tests/browser_workspace_project.cjs`; existing `app.py`, `metadata.py`, `static/workspace-real.js`, `static/workspaces.js`, `static/workspaces.css`, `static/workspaces.html`, `static/workspace-service.js`, workspace contract tests; this plan directory. Project registration in existing metadata requires explicit upgrade/restore coverage; do not introduce sidecar database authority. + +**Tasks:** Define manifest/reference boundaries; add read-only validation/inspection and explicit registration/opening; display project context and externally supplied receipt metadata as unverified until checked; preserve ordinary dataset-only entry; handle missing runtime/artifacts without execution. + +**Acceptance gate:** G1; real desktop/narrow/keyboard walkthrough; no spawned project code, no source mutations, traversal/symlink rejection, cross-project isolation and restart; existing workspace navigation regressions pass. First slice does not parse arbitrary checkpoints or claim scientific results verified. + +**Status:** `Planned` + +### P2: Lossless scientific data and retained compatibility + +**Goal:** Inspect a real lab field/frame sequence in Data, review/freeze it and download an exactly recoverable generic release. + +**Allowed write set:** proposed `scientific_fields.py`, `scientific_release.py`, `docs/contracts/scientific-fields.md`, `tests/test_scientific_fields.py`, `tests/test_scientific_release.py`, `static/scientific-fields.js`; existing immutable asset, Workbench, release, metadata/upgrade/backup owners, `app.py`, workspace UI and affected tests; scoped compatibility modules identified before edits. + +**Tasks:** Acquire actual sample evidence; fix bounds/layout contract; import via bounded file/stream interface; frame/field/slice viewer with units and sampling disclosure; preserve whole-trajectory families; explicit conversion links; qualify retained v1/v2 readers and schema upgrade. + +**Acceptance gate:** G2 and data portion of G7; independent cold read, exact bytes, geometry/time checks, malformed/oversize inputs, restore and retained-history tests. Actual sample unavailability leaves G2 pending while negative fixtures can proceed. + +**Status:** `Planned` + +### P3: Authorized external recipe execution + +**Goal:** Pipeline preflight and Runs execute/cancel a bounded real local process using the shared permission and lifecycle boundary. + +**Allowed write set:** proposed `access.py`, `recipe_process.py`, `docs/contracts/actors.md`, `docs/contracts/external-recipes.md`, corresponding unit/integration tests; existing `pipeline.py`, `runs.py`, `app.py`, metadata and artifact owners, all affected mutation endpoints, workspace UI/tests. Enumerate the complete privileged endpoint population before implementation. + +**Tasks:** Owner session and agent credential scopes; domain admission checks; bound approvals to material inputs; structured process protocol; execution preflight; descendant/restart qualification; update existing callers without bypass; expose queued/running/cancelling/failed/unknown states and bounded structured diagnostics. + +**Acceptance gate:** G5 shared service portion and G6. Use a real bounded executable for lifecycle tests, clearly labelled non-scientific; actual native lab probe decides admission of P4. No false success after cancellation or partial outputs; stale authorization and forged review denied across direct HTTP and UI. + +**Status:** `Planned` + +### P4: Optional Rheon generation controls + +**Goal:** Enable an optional adapter, edit supported parameters in Pipeline, run native generation, and inspect generic output in Data. + +**Allowed write set:** proposed optional `adapters/rheon/` package (or separately distributed equivalent selected before implementation), its contract/fixtures/tests, generic registration/form renderer within existing owners, relevant browser test. Rheon/lab repositories remain read-only unless separately admitted; wrapper files live in the optional package. + +**Tasks:** Wrap proven native generation scripts; disclose supported grid/physics/control bounds; capture source/binary/command/runtime/seeds; map fields without quantization; retain original diagnostics and raw outputs; register generic data only after complete validation. + +**Acceptance gate:** G3 and P4 UI path. Real generated arrays pass cold generic and producer readers; invalid controls fail before spawn; removal of optional package leaves generic features operational. No fabricated native output or silent re-use of PR32's different fixed profile. + +**Status:** `Planned` + +### P5: Surrogate training and autonomous evaluation + +**Goal:** Existing lab algorithms run through an edited graph, publishing checkpoint, scientific metrics, field comparisons and reproducibility receipts in Runs. + +**Allowed write set:** optional lab wrapper/recipe definitions under the P4 package, result-role projection in existing Runs/UI owners, scientific view reuse, `docs/contracts/scientific-results.md`, targeted integration/browser tests and evidence reports. No algorithm source replacement. + +**Tasks:** First evaluate an existing hash-verified checkpoint in a fresh output directory with exposed-test labels; then generate independent train/validation trajectories, fit the modal model using the existing script, freeze configuration and evaluate autonomous rollouts. Preserve lab-native artifacts for its verifier; link generic field projections as derived outputs. Show units/threshold failures, per-frame errors, baseline comparisons and provenance rather than classification metrics. + +**Acceptance gate:** G4 with real runtime/native solver and actual new outputs, protocol-specific lab verification (CPU correctness separated from CUDA/resource/timing claims as detailed in the evidence report), full-field independent checks, no teacher states in learned rollout, seed/family separation, checkpoint/runtime identity and restart inspection. CPU acceptance is distinct from CUDA acceptance and historical speedup reproduction. Missing runtime or bulk inputs is a named blocker, never substituted with fixtures. + +**Status:** `Planned` + +### P6: Agent domain interface + +**Goal:** An agent discovers and controls the qualified workflow through typed domain operations with the same outcomes as the UI. + +**Allowed write set:** proposed `agent_api.py`, `docs/contracts/agent-operations.md`, versioned operation schemas and a thin local MCP adapter selected/pinned after transport review; existing API/access/domain owners only as needed; public HTTP/MCP integration tests and developer usage documentation. + +**Tasks:** Project/data/recipe discovery; queries and typed mutation tools; release preview/freeze; graph editing/preflight; idempotent submit/status/cancel/retry; provenance/artifact access; scoped credential setup/revocation. Tool descriptions identify side effects and approval requirements. Reuse shared validation/errors; no direct DB or file-management tools masquerading as domain operations. + +**Acceptance gate:** Complete G5; equivalent agent/UI workflow yields equivalent domain evidence; revision conflicts, lost acknowledgements, revoked credentials and human-review spoofing are exercised through both public transports. No new actor-specific privilege bypass. + +**Status:** `Planned` + +### P7: Complete workflow and compatibility qualification + +**Goal:** Deliver an installable local workflow with preserved old data, explicit limitations and meaningful real scientific acceptance. + +**Allowed write set:** existing CI configuration, deployment/dependency documentation, existing backup/upgrade and browser acceptance tests, scoped packaging needed by P1–P6, README and this plan's reports. Runtime fixes remain in the owning milestone's boundary and receive affected re-review. + +**Tasks:** Fresh setup/reopen, full PostgreSQL/artifact restore, old+new results, offline operation after installation, full actual browser workflow, all relevant lint/static/type/unit/integration/UI checks supported by chosen technologies; independent architecture/security/scientific review and applicable hosted CI once publication is authorized. + +**Acceptance gate:** G1–G8 satisfied with linked evidence; no unqualified hardware/performance/model claims; all material review findings resolved. Separate outstanding dataset-workflows A8 and actual legacy-v2 packet qualification remain visible rather than hidden by this plan's completion. + +**Status:** `Planned` + +## Blockers + +- No implementation-authorization blocker remains. Publish the plan first; qualify prerequisite gates before advancing stages. +- Real sample captures, compatible native binaries/PyTorch runtime and actual execution permissions must be verified before dependent scientific acceptance. The lab's ignored bulk `runs/` artifacts are not supplied merely by its Git repository. +- Legacy actual-v2 packet retrieval remains a separate recorded blocker. Current planning does not retry or bypass that transfer. +- Standards Engine is unavailable in the exposed tools; source-based structural/content checks are available and do not imply certification. + +## Re-Plan Triggers + +- Actual lab arrays require unstructured grids, unsupported values or materially different storage/streaming; do not squeeze them into a lossy schema. +- The native process cannot be contained/cancelled under the admitted lifecycle; no external execution until containment is qualified. +- Source/runtime pinning cannot reproduce the lab protocol, or script wrappers would change algorithms/held-out selection; separate integration from scientific modification. +- An existing API path bypasses actor policy, or authentication migration would silently break an admitted client; revise the full boundary rather than patch MCP alone. +- Main/PR state, the user's desired controls, or a new dependency changes ownership, migration or acceptance. Update this plan before dependent work. + +## Concurrent Work + +Current planning is serial. After contract decisions, independent UI, adapter and read-only review work may proceed with disjoint assigned write sets. Core data schemas, access policy, metadata, Runs, shared tests and this plan have one serial integration owner. Do not spawn competing implementations of those owners. + +## Repository Isolation + +Plan publication uses `plan/generic-scientific-projects-20261013` from PR43 in the existing `/workspace/Tuldok-v2` worktree; P1 uses a separate feature branch from that plan. Root owns integration. Both are retained for draft review until separately approved integration or explicit supersession; no owner branch/worktree cleanup is authorized. Future milestone branches isolate reviewable product changes after implementation admission. Refresh main and reconcile PR38–43 before choosing each base; do not assume merge authority. Retain owner branches and historical evidence. + +## Final Acceptance + +- Acceptance status: `pending` +- Deferred follow-ups: broader ASR, arbitrary scientific models/physics, distributed execution and GPU performance reproduction; each requires a separate concrete consumer/qualification decision. +- Final status: `Planned` diff --git a/docs/plans/generic-scientific-projects/reports/design-evidence.md b/docs/plans/generic-scientific-projects/reports/design-evidence.md new file mode 100644 index 00000000..480dbad0 --- /dev/null +++ b/docs/plans/generic-scientific-projects/reports/design-evidence.md @@ -0,0 +1,55 @@ +# Design evidence and qualification limits + +## Inspected sources + +Read-only review on 2026-10-13 established these source facts. No scientific execution was performed during that review or this planning assignment. + +| Source | Observation and consequence | +| --- | --- | +| [Tuldok main pipeline](https://github.com/MrScripty/Tuldok/blob/c522bf6b5eaf94d89811578c9e73aecfdba4ef9e/pipeline.py) | Three closed CPU classification recipes; reusable graph ownership, not arbitrary train/evaluate execution | +| [Draft pipeline](https://github.com/MrScripty/Tuldok/blob/51bec9ea89f03357473aca0d40f2fa8251598a25/pipeline.py) | Adds integrity/export stages for spatial assets; does not train scientific models | +| [Draft Rheon import](https://github.com/MrScripty/Tuldok/blob/51bec9ea89f03357473aca0d40f2fa8251598a25/rheon_sequences.py) | Statically coupled fixed-profile v1/v2 parsing, not a generic producer interface | +| [Volume owner](https://github.com/MrScripty/Tuldok/blob/51bec9ea89f03357473aca0d40f2fa8251598a25/volume_assets.py) | Bounded uint8 NPY volumes cannot losslessly represent floating scientific fields | +| [HTTP handlers](https://github.com/MrScripty/Tuldok/blob/51bec9ea89f03357473aca0d40f2fa8251598a25/app.py) | Useful existing domain operations; localhost/same-origin guard is not authenticated human/agent identity; `--data` opens managed storage | +| [Pipeline contract](../../../contracts/workspace-pipeline.md) / [Runs contract](../../../contracts/workspace-runs.md) | Reuse revision checks, idempotent admission, process identity, artifact custody and holdout exposure; broaden only where external execution demands it | +| [Current lab experiment](https://github.com/MrScripty/rheon-surrogate-lab/blob/e8a90bfef6b2955c502008ccc62443e29dd9f2ec/experiments/trajectory-v2/README.md) | 32³ trajectory model, 12 training trajectories × 8 frames, four 120-frame test rollouts; native simulation outside learned pressure solve; CPU control and GPU results are separately recorded | +| [Modal model](https://github.com/MrScripty/rheon-surrogate-lab/blob/e8a90bfef6b2955c502008ccc62443e29dd9f2ec/scripts/trajectory_model.py) | Fits 32,767 gains by supervised modal least squares; no new architecture needed for initial integration | +| [Experiment driver](https://github.com/MrScripty/rheon-surrogate-lab/blob/e8a90bfef6b2955c502008ccc62443e29dd9f2ec/scripts/trajectory_experiment.py) | Native subprocess interaction, training, freeze, checkpoint and rollout artifacts remain lab-owned | +| [CPU control](https://github.com/MrScripty/rheon-surrogate-lab/blob/e8a90bfef6b2955c502008ccc62443e29dd9f2ec/scripts/trajectory_cpu.py) | Existing same-thread CPU evaluation path; requires actual compatible runtime and native binary, not merely Python installed | +| [Verifier](https://github.com/MrScripty/rheon-surrogate-lab/blob/e8a90bfef6b2955c502008ccc62443e29dd9f2ec/scripts/verify_trajectory_results.py) | Checks original artifacts/hashes, disjoint seeds, memory, full-field errors, physics gates and timing summaries; preserve native artifact layout for this oracle | +| [Pumas runtime launcher](https://github.com/MrScripty/rheon-surrogate-lab/blob/e8a90bfef6b2955c502008ccc62443e29dd9f2ec/scripts/pumas_trajectory.py) | Selects managed PyTorch 2.14 environment; this is a training Python environment, not Tuldok's Pumas inference API | + +Repository inspection found no Tuldok MCP server/OpenAPI catalog or general project opener. HTTP APIs and documented Python domain owners are the existing integration base. This is a scoped source finding, not a promise that undocumented external integrations do not exist. + +## Roadmap dispositions + +The [prior readiness audit](../../rheon-v2-consumer/reports/readiness-20261013.md) already separates bounded accepted workspace work from the persistent overall goal. Preserve that distinction. This plan adds concrete scientific execution and agent authority scope; it does not reopen every historical plan, certify PR43's missing actual producer packet, or count controlled-provider tests as dataset-workflows A8 real-model success. + +Legacy actual v2 qualification needs original recorded frames. The newer lab workflow uses different dimensions, trajectory lengths and artifact schemas. Neither should be passed off as the other's acceptance evidence. Missing bulk lab captures should be reported; a deliberately admitted new bounded generation run may provide new evidence, but cannot recreate a historical receipt or untouched test set. + +## Environment procedure before scientific implementation + +1. Inventory configured local Python/PyTorch/NumPy and Rust toolchains without installing or changing a managed runtime; identify actual device availability. +2. Resolve lab source, its required Rheon revision, native executable identity and tracked checkpoint; separately locate required ignored capture files. Missing inputs remain missing. +3. Run the smallest admitted native subprocess handshake/cancellation probe to settle process ownership before broader execution design. Record descendant and pipe behavior. +4. Inspect a real capture's arrays in a cold reader to settle schema, centering and storage bounds before P2 freezes the wire format. +5. Execute new CPU evaluation in a separate output directory using existing checkpoint and recorded protocol; preserve exposure labels and new provenance. Then qualify a new training/evaluation run. +6. Run GPU/performance qualification only if actual requested hardware/runtime exists. CPU behavior, fixture lifecycle and old published benchmarks cannot satisfy it. + +## Standards basis + +Read the adopted [plan template](https://github.com/MrScripty/Coding-Standards/blob/dcc56f26e884ade260770beceba2501d3746200d/templates/PLAN-TEMPLATE.md), Planning, Architecture and Verification sources, and existing repository verification records. Template and plan-validator files match the available standards checkout. Route future implementation through actual Python/JavaScript, contracts, persistence, process/concurrency, security and accessibility policies when its slice is admitted; this planning document does not certify unwritten code. + +No `AGENTS.md` or `.agents/skills` was present in the inspected Tuldok worktree; workspace `.agents` and `.codex` directories were empty. No Standards Engine route/validation tool is exposed. Source-based checks are recorded in the ledger; no Engine result is fabricated. + +## Scientific acceptance protocol to freeze before P5 + +The reviewed trajectory driver uses training seeds 100–111 (eight reference frames each), validation seeds 500–501 (60 frames), and test seeds 900–903 (120 frames). Those test seeds are historically exposed: replaying them is reproduction/correctness evidence, never a new untouched generalization claim. Do not silently choose fresh seeds to improve an acceptance score. Any new holdout protocol must be selected and frozen before inspection and is a separate scientific claim. + +Retain the recorded sealed uniform 32³ box, density 1.25 kg/m³ and time step 1/60, forcing/tracer conventions and gauge. The pinned verifier checks divergence at most 1e-5, residual at most 1e-8, Courant at most 1, and zero learned pressure iterations; it recomputes full-field velocity/tracer errors using its recorded precision tolerance. Freeze the actual source-defined error tolerances and definitions from the selected driver/verifier before executing P5; this plan does not invent substitute error thresholds. + +Use the existing trajectory verifier only on its matching artifact/protocol schema. Its historical workload/resource/timing expectations do not automatically apply to CPU-control output. Before P5, identify the CPU driver's native checks and any missing cold-reader verification, then add only the necessary independently justified CPU result reader in the optional integration package. CPU acceptance requires the actual field/physics correctness, full native baseline comparison, identity and lifecycle gates. It does not require historical CUDA memory or speedup values. If an unchanged verifier rejects a changed environment/protocol, report that mismatch; do not modify original frozen receipts or skip checks while claiming full verification. + +## Independent review + +Read-only review by `/root/mesh_review` found no blocker to presenting this planning proposal. It requested explicit protocol/seed/exposure separation and a single initial array layout with precise source-byte versus array-byte custody. The plan/report now specify both. Review did not certify runtime feasibility, source-code compliance or scientific results.