Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
e9c107b
Bound source closure scans during independent project forks
MrScripty Oct 24, 2026
25f997a
Reuse verified immutable revisions without disposable rewrites
MrScripty Oct 24, 2026
fecd3fc
Adopt bounded groups of reviewed project files atomically
MrScripty Oct 24, 2026
b21904f
Return structured busy errors from authentication boundaries
MrScripty Oct 24, 2026
c5992d7
Compare compatible native measurement packets with complete subject p…
MrScripty Oct 24, 2026
7d73fe7
Keep applied comparison visible while paging its subject selector
MrScripty Oct 24, 2026
e0b8948
Index extraction diagnostics while retaining their original order and…
MrScripty Oct 24, 2026
0afbb11
Read explicitly bound original manifests without reviving historical …
MrScripty Oct 24, 2026
1d76b47
Reuse owned material validation within each fixed candidate snapshot
MrScripty Oct 24, 2026
a3c9472
Support scientific portfolios within a 2 GiB canonical record budget
MrScripty Oct 24, 2026
195ba9b
Read historical material closure once per selected checkpoint
MrScripty Oct 24, 2026
d582987
Create every captured runtime directory with private permissions
MrScripty Oct 24, 2026
4371b73
Retain legacy activity manifest fields without enabling current activ…
MrScripty Oct 24, 2026
1bbb311
Decode original pre-link Torch manifests as inert retained evidence
MrScripty Oct 24, 2026
7f841a6
Validate original Torch material policy only through retained Run spe…
MrScripty Oct 24, 2026
e74760c
Skip surrogate scans for proven ASCII JSON strings
MrScripty Oct 24, 2026
bf6b39b
Use fixed material views for Study history reconstruction
MrScripty Oct 24, 2026
afa7ef4
feat: preserve exact retained sources for project composition
MrScripty Oct 24, 2026
f1cdb23
feat: publish native multi-source project compositions
MrScripty Oct 24, 2026
84960fb
fix: preserve artifact policies before combining project roots
MrScripty Oct 24, 2026
41974ae
feat: recover committed project publication without payload copies
MrScripty Oct 24, 2026
65f6caf
feat(measurements): admit larger complete comparisons through API v2
MrScripty Oct 24, 2026
dd1feca
fix(projects): support clients for large retained projects
MrScripty Oct 24, 2026
1f1e3bc
perf(projects): reuse decoding after verifying current canonical bytes
MrScripty Oct 25, 2026
bf2746e
perf(projects): select active receipt records before copying
MrScripty Oct 25, 2026
0adcc5b
perf(projects): avoid full snapshots for scalar metadata
MrScripty Oct 25, 2026
9f08a24
perf(projects): verify freshness without discarded record copies
MrScripty Oct 25, 2026
6713ace
perf(projects): avoid full history copies for working file reads
MrScripty Oct 25, 2026
9ff9899
perf(history): reuse decoded facts at a verified checkpoint
MrScripty Oct 25, 2026
0490cb1
perf(project): reduce validated snapshot and text-list copies
MrScripty Oct 25, 2026
d3b8f92
perf(project): validate retained receipts from one owned candidate view
MrScripty Oct 25, 2026
f8ba4ce
feat(project): browse authored model portfolios from retained notes
MrScripty Oct 25, 2026
00a02ee
perf(project): read retained measurement sources by validated family
MrScripty Oct 26, 2026
7e43cde
Merge pull request #122: native project migration (3/3)
MrScripty Nov 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 116 additions & 3 deletions agent_api.py

Large diffs are not rendered by default.

11 changes: 11 additions & 0 deletions app.py
Original file line number Diff line number Diff line change
Expand Up @@ -486,9 +486,15 @@ def _access_body(self):
except (ValueError, workbench.WorkbenchError):
raise access.AccessError('Access requests require a bounded JSON object.', 'invalid', 400) from None

def _catalog_busy(self):
return self.reply({'error': 'Tuldok is busy completing another operation. Try again after it finishes.',
'code': 'storage'}, 503)

def _access_error(self, error):
try:
dataset.access.denial(error, 'http.request', urlsplit(self.path).path[:512], anonymous=True)
except CatalogBusy:
return self._catalog_busy()
except CommitOutcomeUnknown:
return self.reply({'error': 'Commit outcome is unknown; inspect state before retrying.', 'code': 'commit_unknown'}, 503)
except psycopg.Error:
Expand Down Expand Up @@ -583,6 +589,8 @@ def _handle_GET(self):
return self.reply({'error': str(error), 'code': error.code}, error.status)
except access.AccessError as error:
return self._access_error(error)
except CatalogBusy:
return self._catalog_busy()
except CommitOutcomeUnknown:
return self.reply({'error': 'Commit outcome is unknown; inspect state before retrying.', 'code': 'commit_unknown'}, 503)
except psycopg.Error:
Expand Down Expand Up @@ -633,6 +641,8 @@ def _handle_POST(self):
return self.reply({'error': str(error), 'code': error.code}, error.status)
except access.AccessError as error:
return self._access_error(error)
except CatalogBusy:
return self._catalog_busy()
except CommitOutcomeUnknown:
return self.reply({'error': 'Commit outcome is unknown; inspect state before retrying.', 'code': 'commit_unknown'}, 503)
except psycopg.Error:
Expand Down Expand Up @@ -909,6 +919,7 @@ def _get(self):
'/workspaces.js': ('workspaces.js', 'text/javascript'),
'/experiment-graph.js': ('experiment-graph.js', 'text/javascript'),
'/project-progression.js': ('project-progression.js', 'text/javascript'),
'/project-model-portfolio.js': ('project-model-portfolio.js', 'text/javascript'),
'/project-measurements.js': ('project-measurements.js', 'text/javascript'),
'/project-measurements.css': ('project-measurements.css', 'text/css'),
'/experiment-lineage.js': ('experiment-lineage.js', 'text/javascript'),
Expand Down
17 changes: 15 additions & 2 deletions docs/contracts/project-files.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,12 @@ request receipts and a SHA256 checkpoint. Its digest covers canonical bytes of
the document with `checkpoint` omitted. The store supports only its declared
format and family; malformed and future versions fail before projection writes.
The canonical checkpoint index is limited to 64 MiB, separately from the
100,000-record, 40 MiB-per-record and 256 MiB-total-record limits. Commit checks
100,000-record, 40 MiB-per-record and 2 GiB-total-record limits. The aggregate
budget covers canonical record bytes, excluding artifact payloads; scientific
catalogs retain original runtime/file observations across many Runs. Commit
and cold Open enforce the same aggregate budget. Commit checks
the index size before staging immutable records. This format is unchanged;
older installations with the former 16 MiB index limit cannot open larger
older installations with the former 16 MiB index or 256 MiB aggregate limit cannot open larger
checkpoints and must be upgraded before consuming them.

Verified stopped-source backups and ordinary catalog backups share identical
Expand Down Expand Up @@ -102,6 +105,16 @@ annotations; human review still requires the owner. Every project action is boun
to an opaque workspace generation and its expected canonical checkpoint.

Use `ProjectFilesOwnerClient` only for the narrow owner project-file operations.
Its explicit `timeout` configuration may be up to 7,200 seconds for large retained
projects; the default remains 120 seconds. The absolute request watchdog still
applies and a transport timeout never authorizes an automatic mutation retry.
Generic clients retain their 300-second maximum. This wait budget is not an
execution deadline or a cancellation request.

Browser contract tables derive from the local `agent_api` registry. Regenerate
them with `python scripts/generate_project_files_contracts.py`; add `--check`
to verify that the installed v1 pins, all supported version pins and fixed
routes match that registry. Remote schemas remain opaque discovery data.
`ProjectFilesClient` wraps a negotiated client. CLI domain `file-*` commands take
`--arguments` JSON files; lifecycle commands use explicit flags and select
`--owner-project-files` with a
Expand Down
87 changes: 87 additions & 0 deletions docs/contracts/project-model-portfolio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
# Authored model organization

The project workspace can display an adopted `.portfolio.json` note as an
authored guide to model fits/components. This is separate from **Experiment
progression**, which retains genuine Study/Trial ancestry. Group edges mean
authored grouping; they do not establish derivation, admission or scientific
acceptance. Relationship notes have explicit comparison/parameter meanings and
source excerpts. Missing historical Trial ancestry stays unknown.

## Retained document

`project_model_portfolio_v1` is a UTF-8 JSON note of at most1MiB with exactly:

- `format`: the fixed string above; `project_id`: the owning32hex project ID;
- `title` (1–200characters), `description` (1–8,000characters), and
`authorship`: `retrospective`;
- `groups`: at most64records with exact fields `id`, `label`, `parent`,
`description`. IDs are unique, parent is null or another group, and grouping
is acyclic;
- `models`:1–1,000records with exact fields `id`, `label`, `group_id`, `aliases`,
`review`, `card`, `summary`. Each unique ID names an authored model entry,
not necessarily a complete generator, Trial or unique parameter state;
- `relationships`: at most2,000records with exact fields `from`, `to`, `kind`,
`explanation`, `evidence_model`, `excerpt`. All model references exist, endpoints
differ, and each ordered endpoint/kind tuple is unique.

Group/model IDs contain1–160ASCII letters/digits or `._:/-`, beginning with a
letter/digit. Group labels are at most200characters, model labels/aliases500.
Each model has at most128unique aliases. A card reference has exact fields
`file_id` (64hex), `path` (a relative `notes/` path up to512characters without
empty/dot/dot-dot components), positive integer `revision`, and64hex `sha256`.
These are exact captured note references, never source-reading capabilities.

Review is `source_reviewed` or `review_pending`. Reviewed summaries have exactly
six nonempty strings, each up to16,384characters: `method`, `purpose`,
`experiment`, `outcome`, `lesson`, `reproduction`. Pending summaries are null.
This status is an author's source-review statement, not numerical qualification.
Relationship kind is `comparison` or `parameter_derivation`; the latter points
from derived model to parameter source and must be acyclic. Explanation is at
most8,000characters and exact excerpt at most16,384. Comparisons are authored
context, not permission to combine incompatible metrics/populations.

## Ownership and reads

The ordinary project-file owner captures/revises the note, authenticates reads,
verifies captured UTF-8 bytes/hashes and retains previous revisions. There is no
new canonical family, SQL table, execution grant, recipe or mutation interface.
Existing `files_inspect` and `file_get` contracts are unchanged. Save/load keeps
the original guide and referenced note history. A fork whose new project/note
identities differ requires an explicitly re-authored guide; the reader refuses
foreign identities instead of retargeting them silently.

`ProjectModelPortfolioDocument` owns the presentation-document decoder and
cross-reference/literal checks. `ProjectModelPortfolio` owns selection,
presentation and its pending reads. Its injected `read` must be the host's
already negotiated, current-authority, byte-verified project-file reader.
The host owns transport/hash validation and context identity. The controller
checks the exact model card ID/path/revision/SHA/project, and that all six
summary strings and that card's relationship excerpts occur literally in its
captured text before displaying a selected summary. Text is inert; no markup,
scripts, parameter loading, scientific scoring or source execution occurs.

Only explicit actions read notes/cards. Discovery visits40adopted files per
page; no hidden scan/retry/polling occurs. The guide holds at most1,000models,
shows20per graph window with their ancestor groups, and reports exact total,
reviewed and filtered counts. Group/name/alias/review filters, paging and
textual relationship controls select the same entries as graph buttons.
Paragraphs wrap; exact source text remains in a disclosure.

The host's binding/project/checkpoint/credential/generation context owns every
read. A context change, permission loss or teardown aborts pending work, clears
old content and restores focus to the host status when needed. Late replies
cannot publish into another context. Errors show their declared diagnostic and
clear old detail instead of displaying a previous model as the new selection.
Unsupported format or foreign-project binding is `unsupported`; malformed
document/reference/summary is `invalid`; transport/availability failures retain
their owning diagnostic. No success or empty-data fallback substitutes for a
failed read.

## Verification scope

Pure codec tests cover contract decisions using literal fixtures. Controlled
read and real-browser tests cover selection, unavailable/obsolete responses,
repeated invocation, focus, keyboard activation and wrapping on desktop/mobile.
Those fixtures do not establish real project capture or save/load. Native
acceptance additionally requires an actual adopted guide, exact original cards,
current authority, real project reads and retained checkpoint/custody evidence.
54 changes: 54 additions & 0 deletions docs/runbooks/project-composition.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Compose retained projects

The owner operation `project_files.compose` creates one independent project from
an open editable primary and one or more explicit retained project checkpoints.
It preserves the original scientific evidence and forks the primary editable
notes, data and definitions through the same owners as Save As. Secondary drafts
and operation receipts remain inert historical sources. No source project is
modified and no imported program, runtime or credential becomes active.

Negotiate the operation's version-1 contract through the native project owner
client. The route is `POST /api/workspace/project-files/compose`. In addition to
the current `binding`, `expected_checkpoint` and one fresh `request_key`, supply:

- `root`, `directory` and `name` for a new destination under a configured root;
- `sources`, an ordered list of additional selections, each containing `root`,
`directory`, `project_id` and `checkpoint`.

The current project is implicitly the first source. At most 15 additional
sources are accepted by the HTTP contract. All source projects must be idle and
available for native exclusive store ownership. The primary's captured working
files must be clean. The destination cannot contain, lie inside or alias any
source, and existing unrelated destinations are never overwritten.

Equal canonical envelopes share one retained representation. Same-ID conflicts
are rejected except for Run documents that differ only in their historical
approval grant/consumption while retaining the same admission and every other
field. The complete alternate document is retained as a native Run observation;
it does not become a second Run or create a disclosure. Both documents remain
subject to the history owner's proof and payload validation.

The persisted `project_composition_v1` record binds the ordered source identities
and exact canonical envelope hashes to the destination's ordinary fork overlay.
A source membership explicitly identifies any alternate Run observation needed
to recover that source's document. Its decoder rejects missing, altered or
contradictory memberships. Subsequent ordinary forks preserve and archive this
provenance. Older software that does not support this family must reject the
project explicitly; records must not be dropped for compatibility.

Payloads are verified and copied as independent ordinary files. A missing copy
can be supplied by another source with the same verified declaration; a corrupt
selected source is rejected. Unique bytes cannot be omitted. Only dependencies
classified as regenerable by their domain owners can remain explicitly missing.

On success, `state` describes the newly selected project, `fork` reports the
primary origin and copy completeness, and `sources` lists every selected source
identity/checkpoint in order. The local owner installation retains its own
credentials. Existing Save As and lifecycle-state contracts are unchanged.

Retain a request before dispatch. An interrupted publication can leave an inert
stage or a complete unselected destination. Inspect the original request and
owner state before reconciling with the same key and arguments. Native receipts
bind reconciliation to the complete ordered source set. A timeout does not
justify another request key or deleting a retained stage. Selection failure
returns a durable unselected receipt while preserving the current primary.
Loading
Loading