Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,11 @@ jobs:
- run: build/workspace-training/bin/python -m unittest discover -s tests -p test_volume_assets.py
- run: build/workspace-training/bin/python -m unittest discover -s tests -p test_rheon_cpu_training.py
- run: build/workspace-training/bin/python -m unittest discover -s tests -p test_rheon_evaluation.py
- run: python -m venv build/workspace-agent
- run: build/workspace-agent/bin/python -m pip install -r requirements.txt
- run: build/workspace-agent/bin/python -m pip install --require-hashes -r requirements-agent.lock
- run: build/workspace-agent/bin/python -m unittest discover -s tests -p 'test_agent_*.py'
- run: build/workspace-agent/bin/python -m unittest discover -s tests -p test_mcp_stdio.py
- run: python -m unittest discover -s tests
- run: node --check static/auth.js
- run: node tests/test_auth_helpers.cjs
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
.venv/
.venv-training/
.venv-agent/
__pycache__/
*.pyc
data/
Expand Down
39 changes: 39 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,45 @@ complete backup/restore, supported versions and preservation limits.

Open http://127.0.0.1:8091. Change the port or data location with --port and --data. Camera access works on localhost; accessing a machine over the network requires HTTPS for browser camera access. Image import also works without camera access.

## Local agent connection

The optional MCP stdio bridge connects to an already running Tuldok server on a
numeric loopback address. Install its separate dependencies explicitly:

```sh
python3.12 -m venv .venv-agent
.venv-agent/bin/python -m pip install --require-hashes -r requirements-agent.lock
.venv-agent/bin/python mcp_stdio.py --config /private/tuldok-agent.json
```

The supplied dependency lock targets CPython 3.12 on Linux x86-64; it does not
claim other interpreter/platform combinations. The app and training environments
remain separate.

Use a private configuration file with `endpoint`, `catalog_id` and `token_file`:

```json
{
"endpoint": "http://127.0.0.1:8091",
"catalog_id": "YOUR_CATALOG_ID",
"token_file": "/private/tuldok-agent-token"
}
```

The token file must contain an owner-issued **agent** credential and be readable
only by its owner. Keep tokens out of command arguments and MCP configuration
shared with others. Obtain catalog identity from the authenticated workspace
context. The bridge checks that identity, credential kind and operation-registry
digest before use; it never creates a server, catalog or credential.

See [agent operations](docs/contracts/agent-operations.md) for credential setup,
the supported tool matrix, request limits and examples. Human review and exact
execution grants remain owner actions. Tools can save drafts, freeze approved
data, edit graphs and submit explicitly approved external Runs. Artifact segments
require the client to verify the reassembled whole-file hash before use; ordinary
verified HTTP downloads remain available. Ending the MCP connection does not
cancel an admitted Run. Use its scoped cancellation operation when required.

## Real image workflow in Workspaces

[Real workflow screenshots](docs/images/workspaces-m2/README.md) show Data,
Expand Down
635 changes: 635 additions & 0 deletions agent_api.py

Large diffs are not rendered by default.

7 changes: 6 additions & 1 deletion app.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import pipeline
import projects
import access
import agent_api
from access import authorized
from http.cookies import SimpleCookie, CookieError
import runs
Expand Down Expand Up @@ -572,6 +573,8 @@ def do_POST(self):
def _get(self):
path = urlsplit(self.path).path
try:
if path == '/api/agent/operations':
return self.reply(agent_api.discovery())
if path == '/api/workspace/projects' or path.startswith('/api/workspace/projects/'):
self.project_access()
if path.startswith('/api/workspace/'):
Expand Down Expand Up @@ -781,7 +784,7 @@ def _post(self):
if not 0 < length <= limit:
raise ValueError('Request is too large or empty.')
raw_body = self.rfile.read(length)
body = projects.decode(raw_body) if (path == '/api/workspace/projects' or path.startswith('/api/workspace/projects/') or (path.startswith('/api/workspace/runs/') and '/import-data/' in path)) else spatial_assets.parse_request(raw_body) if path in ('/api/workbench/pointcloud-import', '/api/workbench/volume-import') else native_detection_import.parse_request(raw_body) if path in ('/api/workbench/native-detection-import/prepare', '/api/workbench/native-detection-import/row') else saved_searches.parse_request(raw_body) if path == '/api/workbench/searches' or path.startswith('/api/workbench/searches/') else sequence_batch_import.parse_request(raw_body) if path in ('/api/workbench/sequence-import', '/api/workbench/sequence-import-item') else meshes.parse_json(raw_body) if path == '/api/workbench/mesh-import' else json.loads(raw_body)
body = projects.decode(raw_body) if (path == '/api/workspace/projects' or path.startswith('/api/workspace/projects/') or (path.startswith('/api/workspace/runs/') and ('/import-data/' in path or '/artifact-segment/' in path))) else spatial_assets.parse_request(raw_body) if path in ('/api/workbench/pointcloud-import', '/api/workbench/volume-import') else native_detection_import.parse_request(raw_body) if path in ('/api/workbench/native-detection-import/prepare', '/api/workbench/native-detection-import/row') else saved_searches.parse_request(raw_body) if path == '/api/workbench/searches' or path.startswith('/api/workbench/searches/') else sequence_batch_import.parse_request(raw_body) if path in ('/api/workbench/sequence-import', '/api/workbench/sequence-import-item') else meshes.parse_json(raw_body) if path == '/api/workbench/mesh-import' else json.loads(raw_body)
if not isinstance(body, dict):
raise ValueError('Expected an object.')
path = urlsplit(self.path).path
Expand All @@ -800,6 +803,8 @@ def _post(self):
admission = {key: value for key, value in body.items() if key != 'expected_sha256'}
return self.reply(dataset.workbench.scientific.admit_run_artifact(
parts[1], parts[3], body['expected_sha256'], admission), 201)
if len(parts) == 4 and parts[0] == 'runs' and parts[2] == 'artifact-segment':
return self.reply(dataset.runs.artifact_segment(parts[1], parts[3], body))
if parts == ['graphs']:
return self.reply(dataset.pipelines.create(body), 201)
if parts == ['graphs', 'external']:
Expand Down
130 changes: 130 additions & 0 deletions docs/contracts/agent-operations.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
# Scoped local agent operations v1

`agent_api.discovery()` is the versioned operation registry. Its deterministic
SHA256 binds schemas, route projections, scope descriptions and limits. The
application exposes it at authenticated `GET /api/agent/operations` without
importing optional packages. MCP uses the same definitions with fixed underscore
names; dotted operation IDs remain stable registry identifiers. These schemas
bound transport input; current HTTP/domain owners decide source custody,
permissions, revisions, family independence and execution eligibility.

## Configuration and authority

The optional bridge reads `--config /absolute/private/config.json`. The closed
configuration has `endpoint`, `catalog_id`, `token_file`, and optional `timeout`.
Endpoint must be explicit `http://127.0.0.1:PORT` or `http://[::1]:PORT`.
Configuration and token files must be owned regular files, private to the current
user, without symlinks or hard links. Token path is absolute; credential contents
are never CLI arguments or logs. Bootstrap requires a live agent credential and
matching catalog and registry. Mutations recheck authenticated discovery. Its HTTP authorization failures keep
the original validated status/code/message; registry_mismatch is reserved for a
successful discovery response whose registry differs. No
redirects, proxies, host discovery, arbitrary URLs, shell, catalog connection,
scheduler, credential issuance or automatic mutation retries exist in the bridge.

An owner prepares credentials, human reviews and exact execution grants through
existing HTTP/UI. Agent draft tools cannot claim human/programmatic review.
External submit requires the exact approval ID, graph revision, material digest
and request key. Identical-key receipt reconciliation is distinct from a new
attempt. Ordinary submit/retry remains owner-only and unexposed. Explicit test
exposure is supported with `expose-holdout`, not an owner-only fiction: it records
irreversible disclosure through the existing owner, has no preview/grant field,
and is not idempotent. A lost acknowledgement requires state inspection.

## Supported matrix

The registry is the machine-readable exact field/schema inventory:

| Operations | Existing domain surface |
| --- | --- |
| context.get, identity.get | workspace context and authenticated identity |
| projects.list/get/open/evidence | configured project roots and bounded declared evidence |
| recipes.list/inspect | recipe declarations, exact expected recipe SHA |
| data.query/get/history/responses/preferences | all eight record kinds, existing metadata/history |
| data.audio/mesh/pointcloud/volume/scientific | existing bounded typed inspections/slices |
| data.import_image/text/audio/mesh/sequence/pointcloud/volume | bounded exact native imports |
| data.import_project_scientific/import_run_scientific | hash-bound existing owner reference capture |
| data.save_draft, responses.save_draft | CAS draft targets; all twelve annotation tasks |
| proposals.caption_list/get/decide, proposals.classification_list/get/decide | existing owner-started proposal reads and reject/apply_draft |
| releases.list/inputs/preview/freeze | canonical_v1 selection/family/review authority |
| graphs.list/get/create/create_external/save/clone/validate/preflight_external | known graph variants; disconnected drafts allowed, preflight canonical |
| run_inputs.list, runs.list/get/progress/predictions | current Run custody/progress/result projections |
| runs.submit_external/cancel/expose_test | exact granted execution, scoped cancellation/disclosure |
| runs.artifact_chunk | bounded stateless segment of a completed external Run artifact |

Registry restrictions narrower than HTTP are explicit: canonical freeze requires
preview_token; draft annotations only; graph edits describe known node parameter
variants rather than arbitrary future types; inline import bytes are at most
512 KiB aggregate and request at most1 MiB. Larger scientific references use
existing Project/Run capture limits, not client filesystem access. Sequence v1
requires run.json/frames.jsonl; v2 additionally controls.json. All file payloads, including image,
are original raw base64 (not image data URLs); no scientific conversion occurs in the transport.

Unexposed: owner review/grant/admin operations, providers/model execution,
ordinary run submit/retry, backup/restore/snapshots, bulk native import workflows,
saved-search/selection mutations, alternate export formats and general binary
asset/release downloads. Project evidence DOES have its existing HTTP route;
`projects.evidence` invokes it. Metadata parity does not promise binary rendering.

## Limits and errors

Optional schema validation uses the standard pinned jsonschema
Draft202012Validator, with no custom type checker or external schema retrieval.
Standard integer schemas reject bool but admit integral floats. Existing domain
owners may require actual Python integers and reject those floats; the bridge
forwards the original value without conversion. Schema validity is not domain
acceptance. App registry imports remain stdlib-only.
Strict JSON rejects duplicate fields, nonfinite numbers, surrogates/NUL and depth
above64. No trimming/coercion occurs during forwarding; owners retain semantics.

Default absolute HTTP deadline is120 seconds (configuration maximum300). Connect
is at most5 seconds. Header wait may use the remaining absolute deadline because
preflight can take longer than5 seconds before replying; a separate deadline
socket shutdown prevents header trickle extending the call. Payload blocks use
at most5 seconds or remaining deadline. This deliberately supersedes any draft
claim of a5-second complete header phase. Every call closes its own socket/timer.

HTTP receive bound16 MiB and MCP encoded response bound32 MiB are client limits,
not existing global server caps. Large list/detail responses can exceed them and
are explicitly refused. Framing requires one Content-Length, JSON identity
encoding, no compression, chunked encoding or redirects. Execution errors retain
bounded safe HTTP status/code/message, returned as `{status,result}`. Unknown tools and registry argument-schema errors return JSON-RPC -32602
before SDK dispatch. Actual HTTP/domain or transport execution failures return
tool isError with the typed envelope; protocol failures are not tool failures.
Unknown commit is never retried automatically. MCP output schemas describe this
stable envelope; domain result semantics remain with their existing owners.

## Artifact byte parity

`runs.artifact_chunk` takes run_id/name/expected_sha256/offset/max_bytes (1..8 MiB).
The existing Runs segment owner verifies receipt/inventory and reads only the
selected interval, returning its hash and explicit whole-hash-unverified status.
Reassemble ordered offsets, verify each segment and verify the final complete
SHA256 against expected_sha256 BEFORE treating bytes as verified evidence or a
checkpoint. Same-size mutation outside the segment is not detected by that read.
No bridge cache, arbitrary output file, fake full verification or Library exists.
The unchanged full HTTP download uses the existing verified anonymous spool.

An8 MiB chunk encodes to11,184,812 base64 characters; duplicating structuredContent
in compatibility text uses22,369,624 characters plus bounded metadata, within32 MiB.
Other JSON can expand more: the protocol owner checks final encoded response size.
Tests must establish exact multichunk reconstruction and linear payload reads,
revocation and mutation failures, alongside legacy full-download behavior.

Focused HTTP/MCP workflow and negative coverage passed after the benchmark
gate, as recorded in p6-verification.md. Shared real scientific reads and final
full regression/CI remain separate pending acceptance gates.

Output schemas check each operation's success top-level kind and identity fields,
and distinguish HTTP error envelopes from successful results. Nested scientific
or runtime documents are preserved as owner evidence, not independently certified
by these schemas. Artifact segment output alone is fully closed and bounded;
client correlation checks echoed Run/name/hash/offset, decoded size and segment
SHA. `verify_artifact` provides bounded whole-file reassembly verification with
an optional caller callback. Callback bytes are provisional until successful
return and must be discarded on error; it never opens a filesystem path.

Response errors without a legacy code map to `http_error`; malformed/non-string/
secret-bearing code fields are transport failures. HTTP output objects posing as
success (`200` with only an error body) fail the operation output schema. Raw
tracebacks or arbitrary response formats are not accepted as authoritative errors.
Loading
Loading