Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,11 @@ jobs:
- run: node tests/browser_scientific_fields.cjs
env:
BROWSER: /usr/bin/google-chrome
- run: node --check static/workspace-external.js
- run: node tests/test_workspace_external_controller.cjs
- run: node tests/browser_workspace_external.cjs
env:
BROWSER: /usr/bin/google-chrome
- run: node --check static/workspace-projects.js
- run: node tests/browser_workspace_project.cjs
env:
Expand Down
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ current record; saving/reviewing and freezing a new release leaves old results
unchanged. Viewing test results records durable exposure; reused records or
pixels cannot be presented as fresh holdout evidence.

Fresh catalogs use schema 7. Existing PostgreSQL schema-1 through schema-6 catalogs require an
Fresh catalogs use schema 8. Existing PostgreSQL schema-1 through schema-7 catalogs require an
explicit stopped-writer backup and upgrade, never an automatic startup change:

```sh
Expand All @@ -92,6 +92,19 @@ finishes. See [workspace storage](docs/contracts/workspace-storage.md),
[CPU adapter](docs/contracts/workspace-image-training.md) for bounds and recovery.
The original corner studio and classic Workbench remain available.

## Approved local recipes

A project can declare an exact local Python recipe with typed parameters, named
inputs and bounded output files. In Pipeline, select its recipe, edit/save the
graph, review preflight and approve the exact execution. Runs shows durable state,
cancellation and verified output downloads. Owners can issue one-use approvals
for explicitly scoped agents. These are trusted same-user programs with ordinary
filesystem/network authority, not sandboxed plugins. Missing drainage after a
custodian failure remains unknown and blocks further external execution until
cessation is proven. See the [external recipe contract](docs/contracts/external-recipes.md)
for runtime support, limits and recovery. Scientific generation/training remains
separately qualified; neutral executable tests are not model acceptance.

## Text classification and WAV data

The same optional CPU environment supports a bounded **TF-IDF / MultinomialNB**
Expand Down
98 changes: 98 additions & 0 deletions access.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
SESSION_TTL = 8 * 3600
LOGIN_TTL = 600
MAX_TOKEN_TTL = 30 * 86400
MAX_APPROVAL_TTL = 3600
_CURRENT = ContextVar('tuldok_actor', default=None)
HASH = re.compile(r'^[a-f0-9]{64}$')

Expand Down Expand Up @@ -333,6 +334,103 @@ def admit_job(self, kind, job_id, request_digest, effects):
self.audit('access.admit_job', job_id, {'admission_id': admission_id, 'kind': kind, 'request_digest': request_digest, 'effects': effects})
return admission_id

def require_external_submission(self):
"""Check only the dedicated approved external-Run entrypoint.

This is not an execution grant: admission additionally requires atomic
consumption of an exact material/actor approval. Ordinary execute
guards and job admission remain owner-only.
"""
try:
with self.lock:
context = self.require('inspect', 'runs.external_submit')
if context.mode != 'credential':
raise AccessError('External submission requires a live credential.')
row = self._credential(context.credential_id)
if 'execute' not in json.loads(row['scopes_json']):
raise AccessError('External submission requires execute scope.')
return context
except AccessError as error:
self.denial(error, 'runs.external_submit')
raise

@authorized('approve-execution', 'access.grant_execution', owner_only=True)
def grant_execution(self, material_sha256, consumer_actor_id, ttl_seconds=600):
"""Approve an already reviewed material digest; Runs owns its meaning."""
if type(material_sha256) is not str or not HASH.fullmatch(material_sha256):
raise AccessError('Reviewed material digest must be SHA256.', 'invalid', 400)
_text(consumer_actor_id, 'Consuming actor ID')
if type(ttl_seconds) is not int or not 60 <= ttl_seconds <= MAX_APPROVAL_TTL:
raise AccessError('Approval lifetime must be 60 through 3600 seconds.', 'invalid', 400)
with self.lock, self.db.transaction():
context = self.require('approve-execution', 'access.grant_execution', owner_only=True)
if self.db.execute('SELECT id FROM access_actors WHERE id=%s', (consumer_actor_id,)).fetchone() is None:
raise AccessError('Consuming actor does not belong to this catalog.', 'unavailable', 404)
created = int(time.time())
grant = dict(id=uuid.uuid4().hex, catalog_id=self.catalog_id,
approver_actor_id=context.actor_id, approver_credential_id=context.credential_id,
consumer_actor_id=consumer_actor_id, material_sha256=material_sha256,
created_at=created, expires_at=created + ttl_seconds, revoked_at=None)
self.db.execute('INSERT INTO access_execution_grants VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)',
tuple(grant.values()))
self.audit('access.grant_execution', grant['id'], grant)
return grant

@authorized('approve-execution', 'access.revoke_execution', owner_only=True)
def revoke_execution(self, grant_id):
_text(grant_id, 'Approval ID')
with self.lock, self.db.transaction():
row = self.db.execute('SELECT * FROM access_execution_grants WHERE id=%s FOR UPDATE', (grant_id,)).fetchone()
if row is None or row['catalog_id'] != self.catalog_id:
raise AccessError('Execution approval is unavailable.', 'unavailable', 404)
if self.db.execute('SELECT grant_id FROM access_execution_consumptions WHERE grant_id=%s', (grant_id,)).fetchone():
raise AccessError('Approval was consumed; cancel the admitted Run instead.', 'conflict', 409)
grant = dict(row)
if grant['revoked_at'] is None:
grant['revoked_at'] = int(time.time())
self.db.execute('UPDATE access_execution_grants SET revoked_at=%s WHERE id=%s',
(grant['revoked_at'], grant_id))
self.audit('access.revoke_execution', grant_id, {'material_sha256': grant['material_sha256']})
return grant

def _consume_external_run(self, grant_id, material_sha256, run_id, request_sha256):
"""Consume once alongside the Run insertion in the caller's transaction.

Runs reconciles its actor/key/request receipt before this call. This
operation never refunds approvals or creates an alternate queue.
"""
if not self.db.in_transaction:
raise RuntimeError('Execution approval consumption requires the Run transaction.')
_text(grant_id, 'Approval ID')
_text(run_id, 'Run ID')
if any(type(value) is not str or not HASH.fullmatch(value) for value in (material_sha256, request_sha256)):
raise AccessError('Material and request digests must be SHA256.', 'invalid', 400)
with self.lock:
context = self.require_external_submission()
grant = self.db.execute('SELECT * FROM access_execution_grants WHERE id=%s FOR UPDATE', (grant_id,)).fetchone()
if grant is None or grant['catalog_id'] != self.catalog_id:
raise AccessError('Execution approval is unavailable.', 'unavailable', 404)
if grant['consumer_actor_id'] != context.actor_id or grant['material_sha256'] != material_sha256:
raise AccessError('Execution approval does not match this actor and reviewed material.')
if grant['revoked_at'] is not None or grant['expires_at'] <= int(time.time()):
raise AccessError('Execution approval is revoked or expired.', 'stale', 409)
if self.db.execute('SELECT grant_id FROM access_execution_consumptions WHERE grant_id=%s', (grant_id,)).fetchone():
raise AccessError('Execution approval was already consumed.', 'conflict', 409)
# Recheck after the row lock: waiting for a competing consumer must
# not retain stale credential authority.
self.require_external_submission()
admission_id = uuid.uuid4().hex
effects = {'runs.update': [run_id]}
self.db.execute('INSERT INTO access_admissions VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s)',
(admission_id, self.catalog_id, context.actor_id, context.credential_id, 'runs', run_id,
request_sha256, _json(effects), int(time.time())))
self.db.execute('INSERT INTO access_execution_consumptions VALUES (%s,%s,%s,%s,%s,%s)',
(grant_id, run_id, admission_id, context.credential_id, request_sha256, int(time.time())))
self.audit('access.consume_execution', grant_id,
{'run_id': run_id, 'admission_id': admission_id, 'material_sha256': material_sha256,
'request_sha256': request_sha256})
return admission_id

def require_job(self, kind, job_id, admission_id, request_digest):
context = self._context()
if (context.mode != 'job' or context.kind != kind or context.job_id != job_id
Expand Down
16 changes: 16 additions & 0 deletions app.py
Original file line number Diff line number Diff line change
Expand Up @@ -540,6 +540,9 @@ def do_POST(self):
if path.startswith('/api/access/credentials/') and path.endswith('/revoke'):
self._access_body()
return self.reply(dataset.access.revoke(path.split('/')[-2]))
if path.startswith('/api/access/execution-grants/') and path.endswith('/revoke'):
projects.closed(self._access_body(), ())
return self.reply(dataset.access.revoke_execution(path.split('/')[-2]))
return self._post()
except access.AccessError as error:
return self._access_error(error)
Expand All @@ -561,6 +564,9 @@ def _get(self):
return self.reply(dataset.projects.get(parts[1]))
if len(parts) == 4 and parts[0] == 'projects' and parts[2] == 'evidence':
return self.reply(dataset.projects.evidence(parts[1], parts[3]))
if len(parts) == 4 and parts[0] == 'projects' and parts[2] == 'recipes':
query = parse_qs(urlsplit(self.path).query)
return self.reply(dataset.pipelines.inspect_external_recipe(parts[1], parts[3], query.get('expected_sha256', [''])[-1]))
if parts == ['context']:
return self.reply({'version': 1, 'mode': 'real', 'dataset_id': dataset.db.dataset_id,
'recipe': 'image_classification_v1', 'execution': 'optional_local_cpu'})
Expand Down Expand Up @@ -707,6 +713,7 @@ def _get(self):
'/workspaces.css': ('workspaces.css', 'text/css'),
'/workspaces.js': ('workspaces.js', 'text/javascript'),
'/workspace-real.js': ('workspace-real.js', 'text/javascript'),
'/workspace-external.js': ('workspace-external.js', 'text/javascript'),
'/workspace-service.js': ('workspace-service.js', 'text/javascript'),
'/workspace-fixtures.json': ('workspace-fixtures.json', 'application/json'),
'/pipeline-editor.js': ('pipeline-editor.js', 'text/javascript'),
Expand Down Expand Up @@ -766,15 +773,24 @@ def _post(self):
return self.reply(dataset.workbench.scientific.admit(raw, admission, origin), 201)
if parts == ['graphs']:
return self.reply(dataset.pipelines.create(body), 201)
if parts == ['graphs', 'external']:
return self.reply(dataset.pipelines.create_external(body), 201)
if len(parts) == 2 and parts[0] == 'graphs':
return self.reply(dataset.pipelines.save(parts[1], body))
if len(parts) == 3 and parts[0] == 'graphs':
if parts[2] == 'clone':
return self.reply(dataset.pipelines.clone(parts[1], body), 201)
if parts[2] == 'validate':
return self.reply(dataset.pipelines.validate(parts[1], body))
if parts[2] == 'preflight-external':
projects.closed(body, ('revision',))
return self.reply(dataset.pipelines.preflight_external(parts[1], body['revision']))
if parts == ['runs']:
return self.reply(dataset.runs.submit(body), 201)
if parts == ['runs', 'approve-external']:
return self.reply(dataset.runs.approve_external(body), 201)
if parts == ['runs', 'submit-external']:
return self.reply(dataset.runs.submit_external(body), 201)
if len(parts) == 3 and parts[0] == 'runs':
if parts[2] == 'cancel':
return self.reply(dataset.runs.cancel(parts[1]))
Expand Down
60 changes: 58 additions & 2 deletions docs/contracts/actors.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,9 @@ The owner has inspect, edit, execute, cancel, freeze, approve-review,
approve-execution, expose-holdout and admin scopes. An owner may issue an agent
bearer with inspect plus selected edit, execute, cancel, freeze and
expose-holdout scopes. Agents cannot issue credentials, approve human review or
approve execution. Current execution starts remain owner-only even when an
agent has execute; bounded approved agent execution is deferred to P3b.
approve execution. Existing CPU, integrity and provider execution starts remain owner-only even
when an agent has execute. The dedicated external-Run path below admits an
agent only with a matching one-use material approval.

## Local owner login

Expand Down Expand Up @@ -71,3 +72,58 @@ revocation/expiry, concurrent one-use exchange, agent scope limits, denial
rollback evidence, exact closed receipts, system restrictions, private CLI files
and schema-6 upgrade/schema-7 backup restoration. This stage does not claim
provider-process containment, approved agent execution or remote deployment.

## One-use external execution approvals (schema 8)

`require_external_submission()` checks a live credential with inspect and
execute scopes. It does not grant execution itself. The existing general
execute guards and `admit_job` remain owner-only. Only the dedicated external
Runs owner may compose this check with approval consumption.

`grant_execution(material_sha256, consumer_actor_id, ttl_seconds=600)` requires
owner approve-execution authority and creates a new immutable grant identity.
The consumer must be an actor in this catalog; TTL is 60–3,600 seconds. Runs
first verifies the exact previously reviewed material digest, which covers the
project/graph, captured source/runtime/input identities, parameters and declared
permissions/resources. Runs retains matching material evidence. Access owns
actor/digest binding and lifetime; it does not interpret recipe semantics.

`revoke_execution(grant_id)` serializes against consumption by locking the
grant row. Repeating revocation is harmless. A consumed grant cannot be
revoked into reusable authority: cancel its admitted Run instead. Credential
revocation and expiry still deny new submissions independently of grant state.

`_consume_external_run(grant_id, material_sha256, run_id, request_sha256)`
requires the caller's existing Run transaction. It locks the grant, checks
exact catalog/actor/material, unrevoked/unexpired state and absence of prior
consumption, then rechecks live credential authority after waiting for the
lock. It atomically writes a consumption receipt, an existing-kind `runs`
admission with only `runs.update` for this Run ID, and actor audit evidence.
It returns the admission ID; the caller inserts the Run before committing.
Rollback restores the unused grant. Unknown commit acknowledgement never
refunds it or authorizes another attempt.

Runs reconciles its existing request key against both consuming actor and exact
request digest before consuming another grant. A matching committed attempt is
returned even after grant expiry; changed actor/digest conflicts. Retry requires
a fresh grant. Owner submit may create and consume its own approval in the
same transaction only against the exact previously reviewed material digest.
Workers use the retained closed admission and may finish bounded effects after
credential revocation; they do not receive approval/admin or owner authority.

Schema 8 adds `access_execution_grants` and immutable
`access_execution_consumptions`, retaining the exact schema-7 definition.
Complete backups retain unconsumed, revoked and consumed grants with original
expiration and identity. Restoring a pre-consumption backup is a privileged
catalog recovery operation, not a mechanism for refunding live approvals.

Approval authorizes a launch; it does not establish a sandbox. Same-UID code
retains effective filesystem/network authority unless separately restricted.
Reviewed material must disclose those permissions. Source/runtime capture and
verified descendant drainage establish their specific identity/lifecycle
contracts, not filesystem or network confinement.

`tests/test_execution_approvals.py` proves exact actor/material matching,
legacy owner-only behavior, TTL/revocation, two-connection consume/revoke races,
transaction rollback, lost commit acknowledgement with retained Run receipt,
closed continuation and schema-7/8 migration/backup custody.
Loading
Loading