feat(harness): hosted execution guest — provisioner, scheduler, outbound evidence, entrypoint - #65
Open
khushalsonawat wants to merge 20 commits into
Open
Conversation
Phase 1 of the hosted execution guest: the typed world surface (state/put/change/drop/call/query) scenario code receives, backed by a per-world logical database — baseline-measured state caps, read-only sub-handles, SQL statement guards, typed errors, and offline + docker test lanes. call() raises until the http_tool shim wire format is pinned (recorded contract defect). Known defects tracked in the implementation ledger. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…contract check(): empty/whitespace strings count as held; any non-string, non-False value is a broken check, not an agent failure. ready(): bare False and any non-string value are broken ready code; setup stays advisory. Bare state() excludes unmeasured tables the same way it excludes over-cap ones, so nothing the agent does during a call can change which tables raise; the typed refusal stays on the explicit selector. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Typed models for the hosted bundle manifest: runtime kinds with the evidence seam, managed/source processes, seed stores with per-protocol sentinels, capability wiring, and the byte-exact inputs-digest helper. Strategy-engine pairing keys on the capability protocol; the manifest re-gains the resolved-secret sweep, scoped so v2's own secret_purposes field is not mistaken for a credential. Riders: runnable-check predicate parity, broken-value message truncation, cap-exclusion coverage. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
The complete §2e checklist as typed preflight errors — digest and filesystem verification scoped to the real directory, secret material scanning, unknown-field translation, placeholder vocabulary, secret purposes both directions, depends_on graph, engine catalog, reserved names, inputs-digest verification, and the compose gate ahead of all file-level checks so the dispositive code wins. seal_bundle_v2() is the normative §2d digest producer, pinned by a hand-computed byte vector. Model layer gains the user-assignment, engine-agreement, and reserved-configuration-name rules. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
The execution half of the provisioner: per-world/job-shared port allocation, closed-vocabulary env rendering, copy-based build trees, and process spawn — every customer process dropped to its declared unprivileged user, the customer checkout copied with symlinks preserved and escaping links rejected, process names constrained so a name can never escape its build/scratch directory. depends_on waits on all of a dependency's readiness probes; healthy() only ever demotes. Managed engines start with atomically-created credential files and are probed with real queries. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ool, transport clients The guest's three reporting channels per outbound-channels v1.3 and seam contract v1.12: the capabilities loader with its closed rejection vocabulary, byte-exact canonicalization with fixed digest vectors, the crash-safe sequenced spool (fsync-first, watermark clamp, corruption degrade-not-wedge, process registry), and the events/results/artifacts transport clients with the closed HTTP error map, fence latching, deadline-bounded retries, and redaction before emit. Four cold review rounds; survivors recorded as known defects in .claude/harness-alk/reports/outbound-review-r4.md (fork-path lock pairing, refusal-log volume, stale v1.11 pin). Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…eceipts The scheduler that drains a job's scenarios across W provisioned worlds per seam contract v1.12: the world pool with §4.5b-serialized provider calls, degrade-tolerant start, pool-owned world_unhealthy emission and the §5.4 zero-ready grace rule; per-scenario reset-lease-run-grade with the v3.4 receipt vocabulary, retry-once on another world, exact skipped synthesis, and cancel awareness; scenario phases dispatched on a dedicated executor so a leaked thread can never starve the provider. Three cold review rounds, closed by a 23-mutant mutation run; survivors (S1–S10, KD-1–6) recorded in .claude/harness-alk/reports/p9-review-r3.md. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…, conformance The provisioner's second half per seam contract v1.12: seed/migration application from the verified bundle under the store's own user, secrets loaded and deleted before any customer process with purposes from job.json, baseline freeze for all three engines (explicit redis SAVE, node-name-free rabbitmq mnesia path), per-world clone and reset with polled readiness promotion, the two-world conformance gate that never raises, the healthy() port method (demote-only), dead shared-engine respawn, reverse-order termination with engine-appropriate signals, and typed §2f failure codes on every path. Plus the P5-round-2 rider cleanups (bundle_v2, process_preflight) and the run_world_check exception-message truncation in checks.py. Five cold review rounds; the round-5 gate closed on mutation evidence. Known defects (rabbitmq datadir_copy baselines are world-0-only — disclosed in-code with the definitions-export follow-up recorded; sentinel read-only guard is defense-in-depth; promote-poll cost) are consolidated in .claude/harness-alk/reports/p6-review-r5.md. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…suite's mutation blind spots Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
… latch, close serialization Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ode contract, W from job parallelism Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…dent Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…scenario validation, single domain map Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
… scenarios Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…liver Wire register_with_platform() into BundleScenarioSource.build(): one provision + one begin call on the attempt's scenarios endpoint (single POST, body-level operation field — the route the platform actually serves), platform-assigned ids matched to documents BY scenario_key with hard guards (missing/unknown/duplicate key -> typed failure, never a partial assignment), and the full key set sent on begin. Scenario result receipts now carry the platform-assigned scenario_id, so they deliver instead of being dropped for an empty id. The P12 pinning test is split honestly: a delivery proof plus a drop-guard preservation test for the one path that can still see an unregistered scenario. Also guards empty scenario_key before the network call, keeping that deterministic content defect in the environment domain instead of letting the platform's 400 reclassify it as platform_sync. ScenariosClient path defaults collapse to "" (provision/begin suffixes would 404 against the real router). Cold review: CLEARS — 7/7 mutations killed, 823 passed / 0 failed x2. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
Replace the typed not-wired call seam with a real LiveKit runner when the job's connector is livekit: pre-dial validation of credentials and dispatch identity, an in-process SimulationSpec drive of the voice engine, deterministic per-scenario room naming, and a runner-owned call budget (config voice_call_timeout_seconds, default 300). Evidence follows the bundle-declared seam: tool_trace reads the world's database endpoint after the call; http_tool has no guest-side capture surface today and stays a typed stop rather than an invented proxy. Transcripts and recordings upload through the artifacts channel before the receipt references them, and artifact-level refusals degrade to null. Failure semantics are three distinct paths, each pinned by tests: an agent that never joins retires the world; any post-dial failure keeps its measured timing on the receipt; a silent zero-turn agent surfaces as missing evidence rather than a graded verdict or a false infrastructure failure. Voice credentials are captured from the job's secrets before the provisioner deletes them and exported once for the engine's own reads; values never appear in logs and stay inside the redaction set. Cold reviews: round 1 found 1 High (test-honesty) + 2 Medium, all fixed; round 2 CLEARS — 18 mutation runs killed or validated, 863 passed twice. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
The call runner reads runtime.metadata["livekit_agent_name"] as its dial identity, but nothing populated it: the value exists only in the agent process's rendered per-world environment, discarded after spawn. Carry it on the spawned-process handle and mirror it onto the world's runtime — exactly one distinct declared name sets the key; zero or conflicting names leave it absent (loudly, in the log) so the runner's typed pre-dial failure fires instead of dialing an arbitrarily chosen agent. Cold review: CLEARS — 4 mutations killed via the production path (one equivalent-mutant analyzed), 866 passed. Two contract notes ride for the next amendment: §3's metadata-keys wording, and a preflight rule for a static name under W>1. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…eipts, support storeless worlds The platform refuses to acknowledge a complete artifact manifest that omits the build, result, and log kinds, so fully-completed runs were stamped failed. The guest now uploads those run-level artifacts before pushing the manifest, redacting receipt text and build output first so a stored artifact can never be the one copy that leaks a secret. An aborted run's manifest is marked complete, since nothing further will ever upload and the platform accepts an incomplete manifest only from a canceled attempt. A bundle with no SQL store is now runnable: the world factory yields a storeless world whose state accessors raise a typed error, which the scheduler reports as an errored receipt rather than a crash. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
…ss bundles At parallelism greater than one, a static LIVEKIT_AGENT_NAME registers every world's agent under the same identity, so a dispatch lands on an arbitrary world and evidence silently crosses worlds; preflight now requires the per-world placeholder. A stateless bundle (no SQL store) is accepted, but one declaring the tool_trace evidence seam is rejected, since that seam reads its evidence from the world's postgres store. Signed-off-by: khushalsonawat <khushalsonawat@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The complete in-sandbox guest: everything that runs inside the Daytona sandbox from the moment the gateway execs the entrypoint.
What's in here (15 commits)
parallelism(world count W; the gateway caps at admission).Verification
1,030 tests green on this exact tree — including this branch's own suites run together with ours. Reviews included deliberate-bug-planting to prove the tests catch regressions (the planted-bug tables are reproducible from the test suites).
Deliberately not wired (typed errors, not stubs)
scenario_keyper entry; the guest now fails cleanly, not with a crash, if it's absent).Contracts
Built against hosted-execution-seams v1.15, outbound-channels v1.4, world-handle v3.5 (in the platform repo's api_contracts/harness/). Notable for reviewers: exit code 4 is new; failure domains are resolved by the provisioner and carried, never re-derived downstream.