Skip to content

agent: sandbox workers only run immutable environments (P7, worker half) - #46

Merged
rlrs merged 1 commit into
mainfrom
cleanup/no-docker-image-path-on-workers
Oct 24, 2026
Merged

rlrs merged 1 commit into
mainfrom
cleanup/no-docker-image-path-on-workers

Conversation

@rlrs

@rlrs rlrs commented Oct 24, 2026

Copy link
Copy Markdown
Owner

Plan item P7, worker half: sandbox workers no longer have a Docker image path.

What goes

  • build_direct_runtime_service's DockerOverlay2RootfsStore branch, its docker_binary, and the overlay2 rootfs_format in boot_config_sha256. The agent refuses to start without the environment registry and backend socket. serve-direct-node-agent drops --docker-binary (vm_init never rendered it) and its DockerImageRuntime.
  • image_eviction.py (LRU Docker image evictor), the provisioner's image_evictor, and the node agent's pre-pull eviction, post-pull note_used and forget_evicted_images.
  • DockerOverlay2RootfsStore's worker-only methods: list_image_ids, image_tag_times, image_present, image_content_id, evict_image, _docker_time_seconds.
  • Gateway OCI-pull paths: the 2 s create image wait and the 32-pull background cap. Every create now waits up to 30 s for its attach. The gateway imports external images whenever immutable_environments is set.
  • immutable_environments.worker_enabled as a switch.

Code −347/+49, tests −210/+53, including image_eviction.py (145) and test_image_eviction.py (129).

Kept

  • sandbox.max_concurrent_image_pulls, which bounds environment attaches.
  • vm_init is untouched. The sandbox role's Docker setup and docker_quota_image_gb stay, because that quota mount holds UCLOUD_DIRECT_IMAGE_CACHE_ROOT. dockerd and the agent unit's After=/Requires=docker.service also stay: Docker's FORWARD DROP policy must be in place before the agent sets ip_forward=1, and the guest init is docker-init.
  • The DockerOverlay2RootfsStore class stays for builders (R6).
  • Three paths stay because the LocalFleet harness's default Docker-ABI LocalRootfsStore still uses them: the provisioner's non-idle collect_image path, the agent's image_materializer for a non-materializing runtime, and OverlayRootfsManager's overlay2 bundle branches. They should go with R6 once the harness defaults to environments.

Evidence (2026-10-24, read-only)

  • deployment.json has immutable_environments.worker_enabled: true.
  • All 6 live sandbox workers (agent 0.10.0) advertise immutable-environment-host-erofs-v1. The agent advertises that capability only when EnvironmentRootfsStore is selected.
  • So no removed worker path ran in production.

Compatibility and rollback

  • worker_enabled is accepted only as true. A file without the key also loads. Any other value is refused with "delete the key". to_dict writes it as true.
  • 0.10.0 reads a missing key as false, so on a rollback keep worker_enabled: true in deployment.json. Then the same deployment.json loads and workers stay on environments.
  • A deployment without immutable_environments can no longer start sandbox workers.
  • CHANGELOG ## Unreleased has the operator notes.

Checks

  • uv run ruff check sandboxes tests scripts: clean.
  • scripts/check_test_rules.py: baseline lowered for test_direct_runtime_assembly.py, which lost its private-member Docker fixtures.
  • Full suite with PostgreSQL (run_tests.py -j 6): 2446 pass, 26 skip, 0 fail.
  • After rebasing onto shared_control: fold the additive DDL into the schema files #40, I re-ran only the affected modules, not the full suite. They pass.
  • No Rust changes.
  • New test: a worker without immutable environments refuses to start and creates nothing.

🤖 Generated with Claude Code

Sandbox workers no longer have a Docker image path. The direct runtime
always builds EnvironmentRootfsStore and refuses to start without the
environment registry and backend socket.

Removed:
- build_direct_runtime_service's DockerOverlay2RootfsStore branch, its
  docker_binary and the overlay2 rootfs_format in boot_config_sha256;
  serve-direct-node-agent's --docker-binary (vm_init never rendered it) and
  its DockerImageRuntime.
- image_eviction.py (the LRU Docker image evictor), the provisioner's
  image_evictor, and the node agent's pre-pull eviction, post-pull note_used
  and forget_evicted_images.
- DockerOverlay2RootfsStore's worker-only methods: list_image_ids,
  image_tag_times, image_present, image_content_id, evict_image and
  _docker_time_seconds. Builders use none of them.
- Gateway: the 2 s create image wait and the 32-pull cap that served OCI
  pulls. Every create waits up to 30 s for its image attach, and the gateway
  imports external images whenever immutable_environments is set.
- immutable_environments.worker_enabled as a switch.

Kept: sandbox.max_concurrent_image_pulls (it bounds environment attaches);
sandbox.docker_quota_image_gb and the sandbox role's Docker setup in vm_init,
whose quota mount holds the image cache root; dockerd and the agent unit's
ordering after docker.service (FORWARD DROP policy, docker-init). The
DockerOverlay2RootfsStore class stays for builders (R6). The provisioner's
non-idle collect_image path, the agent's image_materializer for a
non-materializing runtime and the overlay manager's overlay2 bundle branches
stay too: the LocalFleet harness's default Docker-ABI store still reaches
them, and they go with the class once the harness moves to environments.

Why it is safe: on 2026-10-24 deployment.json had
immutable_environments.worker_enabled true, and every live sandbox worker
(agent 0.10.0) advertised immutable-environment-host-erofs-v1, which the
agent advertises only when EnvironmentRootfsStore is selected. None of the
removed worker paths ran there.

Config compatibility: worker_enabled is accepted only as true (absent also
loads; anything else is refused with "delete the key"), and to_dict writes it
true, so 0.10.0, which reads a missing key as false, still loads the same
deployment.json on a rollback and keeps workers on environments.

Tests: deleted test_image_eviction.py and the evict_image tests;
test_direct_runtime_assembly assembles the environment store (no more
private-member Docker fixtures) and pins the refusal without environments;
the create image wait tests drop the cap; the config test pins true-only
worker_enabled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rlrs
rlrs force-pushed the cleanup/no-docker-image-path-on-workers branch from 41e8e18 to 294d2bb Compare October 24, 2026 12:15
@rlrs
rlrs merged commit 4d3cbcb into main Oct 24, 2026
4 checks passed
@rlrs
rlrs deleted the cleanup/no-docker-image-path-on-workers branch October 24, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant