feat(harnesses): per-phase network policy for docker-* runs, off by default

dokimi/changelog.d/20261024_120000_napetrov_task-network-policy.md

@@ -0,0 +1,16 @@
+### Added
+
+- **Per-phase network policy for `docker-*` runs** — build, solve and verify
+  each get `none`, `allowlist` (egress sidecar) or `public`. Defaults: build
+  public, solve and verify none. Set from task.toml (`[environment]`,
+  `[agent]`, `[verifier]` `network_mode` / `allowed_hosts`, Harbor's
+  `allow_internet`), or `dokimi tasks run --network PHASE=MODE[:hosts]`.
+  The effective plan and each phase's source land in `metrics.network`; a
+  skill's `metadata.network` is recorded as `metrics.network.skill`, not
+  granted.
+
+### Changed
+
+- **A task whose `network_mode` is not `none`, `allowlist` or `public`, or that
+  lists `allowed_hosts` without `allowlist`, now errors the run** instead of
+  running offline.

dokimi/docs/coding-harnesses.md

@@ -302,16 +302,19 @@ The result is in `metrics.isolation_probe` and `isolation_probe.json`:
       data, and `scripts/check_smoke_runs.py` fails the run on it. No
       `egress.jsonl` is written for an `unknown` cell: a missing file is not
       "no requests". The file is also absent when no sandbox ran
-      (`docker-oracle`, the `-host` variants, a build failure), so use
-      `metrics.egress.status` to tell unknown from a cell that ran no sandbox.
+      (`docker-oracle` and the `-host` variants with no allowlisted phase, a
+      build failure), so use `metrics.egress.status` to tell unknown from a
+      cell that ran no sandbox. The file and `metrics.egress` cover the
+      solve's sidecar only: an allowlisted verifier's decisions are
+      `metrics.network.verify_egress`, also saved in `network.json`.
       `docker-probe` reads `violated` on a passing
       cell, because the probe requests denied hosts (`example.com`) by design;
       judge a probe by `isolation_probe.json`, not by `egress.status`.
-- **`network_mode = "public"`.** It applies to the verifier only. The
-  container joins the default bridge after the agent exits. Every
-  uxlfoundation/skills task declares it, and none of their solutions or tests
-  uses the network, so a task whose solve needs a host must list it under
-  `environment.egress`.
+- **`network_mode = "public"`.** An `[environment]`-level `public` applies
+  to the oracle and the verifier, not to an agent: every uxlfoundation/skills
+  task declares it, and none of their solutions or tests uses the network. A
+  task whose agent needs a host lists it under `environment.egress`. See
+  [Network policy](#network-policy) for the other sources.
 - **Environment.** HOME is its own tmpfs, `/run/abhome`; the image's `/tmp`
   is left alone. The environment is:
     - the provider-credential allowlist;
@@ -520,6 +523,43 @@ A `command:` harness (including a `--command-template` that wraps `harbor`)
 keeps its own rule: a `reward.txt` / `reward.json` under the run dir, else a
 `reward: x` on its output, else exit code 0 is a pass.
 
+### Network policy
+
+Every `docker-*` run has a network per phase, recorded in
+`metrics.network` with the source of each phase's setting, so two runs can
+be checked for the same network before their verdicts are compared:
+
+| Phase | Default | What runs |
+|---|---|---|
+| `build` | `public` (host network, host proxy as build args) | `docker build` |
+| `solve` | `none` (an in-container agent still reaches its LLM through the sidecar) | the oracle, or the agent |
+| `verify` | `none` | `tests/test.sh` |
+
+The modes are Harbor's: `none` (Harbor's `no-network`), `public` (the
+default bridge plus the host's proxy variables), and `allowlist` (the listed
+`host[:port]` entries only, through an egress sidecar; the verifier's
+decisions land in `metrics.network.verify_egress`). Later sources win:
+
+1. The defaults above.
+2. The task: `[environment]` `network_mode` and `allowed_hosts` (or Harbor's
+   deprecated `allow_internet`), then `[agent]` and `[verifier]` overrides,
+   and `environment.egress` for an agent's solve. An unknown mode refuses
+   the cell, and its `network.json` holds the reason.
+3. The operator: `dokimi tasks run --network PHASE=MODE[:host,...]`,
+   repeatable, with `all` for every phase. For example
+   `--network verify=none` grades a public task offline, and
+   `--network solve=allowlist:pypi.org,files.pythonhosted.org` lets every
+   arm install packages.
+
+A skill that needs the network at solve time declares it in its `SKILL.md`
+frontmatter as `metadata.network` (`allowlist:host,...` or `public`), with
+the reason for people in `compatibility`, the Agent Skills spec's field for
+environment needs. The run does not widen the skill arm for it: its baseline
+would then run on another network. `metrics.network.skill.missing` lists the
+declared hosts this run did not grant; pass them with `--network solve=...`
+so every arm gets them. A value that does not parse is recorded as it is, with
+`error` and `missing: null`, and the arm still runs.
+
 ## BenchFlow Cells
 
 Skills are not bind-mounted: BenchFlow copies the `--skills-dir` root into the

dokimi/docs/contributing-terminal-bench-task.md

@@ -115,9 +115,9 @@ Field by field, with the consumer that actually reads it:
 | `environment.build_timeout_sec` | `docker-*` harnesses | Image-build budget; used instead of the default when set (oneAPI/pip-heavy images need > 10 min). |
 | `environment.docker_image` | Harbor | Image tag Harbor expects to exist (`harbor run` does **not** build it for you). The in-repo `docker-*` harnesses build and tag `dokimi/<task-name>:latest` themselves. |
 | `environment.cpus`, `memory_mb` | `docker-*` harnesses | Container limits (`--cpus`, `--memory`); default to 4 CPUs / 2048 MB when absent. A malformed value falls back to the default rather than failing the run. |
-| `environment.network_mode` | `docker-*` harnesses | `"public"` gives the **verifier** NAT'd internet (bridge), and `docker-oracle`'s solve too. The `docker-claude` agent never gets it: it solves on an internal network whose only way out is the egress proxy (see `environment.egress`). Anything else, including absent, means `--network none`. The image *build* always gets host network. Prefer offline. |
+| `environment.network_mode` | `docker-*` harnesses, Harbor | `"no-network"` (absent means the same), `"allowlist"` with `allowed_hosts`, or `"public"` (NAT'd internet, bridge). It applies to the **verifier** and `docker-oracle`'s solve, not to an agent: an agent solves on an internal network whose only way out is the egress proxy (see `environment.egress`), unless `[agent] network_mode` says otherwise. `[agent]` and `[verifier]` take the same keys for their phase. Any other value, or `allowed_hosts` without `"allowlist"`, errors the run. The image *build* gets host network unless the operator passes `--network build=none`. See [Network policy](./coding-harnesses.md#network-policy). Prefer offline. |
 | `environment.egress` | `docker-claude` | Hosts the agent's solve may reach through the egress proxy, e.g. `["pypi.org", "files.pythonhosted.org"]`. A bare host means `host:443`; `host:port` is accepted. No wildcards (`*.pythonhosted.org`), commas or leading/trailing dots: list each host. A plain-HTTP host is reached directly, so it must be in the operator's `NO_PROXY`. |
-| `environment.allow_internet` | Harbor | Harbor's equivalent policy flag; set it `false` and keep it consistent with `network_mode`. |
+| `environment.allow_internet` | `docker-*` harnesses, Harbor | Harbor's deprecated flag: `false` is `"no-network"`, `true` is `"public"`, read only when `network_mode` is absent. Keep it consistent with `network_mode`. |
 | `environment.storage_mb`, `gpus` | Harbor | Declared resource envelope. |
 
 Unknown tables and keys are ignored, so a task may carry extra fields the

dokimi/dokimi/commands/tasks.py

@@ -189,6 +189,7 @@ def cmd_tasks_run(args: argparse.Namespace) -> None:
         command_template=args.command_template,
         dry_run=args.dry_run,
         repeats=args.repeats,
+        network=tuple(args.network),
     )
     try:
         runner.validate_harnesses()
@@ -269,6 +270,14 @@ def register(sub, positive_int) -> None:
             "{model}, {output_dir}, {prompt}, {prompt_quoted}."
         ),
     )
+    run_p.add_argument(
+        "--network", action="append", default=[], metavar="PHASE=MODE",
+        help=(
+            "Network for a docker-* phase (build, solve, verify, or all): none, public, or "
+            "allowlist:host[:port],... Repeatable; overrides the task's own network_mode. "
+            "Default build=public solve=none verify=none"
+        ),
+    )
     run_p.add_argument(
         "--dry-run", action="store_true", help="Build artifact without invoking agents"
     )

dokimi/dokimi/harnesses/container_agent.py

@@ -139,14 +139,17 @@ class ContainerSandbox:
     @classmethod
     def open(cls, run_id: str, docker: Docker, log, *, source_env: dict[str, str],
              task_egress: Optional[list[str] | str] = None,
-             with_agent: bool = True, runner: process.Runner = process.run) -> "ContainerSandbox":
+             with_agent: bool = True, include_llm: bool = True,
+             runner: process.Runner = process.run) -> "ContainerSandbox":
+        """The network and sidecar for one run; ``include_llm`` off (the oracle) allows only *task_egress*."""
         if with_agent:
             preflight(source_env)
         binary = claude_binary() if with_agent else None
         extra_hosts = [task_egress] if isinstance(task_egress, str) else list(task_egress or [])
         # Every entry, not only the task's: a malformed one (a Vertex region with a comma)
         # fails here with its name, before the sidecar splits it into extra allowed hosts.
-        allow = [normalize_entry(h) for h in llm_hosts(source_env) + [str(h) for h in extra_hosts]]
+        llm = llm_hosts(source_env) if include_llm else []
+        allow = [normalize_entry(h) for h in llm + [str(h) for h in extra_hosts]]
         files = {v: Path(source_env[v]) for v in _FILE_VARS
                  if source_env.get(v) and Path(source_env[v]).is_file()}
         sandbox = cls(network=f"dokimi-net-{run_id}", proxy_name=f"dokimi-egress-{run_id}",
@@ -183,6 +186,21 @@ def _start_proxy(self, docker: Docker, log) -> None:
             time.sleep(0.2)
         raise RuntimeError(f"egress proxy {self.proxy_name} did not start; see run.log")
 
+    def verifier_sidecar(self, docker: Docker, log, allow: list[str]) -> "ContainerSandbox":
+        """A fresh sidecar on this network for the verifier's allowlist, after :meth:`stop_proxy`.
+
+        Its own name and decision log; no LLM hosts. The caller stops it before :meth:`close`.
+        """
+        side = ContainerSandbox(network=self.network, proxy_name=f"{self.proxy_name}-verify",
+                                allow=sorted(allow), binary=None, label=self.label, runner=self.runner)
+        docker(["rm", "-f", side.proxy_name], log, check=False)
+        try:
+            side._start_proxy(docker, log)
+        except BaseException:  # started, it holds the upstream proxy env and the network: close() can't see it
+            docker(["rm", "-f", side.proxy_name], log, check=False)
+            raise
+        return side
+
     @property
     def proxy_url(self) -> str:
         return f"http://egress:{PROXY_PORT}"