Spec REMOTE-3452: self-hosted Kubernetes network policies

specs/REMOTE-3452/PRODUCT.md

@@ -0,0 +1,35 @@
+# Self-hosted Kubernetes network policies
+## Summary
+Enforce the server-resolved egress policy for each self-hosted Kubernetes execution. Refuse restricted executions when enforcement is unavailable. Preserve the existing behavior for executions without a policy.
+- Work item: [REMOTE-3452](https://linear.app/warpdotdev/issue/REMOTE-3452/self-hosted-kubernetes-network-policy-support).
+- Status: [approved by Ben Navetta in the origin thread](https://warpdev.slack.com/archives/C0A3CQ638NQ/p1792589633164009). This PR does not implement the feature.
+- The companion [TECH.md](TECH.md) defines the implementation and validation criteria.
+## Behavior
+1. **Policy presence controls enforcement.** An absent or null resolved policy adds no worker restriction. A present policy with zero rules denies all policy-controlled egress. Do not treat an empty policy as absent. A policy declaration that the server combines with system allowances is not an empty resolved policy.
+2. **The server owns policy meaning.** Enforce the resolved ALLOW rules supplied for this execution. Do not reload agent or Factory declarations in the worker. Do not add package repositories, arbitrary internal networks, wildcards, port restrictions, or new system hosts. Leave ingress unchanged. Preserve the server's existing feature gate and nil-policy behavior.
+3. **Operators must opt in.** Set `backend.kubernetes.network_policy.enabled` to `true` to enable support; its default is `false`. All network-policy settings belong under `backend.kubernetes.network_policy`, with DNS settings under `dns`. The Helm chart supplies the persistent restricted-Pod default-deny baseline. An operator may enable support only after the documented cluster prerequisites are met and the baseline is enforced. The option declares that the operator has verified the cluster; it is not an automatic CNI security certification.
+4. **Use the available enforcement engine.** Use Cilium's DNS-aware policy when its policy API is available. Otherwise, use native Kubernetes NetworkPolicy and resolve each allowed DNS name to IP addresses once before launch. Do not refresh that snapshot during the execution. Do not downgrade to native policy after a Cilium permission or application failure.
+5. **Use cluster DNS without requiring ordinary in-cluster configuration.** Autodetect resolver addresses when the worker uses its own cluster's in-cluster configuration. Explicit resolver settings override discovery. Out-of-cluster workers require explicit addresses. Operators must align worker and task DNS views and configure overrides for NodeLocal or custom DNS topologies. Both enforcement modes grant network destinations, not HTTP Host or TLS identity. A permitted IP can serve more than one DNS name. The native snapshot can become stale. Trusted DNS access is not a general data-loss-prevention guarantee.
+6. **Fail instead of weakening isolation.** Restricted executions fail with an actionable terminal error on unsupported workers, disabled support, invalid policy, DNS-resolution failure, missing permissions, or policy application failure. Docker, direct, and command backends do not support this feature. Never run the task unrestricted as a fallback. Capacity rejection and a disconnected worker retain their existing queue behavior.
+7. **Upgrade the pool before enabling restricted assignments.** This change does not add capability negotiation or server-side worker-version filtering. Old workers ignore unknown JSON fields and can therefore ignore a policy. Operators must upgrade every worker in the target pool before the server sends policy-bearing assignments. Do not claim fail-closed compatibility with old workers or mixed old/new pools.
+8. **Protect the entire execution within the operator-configured boundary.** Create or validate the policy and durably register the execution before creating its ordinary unsuspended Job. Isolation covers setup, init containers, regular containers, sidecars, teardown, and debug-idle time. Worker control-plane traffic and node image pulls are outside the task-Pod network boundary. Reuse one immutable policy when its final native Kubernetes or Cilium enforcement content matches, within that resource kind and namespace. Native DNS answers matter through the IP rules they produce. A shared policy fingerprint identifies enforcement content; existing run/execution IDs identify its consumers. Reserve `warp-` metadata for Warp, not operator labels.
+9. **Trust the operator-managed cluster boundary.** Enforcement requires an enforcing CNI, an already-enforced restricted-Pod default-deny baseline, trusted DNS, and no other policies that broaden restricted Pods' egress. Operators remain responsible for task credentials and container settings. The worker must not enable host networking, privileged mode, or added `NET_ADMIN`/`SYS_ADMIN` capabilities itself. Preserve operator-supplied settings and do not reject a run solely because the operator chose these options. Such choices can defeat network-policy isolation and fall outside its guarantee. Do not add another opt-in gate or per-execution connectivity probes.
+10. **Keep isolation during uncertainty.** Attach a consumer-aware finalizer when creating the policy. Worker shutdown, a lost watch, a logical run outcome, a cancellation acknowledgment, and main-container exit do not authorize removal. Pending launches, active Pods, and Jobs that can still create Pods all count as consumers. Fence registration against deletion so cleanup cannot remove the policy between registration and Job creation. Reuse is not proof of CNI readiness on a new node.
+11. **Separate execution cleanup from object retention.** Release a consumer only when its Job cannot start more Pods and all its containers have stopped. Automatically delete the shared policy only after zero execution consumers remain. With `--no-cleanup`, retain terminal Job/Pod objects for inspection but release their consumers. Retained terminal objects alone must not keep a policy. Deliberately restarting retained work requires a newly registered execution.
+12. **Recover policy cleanup after disruption.** A replacement worker reconciles durable registrations and policies, including executions that completed while the original worker was absent. Recover a matching Job after an ambiguous launch response. Retry cleanup on API errors. Retain unresolved launch reservations and policies when launch or termination cannot be verified. Do not use an in-memory reference count or expire an ambiguous reservation on a timer. Document verified operator recovery for stuck finalizers and the unsupported-CNI limitations.
+## Decisions from the interview
+- Ben chose Cilium when available and one-time DNS-to-IP resolution otherwise. He accepted snapshot staleness for the expected run duration.
+- Ben required terminal failure when the policy cannot be applied.
+- Ben chose operator prerequisites and explicit opt-in through `network_policy.enabled`, with DNS settings in `network_policy.dns`.
+- Ben preferred removing policies after completion under `--no-cleanup`. Completion means terminal execution, not only a finished agent conversation.
+- Ben explicitly excluded capability negotiation. A coordinated operator-managed upgrade is required instead.
+- In review, Ben requested in-cluster DNS autodetection, a Helm-provided baseline, reserved `warp-` labels, existing run/execution identities, and policy creation before Job creation. He accepted operator-controlled weakening of container settings without worker rejection.
+- Ben selected shared immutable fingerprint policies, a consumer-aware finalizer, durable launch registration, and automatic safe cleanup. Run/execution identity belongs on each registration, not on the shared policy ID.
+- In review, Ben chose content-addressing based only on the final native Kubernetes or Cilium configuration. Existing authenticated routing is sufficient; no additional assignment scope is required.
+## Out of scope
+- Docker, direct, and custom-command policy enforcement.
+- Changes to agent/Factory policy composition, inheritance, `extends`, or the system allowance list.
+- Native DNS refresh, strict application-host enforcement, an egress proxy, or new wildcard/CIDR/port policy syntax.
+- A full execution reattach/finalization service, CNI installation, or automatic cluster certification.
+- Capability negotiation, worker-version dispatch filtering, or an equivalent compatibility handshake.
+- Ingress controls, protection from a malicious cluster administrator, and rendered UI changes.