diff --git a/.github/fork-fleet.md b/.github/fork-fleet.md index 08ed73e9bdffd..613bee1dcc0e1 100644 --- a/.github/fork-fleet.md +++ b/.github/fork-fleet.md @@ -498,7 +498,10 @@ that uses the router (or the `fleet-route` action) may also set: the config must say `"private": true`, and `route` uses no static pool unless the event says `repository.private: true` (unknown refuses, with an error on the run page). A static pool's labels may not be a dispatcher pool's label or - a retired label. + a retired label. The router cannot stop a workflow file from naming those + labels in `runs-on` itself, so on a private repository with static pools keep + **Run workflows from fork pull requests** turned off, and remove the static + runners (or their registration) before the repository is made public. - **`"hosted": false`** (no GitHub-hosted runners). Each kind then needs a static pool, and `upstream_sync` is refused. Every job that is not routed (a change gate off, a manual, scheduled or bot run, an untrusted run of this diff --git a/docs/fork/anchor-machine.md b/docs/fork/anchor-machine.md index 445b25162462a..1ebf2f727e377 100644 --- a/docs/fork/anchor-machine.md +++ b/docs/fork/anchor-machine.md @@ -15,8 +15,9 @@ New threads in a project that several machines share then start on the preferred ## Threads without a project: not covered yet Until upstream fixes its issue #31048, threads without a project ignore the **Prefer** and **Manual only** weights. -They start on the primary machine (or the machine you start them from), even when it is set to **Manual only**. The -composer offers no **Auto balance**, and **Send** goes to that machine without a warning. To use another machine, pick it in the composer; its "No +They start on the primary machine whenever it offers a "No project" folder, even when it is set to **Manual only**; +only when it offers none do they start on the machine you start them from. The composer offers no **Auto balance**, +and **Send** goes to that machine without a warning. To use another machine, pick it in the composer; its "No project" folder opens first. The fix exists as upstream PR #31049. The fork does not carry it yet, because it conflicts with upstream code that @@ -25,27 +26,32 @@ is already released (upstream PR #35658). It is parked in ## What happens when (threads in a shared project) -| Situation | Result | -| ----------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Preferred machine connected, with CPU and memory free | The draft shows **Auto balance** and moves there. | -| Preferred machine offline, or busy (CPU at 95% or more, or 5% memory or less) | The draft stays where it opened. **Send** refuses with "Choose a machine to continue"; pick a machine in the composer. | -| A file is attached before the draft has moved | The draft stays on the machine it opened on, and **Send** goes there, even if that machine is **Manual only** (upstream rule: attachments stay on their machine). | -| A thread is referenced that lives on a **Prefer** or **Normal** machine | That machine is the only one that can read it, so the draft moves there. | -| A thread is referenced that lives on a **Manual only** machine | No machine is picked. **Send** refuses with "Choose a machine to continue"; pick that machine in the composer. | -| A thread on another machine is referenced after the draft has moved | The draft does not move again. **Send** refuses with "Attached thread is on another machine"; remove the thread or pick its machine. | -| You pick a machine, branch or worktree by hand | That choice wins until you choose **Auto balance** again. | -| Only one machine has the project | No balancing; the draft uses that machine. | -| Load balancing off | Upstream's order: the primary machine first. | -| Thread without a project | Not balanced; see above. | +| Situation | Result | +| --------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Preferred machine connected, with CPU and memory free | The draft shows **Auto balance** and moves there. | +| Preferred machine offline, busy (CPU at 95% or more, or 5% memory or less), without a usable provider, or without a recent CPU/memory reading | The draft stays where it opened. **Send** refuses with "Choose a machine to continue" ("Checking machine resources" while the check still runs); pick a machine in the composer. "Without a usable provider": the chosen provider is not installed there, signed out, in error, unavailable, or turned off for this project. When the reading cannot be fetched, the composer shows **Auto balance unavailable**. | +| A file is attached before the draft has moved | The draft stays on the machine it opened on, and **Send** goes there, even if that machine is **Manual only** (upstream rule: attachments stay on their machine). | +| A thread is referenced that lives on a **Prefer** or **Normal** machine | That machine is the only one that can read it, so the draft moves there. | +| A thread is referenced that lives on a **Manual only** machine | No machine is picked. **Send** refuses with "Choose a machine to continue"; pick that machine in the composer. | +| A thread on another machine is referenced after the draft has moved | The draft does not move while you type. Without a file attached, **Send** picks again first (see below): it moves the draft to the thread's machine when that machine qualifies as in the first row, else refuses with "Choose a machine to continue". With a file attached, or after a hand pick, **Send** refuses with "Attached thread is on another machine"; remove the thread or pick its machine. The second refusal does not happen when the thread's machine is linked to the draft's machine: **Send** goes ahead. | +| You pick a machine, branch or worktree by hand | That choice wins until you choose **Auto balance** again. | +| Only one machine has the project | No balancing; the draft uses that machine. | +| Load balancing off | Upstream's order: the primary machine first. | +| Thread without a project | Not balanced; see above. | + +**Send** picks again. Just before an automatically moved draft without a file sends, the client reads the machines' +load again and picks again among the machines that qualify (upstream PR #35658). With the setup above that is only +the preferred machine: if it has gone offline or become busy since the draft moved, **Send** refuses with "Choose a +machine to continue" rather than send to a **Manual only** machine. ## After upstream fixes #31048 Threads without a project will then follow the same weights. The preferred machine's "No project" folder opens first, then the draft moves. One case can still send to a machine you did not choose: if that open fails or is cut -off (for example, the connection drops while it opens), the draft settles on the machine it is on. The composer -shows **Auto balance**, and **Send** goes to that machine even if it is **Manual only**. A failed open shows "Could -not switch machine"; a cut-off open shows nothing. Pick the machine by hand if it matters. This is upstream's -behaviour in #31049; the fork does not change it. +off (for example, the preferred machine fails to create the folder), the draft can settle on the machine it is +on. The composer shows **Auto balance**, and **Send** goes to that machine even if it is **Manual only**. A failed +open shows "Could not switch machine"; a cut-off open shows nothing. Pick the machine by hand if it matters. This is +upstream's behaviour in #31049; the fork does not change it. ## Removed fork setting diff --git a/docs/fork/gate-trust.md b/docs/fork/gate-trust.md index afd509caec4f4..70b70f864aa11 100644 --- a/docs/fork/gate-trust.md +++ b/docs/fork/gate-trust.md @@ -58,8 +58,9 @@ check runs (`head_check_app`, [pr-queue](../../fork-ci/actions/pr-queue/README.m ([issue 83](https://github.com/scuttletoby/t3code/issues/83)). The gate is not affected: it runs on a GitHub-hosted runner and fetches into a new bare repository each run. - **The host.** The gate keeps its own key off every machine. The agent Apps' keys, their token cache - and the `gh` tool on the agent machine are a separate part (the host install kit, not in the fork - yet). Until it lands, an agent can read those keys. + and the `gh` tool on the agent machine are a separate part: the host install kit, + [`fork-kit/gh-apps`](../../fork-kit/gh-apps/README.md). Until the owner installs it on the agent + machine and makes new keys (steps 4 and 5), an agent can read those keys. ## What an agent can still do @@ -101,7 +102,7 @@ agent sees (not a terminal an agent can read). | `PATH_TO_DOWNLOADED_KEY` | the `.pem` file the App's page downloaded | | `POLICY_ID` | step 3 on an existing Environment: the id of a deployment rule to delete | | `FORK_OWNER/FORK` | step 4: the fork that ships the host install kit | -| `RELEASE_SHA`, `KIT_PATH` | step 4: the kit's release commit and its folder in the fork | +| `RELEASE_SHA` | step 4: a release commit of the kit (on the fork's protected `ci-release` branch) | | `AGENT_USER` | step 4: the system user the agents run as | | `APP_NAME`, `AGENT_APP_ID` | step 5: an agent App's name and its App ID | @@ -151,22 +152,26 @@ Then delete the downloaded `.pem`. Keep no repository or organisation secret nam `HEAD_GATE_APP_KEY`. Then add the caller workflow on the default branch and dispatch it once without a number (the backfill), as the README's setup says. -**4. Host install** (as root; not available yet). The host install kit is not in the fork yet. Its -planned shape, which the kit's pull request fixes exactly. The first line deletes a file an earlier -try left, and `&&` stops the chain at the first failure, so a failed download never runs an older -script: +**4. Host install** (as root, in a shell no agent sees; Linux only for now). The kit is +`fork-kit/gh-apps`; its [README](../../fork-kit/gh-apps/README.md) has the details and the upgrade +and removal steps. `RELEASE_SHA` must be a commit on the fork's protected `ci-release` branch; the +bootstrap refuses any other. The first line deletes a file an earlier try left, and `&&` stops the +chain at the first failure, so a failed download never runs an older script: ```sh rm -f /root/gh-apps-bootstrap.sh && -curl -fsSL https://raw.githubusercontent.com/FORK_OWNER/FORK/RELEASE_SHA/KIT_PATH/bootstrap.sh -o /root/gh-apps-bootstrap.sh && +curl -fsSL https://raw.githubusercontent.com/FORK_OWNER/FORK/RELEASE_SHA/fork-kit/gh-apps/bootstrap.sh -o /root/gh-apps-bootstrap.sh && sh /root/gh-apps-bootstrap.sh --repo FORK_OWNER/FORK --sha RELEASE_SHA --agent-user AGENT_USER ``` -It is to print the sha and file hashes it installed; compare the sha with the release page. +It prints the sha, the hash of the download and of every installed file; compare the sha with the +release. Add `--gh-path /path/to/gh` if `gh` is not `/usr/bin/gh`. -**5. New agent App keys** (as root; not available yet, comes with step 4). Keys an agent could ever -read count as leaked: per agent App, in the browser, generate a new key and delete the old one. Then -once per App, with the key text on stdin, and delete the download: +**5. New agent App keys** (as root, after step 4). Keys an agent could ever read count as leaked: +per agent App, in the browser, generate a new key and delete the old one. The install does not +import key files an agent kept in its own home folder; it only reports them (the kit's README, "Old +copies"): delete them once the new keys work. Then once per App, with the key text on stdin, and +delete the download: ```sh gh-apps-admin add-key APP_NAME --app-id AGENT_APP_ID --repo OWNER/REPO < PATH_TO_DOWNLOADED_KEY diff --git a/fork-ci/actions/fleet-route/README.md b/fork-ci/actions/fleet-route/README.md index f7c2d7cd7890b..0a22fc6b78216 100644 --- a/fork-ci/actions/fleet-route/README.md +++ b/fork-ci/actions/fleet-route/README.md @@ -87,6 +87,14 @@ The `runs-on` reads, in order: event may run another repository's code, so they never get a self-hosted label here; 4. otherwise `ubuntu-latest`. +Step 3 is **wider than the trust rule**. It does not check the config's `trusted` actors, workflows and branches, +bots, `private` or `hosted`: when the route job fails, any push and any pull request from this repository (say a +dependency bot's) gets the self-hosted label, where the router sends an untrusted one to `hosted`. A **public** +repository, and any project whose config has a `hosted` label, puts its hosted label there instead of the +self-hosted one (`fromJSON('["ubuntu-24.04"]')`), so a failed route never gives anyone a self-hosted runner. Only a +private project with `"hosted": false` needs a self-hosted label in step 3 (its router sends untrusted runs of this +repository to the always-on pool too); the example above is one. + `fleet_route.test.py` runs this exact expression (copied from this README) for each of those cases. Without GitHub-hosted runners at all (`"hosted": false`), the route job is red for a run that may hold another @@ -102,6 +110,14 @@ repositories only**: the config must say `"private": true`, and the router uses says the repository is private (unknown counts as public). Keep "Require approval for all outside collaborators" on as well. +The router cannot stop a workflow file from naming a static pool's labels in `runs-on` itself. So on a private +repository with static pools: + +- keep **Run workflows from fork pull requests** (Settings, Actions, General) turned off, or a fork's pull request + can name those labels and run on the static runners; +- remove the static runners (or their registration) before the repository is made public. From then on `route` + refuses static pools (red), but any fork's pull request could still name their labels. + ### Inputs | Input | Default | Meaning | @@ -120,7 +136,7 @@ as well. | `attempt` | the run attempt the labels were made for | | `fallback` | JSON runs-on for a job of another attempt (a re-run of failed jobs); never a dispatcher pool's label | | `rerun` | JSON, with `reruns`: `actors`, `pool` (label) and `labels` (job key -> run label without `-a`) | -| `watch` | `true` when a job went to a fleet pool | +| `watch` | `true` when a job went to a dispatcher or static pool, else `false`; it starts nothing itself (below) | ## Modes @@ -251,7 +267,9 @@ A config that breaks any rule above fails the step with the list of problems and This action ships `route` only. The fork's own CI also runs the router's `watch` (cancels a run whose fleet job waits too long) and `sweep` (re-runs it) with a token that may cancel runs; they are not part of this action, and a project -that uses only the action has neither. So: +that uses only the action has neither. The `watch` output only says whether any job went to a dispatcher or static +pool: the fork's CI starts its watch job only when it is `true` (`if:` on that job), and nothing starts by itself on +it. So: - **Old runner labels:** keep the old label on its runners, and in `retired_labels`, for a week after a pool's label changes. Runs routed before the change queue on it and still find a runner. diff --git a/fork-ci/actions/fleet-route/action.yml b/fork-ci/actions/fleet-route/action.yml index 2dba31d6460e2..42eede1c66cfd 100644 --- a/fork-ci/actions/fleet-route/action.yml +++ b/fork-ci/actions/fleet-route/action.yml @@ -49,7 +49,9 @@ outputs: starts (the job's runs-on adds -a to its label); {} when re-runs take "fallback". value: ${{ steps.route.outputs.rerun }} watch: - description: true when a job went to a dispatcher or static pool (a job that bounds its wait can then start). + description: >- + true when at least one job went to a dispatcher or static pool, else false. Nothing starts by itself on it: a + project that runs its own job to watch its fleet jobs can start that job only when this is true. value: ${{ steps.route.outputs.watch }} runs: using: composite