diff --git a/fork-kit/golive/README.md b/fork-kit/golive/README.md index efc39020a872c..35088c996a01d 100644 --- a/fork-kit/golive/README.md +++ b/fork-kit/golive/README.md @@ -334,6 +334,28 @@ start: the new launcher then idled and verify waited for minutes. (`handover`: `launcher`, `timeout`, `start failed`, `before the start ...`), for `launcher` the measured delay (`handoverMs`: at most this many ms from the claim to the let-go; with `handoverLate` when it reached 250 ms, so a server that idled on a very busy host is named), and any marker renewal that failed (`renewFailed`). +- **Root cause, a run that ends with T3 stopped (#530).** A W1 (and a switch) may end, fail or be killed with T3 + stopped on purpose (the home may be half-restored). A lock held by the run's own process would be gone with it, + and the next SSH launch would start a server on that home. So right after the stop the kit starts a holder, a + transient unit of its own (`t3-golive-hold-`, running `t3-golive hold --home `), that + takes the owner lock and keeps it until it is stopped; the run adopts it as its own lock. Every start the kit makes + goes through the same handover, so the launcher gets the home from the holder. It goes only to a launcher that + holds its serving lock, or once the service runs: a failed start, a start that returns without T3 running (`t3 + update` exits non-zero or times out, a build that never comes up), a failure before the start, or a kill leaves + the holder holding. The next run of the same `back` takes the holder over (no free moment, never a second one: + there is one unit name per home); a resumed verify stops one that is still there once the service runs T3. + `status` and `check` name the holder (do not start T3 by hand while it holds: its server would idle); `settle` + stops it (the owner says the home is right). It is found by its live process in `/proc/locks`, never by a file, + so nothing stale outlives it. Where `/proc/locks` cannot be read (the kit could never find the holder again) or + no transient unit can be started, the run holds the lock itself, as before, and the stop step says so (`holder: + null`, `holderError`); `status` and `check` name an active holder unit they cannot see there. A copy needs none: + its guard starts T3 again as soon as the copy run dies. +- **Known gap: a reboot ends the holder.** The service stays disabled, so nothing starts T3 at boot, but a T3 + desktop app connected over SSH that reconnects after the reboot finds the home free and may start its own server + on it while it may be half-restored. A holder that outlives a reboot would be an enabled unit on disk, state that + can outlive its reason (what finding the holder by its live process avoids), so the kit keeps the gap and says + it: `status` then fails `home holder` (none, but a run that stopped and disabled T3 has not ended ok) and says to + quit or disconnect that app until the same `back` ends ok. - **Backstop, a foreign server.** An SSH client older than 0.1.10 reads neither, and a server started by hand ignores both. When the runtime record names a live process that the boot service did not start and that itself holds `userdata/server-owner.sqlite` (a stale record whose pid was reused holds nothing and is never named), for @@ -402,7 +424,7 @@ empty state? | `copy --here` from a T3 terminal | Refused: stopping T3 would kill it half-way | | A desktop app's SSH launch while T3 is stopped (the gap) | It waits: the kit's marker is valid from before the stop until the new server is verified, and the kit holds the owner lock until the service's main process (the new launcher) holds its serving lock (an SSH launch's own probe of that lock, from another process, is not the launcher). No server of the old build takes the home (`tests/gap.test.py`, failing before the fix) | | ... the marker cannot be written | Refused before T3 stops, and for W1 before it says it stops T3 (`the update marker ... cannot be written; T3 was not stopped`; W2/W3: `... not restarted`) | -| ... the run is killed with T3 stopped | Nothing renews the marker: it runs out within 60 s and launches go on as without it. The dead run holds no lock. A rerun is refused (`Unfinished`) as for any killed run | +| ... the run is killed with T3 stopped | Nothing renews the marker: it runs out within 60 s. A dead copy holds no lock (its guard starts T3 at once). A W1 or switch that stopped T3 leaves the kit's holder (above): the home stays owned and launches keep waiting until the same `back` (or the switch guard) hands it to T3, `settle` stops it, or the machine reboots. A rerun is refused (`Unfinished`) as for any killed run; a W1's own rerun follows it and takes the holder over | | ... a newer marker (upstream's own update) is there at the end | Left alone: the kit removes only the bytes it wrote last (`marker: left: ...` in the verify step) | | ... the target build takes no serving lock (an older build) | Older than `0.1.10-nightly.20261114.4695`: the kit lets go of the owner lock just before the start, as before; the marker still holds fresh launches | | ... an SSH client older than 0.1.10, or a server started by hand, takes the home | Backstop: verify stops within about 10 s and names the server (pid, command) and the remedy. The copy reports T3 did not come back. The switch's W1 refuses too, so nothing is put back: the switch ends failed, says that the target waits for the home unverified, and ends the remedy with `t3-golive back --way w1 --lose-work-since