Skip to content

cleaner: switch the project retention tool and pruner over to fork-cleaner #491

Closed

Description

@scuttletoby

Follow-up of #470.

Switch the project's own retention tool and worktree pruner over to the general fork-cleaner, so one tool with one set of refusals cleans every host:

  • a thin project config (its folders, patterns and days) in the machine file instead of the project's own script
  • an installer and a daily timer for fork-cleaner run (the first PR has neither: nothing runs it by itself yet)
  • the kit row in fork-kit/README.md and a cleaner block in common/machine.example.json
  • any project rule the general tool does not have yet becomes a general rule with tests, not project code
  • tune the credential-name check if it keeps too much in practice (it is by name only)

Before the switch: run fork-cleaner plan next to the old tool for a few days and compare what each would delete; anything the new one would delete that the old one kept is a bug until explained.

State question: both tools installed at once (only one timer may run), the old tool's log and state left behind (kept, not migrated), an empty project config (nothing happens).

Activity

  1. scuttletoby commented on Nov 18, 2026

    @scuttletoby
    OwnerAuthor

    Note for the wiring, from the review of #499 (fork-cleaner report):

    The report moves its mark (<logFile>.reported) as soon as it prints the line. The next report then counts only the deletes after that point. So if the daily-report wiring loses the printed line (a failed send, or output that is dropped), that line's "freed" figure is gone for good.

    The wiring must do one of these:

    • keep the line until it is delivered, for example by writing it to a file first and sending it from there; or
    • change the report so the mark moves only after delivery is confirmed, for example with a --no-mark run plus a separate step that moves the mark.

    Two reports at once are already safe: the second one exits 1 with "another report is running" and leaves the mark alone.

  2. scuttletoby commented on Nov 18, 2026

    @scuttletoby
    OwnerAuthor

    Phase 1, first compare on the main dev host (2026-11-18). fork-cleaner is installed from #500 (head cc0bd04, green) in compare mode only: one user timer runs fork-cleaner daily (plan + report + compare), and nothing is deleted. The numbers below are from its first run. The old retention timer and the old worktree pruner still run as before. Each old tool is compared through its own dry run (plan, and prune-worktrees without --apply).

    Totals: fork-cleaner would delete 219. The old tools would delete 246: retention 240 temp entries, the pruner 6 worktrees.

    Class Count What it is
    both 205 199 temp entries (12 name families) + 6 merged worktrees
    new only 14 merged worktrees; all explained, see below
    old only 41 all temp entries; fork-cleaner is stricter, see below

    New only (14): explained, not bugs.

    • The old pruner keeps each of these for "a commit that only its HEAD reflog holds" (an amend or rebase).
    • In all 14, a branch's reflog in the shared clone still holds that commit: directly in 13, and in 1 as the parent of a commit a branch reflog names.
    • Removing a worktree removes only its own HEAD log, and fork-cleaner deletes no branches, so nothing is lost. This is the documented commits only in its reflog rule.
    • Phase 2 note: those commits then live only in the branch reflogs (git expires unreachable reflog entries after 30 days). The old pruner also deletes merged branches that have no worktree. That stops with its timer in phase 2, and fork-cleaner has no such rule.

    Old only (41): fork-cleaner keeps them (stricter by design, plus one timing case).

    Count Reason fork-cleaner keeps them
    33 Temp folders that are git worktrees whose main repository is gone: their .git file points to a folder that no longer exists. fork-cleaner refuses with "could not read" (git status fails), since it cannot tell what is only there. The old tool deletes by age alone.
    5 Content is older than the limit, but the inode change time is under it. fork-cleaner counts both times on purpose: a copy keeps old content times.
    1 Crossed the 7-day limit in the minutes between the plan and the old tool's dry run (timing, not a rule).
    1 A file inside has a credential name (.env; guard refusal).
    1 A git repository inside with commits that are on no remote (guard refusal).

    Not compared (the old tools print no path for these):

    • the build-cache trim line (build caches stay, owner decision 11-18);
    • 13 merged branches with no worktree, which the old pruner would delete.

    Found and fixed during the compare:

    • Tool: a worktree whose branch was deleted under it (HEAD 0000…) ended in "could not read", so every daily run would exit 1. It is now kept as no commit, with a test and a red check. This is commit cc0bd04 in fork-kit/cleaner: installer, daily timer (compare mode), compare, report --out #500.
    • Host config: the first run listed only one of the five w4-* temp name families, and 79 entries showed as old only. This was the host's thin project config, not the tool. It is fixed in the machine file.

    Daily exit code stays 1 for now. In compare mode, the 14 new-only entries and the 33 unreadable temp folders each make it 1. That is the intended signal until they are explained or cleared.

  3. scuttletoby commented on Nov 18, 2026

    @scuttletoby
    OwnerAuthor

    Switched to run mode on the main dev host: the two old project timers (retention, worktree pruner) are disabled and cleaner.daily.mode is run. The compare's only new-only items were merged worktrees whose reflog commits are all still reachable elsewhere (checked one by one). The two jobs the old tools did that fork-cleaner has no rule for yet are #519.

  4. scuttletoby commented on Nov 22, 2026

    @scuttletoby
    OwnerAuthor

    (Reposted 2026-11-22 with private details removed; original 2026-11-18.)

    Main dev host, 2026-11-18 (second compare, new version).

    Installed: release ci/v17 (195d604, built from main 49a7c5d, so it has #511 compress-cold and #512 agent-scratch with keep first), installed with bootstrap.sh --timer as the README says. self-check OK. The previous version folder (cc0bd04) is kept.

    Machine file: only cleaner.agentScratch: {"enabled": true} was added (defaults otherwise). compressCold is not set, so that rule does nothing. A dated backup is beside the file.

    Plan (read only): would delete 328 (5.9 GB), kept 118.

    Rule Would delete Kept
    worktrees 21 merged worktrees (5.8 GB) 57: no merged PR 12, branch not listed 15, ignored files 9, no branch 6, uncommitted 6, commits not in the merged PR 3, merged under 24 h 2, no commit 2, base branch 1, commits only in its reflog 1
    tmp 307 entries (28.9 MB), all from the configured name patterns 61: orphaned worktree 59, credential name 1, unpushed git commits 1
    agent-scratch 0 none listed
    compress-cold off (no folders) -

    agent-scratch has nothing to take yet: every session folder (about 5,800) has an inode time from 2026-11-17 or later, so none is 3 days old. The first ones become old enough around 2026-11-20 23:50 UTC. Nothing is kept first yet. No transcript, live session, build cache, unpushed worktree or path outside the roots is in the plan.

    Compare: fork-cleaner 329, old tools 383 (retention 376, prune-worktrees 7). Both 315, old only 68, new only 14.

    • New only (14): all are merged worktrees that the old pruner keeps for "a commit that only its HEAD reflog holds". This is the same explanation as the first compare. I checked again: in all 14, every commit in the worktree's HEAD reflog can be reached from a branch ref or branch reflog of the shared clone, so removing the worktree loses nothing.
    • Old only (68): 59 orphaned temp worktrees, 1 credential name, 1 unpushed git, and 7 too new by inode time.

    Daily run (fork-cleaner daily, compare mode) exited 0: new only 0, old only 84, both 316. All 21 worktrees were kept as in use during that run. A fresh plan right after shows all 21 as deletable again, so something touched them for a short time and the cleaner kept them, as it should. Report line: worktrees 21.7 GB, temp 0.6 GB, agent scratch 42.0 GB, agent tool homes 11.1 GB, T3 data 10.1 GB, cold folders 0.0 GB, build caches no rule yet; freed 0.0 GB.

    Mode now: still compare. Run mode would be refused, because both timers in oldTimers (project-retention.timer, project-prune-worktrees.timer) are still enabled and active as user timers. I did not turn them off. Next step: disable those two timers (systemctl --user disable --now ...), then set cleaner.daily.mode to "run". Note that the old pruner also deletes merged branches that have no worktree (14 in its current dry run). fork-cleaner has no rule for that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions