Repository navigation
feat(plugins): the usage screen shows Claude Code's 5-hour and weekly limits #185
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
Stener1
wants to merge
4
commits into
cosscom:main
Choose a base branch
from
Stener1:usage-claude-limits
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
87dad9c
feat(plugins): the usage screen shows claude code's 5-hour and weekly…
Stener1 a186ba5
feat(box): the usage screen shows claude code's limits from structure…
Stener1 6a48d28
test(skills): verify-berthd drives the usage screen's claude limits o…
Stener1 8bda77d
fix(plugins): claude limits show in today's view after midnight, and …
Stener1 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # Usage: Claude Code's plan limits | ||
|
|
||
| The Usage plugin (`plugins/usage`, off until turned on in Settings → Plugins) shows each agent's 5-hour and weekly plan windows: how much is used, as of when, and when each resets. Codex's come from its transcripts. Claude Code's come from `~/.berth/limits/<account>.json`, which the box writes from every structured turn (each turn's `rate_limit_event`, `internal/box/claudelimits.go`), and which a status line can also write for agents on Claude Code's own screen (the plugin's README). | ||
|
|
||
| ## Sub-features | ||
|
|
||
| - `limits-structured`: a turn of a structured Claude Code agent writes its account's file: `used_percentage` is the event's utilization × 100, `resets_at` in seconds. A newer reading replaces an older one, a window the turn leaves out keeps the file's, and a log read back after berthd restarts writes nothing. All of that is unit-tested in `internal/box/claudedriver_test.go` (`TestAStructuredSessionRecordsItsAccountsPlanLimits`, `TestPlanLimitsKeepTheLatestReading`). | ||
| - `limits-account`: the file is named as the plugin names the account: `default` for `~/.claude`, the folder's name for `~/.berth/accounts/claude/<name>`, and no file for any other folder (`TestAClaudeAccountIsNamedAsTheUsagePluginNamesIt`). | ||
| - `limits-screen`: the Usage tab's Claude Code panel shows the file's windows as meters ("5-hour limit", "Weekly limit"), the same numbers Claude Code's own `/usage` shows. Home's **Agent usage** widget shows them too. | ||
|
|
||
| ## How to get to it (user POV) | ||
|
|
||
| - Settings → Plugins → turn on **Usage & accounts**; then **Usage** (sidebar, under More) or ⌘K **Show agent usage**. **Switch agent account** opens its Accounts tab, where an account's **…** menu sets which login new sessions use. | ||
|
|
||
| ## Driving it with drive.mjs | ||
|
|
||
| - `../scripts/scenarios/usage-limits-live.mjs`, on a real box: it needs a signed-in Claude account and costs one short Haiku turn. The dev box gets a HOME of its own (so it never writes the real `~/.berth/limits`) with the account linked in as `acme-max`: | ||
|
|
||
| ```sh | ||
| export VERIFY_BOX_DIR=/tmp/vb-limits VERIFY_BOX_HOME=/tmp/vb-limits/home VERIFY_CLAUDE_ACCOUNT=/path/to/signed-in/claude-config-dir VERIFY_DRIVER=claude | ||
| .claude/skills/verify-berthd/scripts/dev-box.sh up && eval "$(.claude/skills/verify-berthd/scripts/dev-box.sh env)" | ||
| node .claude/skills/verify-berthd/scripts/drive.mjs usage-limits-live --port $PORT --agent http://127.0.0.1:9378 | ||
| ``` | ||
|
|
||
| It turns the plugin on, picks acme-max for the box in the Accounts tab, starts session `limits` (structured, Haiku, "Reply with just: ok"), waits for `acme-max.json`, reads `/usage` with `../scripts/claude-usage.sh` and checks that the file, `/usage` and the screen's meters agree. It removes the session after; `dev-box.sh down` removes the rest. | ||
| - `../scripts/claude-usage.sh DIR` prints what `/usage` says for the account in `DIR` (`five_hour=45 seven_day=23`). It starts Claude Code's own screen in a tmux server of its own and trusts an empty `/tmp/claude-usage-w` the first time. | ||
|
|
||
| ## Gotchas | ||
|
|
||
| - An agent still running at `dev-box.sh down` writes its folders again as it exits, after the box's folder is removed: under `VERIFY_BOX_HOME`, `acme-max` comes back as a folder signed in nowhere ("Not logged in"). The scenario removes its session even when it fails, and `up` refuses such a leftover; remove `VERIFY_BOX_DIR` a few seconds after `down`. | ||
| - Run with every `BERTH_*` unset (a /ship worker's `BERTH_DRIVER=structured` leaks into the dev box's agents). | ||
| - The linked account is a real one: the Usage screen lists its real sessions and projects. Shoot only the limits rows (`shot(name, locator)`), never the whole screen, and keep `failed.png` out of a PR. | ||
| - `/usage` and the file are two readings a few seconds apart: on an account other agents are using at the same time, they can differ by a point. Run it on an account nothing else is using. | ||
| - The plugin reads limits only for known accounts (`default` and the folders under `~/.berth/accounts/claude`); a file for any other name is ignored. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| #!/usr/bin/env bash | ||
| # claude-usage.sh DIR [WORKDIR]: what Claude Code's own /usage says of the | ||
| # account in DIR (a CLAUDE_CONFIG_DIR), the ground truth for the Usage | ||
| # screen's Claude limits (features/usage-limits.md). It starts Claude Code | ||
| # on its own screen in a tmux server of its own, trusts WORKDIR (default | ||
| # /tmp/claude-usage-w, made empty) if asked, types /usage, and prints | ||
| # five_hour=45 seven_day=23 | ||
| # Nothing else: the panel also lists the account's sessions and MCP | ||
| # servers, which stay out of evidence. | ||
| set -euo pipefail | ||
| dir=${1:?usage: claude-usage.sh CLAUDE_CONFIG_DIR [WORKDIR]} | ||
| # The default folder keeps its path, so Claude Code asks to trust it only | ||
| # once, and is emptied every run so nothing from another one is in it. | ||
| if [[ -n ${2:-} ]]; then work=$2; else work=/tmp/claude-usage-w; rm -rf "$work"; fi | ||
| sock=$(mktemp -d /tmp/claude-usage-tmux.XXXXXX) | ||
| tm() { env TMUX_TMPDIR="$sock" tmux -L usage "$@"; } | ||
| trap 'tm kill-server 2>/dev/null || true; rm -rf "$sock"' EXIT | ||
| mkdir -p "$work" | ||
| tm new-session -d -s u -x 160 -y 60 -c "$work" -e "CLAUDE_CONFIG_DIR=$dir" claude --model haiku | ||
| screen() { tm capture-pane -p -t u -S -80; } | ||
| # Until its prompt shows, saying yes to the folder's trust question. Its | ||
| # cursor starts on "No, exit" and the screen shows before it reads keys, | ||
| # so Enter goes only once the cursor is on yes. | ||
| for _ in $(seq 60); do | ||
| s=$(screen) | ||
| if grep -q "❯ Yes, I trust this folder" <<<"$s"; then | ||
| tm send-keys -t u Enter | ||
| elif grep -q "Yes, I trust this folder" <<<"$s"; then | ||
| tm send-keys -t u Down | ||
| elif grep -q "❯" <<<"$s" && ! grep -q "Enter to confirm" <<<"$s"; then | ||
| break | ||
| fi | ||
| sleep 0.5 | ||
| done | ||
| # A notice can take the first keys: type /usage until its panel shows. | ||
| for _ in 1 2 3; do | ||
| tm send-keys -t u "/usage" && sleep 1.5 && tm send-keys -t u Enter | ||
| for _ in $(seq 30); do | ||
| s=$(screen) | ||
| grep -q "Current week (all models)" <<<"$s" && break 2 | ||
| sleep 0.5 | ||
| done | ||
| done | ||
| pct() { grep -A1 -F "$1" <<<"$s" | grep -o '[0-9]\+% used' | head -1 | tr -dc 0-9; } | ||
| five=$(pct "Current session") | ||
| week=$(pct "Current week (all models)") | ||
| [[ -n $five && -n $week ]] || { echo "no /usage panel on screen" >&2; exit 1; } | ||
| echo "five_hour=$five seven_day=$week" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
102 changes: 102 additions & 0 deletions
102
.claude/skills/verify-berthd/scripts/scenarios/usage-limits-live.mjs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| // Claude Code's plan limits on the Usage screen, from a structured agent | ||
| // (features/usage-limits.md): the Usage plugin is turned on, its Accounts | ||
| // tab puts the box's new sessions on account "acme-max", one cheap | ||
| // structured turn runs there, and the box writes what that turn said of the | ||
| // account's limits to ~/.berth/limits/acme-max.json. The screen's 5-hour | ||
| // and weekly meters show the same numbers as the file, and as Claude Code's | ||
| // own /usage (claude-usage.sh), the ground truth. | ||
| // | ||
| // On a dev box with a HOME of its own and a signed-in account linked in | ||
| // (../dev-box.sh): | ||
| // | ||
| // export VERIFY_BOX_DIR=/tmp/vb-limits VERIFY_BOX_HOME=/tmp/vb-limits/home VERIFY_CLAUDE_ACCOUNT=/path/to/signed-in/claude-config-dir | ||
| // dev-box.sh up && eval "$(dev-box.sh env)" | ||
| // node drive.mjs usage-limits-live --port $PORT --agent http://127.0.0.1:9378 | ||
| // | ||
| // The screen also lists the account's sessions and projects, so the shots | ||
| // keep only the limits. | ||
|
|
||
| import { execFileSync } from "node:child_process"; | ||
| import { readFileSync } from "node:fs"; | ||
| import { dirname, join } from "node:path"; | ||
| import { fileURLToPath } from "node:url"; | ||
|
|
||
| export const live = true; | ||
|
|
||
| const BOX = (process.env.VERIFY_WORKTREE ?? "devq/acme-shop").split("/")[0]; | ||
| const HOME = process.env.VERIFY_BOX_HOME; | ||
| const ACCOUNT = process.env.VERIFY_CLAUDE_ACCOUNT; | ||
| const SESSION = "limits"; | ||
|
|
||
| const limitsFile = () => { | ||
| try { | ||
| return JSON.parse(readFileSync(join(HOME, ".berth", "limits", "acme-max.json"), "utf8")); | ||
| } catch { | ||
| return null; | ||
| } | ||
| }; | ||
|
|
||
| export async function run({ page, shot, step, expect, call }) { | ||
| if (!HOME || !ACCOUNT) throw new Error("set VERIFY_BOX_HOME and VERIFY_CLAUDE_ACCOUNT, as dev-box.sh up had them"); | ||
| if (limitsFile()) throw new Error("the box already has limits for acme-max: start from a fresh dev box"); | ||
|
|
||
| step("turn on Usage & accounts in Settings → Plugins"); | ||
| await page.getByTestId("nav-settings").click(); | ||
| await page.getByTestId("settings-nav-plugins").click(); | ||
| const plugins = page.getByTestId("settings-plugins"); | ||
| await plugins.getByRole("switch", { name: "Turn on Usage & accounts" }).click({ timeout: 15_000 }); | ||
| await expect(plugins.getByRole("switch", { name: "Turn off Usage & accounts" })).toBeVisible(); | ||
| await page.getByRole("button", { name: "Back to app" }).click(); | ||
| await expect(page.getByTestId("nav-home")).toBeVisible(); | ||
|
|
||
| step("open Usage, Accounts: new sessions on the box use acme-max"); | ||
| await page.keyboard.press("Meta+k"); | ||
| await page.keyboard.type("Switch agent account"); | ||
| await page.getByRole("option", { name: /^Switch agent account/ }).click(); | ||
| await page.getByRole("button", { name: "Use acme-max for…" }).click({ timeout: 60_000 }); | ||
| await page.getByRole("menuitem", { name: `Everywhere on ${BOX}` }).click(); | ||
| await expect.poll(async () => (await call(`/v1/boxes/${BOX}/api/env`)).env?.CLAUDE_CONFIG_DIR ?? "", { timeout: 15_000 }).toContain("acme-max"); | ||
|
|
||
| step("one cheap structured turn on acme-max"); | ||
| const started = Date.now(); | ||
| try { | ||
| // Inside the try: a POST that times out may still have started it. | ||
| await call(`/v1/boxes/${BOX}/api/sessions`, { method: "POST", body: JSON.stringify({ name: SESSION, location: "acme-shop", agent: "claude", driver: "claude", model: "haiku", prompt: "Reply with just: ok" }) }); | ||
| await checkLimits({ page, shot, step, expect, started }); | ||
| } finally { | ||
| // Gone before the dev box is: an agent still running when it goes | ||
| // down writes its account folder again (dev-box.sh up refuses it). | ||
| await call(`/v1/boxes/${BOX}/api/sessions/${SESSION}`, { method: "DELETE" }).catch(() => {}); | ||
| } | ||
| } | ||
|
|
||
| async function checkLimits({ page, shot, step, expect, started }) { | ||
| await expect.poll(() => limitsFile()?.at ?? "", { timeout: 180_000, intervals: [1_000] }).not.toBe(""); | ||
| const file = limitsFile(); | ||
| const five = Math.round(file.limits.five_hour.used_percentage); | ||
| const week = Math.round(file.limits.seven_day.used_percentage); | ||
| const at = Date.parse(file.at); | ||
| if (!Number.isFinite(at) || at < started - 5_000) throw new Error(`the file's reading isn't from this turn: ${file.at}`); | ||
| step(`the box wrote acme-max.json: 5-hour ${five}%, weekly ${week}%, as of ${file.at}`); | ||
|
|
||
| const truth = execFileSync(join(dirname(fileURLToPath(import.meta.url)), "..", "claude-usage.sh"), [ACCOUNT], { encoding: "utf8" }).trim(); | ||
| const usage = Object.fromEntries(truth.split(" ").map((kv) => kv.split("=").map((v, i) => (i ? Number(v) : v)))); | ||
| step(`Claude Code's /usage: 5-hour ${usage.five_hour}%, weekly ${usage.seven_day}%`); | ||
| // Two readings seconds apart: an account in use elsewhere can move a | ||
| // point between them. | ||
| if (Math.abs(usage.five_hour - five) > 1 || Math.abs(usage.seven_day - week) > 1) throw new Error(`the file (${five}%, ${week}%) isn't what /usage says (${truth})`); | ||
|
|
||
| step("the Usage screen shows the same"); | ||
| await page.getByRole("group", { name: "Show" }).getByText("Usage", { exact: true }).click(); | ||
| // The screen read the box when it opened, before the turn; it reads again | ||
| // every 5 minutes, or now with Refresh. | ||
| await page.getByRole("button", { name: "Refresh" }).first().click(); | ||
| const fiveMeter = page.getByRole("meter", { name: "5-hour limit" }); | ||
| const weekMeter = page.getByRole("meter", { name: "Weekly limit" }); | ||
| await expect(fiveMeter).toHaveAttribute("aria-valuenow", String(five), { timeout: 90_000 }); | ||
| await expect(weekMeter).toHaveAttribute("aria-valuenow", String(week)); | ||
| const rows = fiveMeter.locator("xpath=ancestor::div[contains(@class,'border-t')][1]"); | ||
| await expect(rows).toContainText(`${five}% used`); | ||
| await expect(rows).toContainText(`${week}% used`); | ||
| await shot("1-usage-claude-limits", rows); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.