Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/acceptance/thread-usage/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Current chat usage acceptance

Verified on Windows with Chrome on 2026-10-22 (UTC+08:00).

Settings → Usage now offers **All chats** and **Current chat**. All chats and
30 days remain the defaults. Scope and reporting period are independent; select
All time to inspect the current chat's entire saved usage. The current-chat
option is disabled when no chat is selected.

## Actual application records

The before image uses the production frontend and Go API built from
`a07575e9ee863f94cf65d5add5623d27b7358d69`. The after images use this change's
production builds, restarted on the same port and SQLite home. Both versions
read the same saved records.

A local Anthropic-compatible HTTP provider returned deterministic replies and
usage receipts. The production API created both chats and executed all six
turns through `POST /threads/{thread_id}/turns`. After two replies in each chat,
the production `run cancel` CLI ended its idle Run; the next API turn created
the successor Run. No API response, DOM, or database record was fabricated for
the screenshots. The local model has no configured pricing or cache details.

| Scope | Runs | Replies / calls | Input | Output | Total | Calls without a recorded cost |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| All chats | 4 | 6 | 1,600 | 320 | 1,920 | 6 |
| Usage chat A | 2 | 3 | 500 | 100 | 600 | 3 |
| Usage chat B | 2 | 3 | 1,100 | 220 | 1,320 | 3 |

The actual Thread bindings were:

- A: `thread-run-20261021210427-a65ab6aa350f`, Runs
`run-20261021210427-a65ab6aa350f` and `run-20261021210813-6efb37dd8a49`.
- B: `thread-run-20261021210653-246739963ecb`, Runs
`run-20261021210653-246739963ecb` and `run-20261021210813-98cf5c82a848`.

Real browser requests to `/api/v1/usage?period=all&utc_offset_minutes=480`
returned HTTP 200. Adding each Thread's `thread_id` returned the corresponding
totals above and exactly one recent conversation. Switching A → B while current
chat remained selected changed 600 → 1,320, including model totals, trend and
activity. Returning to the new-chat settings route disabled Current chat and
restored 1,920. Missing cost and cache coverage remained `—` with the existing
coverage explanation. The 720-pixel viewport wrapped the controls without overlap.

## Screenshots

Before: application-wide usage, with no chat scope control.

![Before: all chats](before-all-chats.png)

After: all three replies across chat A's two Runs.

![After: chat A](after-chat-a.png)

After switching the current chat to B, using the same API and saved data.

![After: chat B](after-chat-b.png)

Current chat at 720 × 1,000.

![After: narrow layout](after-chat-b-narrow.png)

## Focused verification

- The HTTP regression first failed against the baseline: a valid `thread_id`
returned 400. It passes with optional scope support and retains original defaults,
read authorization, invalid-query rejection and the missing-Thread 404.
- `go test ./internal/store ./internal/httpapi -run '^(TestThreadUsageStatistics|TestUsageStatistics)' -count=1`
passed. A real Thread continuation binds two Runs. Scoped receipts include
root, specialist, compaction and fan-out calls; all four sources from another
chat are excluded. Both priced and unpriced settlements are filtered, with
existing period, coverage, untracked-cost and global semantics retained.
Archived chats remain readable; deleted and missing chats return not found.
- `TestUsageStatisticsEmptyReadOnlyAndValidation` passed with both global and
scoped queries under `PRAGMA query_only=ON` and unchanged SQLite write counts.
- `go test ./internal/httpapi -run '^TestOpenAPI' -count=1` passed.
- `npm test -- src/v2/components/usage-settings.test.tsx` passed (10 tests),
including Settings prop wiring, disabled current scope, independent periods,
an aborted A request resolving after B, and fallback to global when no chat is open.
- API generation, `npm run check:api`, and `npm run build` passed. Vite reported
the existing large-chunk warning.

Only the documented live workflow was checked in Chrome. Settlement pricing,
auxiliary sources and delayed-response ordering were covered by the focused tests.
Binary file added docs/acceptance/thread-usage/after-chat-a.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/acceptance/thread-usage/after-chat-b.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
13 changes: 13 additions & 0 deletions docs/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -34769,6 +34769,16 @@
"minimum": -840,
"type": "integer"
}
},
{
"name": "thread_id",
"in": "query",
"description": "Optional exact Thread identity. Restricts all receipt, cost, model, day and activity statistics to every Run bound to this Thread. Omit for application-wide usage. Active and archived Threads are supported; missing or deleted Threads return NOT_FOUND.",
"schema": {
"maxLength": 256,
"minLength": 1,
"type": "string"
}
}
],
"responses": {
Expand Down Expand Up @@ -34811,6 +34821,9 @@
"403": {
"$ref": "#/components/responses/Forbidden"
},
"404": {
"$ref": "#/components/responses/NotFound"
},
"414": {
"$ref": "#/components/responses/RequestTooLarge"
},
Expand Down
3 changes: 2 additions & 1 deletion internal/httpapi/openapi.go
Original file line number Diff line number Diff line change
Expand Up @@ -2352,9 +2352,10 @@ func openAPIOperationSpecs() []openAPIOperationSpec {
NotFound: true, Parameters: append([]openAPIParameter{runID}, paginationParameters()...)},
{Path: "/api/v1/usage", OperationID: "getUsageStatistics", Summary: "Inspect local application usage", Tag: "System",
Description: "Aggregates this application's durable model terminal receipts across root agents, specialists, context compaction, and read-only fan-out; failed sent calls count and not-sent receipts are excluded. Tokens, model groups, recent Threads, sparse days, and untracked_calls use terminal receipt time. Cache tokens are input subsets; usage_reported_calls and cache_usage_reported_calls identify missing coverage. Missing provider/model identities remain empty strings. Costs use reservation settled_at independently of receipt time: known_cost_micros sums only actual priced USD settlements, unpriced_calls counts unknown-cost settlements, and untracked_calls counts receipts without a matched settlement. Reservation estimates never become known cost. Calendar dates use a fixed UTC offset; 30d includes today and the preceding 29 dates, all starts at the earliest receipt or settlement, and empty history starts today. The sparse activity_days always cover the last 182 calendar dates including today, independently of period. Collections are sorted deterministically; recent_sessions includes at most eight Threads with selected-period receipts and uses their latest receipt as last_activity_at. Unbound historical Runs contribute to totals and unassigned_calls.",
DataType: reflect.TypeOf(domain.UsageStatistics{}), Parameters: []openAPIParameter{
DataType: reflect.TypeOf(domain.UsageStatistics{}), NotFound: true, Parameters: []openAPIParameter{
{Name: "period", In: "query", Description: "Calendar reporting period", Schema: map[string]any{"type": "string", "enum": []string{"today", "30d", "all"}, "default": "30d"}},
{Name: "utc_offset_minutes", In: "query", Description: "Fixed UTC offset in minutes, without daylight-saving transitions", Schema: map[string]any{"type": "integer", "minimum": -840, "maximum": 840, "default": 0}},
{Name: "thread_id", In: "query", Description: "Optional exact Thread identity. Restricts all receipt, cost, model, day and activity statistics to every Run bound to this Thread. Omit for application-wide usage. Active and archived Threads are supported; missing or deleted Threads return NOT_FOUND.", Schema: map[string]any{"type": "string", "minLength": 1, "maxLength": 256}},
}},
{Path: "/api/v1/sessions", OperationID: "listSessions", Summary: "List Sessions", Tag: "Sessions",
Description: "Returns a bounded creation-ordered keyset page of persisted Sessions. Updates and later Sessions cannot shift continuation pages.", DataType: reflect.TypeOf(SessionView{}),
Expand Down
29 changes: 22 additions & 7 deletions internal/httpapi/usage_statistics.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import (
func (a *API) usageStatistics(request *http.Request) (any, *Page, error) {
values := request.URL.Query()
for name, items := range values {
if (name != "period" && name != "utc_offset_minutes") || len(items) != 1 || items[0] == "" {
if (name != "period" && name != "utc_offset_minutes" && name != "thread_id") || len(items) != 1 || items[0] == "" {
return nil, nil, apperror.New(apperror.CodeInvalidArgument, "usage query parameter is invalid")
}
}
Expand All @@ -33,13 +33,28 @@ func (a *API) usageStatistics(request *http.Request) (any, *Page, error) {
return nil, nil, apperror.New(apperror.CodeInvalidArgument, "usage UTC offset must be an integer between -840 and 840")
}
}
source, ok := a.store.(interface {
GetUsageStatistics(context.Context, string, int, time.Time) (domain.UsageStatistics, error)
})
if !ok {
return nil, nil, apperror.New(apperror.CodeFailedPrecondition, "usage statistics are unavailable")
var result domain.UsageStatistics
var err error
if threadID := values.Get("thread_id"); threadID != "" {
if err := validateIdentity(threadID, "usage Thread identity"); err != nil {
return nil, nil, err
}
source, ok := a.store.(interface {
GetThreadUsageStatistics(context.Context, string, string, int, time.Time) (domain.UsageStatistics, error)
})
if !ok {
return nil, nil, apperror.New(apperror.CodeFailedPrecondition, "Thread usage statistics are unavailable")
}
result, err = source.GetThreadUsageStatistics(request.Context(), threadID, period, offset, time.Now().UTC())
} else {
source, ok := a.store.(interface {
GetUsageStatistics(context.Context, string, int, time.Time) (domain.UsageStatistics, error)
})
if !ok {
return nil, nil, apperror.New(apperror.CodeFailedPrecondition, "usage statistics are unavailable")
}
result, err = source.GetUsageStatistics(request.Context(), period, offset, time.Now().UTC())
}
result, err := source.GetUsageStatistics(request.Context(), period, offset, time.Now().UTC())
if err != nil {
return nil, nil, err
}
Expand Down
22 changes: 20 additions & 2 deletions internal/httpapi/usage_statistics_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import (
"strings"
"testing"

"cyberagent-workbench/internal/application"
"cyberagent-workbench/internal/domain"
"cyberagent-workbench/internal/store"
)
Expand Down Expand Up @@ -42,7 +43,20 @@ func TestUsageStatisticsReadAPIQueryAndAuthorization(t *testing.T) {
if result.Period != "all" || result.UTCOffsetMinutes != 480 {
t.Fatalf("usage query not forwarded: %+v", result)
}
for _, query := range []string{"period=week", "period=", "period=today&period=all", "utc_offset_minutes=", "utc_offset_minutes=841", "utc_offset_minutes=-841", "utc_offset_minutes=1.5", "utc_offset_minutes=0&utc_offset_minutes=1", "unknown=all"} {
_, run, err := application.NewRunService(st).Create(t.Context(), application.CreateRunRequest{Goal: "usage scope", Profile: "review"})
if err != nil {
t.Fatal(err)
}
threadID := domain.InitialThreadID(run.ID)
response = request(http.MethodGet, "/api/v1/usage?period=all&utc_offset_minutes=480&thread_id="+threadID, testAccessToken)
decodeDataStatus(t, response, http.StatusOK, &result)
if result.Period != "all" || result.UTCOffsetMinutes != 480 || result.Totals.Calls != 0 {
t.Fatalf("scoped usage query not forwarded: %+v", result)
}
if response := request(http.MethodGet, "/api/v1/usage?thread_id=missing-thread", testAccessToken); response.Code != http.StatusNotFound {
t.Fatalf("unknown Thread must not become empty global usage: %d %s", response.Code, response.Body.String())
}
for _, query := range []string{"period=week", "period=", "period=today&period=all", "utc_offset_minutes=", "utc_offset_minutes=841", "utc_offset_minutes=-841", "utc_offset_minutes=1.5", "utc_offset_minutes=0&utc_offset_minutes=1", "thread_id=", "thread_id=a&thread_id=b", "thread_id=%20a", "thread_id=a%2Fb", "thread_id=a%0Ab", "unknown=all"} {
response := request(http.MethodGet, "/api/v1/usage?"+query, testAccessToken)
if response.Code != http.StatusBadRequest {
t.Fatalf("invalid query accepted: %s: %+v", query, response)
Expand All @@ -67,9 +81,13 @@ func TestUsageStatisticsOpenAPIContract(t *testing.T) {
t.Fatal(err)
}
operation := spec.Paths["/api/v1/usage"].Get
if operation == nil || !operation.ReadOnly || operation.OperationID != "getUsageStatistics" || len(operation.Parameters) != 2 {
if operation == nil || !operation.ReadOnly || operation.OperationID != "getUsageStatistics" || len(operation.Parameters) != 3 {
t.Fatalf("usage contract missing: %+v", operation)
}
thread := operation.Parameters[2]
if thread.Name != "thread_id" || thread.In != "query" || thread.Required || thread.Schema["minLength"] != float64(1) || thread.Schema["maxLength"] != float64(256) || operation.Responses["404"] == nil {
t.Fatalf("optional Thread scope contract missing: %+v", thread)
}
for _, name := range []string{"UsageStatistics", "UsageTotals", "UsageCost", "UsageModel", "UsageSession", "UsageDay", "UsageActivityDay"} {
if spec.Components.Schemas[name] == nil {
t.Fatalf("usage response type %s absent from OpenAPI", name)
Expand Down
40 changes: 34 additions & 6 deletions internal/store/usage_statistics.go
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,19 @@ func (r usageReceipt) monetaryIdentity() (string, int64) {
// or acquiring a writer lock. Terminal receipts alone contribute tokens, so
// checkpoints, started events, and monetary ledgers cannot double count usage.
func (s *SQLiteStore) GetUsageStatistics(ctx context.Context, period string, utcOffsetMinutes int, now time.Time) (domain.UsageStatistics, error) {
return s.getUsageStatistics(ctx, "", period, utcOffsetMinutes, now)
}

// GetThreadUsageStatistics includes every Run bound to the selected Thread,
// using the same receipt and settlement rules as application-wide usage.
func (s *SQLiteStore) GetThreadUsageStatistics(ctx context.Context, threadID, period string, utcOffsetMinutes int, now time.Time) (domain.UsageStatistics, error) {
if threadID == "" {
return domain.UsageStatistics{}, apperror.New(apperror.CodeInvalidArgument, "usage Thread identity is required")
}
return s.getUsageStatistics(ctx, threadID, period, utcOffsetMinutes, now)
}

func (s *SQLiteStore) getUsageStatistics(ctx context.Context, threadID, period string, utcOffsetMinutes int, now time.Time) (domain.UsageStatistics, error) {
if period == "" {
period = "30d"
}
Expand All @@ -91,7 +104,15 @@ func (s *SQLiteStore) GetUsageStatistics(ctx context.Context, period string, utc
return result, err
}
defer finish()
rows, err := tx.QueryContext(ctx, `SELECT e.run_id,e.sequence,e.source,e.type,COALESCE(e.subject_id,''),e.created_at,e.payload_json,
var scopeArgs []any
if threadID != "" {
var exists int
if err := tx.QueryRowContext(ctx, `SELECT 1 FROM threads WHERE id=? AND status<>?`, threadID, domain.ThreadDeleted).Scan(&exists); err != nil {
return result, err
}
scopeArgs = append(scopeArgs, threadID)
}
receiptQuery := `SELECT e.run_id,e.sequence,e.source,e.type,COALESCE(e.subject_id,''),e.created_at,e.payload_json,
COALESCE(t.id,''),COALESCE(t.title,''),COALESCE(c.id,0),
CASE WHEN f.usage_recorded=1 THEN f.input_tokens END,
CASE WHEN f.usage_recorded=1 THEN f.output_tokens END,
Expand All @@ -102,10 +123,13 @@ func (s *SQLiteStore) GetUsageStatistics(ctx context.Context, period string, utc
LEFT JOIN readonly_fanout_model_calls f ON e.source='readonly_fanout' AND f.run_id=e.run_id
AND f.execution_id=json_extract(e.payload_json,'$.execution_id') AND f.shard_ordinal=json_extract(e.payload_json,'$.shard')
AND f.attempt_number=json_extract(e.payload_json,'$.attempt') AND f.provider=json_extract(e.payload_json,'$.provider') AND f.model=json_extract(e.payload_json,'$.model')
WHERE (e.source IN ('model_gateway','specialist_model_gateway') AND e.type IN ('model.started','model.completed','model.failed'))
WHERE ((e.source IN ('model_gateway','specialist_model_gateway') AND e.type IN ('model.started','model.completed','model.failed'))
OR (e.source='thread_context_compaction' AND e.type IN ('context_compaction.model_completed','context_compaction.model_failed'))
OR (e.source='readonly_fanout' AND e.type IN ('readonly_fanout.shard_completed','readonly_fanout.shard_failed','readonly_fanout.shard_cancelled'))
ORDER BY e.run_id,e.sequence`)
OR (e.source='readonly_fanout' AND e.type IN ('readonly_fanout.shard_completed','readonly_fanout.shard_failed','readonly_fanout.shard_cancelled')))`
if threadID != "" {
receiptQuery += ` AND binding.thread_id=?`
}
rows, err := tx.QueryContext(ctx, receiptQuery+` ORDER BY e.run_id,e.sequence`, scopeArgs...)
if err != nil {
return result, err
}
Expand Down Expand Up @@ -180,11 +204,15 @@ func (s *SQLiteStore) GetUsageStatistics(ctx context.Context, period string, utc
// Cost is independently dated by the actual reservation settlement. The
// settled amount can contain a conservative reservation estimate, so only
// a priced settlement's actual_micros is a known cost.
rows, err = tx.QueryContext(ctx, `SELECT r.run_id,r.scope,r.attempt_number,r.provider,r.model,r.created_at,r.settled_at,r.settled_micros,
costQuery := `SELECT r.run_id,r.scope,r.attempt_number,r.provider,r.model,r.created_at,r.settled_at,r.settled_micros,
COALESCE((SELECT MIN(sequence) FROM run_events WHERE run_id=r.run_id AND source='monetary_budget' AND type='monetary.budget_reserved' AND subject_id=r.id),0),
COALESCE(e.payload_json,'{}') FROM run_monetary_reservations r
LEFT JOIN run_events e ON e.run_id=r.run_id AND e.sequence=(SELECT MIN(sequence) FROM run_events WHERE run_id=r.run_id AND source='monetary_budget' AND type='monetary.budget_settled' AND subject_id=r.id)
WHERE r.status='settled' ORDER BY r.run_id,r.scope,r.attempt_number`)
WHERE r.status='settled'`
if threadID != "" {
costQuery += ` AND EXISTS (SELECT 1 FROM thread_runs binding WHERE binding.run_id=r.run_id AND binding.thread_id=?)`
}
rows, err = tx.QueryContext(ctx, costQuery+` ORDER BY r.run_id,r.scope,r.attempt_number`, scopeArgs...)
if err != nil {
return result, err
}
Expand Down
Loading
Loading