Add AI usage statistics with SQLite storage and Console UI by ruibaby · Pull Request #96 · halo-dev/plugin-ai-foundation

ruibaby · 2026-08-11T15:22:25Z

What this PR changes

Adds AI usage statistics for calls made through the audited model wrappers (language, embedding, reranking, image generation). Usage is recorded to a local SQLite database and surfaced through a new Console page.

Statistical model

A logical call is one subscribed SDK invocation; an execution is one actual model step (including retries). Calls and executions are recorded separately so retries and failures stay attributable.
Records capture input/output tokens (including cache-read, cache-creation, and reasoning subsets), caller plugin, feature, provider, model, operation, status, and timing.
usageQuality reports how complete the token data is (REPORTED_COMPONENTS, REPORTED_TOTAL, PARTIAL, ESTIMATED, MISSING); accountedTotalTokens is derived from the best available source.
The database never stores prompts, outputs, messages, tool arguments, or raw provider payloads. Errors are reduced to a bounded class name plus optional safe code.

Console API

GET /apis/console.api.aifoundation.halo.run/v1alpha1/usage-statistics/summary — totals with filter dimensions and coverage
GET .../usage-statistics/trends?resolution=DAY|HOUR — time-bucketed execution trends
GET .../usage-statistics/calls?cursor=...&size=... — cursor-paginated call list with filters (status, provider, model, caller, feature, operation, date range)
GET .../usage-statistics/calls/{id} — call detail with per-execution records
GET .../usage-statistics/health — persistence health (restore status, queue depth, dropped events)
POST .../usage-statistics/reset — clear statistics data

Resilience

SQLite file lives under %DATA_DIR%; daily rotation uses the same backup scheme as plugin-seo-tools.
Corrupt or incomplete databases are isolated and, when a valid backup exists, restored. Persistence failures never break model calls: statistics collection is best-effort and failures are surfaced via the health endpoint and a persistent warning banner in the Console.

Test plan

./gradlew build passes (backend tests included).
UI: pnpm lint, pnpm type-check, pnpm test:unit (39 files / 245 tests) pass.
End-to-end verified in the dev container via Chrome: real model call → summary counts + tokens updated, call detail with executions, filters, HOUR trends, and health endpoint all confirmed.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90bb8ea3b1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Introduce end-to-end AI usage statistics for Halo admins. This adds runtime usage instrumentation across language, embedding, rerank, and image generation calls, persists logical calls and execution attempts in SQLite with retention and health tracking, exposes summary/trend/history/reset console APIs, regenerates the OpenAPI client, and adds a Chinese console view with filters, charts, call details, and reset flow.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 62e9002a27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fc1e3a78d9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts: # app/src/main/java/run/halo/aifoundation/service/language/LanguageModelImpl.java

# Conflicts: # api-docs/openapi/v3_0/aiFoundationApis.json # app/build.gradle # app/src/main/java/run/halo/aifoundation/service/image/ImageGenerationModelImpl.java # app/src/main/java/run/halo/aifoundation/service/rerank/RerankingModelImpl.java # app/src/test/java/run/halo/aifoundation/service/embedding/EmbeddingModelImplTest.java # ui/src/api/generated/base.ts # ui/src/api/generated/models/model-parameter-definition-info.ts # ui/src/api/generated/models/provider-type-info.ts

…rience

Copilot

Copilot review overview

🟡 Changes recommended

Direct SDK timeout exceptions are misclassified as failed calls, producing inaccurate status statistics.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 1 · 1

Open (2)

Classify direct SDK timeout exceptions as timed out · New
Document cursor pagination instead of unsupported page parameters · New

What changed in this PR

Adds end-to-end AI usage tracking with provider normalization, resilient SQLite persistence, Console APIs, and a Vue administration interface.

Changes:

Records logical calls, executions, retries, token usage, status, and health.
Adds retention, rollups, recovery, filtering, trends, pagination, and reset.
Adds generated API bindings, Console UI, and extensive regression tests.

File	Description
`app/build.gradle`	Adds SQLite runtime support.
`app/.../endpoint/UsageStatisticsConsoleEndpoint.java`	Exposes statistics APIs.
`app/.../provider/**`	Normalizes provider usage metadata.
`app/.../service/audit/**`	Wraps SDK calls with usage recording.
`app/.../service/{embedding,image,language,rerank}/**`	Records physical model executions.
`app/.../service/observation/**`	Implements call and execution lifecycle accounting.
`app/.../service/usage/**`	Implements SQLite persistence, querying, recovery, and retention.
`app/src/test/**`	Covers providers, lifecycle, durability, APIs, and storage.
`api-docs/openapi/v3_0/aiFoundationApis.json`	Publishes the Console API schema.
`ui/src/api/generated/**`	Adds generated statistics client models and methods.
`ui/src/composables/use-usage-*.ts`	Manages filters, queries, polling, and pagination.
`ui/src/views/components/usage/**`	Implements statistics dashboards and call details.
`ui/src/views/UsageStatisticsView.vue`	Composes the statistics page.
`ui/src/views/ProviderManager.vue`	Adds the usage tab.
`ui/src/views/ProviderDetail.vue`	Standardizes timestamp formatting.
`ui/src/utils/usage*.ts`	Provides usage formatting and chart utilities.
`ui/src//__tests__/`	Covers Console components and composables.
`openspec/changes/add-ai-usage-statistics/**`	Documents requirements, design, and tasks.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

chatgpt-codex-connector Bot reviewed Aug 11, 2026

View reviewed changes

Comment thread ui/src/composables/use-usage-statistics.ts

Comment thread app/src/main/java/run/halo/aifoundation/service/usage/UsageCallSession.java Outdated

ruibaby force-pushed the feat/ai-usage-statistics branch from e63aad5 to 62bf82e Compare August 12, 2026 04:38

ruibaby force-pushed the feat/ai-usage-statistics branch from 62bf82e to 62e9002 Compare August 12, 2026 04:39

chatgpt-codex-connector Bot reviewed Aug 12, 2026

View reviewed changes

Comment thread app/src/main/java/run/halo/aifoundation/service/usage/SqliteUsageStatisticsStore.java Outdated

Comment thread app/src/main/java/run/halo/aifoundation/service/audit/AuditedLanguageModel.java Outdated

refactor: reuse shared UI formatters

fc1e3a7

chatgpt-codex-connector Bot reviewed Aug 12, 2026

View reviewed changes

Comment thread app/src/main/java/run/halo/aifoundation/service/audit/UsageCallRecorder.java

Comment thread app/src/main/java/run/halo/aifoundation/service/rerank/RerankingModelImpl.java Outdated

ruibaby marked this pull request as draft August 12, 2026 05:32

ruibaby added 3 commits August 12, 2026 14:46

Fix usage statistics correctness and recovery

d86ecf5

fix usage statistics lifecycle and UI reuse

566d733

test organize usage specs

19babe5

ruibaby requested a review from LIlGG August 12, 2026 10:09

ruibaby and others added 3 commits August 13, 2026 11:06

Merge remote-tracking branch 'origin/main' into feat/ai-usage-statistics

225206e

# Conflicts: # app/src/main/java/run/halo/aifoundation/service/language/LanguageModelImpl.java

fix: improve usage statistics accuracy, reliability, and console expe…

5beb4ba

…rience

LIlGG requested a balanced review from Copilot September 22, 2026 10:31

Copilot started reviewing on behalf of LIlGG September 22, 2026 10:32 View session

Copilot AI reviewed Sep 22, 2026

View reviewed changes

Comment thread app/src/main/java/run/halo/aifoundation/service/observation/UsageError.java Outdated

Comment thread app/src/main/java/run/halo/aifoundation/endpoint/UsageStatisticsConsoleEndpoint.java

fix: classify usage timeouts and simplify statistics conditions

a6c32d6

LIlGG requested a balanced review from Copilot September 23, 2026 03:28

Copilot AI reviewed Sep 23, 2026

Navigation Menu

Add AI usage statistics with SQLite storage and Console UI - #96

Add AI usage statistics with SQLite storage and Console UI#96
ruibaby wants to merge 9 commits into
mainfrom
feat/ai-usage-statistics

ruibaby commented Aug 11, 2026 •

edited by LIlGG

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ruibaby commented Aug 11, 2026 • edited by LIlGG Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR changes

Statistical model

Console API

Resilience

Test plan

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ruibaby commented Aug 11, 2026 •

edited by LIlGG

Loading