feat: report token usage per API task and per workflow

[timzsu]

Purpose

A caller that runs a workflow had no way to learn what it cost. The API executor already logged per-call token counts, but nothing reached the result or the workflow. This PR reports token and call usage on each API task's result and sums it per workflow on GET /workflows/{id}, so Lumilake can show per-job usage.

Changes

• Typed task usage (src/shared/schemas/result, api_executor.py). APIUsage is a typed model with prompt_tokens, completion_tokens, reasoning_tokens, calls, failures, retries, truncated_calls (calls that stopped on the length limit) and wall_sec. APIResult.usage carries the task's sum, the same numbers the executor's summary line logs. A call whose response has no usage counts in calls and adds no tokens.
• Usage is computed once, at result ingest (routers/v1/results.py, registries/workflow.py, clients/redis.py). Each task's usage is taken from its result envelope when the result arrives and stored in the workflow registry's Redis under task:{id}:usage. GET /workflows/{id} sums the stored values and never opens a result file. vLLM inference tasks map their GenerationUsage: calls from num_requests, wall_sec from latency_sec, reasoning 0.
• Merged dispatches count once. A merged vLLM dispatch reports the batch total on the parent and each child's share in result.children, and each child is ingested separately. The parent records its own share (total minus its children's), so the workflow sum counts every call once. A merged child whose result is copied from the parent (no result of its own) records no usage, because the parent's total already holds its calls.
• Fail closed, never a partial sum. A completed task with no recorded usage, or a model-calling result whose usage can't be mapped (an API or inference result without usage; embedding, agent and RAG results), makes the workflow's usage null, with one log line naming the task. Task types that make no model calls (echo, SSH, data profiling, data retrieval) contribute nothing. When every completed task is recorded, the sum is returned, even when it is all zero.
• Cleanup. Unregistering a workflow deletes its tasks' usage keys. EventMonitor takes the workflow registry as a required argument.
• SDK. The Workflow model gains a typed usage field.

Design

Usage is stored per task at ingest rather than summed on read. Lumilake polls GET /workflows/{id} for progress, and summing on read reopened and re-parsed every completed task's result file on each poll.

Test Plan

The repo's pre-commit gate on the changed files (gitleaks, isort, black, ruff, codespell, mypy), then tests/server, tests/worker and tests/shared. The new tests cover:

• two API tasks and one vLLM task summed exactly;
• an echo task contributing nothing;
• a missing, unmappable or usage-less result giving null;
• a merged parent with two children summing to the batch total, whichever order the results arrive in;
• a mirrored child;
• key cleanup on unregister;
• a GET that never calls read_result.

Each rule was also neutralised in src on its own to confirm its test fails.

Test Result

• tests/server tests/worker tests/shared: 1905 passed, 1 failed. The failure is test_mp_executor_cleans_up_vllm, a GPU test whose vLLM engine does not start in this environment. This PR changes no vLLM worker code.
• Usage, monitor and results tests: 99 passed.
• Pre-commit: all hooks pass on the changed files.
• End to end on a deployed server: pending.

---

Pre-submission Checklist

• I have read the contribution guidelines.
• I have run pre-commit run --all-files and fixed any issues.
• I have added or updated tests covering my changes (if applicable).
• I have verified that uv run pytest tests/ passes locally.
• If I changed shared schemas or proto definitions, I have checked downstream compatibility across Server and Worker.
• If I changed the SDK or CLI, I have verified the affected packages work (uv sync --all-packages --group ci --frozen).
• If this is a breaking change, I have prefixed the PR title with [BREAKING] and described migration steps above.
• I have updated documentation or config examples if user-facing behavior changed.

[timzsu]

This PR is closed because the features are folded to #157.