Skip to content

Answer rate-limit denials and expose the budget - #1

Merged
sergeyfast merged 1 commit into
masterfrom
release/v0.1.4
Sep 29, 2026
Merged

sergeyfast merged 1 commit into
masterfrom
release/v0.1.4

Conversation

@sergeyfast

Copy link
Copy Markdown
Member

The rate limiter answered every denial with a text/plain 429 and
Retry-After: 10, which an MCP bridge reports to the model as "server
unavailable": the model could neither wait sensibly nor ask for less. This
makes a denial something a client can read and act on, and makes the hourly
budget visible to handlers and to Prometheus.

Rate limiting

  • A denied call with an id gets a 429 carrying JSON-RPC error -32010
    (mcp.CodeRateLimited). data holds reason and retry_after, plus
    budget_used, budget_limit and window when the budget is on. A body
    with no id to answer keeps the plain 429; a refusal reads at most 64 KiB
    of the body to find the id.
  • Retry-After is the real wait: until the budget window rolls, until the
    next RPM token, one second for a concurrency slot.
  • Config.Exempt(method, name) spares cheap calls — a help tool, a static
    resource — the hourly budget. They still pay the rate and concurrency
    limits.
  • Remaining(ctx) and Budget let a handler see what is left and narrow its
    work before it runs into the ceiling.
  • ChargeFor(ctx, label, work) with two new metrics:
    app_mcp_ratelimit_charge_seconds{label} (a histogram that adds up to the
    budget) and app_mcp_ratelimit_budget_used_seconds{user}.
  • Fix a slow request refunding its own RPM tokens.

Transport

  • 400 repeated_key: a request that names a member twice in the envelope,
    params or _meta, in any case. fastjson reads the first occurrence by
    exact bytes, while encoding/json (which decodes the call for zenrpc) reads
    the last and ignores case — so Mcp-Name, Mcp-Method and Exempt could
    be checked against one value while another ran.
  • 400 missing_id: a method other than notifications/* sent without an id
    or with a null one. zenrpc ran such a call detached from the request,
    outside the limiter's concurrency slot and budget. MCP requires a non-null
    id on every request, and Streamable HTTP answers input the server cannot
    accept with an HTTP error status.

Behavior changes

  • A 429 for a call with an id is JSON rather than text, and Retry-After is
    no longer a constant 10.
  • A request that repeats a member, case-insensitively, is a 400.
  • ping, initialize or any other non-notifications/* method without an
    id, or with "id": null, is a 400 instead of a 202.
  • app_mcp_transport_rejected_total gains repeated_key and missing_id.

Testing

make test, make test-race and golangci-lint v2.13.2 are clean. New tests
cover each refusal end to end through the transport in both eras, the
case-folded repeated key against the real dispatcher, a body with 100k
members staying linear, and the budget window and gauge accounting.

- Refuse a call that has an id with a 429 carrying JSON-RPC error
  -32010 (mcp.CodeRateLimited), with reason, retry_after and the budget
  in data: the text/plain 429 reached the model as "server unavailable".
  Keep the plain 429 for a body with no id to answer, and read at most
  64 KiB of a refused body to find one
- Replace the constant Retry-After: 10 with the real wait: until the
  budget window rolls, until the next RPM token, a second for a slot;
  roll the window at resetAt, so a caller who waits exactly that long
  is let in
- Add Config.Exempt(method, name): an exempt call is neither refused for
  the budget nor charged to it, and still takes a rate token and the
  concurrency slots
- Add Remaining(ctx) and Budget, so a handler can narrow its work before
  it runs into the ceiling
- Add ChargeFor(ctx, label, work), app_mcp_ratelimit_charge_seconds
  {label} and app_mcp_ratelimit_budget_used_seconds{user}, which follows
  the window rather than its lazy roll; add Group.Histogram to
  internal/metrics
- Fix a slow request refunding its own RPM tokens: price its extra calls
  when it finishes, not when it was admitted
- Refuse with 400 (repeated_key) a request that names a member twice in
  the envelope, its params or their _meta, in any case: fastjson reads
  the first by exact bytes, encoding/json the last regardless of case,
  so Mcp-Name, Mcp-Method and Exempt were checked against one value
  while another ran
- Refuse with 400 (missing_id) a call other than notifications/* sent
  without an id: zenrpc ran it detached, outside the limiter's slot and
  past the settling of its budget
- Build writeFault on zenrpc.NewResponseError, so both refusals share
  the envelope every dispatched error goes out in
- Describe all of it in README
@sergeyfast
sergeyfast merged commit cec4f81 into master Sep 29, 2026
2 checks passed
@sergeyfast
sergeyfast deleted the release/v0.1.4 branch September 29, 2026 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant