Answer rate-limit denials and expose the budget - #1
Merged
Merged
Conversation
- Refuse a call that has an id with a 429 carrying JSON-RPC error
-32010 (mcp.CodeRateLimited), with reason, retry_after and the budget
in data: the text/plain 429 reached the model as "server unavailable".
Keep the plain 429 for a body with no id to answer, and read at most
64 KiB of a refused body to find one
- Replace the constant Retry-After: 10 with the real wait: until the
budget window rolls, until the next RPM token, a second for a slot;
roll the window at resetAt, so a caller who waits exactly that long
is let in
- Add Config.Exempt(method, name): an exempt call is neither refused for
the budget nor charged to it, and still takes a rate token and the
concurrency slots
- Add Remaining(ctx) and Budget, so a handler can narrow its work before
it runs into the ceiling
- Add ChargeFor(ctx, label, work), app_mcp_ratelimit_charge_seconds
{label} and app_mcp_ratelimit_budget_used_seconds{user}, which follows
the window rather than its lazy roll; add Group.Histogram to
internal/metrics
- Fix a slow request refunding its own RPM tokens: price its extra calls
when it finishes, not when it was admitted
- Refuse with 400 (repeated_key) a request that names a member twice in
the envelope, its params or their _meta, in any case: fastjson reads
the first by exact bytes, encoding/json the last regardless of case,
so Mcp-Name, Mcp-Method and Exempt were checked against one value
while another ran
- Refuse with 400 (missing_id) a call other than notifications/* sent
without an id: zenrpc ran it detached, outside the limiter's slot and
past the settling of its budget
- Build writeFault on zenrpc.NewResponseError, so both refusals share
the envelope every dispatched error goes out in
- Describe all of it in README
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The rate limiter answered every denial with a
text/plain429 andRetry-After: 10, which an MCP bridge reports to the model as "serverunavailable": the model could neither wait sensibly nor ask for less. This
makes a denial something a client can read and act on, and makes the hourly
budget visible to handlers and to Prometheus.
Rate limiting
idgets a 429 carrying JSON-RPC error-32010(
mcp.CodeRateLimited).dataholdsreasonandretry_after, plusbudget_used,budget_limitandwindowwhen the budget is on. A bodywith no id to answer keeps the plain 429; a refusal reads at most 64 KiB
of the body to find the id.
Retry-Afteris the real wait: until the budget window rolls, until thenext RPM token, one second for a concurrency slot.
Config.Exempt(method, name)spares cheap calls — a help tool, a staticresource — the hourly budget. They still pay the rate and concurrency
limits.
Remaining(ctx)andBudgetlet a handler see what is left and narrow itswork before it runs into the ceiling.
ChargeFor(ctx, label, work)with two new metrics:app_mcp_ratelimit_charge_seconds{label}(a histogram that adds up to thebudget) and
app_mcp_ratelimit_budget_used_seconds{user}.Transport
400 repeated_key: a request that names a member twice in the envelope,paramsor_meta, in any case. fastjson reads the first occurrence byexact bytes, while encoding/json (which decodes the call for zenrpc) reads
the last and ignores case — so
Mcp-Name,Mcp-MethodandExemptcouldbe checked against one value while another ran.
400 missing_id: a method other thannotifications/*sent without an idor with a null one. zenrpc ran such a call detached from the request,
outside the limiter's concurrency slot and budget. MCP requires a non-null
id on every request, and Streamable HTTP answers input the server cannot
accept with an HTTP error status.
Behavior changes
Retry-Afterisno longer a constant 10.
ping,initializeor any other non-notifications/*method without anid, or with
"id": null, is a 400 instead of a 202.app_mcp_transport_rejected_totalgainsrepeated_keyandmissing_id.Testing
make test,make test-raceand golangci-lint v2.13.2 are clean. New testscover each refusal end to end through the transport in both eras, the
case-folded repeated key against the real dispatcher, a body with 100k
members staying linear, and the budget window and gauge accounting.