Skip to content

The rate limit answers 402 with the offer, not 429 with a description of it - #188

Merged
ralyodio merged 2 commits into
mainfrom
feat/throttle
Sep 9, 2026
Merged

The rate limit answers 402 with the offer, not 429 with a description of it#188
ralyodio merged 2 commits into
mainfrom
feat/throttle

Conversation

@ralyodio

@ralyodio ralyodio commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

What was wrong

The refusal already named the pass, its price and exactly how to buy it — in prose:

"buy": {
  "url": "https://rssamplifier.com/crawl",
  "price": "1.00 USD per day, USDC, settled by CoinPay",
  "how": "Fetch https://rssamplifier.com/crawl with Accept: application/json for an x402 offer, pay it, then send the pass in the x-crawl-pass header."
}

An x402 client cannot act on prose. A program that hit the wall had the same two options it arrived with — slow down, or spread itself over a proxy pool — plus a second round trip to /crawl to find out what to sign.

What changes

The refusal now is the offer. Status 402, with the accepts array spread into the body it already had, so a client can pay and carry on inside the same exchange.

The offer is read off the gateway rather than rebuilt here. payTo, the price, the currency and the network live in one place — a second copy is how a site ends up quoting a number its own /crawl disagrees with. That is what @profullstack/x402-gateway@0.6.0 adds: gateway.offer(), which was already computed inside the closure and is now just not private.

The paid rungs still get 429. A caller at the sponsor ceiling has bought everything there is; a 402 would be asking it to pay twice for nothing.

Tripping the anonymous rung takes 120 requests in a minute against a steady reader's thirty, so the caller being answered here is a machine.

What is deliberately NOT changed

lib/crawlThrottle.js stays exactly as it is, and this site does not adopt @profullstack/throttle.

The rest of the fleet is getting that package because their limiters only watched /api/. This one already meters every route, and it does something the package does not: it keys a declared crawler by the token in its own User-Agent, collapsing meta-externalagent's seventy addresses into one caller. Per-address metering never reaches that fleet. Swapping this for the generic package would be a downgrade.

What it was missing was the sale, and that is all this adds.

Verified

node --test test/*.test.js405 passed, 0 failed (two new). pnpm --filter @rssamplifier/web build clean with the proxy intact.

New tests read the behaviour off the source, the way the matcher test already does — importing proxy.js under node:test fails on a bare next/server specifier, which is why none of the tests here drive the proxy — plus one that actually calls gateway.offer() and checks every accepts entry names a network, a payTo and a positive amount.

🤖 Generated with Claude Code

ralyodio and others added 2 commits September 9, 2026 16:52
… of it

The refusal already named the pass, its price and exactly how to buy it. In
prose. An x402 client cannot act on prose, so a program that hit the wall had
the same two options it arrived with: slow down, or spread itself over a
proxy pool. It had to make a second trip to /crawl to find out what to sign.

Now the refusal carries the offer itself, read off the gateway rather than
rebuilt here -- payTo, the price, the currency and the network live in one
place, and a second copy is how a site ends up quoting a number its own
/crawl disagrees with. That is what @profullstack/x402-gateway 0.6.0 adds:
`gateway.offer()`, which was already computed inside the closure and is now
just not private.

The paid rungs still get 429. A caller at the sponsor ceiling has bought
everything there is, and a 402 would be asking it to pay twice for nothing.

Tripping the anonymous rung takes 120 requests in a minute, against a steady
reader's thirty, so the caller being answered here is a machine. The tiered
throttle in lib/crawlThrottle.js is untouched: it keys a declared crawler by
its own bot token, which collapses a seventy-address fleet into one caller,
and nothing in @profullstack/throttle does that. This is the sale it was
missing, not a replacement for it.

Read off the source, the way the matcher test already is: importing proxy.js
under node:test fails on a bare `next/server` specifier, which is why none of
the tests here drive the proxy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YDGCxTmEPs3ecwjjLJDQXh
The only conflict was the import block at the top of proxy.js, where main
added the challenge middleware and this branch added `gateway` to the
crawl-gateway import so the refusal can quote the same offer /crawl
serves. Both imports are kept; the two features are independent and their
tests pass together.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0139ts1a6Lzj5DXySTCbeG8C
@socket-security

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updated@​profullstack/​x402-gateway@​0.4.0 ⏵ 0.6.076100100 +194 +1100

View full report

@ralyodio
ralyodio merged commit 39804c6 into main Sep 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant