Repository navigation
Send a User-Agent from web_fetch and http_request - #52
Merged
senamakel merged 9 commits intoOct 8, 2026
Merged
Conversation
reqwest sends no `User-Agent` unless told to, and some APIs refuse a request
without one. GitHub's REST API is the clearest case:
403 Request forbidden by administrative rules.
Please make sure your request has a User-Agent header
Reproducible on demand — the same URL in the same second answers 403 with no
header and 200 with one. Every `api.github.com` call through these tools
therefore failed, deterministically, and the 403 was then read downstream as a
credentials problem, which sends whoever is debugging it after the wrong thing
entirely.
Adds `gate::USER_AGENT` (`tinytools/<crate version>`) and sets it on both
clients. Identifying rather than disguised: a server that wants to throttle or
block this traffic should be able to name it, and a version makes builds
distinguishable.
The test asserts the header left the process, read back from `serve`'s raw
request log, rather than asserting on the builder.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Tiny Sweeper reviewTiny Sweeper reviewed this change across 6 lane(s) and found 2 active actionable finding(s). Detailed lane evidence and any incomplete work are listed below. State: Ready for maintainer review Review snapshot
Completeness: Complete What changedThe review could not produce a supported behavioral summary; inspect the cited changed surface and lane details below. FeaturesNone identified with supported citations. TestsNo supported feature-to-test mapping was produced. Test execution is not inferred. FindingsPreviously reported and still active
Resolved this pass
Before merge
How this fits togetherflowchart LR
n0["HttpRequestTool<br/>changed"]:::changed
n1["WebFetchTool<br/>changed"]:::changed
n2["fetch"]:::impacted
n3["test_tool"]:::impacted
n4["execute_request"]:::impacted
n5["...d_once_with_the_hooks_headers_and_settled"]:::impacted
n6["..._is_returned_as_the_error_without_a_retry"]:::impacted
n2 -->|uses| n1
n3 -->|uses| n0
n5 -->|calls| n3
n5 -->|tests| n3
n5 -->|calls| n4
n5 -->|tests| n4
n6 -->|calls| n3
n6 -->|tests| n3
n6 -->|calls| n4
n6 -->|tests| n4
classDef changed fill:#0d4429,stroke:#238636,color:#e6edf3
classDef impacted fill:#161b22,stroke:#6e7681,color:#c9d1d9
classDef flagged fill:#5a1e02,stroke:#d93f0b,color:#ffffff
classDef blocking fill:#67060c,stroke:#f85149,color:#ffffff
Agent review detailscritique
security
tests
commits
description
e2e
Evidence and run details
|
|
Warning Review limit reached
This review includes 5 billable files and costs up to $1.25. Or wait 47 minutes for your next included review. View limit detailsLimit details: You’ve used the included review currently available. Review configuration: ⚙️ Run configuration
📒 Files selected for processing (5)
Comment |
![tinysweeper[bot]](https://avatars.githubusercontent.com/in/4526585?s=60&v=4){kind=small}
There was a problem hiding this comment.
tinysweeper found nothing blocking. Approving.
$0.0028 · 69,009 in / 2,596 out · 6,182 cached (9%) · gpt-5.6-luna, glm-5.3-flash
critique: $0.0017 · 34,605 in / 780 out · 4,284 cached (12%) · gpt-5.6-luna
security: $0.0010 · 20,718 in / 284 out · 1,898 cached (9%) · gpt-5.6-luna
tests: $0.0001 · 5,206 in / 432 out · 0 cached (0%) · glm-5.3-flash
description: $0.0000 · 5,002 in / 332 out · 0 cached (0%) · glm-5.3-flash
Add a test that spins up a local listener, performs a validated fetch, and checks the captured request headers for a tinytools user agent. This guards against regressions where the header is dropped from outgoing requests. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
The USER_AGENT constant is now crate-visible rather than public, since it is only consumed within the network module. The user agent test was reworked to return a Result and propagate errors with ?, replacing the expect calls. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
There was a problem hiding this comment.
tinysweeper found nothing blocking. Approving.
$0.0100 · 110,077 in / 5,973 out · 9,878 cached (9%) · gpt-5.6-luna, glm-5.3-flash
critique: $0.0052 · 51,837 in / 2,747 out · 4,575 cached (9%) · gpt-5.6-luna, glm-5.3-flash
security: $0.0046 · 41,705 in / 2,163 out · 3,575 cached (9%) · gpt-5.6-luna
tests: $0.0000 · 6,179 in / 92 out · 1,536 cached (25%) · glm-5.3-flash
description: $0.0000 · 5,963 in / 99 out · 64 cached (1%) · glm-5.3-flash
The user agent test now extracts the header line and compares it against the exact expected value instead of a substring match, so a wrong or truncated version string is caught rather than passing on the prefix alone. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformat the user agent assertion in the outgoing request test to satisfy rustfmt line width limits. No behaviour change. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
There was a problem hiding this comment.
tinysweeper found nothing blocking. Approving.
$0.0043 · 38,294 in / 2,927 out · 64 cached (0%) · gpt-5.6-luna, glm-5.3-flash
critique: $0.0024 · 13,482 in / 1,115 out · 64 cached (0%) · gpt-5.6-luna, glm-5.3-flash
security: $0.0018 · 8,146 in / 511 out · 0 cached (0%) · gpt-5.6-luna
tests: $0.0000 · 6,199 in / 246 out · 0 cached (0%) · glm-5.3-flash
description: $0.0000 · 5,983 in / 152 out · 0 cached (0%) · glm-5.3-flash
The test server now reads until the end of the header block instead of assuming a single read captures the whole request, so the assertion no longer fails when the request arrives split across packets. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
There was a problem hiding this comment.
tinysweeper found nothing blocking. Approving.
$0.0052 · 61,579 in / 3,817 out · 15,314 cached (25%) · gpt-5.6-luna, glm-5.3-flash
critique: $0.0030 · 27,875 in / 1,268 out · 7,381 cached (26%) · gpt-5.6-luna
security: $0.0021 · 16,569 in / 1,143 out · 4,861 cached (29%) · gpt-5.6-luna
tests: $0.0000 · 6,322 in / 268 out · 1,536 cached (24%) · glm-5.3-flash
description: $0.0000 · 6,106 in / 178 out · 1,408 cached (23%) · glm-5.3-flash
The user agent test previously broke out of the read loop when the peer closed early, which let the assertion run against a truncated request. It now panics with a clear message so the failure points at the incomplete read instead of a confusing mismatch. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Replace the explicit panic on a zero-length read with an assert! that carries the same message, keeping the test's intent while shortening the loop body. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reformatted the assertion that checks the peer has not closed before sending complete HTTP headers so it matches the project's rustfmt style. No behaviour change. Auto-committed-on: dragonfly Co-authored-by: Medulla <medulla@tinyhumans.ai>
This was referenced Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
reqwestsends noUser-Agentunless told to, and some APIs refuse a request without one. GitHub's REST API is the clearest case:Reproducible on demand — the same URL in the same second:
So every
api.github.comcall throughhttp_requestfailed, deterministically — not intermittently. Observed in a live agent run as eight consecutive failures, which were then classified downstream as an authentication problem and sent the operator looking for credentials that were never involved.Several other APIs require a
User-Agenttoo, and anonymous traffic is the first thing a rate limiter penalises.Change
gate::USER_AGENT=tinytools/<crate version>, shared by both toolshttp_requestandweb_fetchclientsIdentifying rather than disguised: a server that wants to throttle or block this traffic should be able to name it, and the version keeps builds distinguishable.
Testing
cargo test -p tinytools-std --lib network::— 92 passed, 0 failed.The new test asserts the header left the process, read back from
serve's raw-request log, rather than asserting on the builder:cargo fmt -p tinytools-std -- --checkclean.Risk
Low. One header added to outbound requests; no public API, schema or behaviour change beyond the header itself.
🤖 Generated with Claude Code