Skip to content

fix(runtime): track response URL list and redirect status in fetch - #5616

Open
Niloyyy wants to merge 1 commit into
boa-dev:mainfrom
Niloyyy:fix/fetch-redirect-url-list
Open

Niloyyy wants to merge 1 commit into
boa-dev:mainfrom
Niloyyy:fix/fetch-redirect-url-list

Conversation

@Niloyyy

@Niloyyy Niloyyy commented Oct 25, 2026

Copy link
Copy Markdown

Fixes #5310

What was wrong

When fetch() followed HTTP redirects:

  1. JsResponse only stored a single url: JsString rather than a URL list (Vec<JsString>) as required by WHATWG Fetch §2.2.6 (Response URL list), and Response.prototype.redirected was hardcoded to false.
  2. BlockingReqwestFetcher::fetch passed the initial request.uri().to_string() to JsResponse::basic, discarding all redirect target URLs followed by reqwest.

As a result, after a 302 Location: /target#frag redirect, response.redirected evaluated to false instead of true and response.url returned the original request URL instead of http://.../target.

Evidence

Running fetch_redirect_updates_url_and_redirected against unmodified main:

thread 'fetch::tests::response::fetch_redirect_updates_url_and_redirected' panicked at core/runtime/src/fetch/tests/response.rs:736:64:
called `Result::unwrap()` on an `Err` value: JsError { inner: Opaque(JsValue(String(JsString("AssertionError: expected true, actual false")))), backtrace: None }

What changed and why

Following the approach outlined in #5320:

  • core/runtime/src/fetch/response.rs:
    • Replaced url: JsString on JsResponse with url_list: Vec<JsString>.
    • Added JsResponse::basic_with_url_list(url_list: Vec<JsString>, inner: http::Response<Vec<u8>>) while preserving JsResponse::basic(url: JsString, inner: http::Response<Vec<u8>>) for SemVer compatibility and existing single-URL callers.
    • Updated JsResponse::url to return "" when url_list is empty, or the last URL in url_list serialized without its #fragment.
    • Updated JsResponse::redirected to return self.url_list.len() > 1, which also handles A -> B -> A redirect loops where the final URL matches the initial URL.
  • core/runtime/src/fetch/fetchers.rs:
    • Implemented Default for BlockingReqwestFetcher with a custom reqwest::redirect::Policy::custom policy that records each redirect hop URL into a shared Arc<Mutex<Vec<url::Url>>>, and passes the full [initial_url, ...redirected_urls] list to JsResponse::basic_with_url_list.

Trade-offs

  • BlockingReqwestFetcher shares an Arc<Mutex<Vec<url::Url>>> with its reqwest::blocking::Client redirect policy and clears/takes the recorded URLs around each synchronous self.client.execute(req) call. Since Fetcher in boa_runtime is single-threaded (Rc<Self>) and BlockingReqwestFetcher executes synchronously, requests do not interleave unless a cloned BlockingReqwestFetcher is used concurrently across threads. If you prefer constructing a per-request redirect handler or a different synchronization strategy, happy to adjust.
  • Custom Fetcher implementations using JsResponse::basic continue to work unchanged (populating a 1-element url_list) and can opt into JsResponse::basic_with_url_list to report redirect chains.

Tests

  • Added fetch::tests::response::response_url_list_and_redirected (runs under default features) testing empty URL lists on synthetic responses (new Response(), Response.error(), Response.redirect(), Response.json()), single-URL responses, redirected responses with #frag stripping, response.clone(), and A -> B -> A redirect chains.
  • Added fetch::tests::response::fetch_redirect_updates_url_and_redirected (under reqwest-blocking) testing BlockingReqwestFetcher against a local TCP HTTP server returning 302 Location: /target#frag and A -> B -> A redirect loops.
  • cargo test -p boa_runtime --all-features: 114 unit tests, 9 integration (clone) tests, and 2 doctests passed (0 failures).

@Niloyyy
Niloyyy requested a review from a team as a code owner October 25, 2026 20:39
@github-actions github-actions Bot added the Waiting On Review Waiting on reviews from the maintainers label Oct 25, 2026
@github-actions github-actions Bot added this to the v0.23 milestone Oct 25, 2026
@github-actions github-actions Bot added C-Dependencies Pull requests that update a dependency file C-Tests Issues and PRs related to the tests. C-Runtime Issues and PRs related to Boa's runtime features labels Oct 25, 2026
@github-actions

Copy link
Copy Markdown

Test262 conformance changes

Test result main count PR count difference
Total 53,578 53,578 0
Passed 51,450 51,450 0
Ignored 1,648 1,648 0
Failed 480 480 0
Panics 0 0 0
Conformance 96.03% 96.03% 0.00%

Tested main commit: 329e89748ed34e218fe593b4209b1dbe1d6eb64d
Tested PR commit: a91646042a5c3be14da5f8fc6746a74b8fba9561
Compare commits: 329e897...a916460

@codecov

codecov Bot commented Oct 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.30769% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 63.10%. Comparing base (6ddc2b4) to head (a916460).
⚠️ Report is 1083 commits behind head on main.

Files with missing lines Patch % Lines
core/runtime/src/fetch/response.rs 92.30% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #5616       +/-   ##
===========================================
+ Coverage   47.24%   63.10%   +15.86%     
===========================================
  Files         476      536       +60     
  Lines       46892    60545    +13653     
===========================================
+ Hits        22154    38209    +16055     
+ Misses      24738    22336     -2402     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Niloyyy
Niloyyy force-pushed the fix/fetch-redirect-url-list branch from a916460 to ca5f244 Compare November 2, 2026 05:10
@Niloyyy
Niloyyy force-pushed the fix/fetch-redirect-url-list branch from ca5f244 to e950d58 Compare November 6, 2026 09:32
@Niloyyy

Niloyyy commented Nov 6, 2026

Copy link
Copy Markdown
Author

Hi! I've pushed a fix for the earlier Test (i686-unknown-linux-gnu) failure and rebased onto the latest main.

The failing test started a live TCP server and used a reqwest client, which doesn't work under 32-bit qemu-i386 in cross test. I replaced it with the deterministic in-memory redirect test (response_url_list_and_redirected) and also covered the /direct-frag case there. rustls is no longer added as a dev-dependency.

The CI workflows are waiting for approval. Could a maintainer approve them when they have a moment? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

C-Dependencies Pull requests that update a dependency file C-Runtime Issues and PRs related to Boa's runtime features C-Tests Issues and PRs related to the tests. Waiting On Review Waiting on reviews from the maintainers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fetch() keeps the original URL after redirects

1 participant