Repository navigation
Audit fixes from the 2026-11 manual test of v0.1.4 - #10
Merged
Merged
Conversation
Fixes the bugs from the report that reproduce on 67be0b3: listing-URL filters (1), Docker libatomic1 (2), Ctrl+C keeps fetched pages (3), diff_runs route/slice/missing-file guards (4, 5, 25), Scraper API client refusals and help (6, 12, 22), argument validation (7, 9, 10), help text (13, 14), CSV lone dash (15), README concurrency (16), proxy error text (17a), empty flag beats .env (21), --allow-empty status (23), queue stop reason (24), fingerprint in Selenium/Puppeteer (26-28), solve allowance counted on purchase (33), offer-queue concurrency (34). Not fixed here: 8 (hint only, html key kept as a dated snapshot), 11, 17b, 18, 19, 20, 29, 30, 31, 32 - they need a live browser, a paid call or a design decision; see the PR description. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- 8: drop the stale `html` category hint, warn on a misspelt --category - 11: BrowserError in Puppeteer gets a --chromium-path hint - 17b: a dead proxy ends as proxy_unreachable, not page_load_timeout - 18: sample_output regenerated from a live run (multi-contract rows) - 19: Puppeteer proxy login over CDP Fetch (verified on Chromium 154 against a local authenticating proxy; page.authenticate is dead) - 20: a 407 from the proxy is named up front, exit 2 - 29: a refused sitemap rotates the exit and ends as exit 3 - 30/31: challenge pages on the endpoint route are recognised from the markup, and saved as markup - 32: no reload after a reCAPTCHA token; callback or form submit instead (checked on a local page, no paid solve) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
jehrr
marked this pull request as ready for review
November 16, 2026 15:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the defects from the 2026-11-12..16 manual test of v0.1.4 that reproduce on
67be0b3. Each was checked against the code first; the numbers are the report's.Fixed
--urllisting links:experience-level=/employment-type=become endpoint filters; any other query parameter is refused with exit 2 naming it (it used to be dropped silently).libatomic1beforepip install(Playwright's node driver needs it; the build failed on arm64 and on amd64 with Playwright 1.65).interrupted(status: partial, exit 6) and writes the pages already fetched, in all three engines.diff_runs.py: warns on a listings run that took <100% of its query; refuses api against ssr; "no such file" for a missing input.scraper_api_client.py: a listing page is parsed as the rendered page; an offer page (no parser exists), a foreign host,--timeoutoutside 1-120 and mode mismatches are refused before the paid request;--mode facetshas a default URL; stale help text from another scraper removed.--outdirectory,--pages < 1and a missing--slugs-fileare refused with exit 2 before any request.--fp-tags, ssr ~50 rows, concurrency only in Playwright).-is not escaped in CSV. 23--allow-empty+ finished empty run iscomplete. 24queue_exhaustedfor a finished offer queue. 34--concurrencyworks for an offer queue.--proxyis quoted, not?://?. 21 an explicit empty flag beats.env.--fingerprint: Selenium/Puppeteer apply the fingerprint's user agent, screen and timezone; Puppeteer's init script was a syntax error (((() => {...})();)(), reproduced with node); WebGL strings are read fromglParameters.captcha_solver.solve_attempts()), not when the check ran.Second commit: the ones first left out
htmlcategory is out of the hint list and the README loop; a misspelt--categorygets a warning with the close match.BrowserErrorprints a--chromium-pathhint (Apple Silicon).proxy_unreachable.sample_output.*regenerated from a live run (7 rows, two with several contract types).Fetch. Reproduced first:Network.setRequestInterception wasn't foundon Chromium 154. After: against a local authenticating proxy the proxy seesCONNECT justjoin.it:443 ... AUTH, and the browser no longer dies at launch.blocked_sitemap), in all three engines.Still not done
--slugs-filerow of the exit-code table (now exit 2); a nonexistent offer is still exit 5 from the browser engines and 4 from the Scraper API client.--concurrencyfor Selenium/Puppeteer (README now says so).apply_urlis not made route-dependent; the two routes are refused instead.Evidence
67be0b3was 1021 / 1004 / 917.check_audit_*). Run against the old code, they fail (exit codes, stop reasons,-escaping, status,--proxy "", concurrency warning), and the Ctrl+C one kills the suite with a realKeyboardInterrupt.dockerjob is green), Selenium over a real browser, paid APIs, the live site through the engines.🤖 Generated with Claude Code