Skip to content

Add AutoScout24 site adapter - #1

Merged
Kianda merged 3 commits into
Kianda:mainfrom
ebaschiera:feat/autoscout24
Oct 12, 2026
Merged

Kianda merged 3 commits into
Kianda:mainfrom
ebaschiera:feat/autoscout24

Conversation

@ebaschiera

Copy link
Copy Markdown
Contributor

Hi! I've been using subitoo to watch used-car listings, and many of them are on AutoScout24 rather than Subito, so I wrote an adapter for it. It's been running on my instance for about two weeks with six hourly queries, and it has been stable. The only failed runs were a handful of network timeouts, and each was retried within the same hour.

How it works

  • You paste the search URL from the address bar. All the filters (make, model, years, price, fuel, gearbox, power…) are already in the path and query string, so nothing has to be translated into subitoo's own fields.
  • No browser needed: it requests the pasted URL with curl_cffi, adds page=N to it, and reads the listings from the page's __NEXT_DATA__ JSON.
  • __NEXT_DATA__ is a Next.js internal, not a stable API, so the parser fails loudly when the shape changes. An empty result would look to the core like every listing had disappeared.

Things worth knowing

  • AutoScout24 listings don't carry a publication date, so the search should be sorted by sort=age&desc=1 to keep new listings on page 1. The README says so.
  • The site drops query parameters it doesn't recognise, so the URL should be copied from the address bar, not built by hand.
  • Redirects can lose filters. During testing AutoScout24 renamed /lst/bmw/3-series to /lst/bmw/serie-3. The old path redirects to the new one but keeps only price, country and sort, so a saved search silently became much wider and started notifying on diesels and cars from any year. The adapter now compares the parameters it asked for with the ones on the page it landed on, and fails the run if any are missing (the fix is to paste the URL again).
  • Sponsored slots repeat an organic result. I had deduplicated them inside the adapter, but your fix in a69c1bc now does that in run_query, so I removed it to keep the adapter in line with the SPEC (adapters never dedup).

Tests

tests/test_autoscout24.py replays a real search page saved as a fixture (trimmed to five cars, seller contacts and addresses removed), with no network access. It covers parsing, price-on-request listings, URL paging, max_pages, blocks (non-200) and the redirect check. The whole suite passes (107 tests).

Happy to change anything to fit how you'd like adapters to look.

Browserless: replays the pasted search URL with a page cursor and reads the
listings from the page's __NEXT_DATA__ JSON. Sponsored slots (Nfm) repeat an
organic result on the same or a later page, so ids are deduplicated within
the run — otherwise the first-run seeding fails on the UNIQUE constraint.
AutoScout24 renamed /lst/bmw/3-series to /lst/bmw/serie-3. The old path
redirects to the new one keeping only price, country and sort, so a saved
search silently widened to every fuel, body and year and notified on cars
outside it. Compare the param names asked for with the ones the site
landed on and raise if any went missing.
a69c1bc makes run_query fold listings by id before seeding, so the
adapter no longer needs its own pass for the sponsored slots that repeat
an organic result. SPEC already says adapters never dedup.
@Kianda
Kianda merged commit 4b8daee into Kianda:main Oct 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants