0.20.0: eight full catalogues, each built and adversarially reviewed on its own branch - #88
Merged
Conversation
…on its own branch Every film and every video game on Wikidata (CC0, walked by numeric id windows over SPARQL), every team and every player TheSportsDB knows (terms permit copying from the official endpoints; players land in People as OpenProfile.md with their socials), every anime on Kitsu (no licence stated, and the description says so), Project Gutenberg's whole catalogue and every LibriVox audiobook (public domain), and every app on Steam through the keyless store query. Each walk is capped per run, resumes from its cursor, ends its pass and starts over, and states its licence. Integration only here: the eight adapters, tests and real-response fixtures come from their branches unchanged except a 15,000-row cap on a Gutenberg run (one run held the whole catalogue at 700 MB) and the startId config field Wikidata films declared in its defaults but not its fields. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2
…ite URL the process configured Main took another 0.20.0 (#87) between this branch's cut and its merge, so the catalogues ship as 0.21.0. Main's CI has also been red since 0.18.0 on two OpenSaaS assertions that compare against a literal https://nichedb.test while the shared test process had already loaded the config with SITE_URL unset; they now build their expectations from config.siteUrl, the same fix the profiles test got in #71. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2
Main's CI has been red since 0.18.0 on two assertions comparing against a literal https://nichedb.test while the shared test process had already loaded the config with SITE_URL unset; the expectations now come from config.siteUrl, the same fix the profiles test got in #71. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Eight more full-catalog sources, from the arr-ecosystem survey. Each was built by one agent on its own branch with real saved responses as fixtures and a live probe, then adversarially reviewed by a second agent that re-ran the tests, hit the real API with the adapter's exact URLs, checked cursor resume with two consecutive pulls, pacing, licence wording and item shape, and fixed what it found in place. Seven were fixed by review, one was clean, none blocked.
wikidata-filmswikidata-gamessportsdb-teamssportsdb-playerskitsu-animegutenberg-cataloglibrivox-catalogsteam-catalogReview findings worth knowing: Wikidata
wdt:P577drops date precision so year-only dates arrived as 1 January (fixed to year precision); the Wikidata games inner subquery needed DISTINCT or a doubly-typed item shortened a page and skipped rows; Gutenberg's If-Modified-Since compared against the wrong stamp; LibriVox lists projects still being recorded (skipped); three licence sentences overstated what the terms say and were corrected.Integration changes only: registration in
index.js, README rows, a 15,000-row cap on a Gutenberg run (one run held all 90k at ~700 MB), and thestartIdconfig field Wikidata films used but did not declare. Version 0.20.0.bun test1323 pass; lint clean for the new files.🤖 Generated with Claude Code
https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2