0.23.0: every series on TheTVDB, with the attribution link its terms require - #92
Merged
Conversation
TVmaze rows already carry data.thetvdbId and the *arr family keys on it, so the screen collection wanted the catalogue that id belongs to: 173,000 series against TVmaze's 90,000. The adapter walks /v4/series 500 a page (capped at 20 pages a run, resuming from the cursor, done on an empty page or a null links.next since the API answers 200 past the end), then keeps current from /v4/updates?type=series, re-reading only the changed ids from /series/<id>/extended?short=true, the one endpoint that carries genres (which decide tv against anime), the IMDb and TMDB ids and the network. The v4 login JWT rides in the cursor with its expiry, the way igdb keeps its Twitch token, renewed within a day of expiry or on a 401. The key is redacted from every log line, note and error, and a refused login ends the run at once rather than counting as three page failures. TheTVDB's terms require attribution with a direct link shown to end users, so every row carries data.attribution and data.attributionUrl. Config and ingest gain THETVDB_API_KEY plumbing; the adapter is not yet registered in the index, which integration does. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2
…k began A walk is many runs over a few hours, so a series changed after its page was read but before the last page landed was missed until its next change. The cursor now carries startedAt from the first page and the completed walk commits it as refreshedAt; /updates takes since inclusively and the ids are deduped, so the overlap costs a few listing pages and loses nothing. Also: a delta run whose deadline is already past reads no listing page and now keeps the previous mark instead of committing "now" with nothing read. Reviewed live against api4.thetvdb.com: login, /series pages 0 and 1 (500 rows each, parsed to 500 items each with no shape violations), one /updates listing (since is inclusive, reaches 40 days back), one extended row, the past-the-end 200 and the bad-bearer 401 all match the fixtures. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2
…require The screen collection gains thetvdb-catalog: /v4/series walked 500 a page once, then /v4/updates for what changed, the month-long bearer kept in the cursor and renewed on a 401. TheTVDB's API terms allow commercial use free under $50k a year and require attribution with a direct link, so every row carries "Metadata provided by TheTVDB.com" and the link. The key is THETVDB_API_KEY, found in the bittorrented vault and copied to nichedb's. Built on its own branch and adversarially reviewed: the review moved the first delta mark back to when the walk began (a series changed during the three-hour walk was otherwise missed until its next change) and stopped a zero-page delta run from reporting itself caught up. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The screen collection gains
thetvdb-catalog: every series on TheTVDB,/v4/serieswalked 500 a page once, then/v4/updatesfor what changed, the month-long bearer kept in the cursor and renewed on a 401.TheTVDB's API terms (thetvdb.com/api-information) allow commercial use free under $50k a year and require attribution with a direct link, so every row carries
data.attribution"Metadata provided by TheTVDB.com" anddata.attributionUrl. The key isTHETVDB_API_KEY, which lived in the bittorrented vault; it is now innichedb--prodand on the Railway service, so this source runs on deploy.Built on its own branch and adversarially reviewed. The review fixed two real gaps: the first delta mark was the walk's end time, so a series changed during the ~3-hour walk was missed until its next change (now the mark is when the walk began), and a delta run with its deadline already past reported itself caught up after reading nothing. Live probe: login, page 0 and 1 (500 rows each) and one updates call through the adapter's own URL builders.
Version 0.23.0.
bun testall green, lint clean for the new files.🤖 Generated with Claude Code
https://claude.ai/code/session_01CDEiDss9RWYibtmxSk5Gr2