Conversation
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (el.type !== 'tag') return; | ||
| for (const attr of Object.keys(el.attribs)) { | ||
| const value = el.attribs[attr]; | ||
| if (/^on/i.test(attr) || /^\s*javascript:/i.test(value)) { |
There was a problem hiding this comment.
A chapter link such as href="java script:alert(1)" keeps a newline inside its URL after parsing. This check only catches whitespace before javascript:, so it leaves the link in the chapter HTML. A browser treats it as a script URL when a reader clicks it. Remove control characters before checking URLs, and reject unsafe URL schemes.
How this was verified: The URL check leaves embedded control characters intact, and the reader inserts the returned chapter HTML without another safety check.
There was a problem hiding this comment.
Fixed in 4993319. Before any scheme check, isUnsafeUrl strips ASCII control characters and whitespace (U+0000–U+0020, U+007F) and lowercases the value. It then rejects javascript:, vbscript: and any data: URL that isn't data:image/. The parser has already decoded entities, so java script: is caught. Tested against synthetic chapter HTML with the , mixed-case/tab, vbscript: and data:text/html cases.
|
|
||
| private async fetchCatalog(): Promise<BeastBook[]> { | ||
| const html = await this.fetchText(this.site + 'novels'); | ||
| const match = html.match(/allBooks:\s*(\[.*\]),\s*\n/); |
There was a problem hiding this comment.
fetchCatalog requires the whole allBooks array on one line, followed by a comma and newline. If the site spreads the same valid array across lines or changes that punctuation, parsing throws and readers cannot browse or search. Parse the embedded data without relying on its whitespace layout.
There was a problem hiding this comment.
Fixed in 4993319. fetchCatalog now finds allBooks:, then scans from the opening [ to its matching bracket, skipping brackets inside JSON strings, and passes that slice to JSON.parse. It no longer depends on whitespace or on what comes after the array. Tested on the live page (36 novels) and on synthetic pretty-printed, trailing-semicolon and no-space variants whose novel title contains [, ], } and escaped quotes.
| const [html, catalog] = await Promise.all([ | ||
| this.fetchText(this.site + novelPath), | ||
| this.fetchCatalog(), | ||
| ]); |
There was a problem hiding this comment.
parseNovel waits for /novels before it parses the series page. If /novels alone fails, Promise.all throws even when the series page loaded and contains the chapters. Let readers open the novel and its chapters without catalog status when that extra request fails.
There was a problem hiding this comment.
Fixed in 4993319. In parseNovel, a failed catalog request now falls back to an empty list, so the novel details and chapters still come from the series page, just without the catalog-derived status. Tested by stubbing /novels to return HTTP 500: Ode of the Brave still returned its name, summary and 56 chapters, with no status.
- Strip control characters and whitespace before checking URL schemes; also reject vbscript: and non-image data: URLs. - Parse the embedded catalog by bracket matching instead of a layout-dependent regex. - Keep parseNovel working when the catalog request fails.
This branch has not been deployed
1 participant
Closes #2427
Adds a standalone plugin for Beast Novels, an English translation site. It's a custom Laravel site, not a WordPress theme, so no multisrc template fits.
How it works
/novelsembeds the site's whole catalog (36 novels right now) as a JSON array that the page sorts and filters in the browser. The plugin parses that array for browsing, search and filters, so everything comes from one page. It finds the array by matching brackets, so the page's formatting around it doesn't matter. Pages after the first return nothing./search/booksJSON endpoint returns at most 7 results./series/<slug>. That page comments out its status badge, so status comes from the catalog entry. If the catalog request fails, the novel still loads, just without a status.data-premium="1") redirect to the login page, so they aren't listed. Release dates are left out because the site only shows relative times like "1 week ago"..chapter-content, withscript/style/iframeelements andon*attributes removed. Attributes whose URL scheme isjavascript:,vbscript:or a non-imagedata:are also removed; control characters and whitespace are stripped before that check, sojava script:doesn't slip through.Testing
npm run check:plugin -- plugins/english/beastnovels.ts: PASS for all four checks (popularNovels 36 novels, searchNovels, parseNovel 156 chapters, parseChapter 17700 chars).parseNovelreturns (for Ode of the Brave: 56 free chapters out of 240),resolveUrl, and that the chapter HTML has noon*/javascript:.fetchto check the catalog parser on pretty-printed, trailing-semicolon and no-space variants (with[,],}and quotes inside a title),parseNovelwhen/novelsreturns HTTP 500, and the URL sanitizer on the , mixed-case/tab,vbscript:anddata:text/htmlcases. I then re-rancheck:plugin(all four PASS again).eslintandprettier --checkon the new file, andnpm run build:compile, all pass.This PR was written by an AI agent (Claude). It has not had human review yet.