Skip to content

fix(english): harden We Tried TLS sanitizer and chapter parsing - #2621

Open
RevDev909 wants to merge 2 commits into
lnreader:masterfrom
RevDev909:fix/wetriedtls-audit-hardening
Open

RevDev909 wants to merge 2 commits into
lnreader:masterfrom
RevDev909:fix/wetriedtls-audit-hardening

Conversation

@RevDev909

Copy link
Copy Markdown
Contributor

Follow-up to #2604, which was merged before the last Greptile review findings on it could be addressed. This PR applies a full audit hardening pass to plugins/english/wetriedtls.ts:

  • Chapter HTML is now sanitized by an allowlist parser instead of pattern matching — closes the SVG xlink:href and entity-smuggled javascript: URL bypasses; dangerous elements are dropped together with their content, and unknown tags are unwrapped.
  • Malformed chapter-list responses now fail loudly (bad last_page metadata, or a non-empty page with zero usable entries) instead of silently truncating the chapter list.
  • The block splitter preserves list/table wrapper markup instead of dropping it.
  • The chapter-title cache is bounded (4,000 entries, oldest evicted first) instead of growing for the whole session.
  • Response size caps and a pagination runaway guard were added.
  • Entity decoding is single-pass throughout, so titles, summaries and attributes are no longer double-decoded.

Verified against captured fixtures: for normal content the parsed chapter output text is byte-identical to before (only dropped attributes/elements differ), and the file is formatted with the repo's Prettier config.

@greptile-apps

greptile-apps Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

[Medium risk] Adds safety limits to a novel-scraping plugin.

The PR is not ready to merge because promo trimming can remove chapter story text.

Findings

  1. P1 Story text gets removed ▶
  2. P2 Nested chapter markup breaks ▶
  3. P2 Large replies load before rejection ▶

Summary

We Tried TLS now rebuilds chapter HTML from allowed tags and attributes, and checks chapter-list replies before treating a list as complete. It also keeps list and table markup together, decodes text once, and bounds response sizes, pagination, and its title cache.

  • Chapter pages use an allowlist sanitizer with URL checks.
  • Broken chapter-list replies and runaway pagination stop with errors.
  • Chapter parsing keeps wrapper markup and limits cached titles.

Reviews (1) · Last reviewed commit: "fix(english): rebuild We Tried TLS sanit..."

const blocks: string[] = [];
const blockRe =
/<p[\s\S]*?<\/p>|<h[1-6][\s\S]*?<\/h[1-6]>|<figure[\s\S]*?<\/figure>|<img[^>]*>/gi;
/<ul[\s\S]*?<\/ul>|<ol[\s\S]*?<\/ol>|<table[\s\S]*?<\/table>|<blockquote[\s\S]*?<\/blockquote>|<div[\s\S]*?<\/div>|<p[\s\S]*?<\/p>|<h[1-6][\s\S]*?<\/h[1-6]>|<figure[\s\S]*?<\/figure>|<img[^>]*>/gi;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Story text gets removed

If a chapter ends with a <div> containing both story text and a Discord plug, the new blockRe groups them into one block. isPromoParagraph marks that whole block as a plug, so the reader loses the story text too. Trim the plug without removing the story paragraphs beside it.

const blocks: string[] = [];
const blockRe =
/<p[\s\S]*?<\/p>|<h[1-6][\s\S]*?<\/h[1-6]>|<figure[\s\S]*?<\/figure>|<img[^>]*>/gi;
/<ul[\s\S]*?<\/ul>|<ol[\s\S]*?<\/ol>|<table[\s\S]*?<\/table>|<blockquote[\s\S]*?<\/blockquote>|<div[\s\S]*?<\/div>|<p[\s\S]*?<\/p>|<h[1-6][\s\S]*?<\/h[1-6]>|<figure[\s\S]*?<\/figure>|<img[^>]*>/gi;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Nested chapter markup breaks

If a chapter has a list, table, or <div> inside another of the same kind, the new blockRe stops at the inner closing tag. The splitter then drops outer closing tags left in text-free gaps. The reader gets broken markup even though the source was balanced. Match the full outer block before discarding gaps.

Comment on lines +29 to +32
async function fetchApiText(url: string): Promise<string> {
const text = await fetchText(url);
if (text.length > MAX_RESPONSE_CHARS)
throw new Error("Response too large to parse safely");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Large replies load before rejection

fetchApiText checks the length only after fetchText has downloaded and decoded the whole reply. A very large API reply can still use the reader’s data and memory before this limit throws. Check the size while reading the reply.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant