Skip to content

Read imports from the AST, and retire a stripper that quietly stopped working - #109

Merged
DKoenig9 merged 1 commit into
mainfrom
fix/import-detection-without-comment-stripping
Sep 21, 2026
Merged

DKoenig9 merged 1 commit into
mainfrom
fix/import-detection-without-comment-stripping

Conversation

@DKoenig9

Copy link
Copy Markdown
Contributor

Closes the stripComments finding from #108 by making the helper unnecessary at both of its call sites, rather than by improving it.

The finding, measured

Two detectors asked "does this file still import X?" by searching the raw source with comments blanked first — because a docblock that DOCUMENTS the conversion quotes the very syntax they look for (nest-server-starter/tests/unit/bootstrap-diagnostics.spec.ts names from '@lenne.tech/nest-server' in prose, and the detector told the user to rewrite imports that file does not have).

The blanking used a standalone ts.createScanner in a plain scan() loop. That loop cannot read a template literal with a substitution: the scanner returns TemplateHead at `x${ and the caller must call reScanTemplateToken() to continue. Nothing did. From that token on it is desynchronised, stops classifying comment trivia as trivia, and every comment below passes through verbatim.

Ran over the exact file sets both call sites scan:

Call site files scanned comment stripping degraded surviving comment quoting the needle
frontend (nuxt-base-template app + tests) 76 23 (3 of them .vue) 0
backend (nest-server-starter scanned globs) 50 21 0

So roughly a third of real files degrade — template literals are everywhere — while the harmful combination is currently absent. A live trap, not a live bug: it fires the day someone writes a docblock quoting an import below a template literal, and it will read as the detector regressing rather than the stripper.

The fix

src/lib/module-specifiers.ts asks the parser instead. importedSpecifiers walks the AST for import, import 'x', export … from, export * from, import x = require(), dynamic import() and require(). Both call sites now take a predicate over a specifier instead of a text needle:

  • findStaleImports(dest, matches)(s) => isPackageImport(s, '@lenne.tech/nest-server') and isRelativeCoreImport
  • findStaleFrontendImports(appDir, matches, skip?)isRelativeCoreImport and (s) => isPackageImport(s, '@lenne.tech/nuxt-extensions')

A comment is not part of the AST, so there is nothing to strip and nothing to get wrong. Two precision gains fall out: a path in a string literal ('node_modules/@lenne.tech/nest-server/dist/x') no longer counts as an import, and @lenne.tech/nest-server-extras no longer matches @lenne.tech/nest-server.

.vue is handled deliberately, not incidentally

A .vue SFC is not valid TypeScript as a whole, so handing it to createSourceFile would parse the <template> block as expression syntax and recover badly — which is also why the scanner had even less to work with there (3 of the 23 degraded frontend files are .vue). fileImportedSpecifiers therefore extracts the <script> / <script setup> blocks and parses each one on its own, naming them <file>.ts so the TS dialect is picked. <template> and <style> are skipped on purpose: a commented-out import inside a template is prose by definition, and the test pins exactly that (an HTML comment quoting @lenne.tech/nuxt-extensions in <template> must not be reported, while the real import in <script setup> must).

stripComments itself

Deleted, with its test. After this change it has no callers, and its docblock's promise — "string literals, template literals and regex literals containing // or /* are handled correctly by construction" — is true only for the case it names (a comment marker INSIDE a literal) and false for a template that interpolates. Keeping it for a hypothetical third consumer would mean keeping that defect behind a warning label. The two places that pointed at it (heal-vendor-migrate-store.ts's docblock and the matching CLAUDE.md rule) now say: parse, a scanner is not enough either.

A correction I owe this PR

The cause I published for this in #108"a regex literal followed by a division, which a scanner has no parser context to tell apart" — is wrong. I read it off the offset where blanking stopped; it was plausible and it was not tested. The counter-test built from it passed, which is how it came out.

Bisecting growing prefixes of src/templates/check/check.mjs, each with a known comment appended:

prefix ends at last line included comment
63 rel.replace(/^projects\//, '') — regex literal blanked
66 const s = ms / 1000; — division blanked
67 if (s < 60) return `${s.toFixed(1)}s`; leaks

Isolated per construct: interpolating template → leaks; plain template → clean; division alone → clean; regex-then-division → clean. #108 carries its own correction commit for Rule 1b. Write the counter-test before publishing the cause.

Checks

  • tsc --noEmit, eslint clean
  • Jest, serially: 71 suites / 1073 tests, plus the four slow suites separately (4 / 65). git-commands excluded — it makes real network calls (documented in CLAUDE.md), unrelated to this change.
  • Three mutations of the new module (don't split .vue, drop the relative-path check, loosen isPackageImport to includes) each turn exactly one test red.

Draft until the cli release goes out, as agreed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01N8cvaEziSrKGHv3Jcp59JH

…ietly stopped working

Two detectors asked "does this file still import X?" by searching the raw text with
comments blanked first, because a docblock that DOCUMENTS the conversion quotes the
very syntax they look for. The blanking used a standalone `ts.createScanner` in a
plain `scan()` loop — and that loop cannot read a template literal with a
substitution: the scanner returns `TemplateHead` at `` `x${ `` and the caller must
call `reScanTemplateToken()` to continue. Nothing did. From that token on it was
desynchronised, stopped classifying comment trivia as trivia, and every comment below
passed through verbatim.

Measured over the exact file sets both call sites scan: degraded in 23 of 76 real
frontend files (3 of them `.vue`) and 21 of 50 backend files — template literals are
everywhere. Files where a SURVIVING comment also quotes a needle, i.e. real false
alarms: 0. A live trap, not a live bug: it fires the day someone writes a docblock
quoting an import below a template literal, and it will look like the detector
regressed rather than like the stripper did.

`src/lib/module-specifiers.ts` removes the need instead of improving the stripper.
`importedSpecifiers` walks the AST for `import`, `export … from`, `import x =
require()`, dynamic `import()` and `require()`; both call sites now take a predicate
over a specifier (`isPackageImport`, `isRelativeCoreImport`) instead of a text needle.
A comment is not part of the AST, so there is nothing to strip and nothing to get
wrong — and the check gets stricter as a side effect: `'node_modules/@lenne.tech/
nest-server/dist/x'` in a string no longer counts, `@lenne.tech/nest-server-extras`
no longer matches `@lenne.tech/nest-server`.

`.vue` is not an afterthought here. An SFC is not valid TypeScript as a whole, so the
scanner had even less to work with — 3 of the 23 degraded frontend files were `.vue`.
`fileImportedSpecifiers` splits the `<script>` blocks out and parses each one, and
skips `<template>`, where a commented-out import is prose by definition.

`stripComments` had no callers left, so it is deleted rather than documented. Its
docblock promised that "string literals, template literals and regex literals … are
handled correctly by construction" — true for a comment marker INSIDE a literal, false
for a template that interpolates. A helper kept for a future consumer would have
carried that defect with a warning label; the counter-test records it better.

A correction worth keeping: the first published cause for this was "a regex literal
followed by a division, which a scanner has no parser context to tell apart", read off
the offset where blanking stopped. It is wrong. Bisecting growing prefixes of
`src/templates/check/check.mjs` puts the regex (line 63) and the division (line 66)
both on the clean side and the break at line 67's `` `${s.toFixed(1)}s` ``; isolated
per construct, interpolating template leaks while plain template, division alone and
regex-then-division are all clean. The counter-test written from the wrong theory is
what failed and exposed it. Write the counter-test before publishing the cause.

Tests: 71 suites / 1073 tests plus the four slow suites separately (4 / 65), run
serially — `git-commands` excluded, it makes real network calls (see CLAUDE.md).
Three mutations of the new module each turn exactly one test red.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N8cvaEziSrKGHv3Jcp59JH
@DKoenig9
DKoenig9 marked this pull request as ready for review September 21, 2026 12:45
@DKoenig9
DKoenig9 merged commit c5cc9d1 into main Sep 21, 2026
2 checks passed
@DKoenig9
DKoenig9 deleted the fix/import-detection-without-comment-stripping branch September 21, 2026 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant