Skip to content

fix(completion): surface EQUATEs declared in another file - #554

Merged
msarson merged 1 commit into
msarson:version-1.0.4from
geircodes:fix/word-completion-crossfile-equates
Sep 16, 2026
Merged

msarson merged 1 commit into
msarson:version-1.0.4from
geircodes:fix/word-completion-crossfile-equates

Conversation

@geircodes

Copy link
Copy Markdown
Contributor

Surface EQUATEs declared in another file in word completion

What happened

Reported: typing the leading characters of an EQUATE constant produced nothing in completion,
even though the constant resolved fine everywhere else (hover, compilation) — it was declared in a
separate .inc file, pulled into the module being edited only indirectly (via the PROGRAM file's
own INCLUDE, itself outside any MAP).

Root cause

Two gaps stacked:

  1. WordCompletionProvider.collectEquates read only DocumentStructure.getEquates(), which is
    built by linkEquatesPass() walking the CURRENT document's own tokens. No cross-file tier
    existed for equates, unlike variables (collectProgramGlobalDataSymbols already reaches into the
    PROGRAM file for a MEMBER module).
  2. Even that tier wouldn't have reached this case: a data-section INCLUDE('Constants.inc')
    (outside a MAP) is never inlined into the including document's token stream — only MAP-nested
    INCLUDEs are followed, via ScopeAnalyzer.getMapTokensWithIncludes. A plain INCLUDE only
    leaves a referencedFile on the token, used for go-to-definition, so even the PROGRAM file's own
    token list doesn't contain the .inc's declarations.

StructureDeclarationIndexer (SDI) already solves exactly this: it regex-scans every file on the
project's search paths and indexes EQUATE/ITEMIZE_EQUATE declarations solution-wide, disk-cached
and background-revalidated. It's used today by the missing-constants diagnostic
(ProjectConstantsChecker.isConstantSatisfied) — completion simply never queried it.

Fix

collectEquates (now async) keeps the existing document-local tier first, then adds a new
collectProjectEquates tier reading StructureDeclarationIndexer, following the same
SolutionManager.findProjectForFile → sdi.getOrBuildIndex(project.path) idiom HoverProvider
already uses for its own SDI tier:

private async collectProjectEquates(
    document: TextDocument,
    partial: string,
    add: (label: string, kind: CompletionItemKind, detail?: string, documentation?: string) => void
): Promise<void> {
    if (partial.length < PROJECT_EQUATE_MIN_PREFIX) return;

    const project = SolutionManager.getInstance()?.findProjectForFile(docPath);
    if (!project?.path) return;

    const sdi = StructureDeclarationIndexer.getInstance();
    if (!sdi.isIndexed(project.path)) return;   // never await a cold scan on a keystroke
    const index = await sdi.getOrBuildIndex(project.path);

    const needle = partial.toLowerCase();
    let emitted = 0;
    for (const [key, decls] of index.byName) {
        if (!key.startsWith(needle)) continue;
        const equate = decls.find(d => d.structureType === 'EQUATE' || d.structureType === 'ITEMIZE_EQUATE');
        if (!equate) continue;
        add(equate.name, CompletionItemKind.Constant, /* EQUATE(value) */ ..., `Declared in ${path.basename(equate.filePath)}`);
        if (++emitted >= PROJECT_EQUATE_LIMIT) break;
    }
}

Two deliberate gates:

  • PROJECT_EQUATE_MIN_PREFIX = 2 — the index spans every search path plus libsrc, so on an empty
    prefix this would dump thousands of constants onto an already-large candidate list; the
    document's own equates are never gated.
  • PROJECT_EQUATE_LIMIT = 300 — cap per response.
  • !sdi.isIndexed(...) short-circuits rather than awaiting a cold build, so a keystroke can never
    block on a solution-wide scan.

The document-local tier still runs first, so add()'s existing first-write-wins dedup keeps a local
declaration ahead of an index entry of the same name.

Testing

New suite, WordCompletionProvider.ProjectEquates.test.ts (10 tests) — StructureDeclarationIndexer
prototype-stubbed and SolutionManager replaced with a minimal stand-in (no disk fixture), covering:
the cross-file EQUATE appearing for a typed prefix, its EQUATE(value) detail and declaring-file
documentation, a valueless EQUATE's bare EQUATE detail, an ITEMIZE_EQUATE resolving through the
existing qualifier-completion branch, and six guards (local declaration wins on name collision, empty
prefix excluded, single-character prefix excluded, non-EQUATE index entries excluded, an unbuilt
index skipped rather than awaited, no owning project handled without throwing).

Non-vacuity proven on clean builds both ways (rm -rf out before each, compiled JS grepped to
confirm which state was under test): without the fix, 4 failing — exactly the 4 new bug-repro tests,
symptom an empty candidate list, matching the report; all 6 guards passed in both states. With the
fix: full suite 2824 passing / 0 failing / 4 pending (pre-existing, unrelated).

Scope

One file changed (server/src/providers/WordCompletionProvider.ts), one new test file. No other
provider touched.

Known follow-up (not part of this PR)

Ranking: project-wide equates are currently interleaved with local/document-scope completions by
the client's default sort (this codebase sets no sortText anywhere in WordCompletionProvider),
so a cross-file equate can rank ahead of a local variable purely by alphabetical luck on a shared
prefix. A natural refinement is giving the collectProjectEquates tier a sortText that sorts
after local-scope items, so project-wide equates stay in the list (they're valid to the compiler,
so they should be valid here) without competing for the top of it. Deferred: touches ordering
semantics for the whole provider, not just this bug, and deserves its own review.

Typing the leading characters of an EQUATE constant produced nothing in
completion whenever it was declared in a separate .inc file and only
reachable indirectly, e.g. through the PROGRAM file's own INCLUDE sitting
outside any MAP — even though the constant resolved fine everywhere else
(hover, compilation).

Two gaps stacked:

1. WordCompletionProvider.collectEquates read only
   DocumentStructure.getEquates(), built by linkEquatesPass() walking the
   current document's own tokens. No cross-file tier existed for equates,
   unlike variables (collectProgramGlobalDataSymbols already reaches into
   the PROGRAM file for a MEMBER module).
2. Even that tier wouldn't have reached this case: a data-section
   INCLUDE('x.inc') outside a MAP is never inlined into the including
   document's token stream — only MAP-nested INCLUDEs are followed, via
   ScopeAnalyzer.getMapTokensWithIncludes. A plain INCLUDE only leaves a
   referencedFile on the token (used for go-to-definition), so even the
   PROGRAM file's own token list doesn't contain the .inc's declarations.

StructureDeclarationIndexer (SDI) already solves exactly this: it
regex-scans every file on the project's search paths and indexes
EQUATE/ITEMIZE_EQUATE declarations solution-wide, disk-cached and
background-revalidated. It's used today by the missing-constants
diagnostic (ProjectConstantsChecker.isConstantSatisfied) — completion
simply never queried it.

Fix: collectEquates keeps the existing document-local tier first, then
adds collectProjectEquates, reading SDI via the same
SolutionManager.findProjectForFile -> sdi.getOrBuildIndex(project.path)
idiom HoverProvider already uses for its own SDI tier. Gated on a 2+
character typed prefix (the index spans every search path plus libsrc, so
an empty prefix would dump thousands of constants onto an already-large
list) and capped at 300 results; skips rather than awaits when the index
isn't built yet, so a keystroke never blocks on a solution-wide scan. The
document-local tier still runs first, so the existing first-write-wins
dedup keeps a local declaration ahead of an index entry of the same name.

Tests: new WordCompletionProvider.ProjectEquates.test.ts (10 tests) —
SDI prototype-stubbed, SolutionManager replaced with a minimal stand-in,
no disk fixture. Covers the cross-file EQUATE appearing for a typed
prefix, its EQUATE(value) detail and declaring-file documentation, a
valueless EQUATE's bare EQUATE detail, an ITEMIZE_EQUATE resolving through
the existing qualifier-completion branch, and six guards (local
declaration wins on name collision, empty/1-char prefix excluded,
non-EQUATE index entries excluded, an unbuilt index skipped rather than
awaited, no owning project handled without throwing).

Non-vacuity proven on clean builds both ways: without the fix, 4 failing
(exactly the 4 bug-repro tests, symptom an empty candidate list, matching
the report), all 6 guards passing. With the fix: full suite 2959 passing,
0 failing, 4 pending (pre-existing, unrelated).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@msarson
msarson merged commit 16db643 into msarson:version-1.0.4 Sep 16, 2026
1 check passed
msarson added a commit that referenced this pull request Sep 16, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
geircodes added a commit to geircodes/Clarion-Extension that referenced this pull request Sep 17, 2026
…on#555)

WordCompletionProvider set no sortText anywhere, so the client ordered
every candidate alphabetically against every other one. A project-wide
EQUATE from the declaration index (msarson#554) could therefore outrank a
local variable on a shared prefix by pure alphabetical luck.

Assigns one tier per source, stamped as a padded sortText prefix:
locals and parameters, then module and PROGRAM data, then MAP
procedures, then project-wide index entries, then the static catalogs
(keywords, built-ins, data types, controls, attributes, directives),
each tier alphabetical within itself.

Two details worth calling out:

Every candidate gets a tier, including the catalogs the issue does not
name. Leaving any of them unstamped would not preserve their position:
the client compares an unstamped item's label against a stamped item's
sortText, and since digits precede letters, every unstamped candidate
would sink below every stamped one. The catalogs reach `seen` without
going through add(), so a post-pass stamps whatever is left.

The tier number is zero-padded to two digits and the label half
lowercased. Without padding a tenth tier would sort between 1 and 2;
without lowercasing, Clarion being case-insensitive, MyConst and
myconst would swap places depending on capitalisation. Two digits is
deliberate — the tiers are spaced by ten, leaving room for nine more
between any two existing ones against a ceiling of 99.

add() stamps sortText only on first insertion, never on the merge path,
so the first (higher-priority) tier to claim a label keeps its rank —
the same first-writer-wins rule the closure already followed. A small
addIn(tier) wrapper means none of the 13 collector signatures changed.

Tests: 3 new cases — the issue's stated ordering (local, PROGRAM global,
project-wide EQUATE on a shared prefix), each tier forming one
contiguous alphabetical block under case-insensitive comparison, and
every candidate carrying a sortText. All 3 fail against the unstamped
provider and pass with it.

Full suite: 3053 passing, 4 pending (pre-existing, unrelated), 0 failing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@geircodes
geircodes deleted the fix/word-completion-crossfile-equates branch September 18, 2026 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants