[Sanjay Singh (san360)]

Description

Fixes the anti-pattern Why? button that stays on "Asking AI for an explanation…" forever (#278). It also makes every AI feature use GitHub Copilot's Auto model, fail fast with a clear message, and appear only when GitHub Copilot is available.

Code changes

File	Change
src/core/copilot-model.ts (new)	Shared { vendor: 'copilot', id: 'auto' } selector, the consent-dialog justification text, and the 120s request budget (COPILOT_REQUEST_BUDGET_MS).
src/webview/panel-llm-model.ts (new)	Model access: selectModel() (Copilot Auto only; at most 15s and never past the request's deadline), isCopilotModelAvailable() probe, withTimeout(), and classification of errors that must not be retried.
src/webview/panel-llm.ts	Removed the hard-coded model families and old selection code. callLlm gets a 120s budget that includes model selection and can be shared across calls (llmDeadline() and a deadline option), no retries for non-recoverable errors, empty reply treated as failure, and the justification. callLlmJson only gains the justification.
src/webview/panel-rpc.ts	generateRule passes one deadline to all of its model calls (up to three) and keeps the latest draft if a follow-up runs out of time. Its retry loop is now the testable generateRuleMarkdown.
src/webview/shared.ts	Generate SKILL.md (generateSkillContent) gets the 300s webview timeout used for Copilot-backed requests, like Why? and Generate rule. The list is type-checked against the RPC method names.
src/core/rule-compiler.ts	{ family: 'gpt-4.1' } replaced by the shared Auto selector, plus the justification.
src/webview/occurrence-explain.ts (new)	Why? flow moved out of the page module. Every state is rendered through Preact.
src/webview/page-antipatterns.ts	Inline explainOccurrence removed; the page uses the new module. The Why? button is always rendered and marked llm-only.
src/webview/panel.ts	getCapabilities actually probes Copilot. The panel pushes capabilities messages on lm.onDidChangeChatModels.
src/core/types/rpc-types.ts	New HostCapabilities and HostCapabilitiesMessage types.
src/webview/capabilities.ts	Applies pushed capabilities and reflects them as an llm-unavailable class on the root element, so CSS shows or hides AI controls without rebuilding the page. Adds a host-aware llmUnavailableNote() (replaces LLM_UNAVAILABLE_NOTE).
src/webview/dashboard-shell.ts, page-dashboard.ts, page-config.ts, styles.css	AI controls are always rendered and marked llm-only: the Skill Finder and Level Up nav entries, the Dashboard Skill Finder card, and Review Context Files with its workspace count. A disabled Review Context Files carrying the sign-in hint is marked llm-fallback. Two CSS rules keyed on the root class show or hide them.
Tests	New panel-llm-call.test.ts, occurrence-explain.test.ts, capabilities.test.ts; extended panel.test.ts; new shared-rpc.test.ts; Playwright cases in antipatterns.spec.ts and the new copilot-availability.spec.ts (harness.html mirrors the nav markers).
Docs	README privacy note, CHANGELOG.

Impacted functionality: before → after

Functionality (where in the UI)	Before	After
Why? result box, Anti-Patterns → finding → occurrences	The loading text stayed after the answer arrived (<answer>Asking AI for an explanation…). An empty answer left only the loading text. Asking again after hiding the answer stuck on "Asking AI for an explanation…" forever, with no error.	The loading text is replaced by the answer. Asking again shows the new answer. An empty answer shows "The AI returned an empty explanation. Try again." Host errors appear in the box.
Model used by every AI feature: Why?, Rule Editor Generate rule, Skill Finder (Analyze, catalog picks, Generate SKILL.md), Learning Center (quiz, Slop or Not, Did you know, resources), Context Health Review Context Files	Tried gpt-5.4-mini → gpt-5-mini → gpt-4.1-mini → gpt-4.1, then the first model from any vendor. A model could be selectable but disabled for the user's plan, which produced empty replies (#91).	Only Copilot Auto, which routes to a model enabled for the user's plan and org. There is no fallback. If Auto is unavailable: "GitHub Copilot isn't available. Sign in to GitHub Copilot in VS Code to use AI features."
Wait time and retries for plain-text AI calls: Why?, Generate rule, Generate SKILL.md	Model selection had no time limit. Every failure except cancellation was retried 3 times with a 90s timeout each, so a hung request took up to 270s. Denied access and quota errors were retried too. Empty replies counted as success. Generate SKILL.md gave up on the webview side after 120s and dropped a late reply.	The whole request, model selection included, is limited to 120s, and selection to 15s. Generate rule's up to three model calls share that one budget and keep the latest draft if time runs out. Generate SKILL.md now waits up to 300s on the webview side, so the reply or the error always arrives. Timeouts, denied access (NoPermissions), quota or rate limits (Blocked) and NotFound fail immediately with an actionable message (for example "Run 'Chat: Manage Language Model Access'…"). Transient errors and empty replies are retried within the budget.
JSON-based AI features: Skill Finder, Learning Center, Context Health review	Hard-coded families with any-vendor fallback; no consent justification.	Copilot Auto with the justification. Retry and JSON-repair logic are unchanged; that area is handled in #253 and #277.
Rule Playground: natural-language rule compile	{ family: 'gpt-4.1' } only, falling back to the heuristic template when that model wasn't available.	Copilot Auto; the heuristic fallback is unchanged.
Language-model consent dialog, first AI use	No reason given.	Shows "AI Engineer Coach uses GitHub Copilot to explain anti-pattern detections, review context files, and generate skills, rules, and quizzes when you ask."
AI feature visibility in VS Code	getCapabilities always returned llm: true. Why?, Skill Finder (nav entry and dashboard card), Level Up and Review Context Files were always shown, even without Copilot, and then failed or hung when used.	The dashboard probes Copilot Auto at load (up to 3s; treated as available if the probe is slow or errors). Without Copilot, Why? buttons, the Skill Finder card, and the Skill Finder and Level Up nav entries are hidden. Opening those pages directly redirects to Dashboard. Review Context Files is disabled with a hint.
Copilot sign-in or sign-out while the dashboard is open	Not tracked; AI features always looked available.	The panel listens to lm.onDidChangeChatModels and pushes the change. AI controls appear or disappear in place, with no need to reopen. The page isn't rebuilt, so an open Rule Editor and its unsaved draft, expanded findings, and loaded results are kept. If you're on Skill Finder or Level Up when Copilot becomes unavailable, you stay on the page and AI actions show the sign-in message.
Hint next to unavailable AI features	Only the canvas text existed: "Open in VS Code with the local Copilot agent to use this."	In VS Code: "Sign in to GitHub Copilot in VS Code to use this." The canvas text is unchanged.
Bring-your-own-key or third-party-only setups (no Copilot)	AI features could run on any registered model through the any-vendor fallback.	Not supported: AI features are hidden unless Copilot Auto is available. This is an intentional scope decision.
Copilot app canvas mode	AI features hidden (llm: false).	Unchanged.
Diagnostics, AI Engineer Coach Output channel	Only call-failed from JSON calls.	Local [panel-llm] lines: model-selected (vendor, id, family), model-unavailable, call-failed (with a retryable flag), empty-response, probe-inconclusive, plus [panel] capabilities-changed. Prompts and keys are never logged.

Not changed:

• prompts and secret redaction;
• callLlmJson retry and JSON-repair logic;
• the @aicoach chat participant, which uses the model selected in chat;
• MCP tools;
• log parsing and analytics, and every page driven by logs;
• the canvas host.

Root cause (reproduced)

explainOccurrence wrote the loading text with textContent and then called Preact render() on the same element. Preact never removes DOM nodes it didn't create, and after a textContent wipe it keeps patching its old, detached nodes.

The new tests reproduce the bug against the original code:

• jsdom, first answer: "The session ran long.Start a new chat per task.Asking AI for an explanation..."
• jsdom, asking again: "Asking AI for an explanation..." forever
• real Chromium (Playwright): "First explanation.Asking AI for an explanation..."

Not adopted from the issue

• contributes.languageModels: VS Code has no such contribution point. The only one is languageModelChatProviders, which is for providing models. Calling vscode.lm needs no contribution; consent is an auth dialog shown on the first sendRequest.
• gpt-4o / o1 / o3-mini family list: these models are retired too. Copilot Auto replaces hard-coded names.
• Warm-cache pendingMessages flush: already fixed on main by publishReady() (68ceef6) and covered by an existing panel.test.ts case.

Validation

Command	Result
npm run check (CI verify)	✅ passed in CI: npm run check, build, check-size, package
npm run check (local)	typecheck ✅ · lint ✅ 0 errors (426 warnings, same as baseline) · spellcheck ✅ · knip ✅ · lockfile-lint ✅ · tests: 1694 passed, 5 skipped. 7 src/core/github-app-analytics.test.ts failures are specific to this machine: they also fail on unmodified main and pass in CI.
npm run test:e2e	✅ 130 passed, 7 skipped (Burndown/Token usage feature flags)
npm run check-size	✅ 1.64 MB, within the 2 MB budget

New tests:

• panel-llm-call.test.ts (18): selection, probe, retry, timeout and budget behaviour, including model selection counting toward the budget and deadlines shared across calls.
• shared-rpc.test.ts (4): every Copilot-backed RPC waits longer than the host budget plus a 60s delivery margin; other RPCs keep 120s.
• panel-rpc.test.ts (+5): generateRuleMarkdown retry feedback and keeping the latest draft.
• occurrence-explain.test.ts (6) and capabilities.test.ts (5): jsdom.
• 4 host-capability tests in panel.test.ts.
• 4 Playwright tests in antipatterns.spec.ts, including a regression test that an unsaved Rule Editor draft survives a capability change.
• 2 Playwright tests in copilot-availability.spec.ts: the Dashboard Skill Finder card and the Context Health fallback button.

Manual verification with real Copilot (still to do before marking ready)

Launch an isolated dev host from the repo root with code --user-data-dir "$env:TEMP\aiec-test" --extensionDevelopmentPath "$PWD". It starts with fresh GitHub sign-in and consent state and still reads your real session logs. Watch View → Output → AI Engineer Coach.

• Signed in: Why? returns an explanation, and the log shows a [panel-llm] model-selected line with vendor=copilot id=auto. Hide it, then ask again: a new explanation appears.
• Consent: on the first use the dialog shows the justification. Deny gives an immediate actionable error; after allowing access, Why? works.
• Signed out: the Why? buttons, the Skill Finder card, and the Skill Finder and Level Up nav entries are hidden, and Review Context Files is disabled with a hint. Sign in and they come back without reopening the dashboard.
• With the Rule Editor open and a draft typed, sign out of Copilot and back in: the editor and the draft stay.
• Offline: you get a bounded error, never an endless spinner.
• Skill Finder scan and the Level Up quiz still work through Copilot Auto.

Related Issues

Resolves #278. Related: #91 (empty responses from selectable-but-disabled models). This PR supersedes the model-selection part of #253, while #253's callLlmJson empty-response handling is still complementary. It is also compatible with #276 / #277.

Checklist

• npm run check passes (typecheck + lint + spellcheck + knip + tests). Green in CI verify; see Validation for local notes.
• Changes are covered by tests (if applicable)
• Documentation updated (if applicable): README privacy note and CHANGELOG

[github-actions]

⚠️ Deprecation Warning: The deny-licenses option is deprecated for possible removal in the next major release. For more information, see issue 997.

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

[Copilot]

🟡 Changes recommended

Capability-triggered page replacement silently discards unsaved Rule Editor drafts.

1 open issue

• Copilot changes discard unsaved Rule Editor drafts · New

What changed in this PR

Fixes stuck anti-pattern explanations and standardizes optional AI features on GitHub Copilot Auto.

Changes:

• Renders explanation states consistently through Preact.
• Adds bounded model selection and improved plain-text request error handling.
• Updates AI feature visibility when Copilot availability changes, with regression coverage.

File	Description
tests/​e2e/​harness.html	Adds AI navigation markers.
tests/​e2e/​antipatterns.spec.ts	Tests explanations and capability changes.
src/​webview/​styles.css	Hides unavailable AI navigation items.
src/​webview/​panel.ts	Probes and pushes Copilot availability.
src/​webview/​panel.test.ts	Tests host capability responses and updates.
src/​webview/​panel-llm.ts	Updates model access, retries, and timeouts.
src/​webview/​panel-llm-model.ts	Centralizes model selection and failure classification.
src/​webview/​panel-llm-call.test.ts	Tests model access and request failures.
src/​webview/​page-config.ts	Uses host-specific availability guidance.
src/​webview/​page-antipatterns.ts	Delegates explanation rendering.
src/​webview/​occurrence-explain.ts	Manages explanation states through Preact.
src/​webview/​occurrence-explain.test.ts	Tests explanation rendering and recovery.
src/​webview/​dashboard-shell.ts	Marks AI-dependent navigation items.
src/​webview/​capabilities.ts	Handles capability updates and subscriptions.
src/​webview/​capabilities.test.ts	Tests capability state and notifications.
src/​webview/​app.ts	Refreshes navigation and pages on capability changes.
src/​core/​types/​rpc-types.ts	Defines shared capability message types.
src/​core/​rule-compiler.ts	Uses Copilot Auto and consent justification.
src/​core/​copilot-model.ts	Defines shared selector and justification.
README.md	Documents Copilot requirements and availability.
CHANGELOG.md	Records explanation and model-access changes.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Host and webview request timeouts remain misaligned, and the planned real-Copilot integration verification is still pending.

1 open issue

• Start LLM deadline before selection and extend skill-generation RPC timeout · New

1 resolved since last review

• Copilot changes discard unsaved Rule Editor drafts

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Real Copilot Auto routing, consent, sign-in transitions, and offline behavior still require the manual integration checks identified in the PR.

No open issues 🎉

1 resolved since last review

• Start LLM deadline before selection and extend skill-generation RPC timeout

_{Review effort: Balanced}