Skip to content

D-CODEGRAPH-RECON-1: Codegraph over real Charon stores, Mermaid reconstruction, review evidence packet - #2000

Merged
AdaWorldAPI merged 11 commits into
mainfrom
ccr-ca0562e7-hudvlz-codegraph-recon
Nov 11, 2026
Merged

AdaWorldAPI merged 11 commits into
mainfrom
ccr-ca0562e7-hudvlz-codegraph-recon

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Nov 11, 2026 •

Copy link
Copy Markdown
Owner

This probe tests whether an agent can get source-backed answers about the DT64 / R2IL / RBAC subsystems from compiler facts. Those answers cover what exists, what calls what, impact, and what is unknown. Board: EXO-229 (intake), EXO-230 (navigation).

Changes

  • crates/lance-graph-codegraph/examples/cg_query.rs: a new read-only command-line tool over Codegraph (symbols, callers, callees, impact, refs, sweep).
    • Every line carries its grade.
    • Refusals print verbatim.
    • A PUBLIC symbol gets an UNKNOWN external-callers-unobserved line, and every answer lists the files the store observed.
    • The library API is unchanged.
  • .claude/probes/2026-11-11-codegraph-recon-1/:
    • corpus.tsv: 444 relationships from .mermaid/ pages 03, 05 and 06, each verified against source.
    • recon.py: builds the machine-recovered diagram from store rows only, scores recall, writes the reconciled diagram and the evidence packet, and carries 15 negative tests.
    • The outputs, the measurement and REPORT.md.
  • .codegraph/coverage.tsv is regenerated for the new example, and there is a board entry.

Findings (all measured; details in REPORT.md)

  • Intake: Charon → ruff_charon ran clean on dt64, contract, quack and r2il, with 1 residual. The four-crate store has 9,138 positions. impact + callers over all of them takes 1.2 s with 0 refusals.
  • Correction to the brief's limits: ClosureWidth(64) and the 256-id tile are single-tile limits. Codegraph answers through closure_wide.
  • False unaffected: impact returns an empty set for public symbols whose callers are in other crates. On dt64 this is 261 of 370 public symbols, including BindStore::admit and method_ref.
  • rust-analyzer coverage: it drops calls inside macro arguments but marks the regions, does not index #[cfg(test)] modules, and counts 6,280 constructions as calls.
  • Reconstruction: recall of the curated call edges over observed files is Charon 61/87 and rust-analyzer 91/99. Most of the gap is variant construction, which Charon does not model as a call. Only 3 true call sites are missed.
  • Retrieval: 4 paths × 2 runs × 10 questions were graded blind against a source-checked key. There were 0 wrong answers and all 80 were coverage-honest. Compiler facts were the most precise on references and calls, and could not see test callers. With n = 2, no path is more accurate overall.
  • Evidence packet: a read-only ReviewEvidencePacket prototype for a change to LaneBinding::check, using the crai-rs fields and axes. It holds evidence, not a verdict.

Review fixes

Codex raised three findings in its review. Each now has a test that failed first and goes red again when the fix is disabled:

  • A caller reached only through stale evidence is no longer confirmed (test 11).
  • possible now joins IMPLEMENTS to MAY_DISPATCH, as Codegraph::callers does (test 12).
  • An edge from a missing source file is never drawn as verified (test 13).

Reproduction

The documented Charon and ruff_charon commands were re-run from scratch into a fresh directory.

  • machine-recovered.mmd, reconciled.mmd and score.tsv regenerate byte-identical.
  • The sweep counts are identical, and so is the rust-analyzer graph apart from its timing rows.
  • One finding: the store is reproducible in content but not location-independent. Build-script outputs and the LLBC path are recorded as absolute paths, so the same input in a different target directory gets a different digest. Owner: EXO-229.

Open

  • Coverage-aware impact: the follow-up branch adds Codegraph::impact_report additively. Codegraph::impact is unchanged (D-CODEGRAPH-RECON-2).
  • Charon artifacts for test targets and consumer crates. (EXO-229)
  • Readable R2IL per-method step reports are not done; this would need a new r2il example.

Checks

  • cargo clippy --release --example cg_query -- -D warnings and cargo fmt -- --check (in crates/lance-graph-codegraph)
  • python3 .claude/probes/2026-11-11-codegraph-recon-1/recon.py --self-test: 15 tests, each disable-verified red
  • python3 .claude/tools/codegraph_index.py --check: current

🤖 Generated with Claude Code

https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p

…s empty

Prints callers, callees, impact and references with their grade, prints
refusals verbatim, and adds an UNKNOWN external-callers-unobserved line
for any PUBLIC symbol plus the files the store observed. Measured on a
Charon store of lance-graph-dt64: 261 of 370 public symbols otherwise
return an empty impact although their callers live in other crates.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
… process

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
…corpus and seed list

recon.py rebuilds a diagram from a ruff_charon store only, scores the
curated corpus' call edges against compiler call sites, overlays labelled
human rows, and emits a read-only review evidence packet. --self-test
holds the eleven negatives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
…possible dispatch

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
…Mermaid, recall scores, evidence packets, report draft

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
…ETHOD, body UNKNOWN)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
… board entry

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
@coderabbitai

coderabbitai Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5f2f732a-00c6-4da4-a0cc-5f6fb32853f5
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review November 11, 2026 06:08
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-11T06:10:32.276450Z e6c6e4e Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e6c6e4ec7a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…atch to implementations

A caller reached only through a stale edge is no longer confirmed; it is
listed under affected_symbols.stale and flags dependency_evidence_stale.
possible now joins IMPLEMENTS to MAY_DISPATCH, as Codegraph::callers does,
and propagates through callers. An edge from a missing source file is drawn
as MISSING_SOURCE, never as a verified edge. Addresses review on #2000.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAqKMPJjVvasjwqhjJ6c9p
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants