Skip to content

VALIDATION.md

Latest commit

 

History

History
46 lines (34 loc) · 5.6 KB

VALIDATION.md

File metadata and controls

46 lines (34 loc) · 5.6 KB
Raw

Validation of the Rust migration (0.5.0)

Version 0.9 release checks are recorded separately in release notes and final local validation. Historical measurements below are unchanged. Lean native site parity is checked across the prepared language groups; CI also runs the Unicode/CRLF relocation and strict-scope probe, plus relocated package/client-adapter checks.

Local reference: immutable v0.4.0, initially commit 2501034b84fab0ca0c0194d95be74a7660bb8a56. Both servers use the retained semantic adapters; the comparison checks the new orchestration/storage/query/transport against the old core. Stable IDs, complete fixture searches, architecture/diagnostics, callers, callees, references, implementations, neighbors, dependencies, affected_by_change, blast_radius and snippets are compared for every fixture symbol. Only generation is normalized in graph comparisons; initialize version and new metadata are checked separately. Existing status fields and unchanged index/change reports are compared; provider-health field shapes are checked because runtime availability/locations change with the native launch. After a source edit, source-change reports and resulting graph data are compared; reindexed work sets may differ because scope invalidation is intentionally different.

Local checks (Linux x64, 2026-10-01)

  • Rust formatting, Clippy with warnings denied, 30 core/transport/storage/incremental tests.
  • Dart provider formatting, analysis with infos denied and three Analyzer tests, including generics/accessors/operators, conditional dependencies and emission with unchanged context.
  • Ruff and strict mypy on the retained Python adapters; Go vet on the retained Go adapter.
  • Differential native MCP: Dart fixture 37 symbols / 99 edges; polyglot 41 / 108; Python/Rust 32 / 81; mobile 75 / 131; actual Flutter 24 / 75. Mixed ten-language repository: 185 / 419.
  • Actual Android SDK classpath: 5 symbols / 6 edges; TypeScript server with no Dart SDK on PATH.
  • Relocated native Linux package, including selective provider setup and adjacent-asset discovery, tested in paths containing spaces.
  • Release synthetic benchmark: 100,000 symbols / 500,000 relations, repeated queries plus reconciliation, and separate persistence measurements. End-to-end fixture timing includes provider process memory. See docs/benchmarks.

Regression tests cover watcher overflow/loss recovery, duplicate source events, atomic replacement, rename/delete, independent scopes and dependency invalidation, source changes during analysis, concurrent writers, failed provider updates, interrupted SQLite publication, cache corruption at startup/during service, sidecar symlinks, external prepared context edits, Unicode/CRLF snippets, stale source, bounds/timeouts/output overflow, descendant termination, cancellation/duplicate IDs/queue limits, framing, lifecycle and EOF draining. A no-change index reuses its graph and performs no extraction; warm queries between reconciliations perform no full scans.

Cross-platform acceptance

CI requires core and native semantic/provider smoke checks on Linux, Windows and macOS, Flutter on each host, Android SDK on each host and real UIKit/bridging-header resolution on macOS/Xcode. Package builds and relocation/setup smoke tests cover Linux x64, Windows x64, macOS x64 and ARM64, exercising TypeScript plus the bundled native Dart/Go workers. Core tests also run on the Intel macOS package runner. The workflow uses v0.4.0 as its compiled oracle and uploads benchmark reports.

The complete 20-job cross-platform matrix passed for bef8832, including all four packages, Flutter, Android and macOS UIKit. Later commits must also pass their exact CI run before delivery. Local Linux checks do not verify UIKit, Windows or macOS. The v0.5.0 native release was published separately after all 20 jobs passed for 45fa910, with the four verified archives and SHA-256 checksums.

Limits

Watcher loss can leave stale data until the 30-second reconciliation. Use an explicit index scan for sensitive work, or watch:false for per-query verification. External artifacts beyond declared/tracked source contexts still need forced indexing after preparation. Provider analysis can read wider semantic context than the records emitted; scope tests guarantee isolated publication, not compiler incremental compilation. Dynamic/runtime and omitted cross-language relationships remain incomplete. The benchmark does not demonstrate token savings, a universal speed ratio or a memory reduction.

Local 0.8 candidate

The clean v0.7 branch, hashes, new regression matrix, Linux native checks and unverified cross-platform/AI gates are recorded separately in the 0.8 report. This entry does not change historical 0.5/0.6/0.7 gates. The new Python accounting/evaluator tests are covered by the existing CI test_*.py discovery; Rust regressions remain in cargo xtask check. That dated report describes the uncommitted rc.1; subsequent publication is covered by 0.8 release notes. Final 0.8 additionally fixes request metadata compatibility, tested in real framing/EOF and native profile checks. The application pilot observes lower uncached input than 0.7, but G2 remains failed and the no-graph target unmet. Exact release CI and package artifact identities are recorded in the GitHub release.