Skip to content

Carry over the drift-sync residue PR #9 found - #11

Merged
jamiewest merged 1 commit into
mainfrom
drift/sync-residue-2026-09-21
Sep 21, 2026
Merged

jamiewest merged 1 commit into
mainfrom
drift/sync-residue-2026-09-21

Conversation

@jamiewest

Copy link
Copy Markdown
Owner

Supersedes #9. Two /drift sync runs overlapped — the 2026-09-07 run stalled mid-session and landed as #8 at 16:02Z today, while the scheduled 2026-09-21 run opened #9 at 12:24Z off the pre-merge main and re-ported the same three commits. Rather than rebase #9 (all nine of its files conflict against merged main, including a duplicate test at a different path), this carries over only what #8 lacked.

⚠️ The fix that matters — lib/src/ai/, affects jamiewest/agents

#8 shipped the fail-closed change half-wired. It fixed interpretScore() but left quality_evaluator_base.dart assigning the interpretation only in the parse-success branch:

if (!metric.tryParseEvaluationResponseWithTags(evalResponse, duration)) {
  metric.addDiagnostic(EvaluationDiagnostic.error('Could not parse a score…'));
} else {
  metric.interpretation = metric.interpretScore();   // ← only on success
}

So the one case upstream #7735 exists for — the judge's reply can't be parsed — still produced interpretation == null, and a caller gating on EvaluationMetricInterpretation.failed read that null as a pass. The interpretation is now assigned unconditionally.

If you're tracking the downstream impact: this widens what #8 already changed. More metrics now report failed: true — specifically, every unparseable evaluation, which previously reported nothing at all.

Also carried over

Pins → d13c2266 (runtime) / d2364e51 (extensions)
#133494 d13c226 skip — List capacity hints and a collection-expression rewrite; no behavior change. New N/A row covers the recurring class of allocation-tuning edits
#7749 d2364e5 out of scope — Microsoft.Extensions.Http.Diagnostics, which no scope row covers (the http row maps to dotnet/runtime's Microsoft.Extensions.Http)
#7738 c74cd51 skip — TypeScript frontend of the HTML report; the Dart port has no HtmlReportWriter
New open priority RelevanceTruthAndCompletenessEvaluator returns early on an inconclusive rating — now the only evaluator path that can still leave a metric unfailed
Tests explicit NaN case, the full 13-entry band table, a sub-threshold sweep, and a case pinning the wiring above
Cleanup _refreshWriteTime() extracted, removing the block duplicated between hasChanged and _checkForChanges

I verified the pins and the evaluator finding against source rather than taking #9's word: d13c2266 is the only in-scope runtime commit after 6fa6b31, and relevance_truth_and_completeness_evaluator.dart:106 does return early exactly as described.

Not taken from #9

It dropped the per-file try/catch in polling_wildcard_change_token.dart. That would let a single unreadable file abort the whole scan and report "no change" — and a permanently unreadable file would make the token blind to every other file indefinitely. Upstream only looks similar because its GetLastWriteUtc swallows per-file errors, so it doesn't actually abort either. main's behavior is kept.

Two commits landed after #9's run

Found while verifying the pins — both in AI.Evaluation.NLP, which is ported, and both deliberately left for the next increment:

  • e77687c (#7766) — fix tokenizer attaching punctuation to number tokens
  • 561bdf7 (#7768) — score GLEU against the best matching reference

They're recorded in the ledger as the next run's first items.

Release sequencing

Version stays 0.8.1 — it isn't on pub.dev yet, so this folds into that entry instead of burning a version. Merge this before pushing extensions-v0.8.1, or the tag ships without the evaluator fix.

Verification

Flutter 3.47.2 / Dart 3.13.2, matching CI:

Check Result
dart analyze --fatal-infos . No issues found
dart format --set-exit-if-changed . 883 files, 0 changed
dart test 1050 passing (+3)
flutter analyze --fatal-infos No issues found
flutter test 47 passing

The first run caught a real error — a const map keyed on double doesn't compile — which is fixed here as a list of records.

🤖 Generated with Claude Code

https://claude.ai/code/session_01P889JtFvr4ypqnHYQZCPag


Generated by Claude Code

Two /drift sync runs overlapped: the 2026-09-07 run stalled mid-session and
landed as #8 at 16:02Z today, while the scheduled 2026-09-21 run opened #9 at
12:24Z off the pre-merge main and re-ported the same three commits. #9 is
closed as superseded; this carries over only what #8 lacked.

The fix that matters:

* Quality evaluators now set `interpretation` unconditionally.
  #8 ported `interpretScore`'s fail-closed behavior but left
  `quality_evaluator_base.dart` assigning the interpretation only in the
  parse-success branch, so an unparseable judge reply still produced
  `interpretation == null` -- the exact case upstream #7735 exists to fix.
  A caller gating on EvaluationMetricInterpretation.failed read that null
  as a pass.

Also carried over:

* Pins advanced to d13c2266 (runtime) / d2364e51 (extensions), with the
  three commits in that window recorded: #133494 skipped as allocation
  tuning (new N/A row covering the recurring class), #7749 out of scope
  (Http.Diagnostics, which no scope row covers), #7738 skipped as
  TypeScript frontend.
* New open priority: RelevanceTruthAndCompletenessEvaluator returns early
  on an inconclusive rating, so it is now the only evaluator path that can
  still leave a metric unfailed. Verified against the Dart source and
  recorded rather than guessed at -- the fix needs a decision on whether to
  mirror upstream's sentinel values or leave `value` null.
* Test coverage folded in from #9: an explicit NaN case, the full 13-entry
  rating-band table, the sub-threshold sweep, and a case pinning the
  unscored-metric wiring above.
* `_refreshWriteTime()` extracted in polling_file_change_token.dart,
  removing the block duplicated between hasChanged and _checkForChanges.

Deliberately not taken from #9: it dropped the per-file try/catch in
polling_wildcard_change_token.dart, which would let one unreadable file
abort the whole scan and report "no change" -- a permanently unreadable
file would make the token blind to every other file indefinitely. Upstream
only looks similar because its GetLastWriteUtc swallows per-file errors.

Version stays 0.8.1: it is not on pub.dev yet, so this folds into that
entry rather than burning a version. Merge this before pushing the
extensions-v0.8.1 tag, or the tag ships without the evaluator fix.

Verified on Flutter 3.47.2 / Dart 3.13.2: dart analyze --fatal-infos clean,
dart format clean, 1050 tests pass (+3); flutter analyze clean, 47 tests
pass, format clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01P889JtFvr4ypqnHYQZCPag
@jamiewest
jamiewest merged commit 33cabbc into main Sep 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants