Repository navigation
Carry over the drift-sync residue PR #9 found - #11
Merged
Merged
Conversation
Two /drift sync runs overlapped: the 2026-09-07 run stalled mid-session and landed as #8 at 16:02Z today, while the scheduled 2026-09-21 run opened #9 at 12:24Z off the pre-merge main and re-ported the same three commits. #9 is closed as superseded; this carries over only what #8 lacked. The fix that matters: * Quality evaluators now set `interpretation` unconditionally. #8 ported `interpretScore`'s fail-closed behavior but left `quality_evaluator_base.dart` assigning the interpretation only in the parse-success branch, so an unparseable judge reply still produced `interpretation == null` -- the exact case upstream #7735 exists to fix. A caller gating on EvaluationMetricInterpretation.failed read that null as a pass. Also carried over: * Pins advanced to d13c2266 (runtime) / d2364e51 (extensions), with the three commits in that window recorded: #133494 skipped as allocation tuning (new N/A row covering the recurring class), #7749 out of scope (Http.Diagnostics, which no scope row covers), #7738 skipped as TypeScript frontend. * New open priority: RelevanceTruthAndCompletenessEvaluator returns early on an inconclusive rating, so it is now the only evaluator path that can still leave a metric unfailed. Verified against the Dart source and recorded rather than guessed at -- the fix needs a decision on whether to mirror upstream's sentinel values or leave `value` null. * Test coverage folded in from #9: an explicit NaN case, the full 13-entry rating-band table, the sub-threshold sweep, and a case pinning the unscored-metric wiring above. * `_refreshWriteTime()` extracted in polling_file_change_token.dart, removing the block duplicated between hasChanged and _checkForChanges. Deliberately not taken from #9: it dropped the per-file try/catch in polling_wildcard_change_token.dart, which would let one unreadable file abort the whole scan and report "no change" -- a permanently unreadable file would make the token blind to every other file indefinitely. Upstream only looks similar because its GetLastWriteUtc swallows per-file errors. Version stays 0.8.1: it is not on pub.dev yet, so this folds into that entry rather than burning a version. Merge this before pushing the extensions-v0.8.1 tag, or the tag ships without the evaluator fix. Verified on Flutter 3.47.2 / Dart 3.13.2: dart analyze --fatal-infos clean, dart format clean, 1050 tests pass (+3); flutter analyze clean, 47 tests pass, format clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01P889JtFvr4ypqnHYQZCPag
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #9. Two
/drift syncruns overlapped — the 2026-09-07 run stalled mid-session and landed as #8 at 16:02Z today, while the scheduled 2026-09-21 run opened #9 at 12:24Z off the pre-mergemainand re-ported the same three commits. Rather than rebase #9 (all nine of its files conflict against mergedmain, including a duplicate test at a different path), this carries over only what #8 lacked.lib/src/ai/, affectsjamiewest/agents#8 shipped the fail-closed change half-wired. It fixed
interpretScore()but leftquality_evaluator_base.dartassigning the interpretation only in the parse-success branch:So the one case upstream #7735 exists for — the judge's reply can't be parsed — still produced
interpretation == null, and a caller gating onEvaluationMetricInterpretation.failedread that null as a pass. The interpretation is now assigned unconditionally.If you're tracking the downstream impact: this widens what #8 already changed. More metrics now report
failed: true— specifically, every unparseable evaluation, which previously reported nothing at all.Also carried over
d13c2266(runtime) /d2364e51(extensions)d13c226Listcapacity hints and a collection-expression rewrite; no behavior change. New N/A row covers the recurring class of allocation-tuning editsd2364e5Microsoft.Extensions.Http.Diagnostics, which no scope row covers (thehttprow maps to dotnet/runtime'sMicrosoft.Extensions.Http)c74cd51HtmlReportWriterRelevanceTruthAndCompletenessEvaluatorreturns early on an inconclusive rating — now the only evaluator path that can still leave a metric unfailed_refreshWriteTime()extracted, removing the block duplicated betweenhasChangedand_checkForChangesI verified the pins and the evaluator finding against source rather than taking #9's word:
d13c2266is the only in-scope runtime commit after6fa6b31, andrelevance_truth_and_completeness_evaluator.dart:106does return early exactly as described.Not taken from #9
It dropped the per-file
try/catchinpolling_wildcard_change_token.dart. That would let a single unreadable file abort the whole scan and report "no change" — and a permanently unreadable file would make the token blind to every other file indefinitely. Upstream only looks similar because itsGetLastWriteUtcswallows per-file errors, so it doesn't actually abort either.main's behavior is kept.Two commits landed after #9's run
Found while verifying the pins — both in
AI.Evaluation.NLP, which is ported, and both deliberately left for the next increment:e77687c(#7766) — fix tokenizer attaching punctuation to number tokens561bdf7(#7768) — score GLEU against the best matching referenceThey're recorded in the ledger as the next run's first items.
Release sequencing
Version stays 0.8.1 — it isn't on pub.dev yet, so this folds into that entry instead of burning a version. Merge this before pushing
extensions-v0.8.1, or the tag ships without the evaluator fix.Verification
Flutter 3.47.2 / Dart 3.13.2, matching CI:
dart analyze --fatal-infos .dart format --set-exit-if-changed .dart testflutter analyze --fatal-infosflutter testThe first run caught a real error — a
constmap keyed ondoubledoesn't compile — which is fixed here as a list of records.🤖 Generated with Claude Code
https://claude.ai/code/session_01P889JtFvr4ypqnHYQZCPag
Generated by Claude Code