Repository navigation
fix(engine): surface PR read degradation in metrics and the report - #281
Merged
Merged
Conversation
The read now knows which steps degraded; this carries that into the metrics. ReportMetrics gains pr_enrichment_degraded, omitted when nothing degraded so a clean run's metrics.json is unchanged, and both entry points pass the read's degradation to aggregate(). Merge strategy changes behavior on purpose. With enrichment degraded, merge PRs have no parent count and no commit refs and fall through to unknown, while squash-stamped ones still classify - so the fallback could only ever answer squash. A mixed repo read as 100% squash with its per-commit metrics flagged unreliable. The detector now reports unknown, with the reason. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
The recent-window and pre/post-adoption metrics are aggregated from slices of the same PR list, but were not told the read degraded. The adoption split is written into metrics.json, so a degraded run still carried a guessed merge strategy there, nested under the top-level unknown. Every aggregate() over this run's PRs now receives the read's degradation. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
The metrics now carry the degradation, but the report still presented partial PR numbers with the same confidence as complete ones. It states the failed steps at the top, next to the non-code-repository caveat - not in the PR section, which a read that lost every PR does not render. English and Portuguese. docs/METRICS.md documents the field in section 15 and the merge-strategy rule in section 28. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
The caveat promised that a re-run usually recovers the missing PR data, which is false when gh is installed but not authenticated: then every run fails the basic step. It also called the affected metrics understated, but with the reviews step failed, review coverage reads 0% and the single-pass rate 100% - skewed, not understated. The caveat and the METRICS.md entry now say so, and point to gh auth status when every run fails. The console no longer reports a failed read as skipped. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
2 tasks done
This was referenced Oct 5, 2026
cpenaforte-clickbus
marked this pull request as ready for review
October 5, 2026 19:16
cpenaforte-clickbus
requested review from
clickmatos,
lucastribioliclickbus,
rafaeldearaujop and
trentas
and removed request for
trentas
October 5, 2026 19:28
clickmatos
changed the base branch from
fix/pr-read-degradation-signal
to
main
October 8, 2026 13:35
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contexto e Motivação
A leitura de PRs já sabe quais passos degradaram (#280), mas nada fora dela fica sabendo: o
metrics.jsone o relatório seguem apresentando dado parcial com a mesma confiança do completo. E a merge strategy faz pior do que omitir: sob enriquecimento degradado, a heurística de fallback só consegue respondersquash, então um repositório misto aparece como squash 100%, com as métricas por commit marcadas como não confiáveis — o oposto da verdade. Como o dashboard mostra o run mais recente, um único run degradado basta para o repositório aparecer assim.Resolver agora porque é esta parte que torna o sinal visível: sem ela, a anterior não muda nada para quem lê o relatório.
Refs #236
Mudanças
ReportMetrics.pr_enrichment_degraded— os passos que falharam, em ordem e sem repetição. Omitido do JSON quando nada degradou, então um run limpo gera o mesmometrics.jsonde antes.aggregate(..., pr_fetch_degraded=...)emite o campo, ecli.pyeorg_runner.pypassam o que a leitura reportou — também nos agregados da janela recente e de antes e depois da adoção, porque ometrics.jsonserializa esses dois últimos emadoption_timeline.detect_merge_strategy(..., enrichment_degraded=True)devolveunknowncom o motivo,commit_metrics_reliable = truee semdominant_share, em vez de classificar com uma heurística que só sabe dizer squash.ghinstalado mas sem autenticação, toda execução falha, e o texto apontagh auth status.docs/METRICS.md§15 documenta o campo, e §28 a regra da merge strategy.Plano de Teste
pytest tests/test_pr_fetch_degradation.py tests/test_merge_strategy_detector.py tests/test_pr_enrichment_caveat.py -v— o campo no payload e a ausência dele num run limpo;unknownsob degradação contrasquashcom os mesmos dados; os dois pontos de entrada,clieorg_runner, escrevendo o campo nummetrics.jsonreal a partir de um repositório git temporário, inclusive nos agregados de adoção; a linha de console de falha; o aviso em EN e PT-BR, inclusive sem seção de PR — rodado em10441c4: 36 passedpytest tests/ -q— a suíte inteira, com o portão de determinismo da [FEAT] Comparador de paridade do metrics.json, campo a campo #214 — rodado em10441c4: local: 371 passed; a única falha é atest_status_reflects_state([BUG] test_agent_settings_hook lê o spool real — falha em máquina com registros pendentes #146), que só ocorre fora do CI, e otest_hook_binders.pyficou de fora por depender de rede; no CI o comando literal passapython scripts/check_analysis_chain.py— nenhum módulo de análise novo — rodado em10441c4: Analysis chain OKA verificação roda fora da máquina de quem construiu: o job
clido CI executa a suíte a cada push (RFC 0029, PM-4).Impacto e Risco
docs/METRICS.md.metrics.jsone o mesmo relatório. O/api/ingestaceita o campo novo sem alteração (.passthrough(),route.ts:45). Mudança de comportamento deliberada: com o enriquecimento degradado,merge_strategypassa aunknownem vez de um valor enviesado, e o repositório mostra estratégia desconhecida naquele run em vez de squash 100%.reviewsdegradado, as métricas de review ainda saem distorcidas aqui — cobertura 0%, single-pass 100% —, e o aviso diz isso. O fix(engine): omit review-derived metrics when the reviews read fails #284, empilhado sobre este, passa a omiti-las.Sobre o tamanho (RFC 0028 §2.4): 412 linhas, faixa 251–500. Separar o campo do aviso deixaria o relatório calado sobre algo que o JSON já carrega.
PR empilhado: a base é a branch do #280, então o diff mostra só esta parte. Depois do merge do #280, a base passa a ser a
main.Plano de Rollback
git revertdo merge deste PR. O campo some do payload e a merge strategy volta a ser classificada como antes; nada a migrar.Referências
docs/METRICS.md§15 e §28Autoria Assistida
unknownpartiu do autor; desenho e implementação guiada por testes foram assistidos🤖 Generated with Claude Code