Repository navigation
fix(engine): stop fabricating acceptance and funnel on degraded reads - #289
Merged
cpenaforte-clickbus merged 22 commits intoOct 8, 2026
Conversation
Acceptance matches commits to PRs through commit_refs, and open-PR staleness reads the last commit push from them. Both come from the enrichment pass. With it failed a real run reported 0 commits in PRs against 30 clean, and underestimated last activity, inflating stalled PRs. Omit acceptance_by_origin/by_tool whole and the four staleness fields; open-PR count and ages stay. Documented in METRICS.md. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A null data.repository, or a node with commits: null, raised AttributeError, which dropped the whole PR read to the fetch step. A non-dict at those points is now a missing page (read incomplete, what was already read is kept) or a node with no commits. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gh api -F reads a file when the value starts with "@", so a cursor beginning with that character would be misread. -f sends plain text. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The console said "failed ... continuing without PR data" when only a secondary pass failed on a window with no PRs, and said nothing when PRs were found but the read had degraded. Now PRs plus degradation print the found count with the incomplete steps; no PRs with only secondary steps degraded say the read was incomplete; basic or fetch degraded keep the failure message. Clean runs print as before. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The by-origin and by-tool blocks repeated the same comprehension plus an update loop for the review-derived keys. One small helper now builds both. No behavior change: the key order in each group is unchanged, so a clean run stays byte-identical. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The funnel's "In PR" stage reads acceptance_by_origin and, when it is absent, assumes every commit is in a PR. With the enrichment failed that field is now omitted, so the funnel reported 100% In PR and inflated every later conversion. Return no funnel then (the stages chain, so dropping one would break its meaning). Repos with no PR data keep the old default. Also fix two METRICS.md rows that claimed the review-round and single-pass fields drop on enrichment; only reviews gates them. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The funnel returned nothing only when enrichment failed. When every state's list failed (basic) or the whole read raised (fetch), there are no PRs, acceptance is absent, and the "In PR" stage fell back to every commit being in a PR: 100% In PR in metrics.json and the report. Now the funnel is omitted whenever the read was degraded and acceptance is absent. PRs absent by design (no gh, no GitHub remote) mark nothing as degraded and keep the old default. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The enrichment query sent owner and name with `-F`, which lets gh coerce the value: a repo named 2048 went out as the number 2048 (and `true`/`null` as a boolean/null), so the `String!` variable failed on every page and the enrichment always degraded. Every variable of the query is a string, so all of them now go with `-f`, as the cursor and states already did. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A page with `"nodes": null` raised a TypeError, and a null PR node or a null commit entry raised an AttributeError; any of them crashed the whole PR read into `fetch`. The enrichment pass now reads a null list as empty and skips nodes and commit entries that are not objects, keeping the rest of the page. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The console printed "{count} merged PRs found", but the count is every
PR read in the window: merged, closed and open. Both the clean and the
degraded line now say "{count} PRs found" ("{count} PRs encontrados"
in pt-BR).
Refs #236
Assisted-by: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A page whose `nodes` was null, or not a list at all, was read as an empty page: the enrichment pass ended "complete" with none of that page's PRs. It is now an incomplete page, like a null `pullRequests`: the pass keeps what earlier pages collected and reports enrichment as failed. Null PR nodes and commit entries inside the list are still skipped. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The enrichment retry waited 2 s on any gh failure, deterministic ones included: a 401, a 404 or a GraphQL error fails the same way on the second try, so every run paid the pause per page and per PR state for nothing. The retry now reads gh's stderr and only tries again on a transient error (HTTP 5xx, a timeout, a dropped connection). The reviews pass, which the code documents as prone to 504s, had no retry at all, while one failure of it now drops review metrics, the flow block and staleness from the report. It gets the same single transient-only retry. The basic list stays as it was. The pause constant becomes _RETRY_DELAY_S, since both passes share it. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PR read fetches merged, closed and open PRs separately, but the aggregator gated on the union of the three: a reviews or enrichment failure only for open PRs wiped review, flow, acceptance and merge strategy, which read merged PRs alone. That threw away good data. PullRequestFetch now keeps the failed steps per state (degraded_by_state), the window-cache slice inherits them, and every aggregate() call in the CLI and the org runner passes them. Review, flow, acceptance and merge strategy follow the merged state; open-PR staleness follows the open state; a closed-state failure omits nothing, since closed PRs only feed counts. Without the per-state map (a fetch that never reached the states, or no map at all) every state takes the union, as before. The origin funnel cannot tell the states apart from the union alone, so it now skips when enrichment failed and acceptance is absent. The aggregator always omits acceptance when the merged state's enrichment failed, so the funnel's "In PR" stage still never falls back to a made-up "every commit is in a PR" then, and the funnel survives a failure that hit only open or closed PRs. pr_enrichment_degraded keeps listing the union, so the JSON contract does not change, and a clean run's outputs stay identical. The report caveat and METRICS.md now say the omissions follow the PR state. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When `gh pr list` failed for one state while the others were read, the PR list was not empty, so everything the missing state feeds read as a real zero: with the merged list gone, acceptance showed 0% for every group, the activity timeline 0 PRs merged a week, and merge strategy "unknown" with reliable commit metrics; with any list gone, Flow Load undercounted the WIP. A missing merged list now omits what merged PRs feed (PR lifecycle, review, Flow Efficiency, human coverage, acceptance, merge strategy) and leaves the timeline's prs_merged unknown; a missing open list omits every open-PR field; a missing list in any state omits Flow Load. Without the per-state map the union applies, and `fetch` counts as every list failed. Clean runs are unchanged. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The transient-error check matched anywhere in gh's stderr: "EOF" matched "thereof", and "timeout" matched any error about a repo named like timeout-svc, so a deterministic failure there was retried with the pause on every page. The repo's owner/name, name and owner are now blanked out of stderr first (only where they stand alone, so a repo named "out" leaves "timeout" whole), and the patterns match whole words: HTTP 5xx, timeout/timed out, deadline exceeded, connection reset/refused, EOF. GitHub's GraphQL "may be the result of a timeout" error is still retried. Also refresh the enrichment docstring, which still said a page failed "even after one retry". Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nullable-when cells said "`reviews` degraded" or "`enrichment` degraded" while the prose already gated each field on the PR state that feeds it; the cells now carry that state, and the ones a failed list omits say so (acceptance, funnel, timeline, PR lifecycle, Flow Load, Flow Efficiency, coverage, open PR aging, merge strategy). §15 documents the failed-list rule, defines a stopped enrichment pass as failed even when earlier pages came back, and says a 4xx or other GraphQL error is not retried (GitHub's "may be the result of a timeout" error is). Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The report caveat listed flow, review and in-PR metrics as what a
degraded read may leave missing, but a failed open list, or a failed
enrichment or reviews pass for open PRs, omits open-PR fields too. The
caveat now names them ("open-PR metrics" / "de PRs abertos"); its
opening sentence is unchanged.
Refs #236
Assisted-by: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
§24 listed a failed list only in the nullable-when cell. It now says the whole series goes when `gh pr list` failed for any state (or `fetch`), author_concurrency included, and why: the missing list would undercount the WIP in every bucket it overlaps, and keeping only the commit-based author count would emit buckets in a shape the platform does not read. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Several notes still described the older PR read. The aggregate() docstring called `prs` merged PRs, but they come in all three states. METRICS.md §10 and the funnel module said Iris only fetches merged PRs; it reads merged, closed and open PRs, and the limitation is that acceptance counts only commits in merged PRs. §10 now says a failure confined to closed or open PRs leaves acceptance and the funnel where a clean run would have them, without implying they are always present. §15 now says that with merged enrichment failed, Flow Efficiency is computed over the PRs whose commit refs came back, so it is skewed rather than missing. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed merged list omits the PR lifecycle fields (counts, time to merge, cycle time), and a partial read skews them, but the report caveat did not list them among what may be missing or skewed. It now does, in both languages; its opening sentence is unchanged. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This was referenced Oct 8, 2026
…-list fix(engine): omit what a failed PR list feeds, per state
fix(engine): gate degradation per PR state and retry transient errors
cpenaforte-clickbus
merged commit Oct 8, 2026
0f5d438
into
fix/pr-read-degradation-org-report
4 of 5 checks passed
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contexto e Motivação
Uma varredura antes do merge da série (#280 a #285), com o engine rodando sobre dados reais do GitHub e falhas simuladas em cada passo da leitura de PRs, mostrou que ainda saíam números inventados quando o enriquecimento falhava. Com a passada de GraphQL derrubada, a acceptance do repositório afirmava 0 commits em PR e 0% de PR rate, contra 30 commits e 33% no run limpo. E, uma vez sem acceptance, o funil de origem caía no valor padrão e mostrava 100% "em PR". Valores nulos no GraphQL ainda derrubavam a leitura inteira para
fetch, e ogh api -Fconvertia tipos e lia arquivo a partir de valores das variáveis.Resolver agora porque são exatamente o tipo de número que a #236 existe para não publicar, e a série ainda não entrou na
main.Refs #236
Mudanças
acceptance_by_origineacceptance_by_toolsão omitidos (casar commit com PR depende das commit refs), e também os campos de PR aberto parado, cuja última atividade usa os pushes de commit.ghou sem remote — mantém o comportamento de antes.repository,pullRequestsounodesnulos como página incompleta e ignora nós e commits nulos, em vez de levantar exceção.gh api -f: com-F, um repositório de nome só numérico virava número e quebrava o enriquecimento em todo run, e um valor começando com@era lido como caminho de arquivo.Plano de Teste
pytest tests/test_pr_fetch_degradation.py -v— acceptance, staleness e funil ausentes com o enriquecimento degradado e presentes no run limpo; funil ausente combasicefetch;repository,pullRequests,nodes, nó e commit nulos sem exceção;-fpara owner, name e cursor, inclusive nome2048,trueenull; as linhas de console — rodado em0e0bf9d: 54 passedpytest tests/ -q— a suíte inteira, com o portão de determinismo da [FEAT] Comparador de paridade do metrics.json, campo a campo #214 — rodado em0e0bf9d: local: 408 passed; a única falha é atest_status_reflects_state([BUG] test_agent_settings_hook lê o spool real — falha em máquina com registros pendentes #146), que só ocorre fora do CI, e otest_hook_binders.pyficou de fora por depender de rede; no CI o comando literal passaA verificação roda fora da máquina de quem construiu: o job
clido CI (RFC 0029, PM-4).Impacto e Risco
docs/METRICS.md.metrics.jsone o mesmo relatório, byte a byte. Runs com o enriquecimento degradado deixam de trazer acceptance, staleness e funil.gh: nenhum valor vindo da resposta do GitHub pode mais ser interpretado como caminho de arquivo.Sobre o tamanho (RFC 0028 §2.4): 482 linhas, faixa 251–500. São correções da mesma varredura, cada uma num commit próprio; a parte de degradação por estado foi separada no PR seguinte justamente para caber.
Plano de Rollback
git revertdo merge. Volta o comportamento da #285; nada a migrar.Referências
Autoria Assistida
🤖 Generated with Claude Code