Repository navigation
feat(engine): flag repos with a degraded PR read in org outputs - #285
Merged
Merged
Conversation
A repository whose PR read failed showed 0 PRs in the org report and in org-metrics.json with no mark, while its own report carried the caveat. The org report now names those repositories next to the non-code note, and each org-metrics.json repo entry carries pr_enrichment_degraded when its read degraded. Clean runs produce the same outputs as before. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
"1 repositories" read wrongly for a single repository; the note now says "repository(ies)" like the sibling non-code note. "below" becomes "in this report", since the trend sections above the note can also carry PR deltas. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
cpenaforte-clickbus
added
type: tech-debt
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
3 tasks done
cpenaforte-clickbus
marked this pull request as ready for review
October 5, 2026 19:17
cpenaforte-clickbus
requested review from
clickmatos,
lucastribioliclickbus,
rafaeldearaujop and
trentas
October 5, 2026 19:29
Acceptance matches commits to PRs through commit_refs, and open-PR staleness reads the last commit push from them. Both come from the enrichment pass. With it failed a real run reported 0 commits in PRs against 30 clean, and underestimated last activity, inflating stalled PRs. Omit acceptance_by_origin/by_tool whole and the four staleness fields; open-PR count and ages stay. Documented in METRICS.md. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A null data.repository, or a node with commits: null, raised AttributeError, which dropped the whole PR read to the fetch step. A non-dict at those points is now a missing page (read incomplete, what was already read is kept) or a node with no commits. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
gh api -F reads a file when the value starts with "@", so a cursor beginning with that character would be misread. -f sends plain text. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The console said "failed ... continuing without PR data" when only a secondary pass failed on a window with no PRs, and said nothing when PRs were found but the read had degraded. Now PRs plus degradation print the found count with the incomplete steps; no PRs with only secondary steps degraded say the read was incomplete; basic or fetch degraded keep the failure message. Clean runs print as before. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The by-origin and by-tool blocks repeated the same comprehension plus an update loop for the review-derived keys. One small helper now builds both. No behavior change: the key order in each group is unchanged, so a clean run stays byte-identical. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The funnel's "In PR" stage reads acceptance_by_origin and, when it is absent, assumes every commit is in a PR. With the enrichment failed that field is now omitted, so the funnel reported 100% In PR and inflated every later conversion. Return no funnel then (the stages chain, so dropping one would break its meaning). Repos with no PR data keep the old default. Also fix two METRICS.md rows that claimed the review-round and single-pass fields drop on enrichment; only reviews gates them. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The funnel returned nothing only when enrichment failed. When every state's list failed (basic) or the whole read raised (fetch), there are no PRs, acceptance is absent, and the "In PR" stage fell back to every commit being in a PR: 100% In PR in metrics.json and the report. Now the funnel is omitted whenever the read was degraded and acceptance is absent. PRs absent by design (no gh, no GitHub remote) mark nothing as degraded and keep the old default. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The enrichment query sent owner and name with `-F`, which lets gh coerce the value: a repo named 2048 went out as the number 2048 (and `true`/`null` as a boolean/null), so the `String!` variable failed on every page and the enrichment always degraded. Every variable of the query is a string, so all of them now go with `-f`, as the cursor and states already did. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A page with `"nodes": null` raised a TypeError, and a null PR node or a null commit entry raised an AttributeError; any of them crashed the whole PR read into `fetch`. The enrichment pass now reads a null list as empty and skips nodes and commit entries that are not objects, keeping the rest of the page. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The console printed "{count} merged PRs found", but the count is every
PR read in the window: merged, closed and open. Both the clean and the
degraded line now say "{count} PRs found" ("{count} PRs encontrados"
in pt-BR).
Refs #236
Assisted-by: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A page whose `nodes` was null, or not a list at all, was read as an empty page: the enrichment pass ended "complete" with none of that page's PRs. It is now an incomplete page, like a null `pullRequests`: the pass keeps what earlier pages collected and reports enrichment as failed. Null PR nodes and commit entries inside the list are still skipped. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The enrichment retry waited 2 s on any gh failure, deterministic ones included: a 401, a 404 or a GraphQL error fails the same way on the second try, so every run paid the pause per page and per PR state for nothing. The retry now reads gh's stderr and only tries again on a transient error (HTTP 5xx, a timeout, a dropped connection). The reviews pass, which the code documents as prone to 504s, had no retry at all, while one failure of it now drops review metrics, the flow block and staleness from the report. It gets the same single transient-only retry. The basic list stays as it was. The pause constant becomes _RETRY_DELAY_S, since both passes share it. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The PR read fetches merged, closed and open PRs separately, but the aggregator gated on the union of the three: a reviews or enrichment failure only for open PRs wiped review, flow, acceptance and merge strategy, which read merged PRs alone. That threw away good data. PullRequestFetch now keeps the failed steps per state (degraded_by_state), the window-cache slice inherits them, and every aggregate() call in the CLI and the org runner passes them. Review, flow, acceptance and merge strategy follow the merged state; open-PR staleness follows the open state; a closed-state failure omits nothing, since closed PRs only feed counts. Without the per-state map (a fetch that never reached the states, or no map at all) every state takes the union, as before. The origin funnel cannot tell the states apart from the union alone, so it now skips when enrichment failed and acceptance is absent. The aggregator always omits acceptance when the merged state's enrichment failed, so the funnel's "In PR" stage still never falls back to a made-up "every commit is in a PR" then, and the funnel survives a failure that hit only open or closed PRs. pr_enrichment_degraded keeps listing the union, so the JSON contract does not change, and a clean run's outputs stay identical. The report caveat and METRICS.md now say the omissions follow the PR state. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When `gh pr list` failed for one state while the others were read, the PR list was not empty, so everything the missing state feeds read as a real zero: with the merged list gone, acceptance showed 0% for every group, the activity timeline 0 PRs merged a week, and merge strategy "unknown" with reliable commit metrics; with any list gone, Flow Load undercounted the WIP. A missing merged list now omits what merged PRs feed (PR lifecycle, review, Flow Efficiency, human coverage, acceptance, merge strategy) and leaves the timeline's prs_merged unknown; a missing open list omits every open-PR field; a missing list in any state omits Flow Load. Without the per-state map the union applies, and `fetch` counts as every list failed. Clean runs are unchanged. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The transient-error check matched anywhere in gh's stderr: "EOF" matched "thereof", and "timeout" matched any error about a repo named like timeout-svc, so a deterministic failure there was retried with the pause on every page. The repo's owner/name, name and owner are now blanked out of stderr first (only where they stand alone, so a repo named "out" leaves "timeout" whole), and the patterns match whole words: HTTP 5xx, timeout/timed out, deadline exceeded, connection reset/refused, EOF. GitHub's GraphQL "may be the result of a timeout" error is still retried. Also refresh the enrichment docstring, which still said a page failed "even after one retry". Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nullable-when cells said "`reviews` degraded" or "`enrichment` degraded" while the prose already gated each field on the PR state that feeds it; the cells now carry that state, and the ones a failed list omits say so (acceptance, funnel, timeline, PR lifecycle, Flow Load, Flow Efficiency, coverage, open PR aging, merge strategy). §15 documents the failed-list rule, defines a stopped enrichment pass as failed even when earlier pages came back, and says a 4xx or other GraphQL error is not retried (GitHub's "may be the result of a timeout" error is). Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The report caveat listed flow, review and in-PR metrics as what a
degraded read may leave missing, but a failed open list, or a failed
enrichment or reviews pass for open PRs, omits open-PR fields too. The
caveat now names them ("open-PR metrics" / "de PRs abertos"); its
opening sentence is unchanged.
Refs #236
Assisted-by: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
§24 listed a failed list only in the nullable-when cell. It now says the whole series goes when `gh pr list` failed for any state (or `fetch`), author_concurrency included, and why: the missing list would undercount the WIP in every bucket it overlaps, and keeping only the commit-based author count would emit buckets in a shape the platform does not read. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Several notes still described the older PR read. The aggregate() docstring called `prs` merged PRs, but they come in all three states. METRICS.md §10 and the funnel module said Iris only fetches merged PRs; it reads merged, closed and open PRs, and the limitation is that acceptance counts only commits in merged PRs. §10 now says a failure confined to closed or open PRs leaves acceptance and the funnel where a clean run would have them, without implying they are always present. §15 now says that with merged enrichment failed, Flow Efficiency is computed over the PRs whose commit refs came back, so it is skewed rather than missing. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed merged list omits the PR lifecycle fields (counts, time to merge, cycle time), and a partial read skews them, but the report caveat did not list them among what may be missing or skewed. It now does, in both languages; its opening sentence is unchanged. Refs #236 Assisted-by: Claude Code <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 tasks done
clickmatos
changed the base branch from
fix/pr-read-degradation-review-metrics
to
main
October 8, 2026 13:42
…-list fix(engine): omit what a failed PR list feeds, per state
fix(engine): gate degradation per PR state and retry transient errors
fix(engine): stop fabricating acceptance and funnel on degraded reads
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contexto e Motivação
Num run de organização, um repositório cuja leitura de PRs falhou aparece no
org-report.mde noorg-metrics.jsoncom 0 PRs, sem marca nenhuma — enquanto o relatório do próprio repositório avisa. Quem lê só o relatório da organização compara repositórios e soma PRs sem saber que um deles veio incompleto.Resolver agora porque é a última saída do engine que ainda apresenta a leitura degradada como completa; com ela, a #236 fica completa.
Refs #236
Mudanças
org-report.md— logo depois do aviso de repositórios não-código, uma linha nomeia os repositórios cuja leitura de PRs degradou nesta execução e diz que as contagens e métricas de PR deles, neste relatório, podem faltar ou sair distorcidas. EN e PT-BR.org-metrics.json— cada repositório trazpr_enrichment_degradedquando a leitura dele degradou; ausente caso contrário.Plano de Teste
pytest tests/test_org_pr_read_degraded.py tests/test_org_non_code_exclusion.py -v— o aviso nomeando só o repositório degradado; nenhum aviso num run limpo; o campo só no repositório degradado do JSON; o texto em PT-BR — rodado em3969f6e: 16 passedpytest tests/ -q— a suíte inteira, com o portão de determinismo da [FEAT] Comparador de paridade do metrics.json, campo a campo #214 — rodado em3969f6e: local: 380 passed; a única falha é atest_status_reflects_state([BUG] test_agent_settings_hook lê o spool real — falha em máquina com registros pendentes #146), que só ocorre fora do CI, e otest_hook_binders.pyficou de fora por depender de rede; no CI o comando literal passaA verificação roda fora da máquina de quem construiu: o job
clido CI (RFC 0029, PM-4).Impacto e Risco
iris/reports/org_writer.pye as strings EN/PT-BR do engine.org-report.mdeorg-metrics.jsonde um run sem degradação saem idênticos; a chave nova só aparece em repositório degradado.Tamanho (RFC 0028 §2.4): 128 linhas, faixa 51–250.
Plano de Rollback
git revertdo merge. O aviso e a chave somem; nada a migrar.Referências
Autoria Assistida
🤖 Generated with Claude Code