Repository navigation
Description
Problem
Intake has no counter of placed, skipped, failed or duplicate-flagged candidates. The implement job has no timer for the scoped checks, the publish step and the watch of the PR checks. The scoped checks take about 3.5 minutes. The watch takes up to 3600 s. Nobody can see which of them is slow.
Expected
Intake. Add the counter devops_cli_roadmap_intake_candidates_total with the labels repo and result. The values of result are placed, skipped, failed and duplicate. apply_intake adds 1 for each candidate. Series count: 4 per repo.
Implement. Use the histogram devops_cli_roadmap_step_seconds of the issue #4066, with row set to implement. Add three StepName members: checks, publish and watch. The implement job times:
checks: the scoped checks (_scoped_checks);publish: the push and the draft PR;watch: the watch of the PR checks (watch_pull).
result is ok or error. Series count: 3 steps x 2 results = 6 per repo, times the histogram buckets.
Document both series in docs/TELEMETRY.md. State the series counts there.
The implement Job runs in a runner pod. Check that the Job sends its samples to the collector before it ends. If it does not, the pull request says so and times the steps in the Service from the log of the Job instead.
No live model run (AGENTS.md rule 13).
Acceptance criteria
-
uv run pytest tests/telemetry/test_roadmap_intake_implement_metrics.py -k intake_outcome- intake emits one outcome counter for each candidate, with itsresult. -
uv run pytest tests/telemetry/test_roadmap_intake_implement_metrics.py -k implement_step- the scoped checks, the publish step and the watch each emit one histogram sample withrow=implement. -
uv run pytest tests/telemetry/test_roadmap_intake_implement_metrics.py -k step_names- the step values are members ofStepName. -
grep -q 'devops_cli_roadmap_intake_candidates_total' docs/TELEMETRY.md- the documentation lists the new counter (it prints nothing today).
Every command above fails on the unchanged code. The pull request cites the fail-to-pass run (devops ci fail-to-pass --base <ref>).
Feasibility
I read these files at origin/main: intake.py (apply_intake), implement_run.py (_scoped_checks, _push, _checks), implement_pull.py (watch_pull), telemetry/instruments.py and docs/TELEMETRY.md. I did not run the code. I did not check how a runner pod sends telemetry. That is why the pull request must check it.
The change makes no GitHub request.
Existing tool considered
telemetry/instruments.py (emit) and tool_metrics.py as the pattern. No new library.
Cost
Build: about 45k tokens and 2.5 h. Keep: about 3k tokens per quarter. Fix: about 10k tokens and 0.5 h.
Related
Part of #3027. #3069, #3650, #3655, #3076. Order: after the issues #4061 and #4066.
Activity
dan-petty commentedon Nov 7, 2026
Intake placed this item to the backlog: intake puts every new item in the backlog. A person plans the milestones.
featprefix of the title gives the type.(telemetry)scope of the title gives the scope.It improves observability of the implement job but does not block a release or fix a defect.Operators gain per-step timing and an intake outcome count, which aids debugging slow runs.The body estimates about 2.5 hours of build work across intake, implement, and docs files.Both use the histogram devops_cli_roadmap_step_seconds and the StepName enum; #4066 creates the instrument for stage, while this candidate adds checks, publish, and watch members and wires the implement job to emit samples.