Skip to content

feat(telemetry): step seconds and a gate counter for the stage row #4066

Open

Description

@dan-petty

Problem

Stage has one metric, devops_cli_roadmap_stage_total, with the label result only (stage.py, emit_stage_results). No metric shows how long the steps of a stage round take. No counter shows which gate refused a plan. check_proposal joins the problems of all gates, and the gate name is lost.

Expected

Add one histogram devops_cli_roadmap_step_seconds with the labels repo, row, step and result. The step values come from StepName (roadmap/failure.py). The row name is passed as an argument, as tool_metrics.start_run(row=...) takes it. So no shared scope is needed.

In this issue row is stage. The issue #4067 reuses the histogram for the implement row.

StepName members for the stage:

What each step times:

  • select: read_stage_selection.
  • plan: the model calls of the planner for one item.
  • gate: the calls of the gates for one item.
  • review: the model review of the verifier.
  • apply: the write of one item.

result has two values: ok and error.

Series count: 1 repo x 1 row x 5 steps x 2 results = 10 series, times the histogram buckets.

Add the counter devops_cli_roadmap_stage_gate_refusals_total with the labels repo and gate. The planner calls check_gates where it calls check_proposal, and adds 1 for each gate that found a problem. The value of gate is a ProposalGate value. Series count: 14 gates x 1 repo = 14.

Document both series in docs/TELEMETRY.md. State the series counts there.

No live model run (AGENTS.md rule 13).

Acceptance criteria

  • uv run pytest tests/telemetry/test_roadmap_stage_steps.py -k step_label - a stage round emits a step sample for select, plan, gate, review and apply.
  • uv run pytest tests/telemetry/test_roadmap_stage_steps.py -k gate_refusal - a refused plan increments the counter of its gate name, and only that gate.
  • uv run pytest tests/telemetry/test_roadmap_stage_steps.py -k step_names - the step values are members of StepName. No other value appears.
  • grep -q 'devops_cli_roadmap_step_seconds' docs/TELEMETRY.md && grep -q 'devops_cli_roadmap_stage_gate_refusals_total' docs/TELEMETRY.md - the documentation lists both series (it prints nothing today).

Every command above fails on the unchanged code. The pull request cites the fail-to-pass run (devops ci fail-to-pass --base <ref>).

Feasibility

I read these files at origin/main: stage.py (emit_stage_results, _settle, read_stage_selection), stage_change.py (check_gates, check_proposal, ProposalGate), roadmap/tool_metrics.py, telemetry/instruments.py and docs/TELEMETRY.md. I checked that check_proposal calls check_gates and joins its lists. I did not run the code. The change makes no GitHub request.

Existing tool considered

telemetry/instruments.py (emit), tool_metrics.py as the pattern, and the OpenTelemetry instruments that exist. No new library.

Cost

Build: about 50k tokens and 2.5 h. Keep: about 3k tokens per quarter. Fix: about 10k tokens and 0.5 h.

Related

Part of #3027. #3069, #3650, #3655, #3162, #3566. Order: after the issues #4056 and #4064. All of them change stage.py or the telemetry files.

Activity

  1. added
    type/featureNew user-facing functionality or capability addition
    scope/telemetryOpenTelemetry tracing spans, Prometheus metrics, and Jaeger
    status/newStatus New: the item is not ready yet
    on Nov 7, 2026
  2. dan-petty commented on Nov 7, 2026

    @dan-petty
    OwnerAuthor

    Intake placed this item to the backlog: intake puts every new item in the backlog. A person plans the milestones.

    • Type: type/feature. The feat prefix of the title gives the type.
    • Scope: scope/telemetry. The (telemetry) scope of the title gives the scope.
    • Priority: P2-Medium. Observability improvement for operators; no user-facing defect or security risk.
    • Value: Medium. Gives operators step-level timing and gate refusal visibility, aiding tuning and debugging.
    • Effort: Medium. Adds one histogram, one counter, three StepName values, stage.py changes, and a docs update; about 2.5 h per the body.
    • Related: feat(telemetry): measure the seconds of each row #4058, which asks for a different change. Both add timing histograms to the roadmap telemetry, but #4058 measures the full row duration while this candidate measures individual steps within a stage row.
  3. added
    priority/p2-mediumStandard priority feature work and architectural improvements
    on Nov 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    effort/mediumEffort Mediumpriority/p2-mediumStandard priority feature work and architectural improvementsscope/telemetryOpenTelemetry tracing spans, Prometheus metrics, and Jaegersource/agentFiled by an agentstatus/newStatus New: the item is not ready yettype/featureNew user-facing functionality or capability additionvalue/mediumValue Medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions