Skip to content

feat(backport): run backport-fix-imports with an ai pass gated on go vet - #330

Open
vcauesantos wants to merge 3 commits into
devops-1648/backport-fix-importsfrom
devops-1648/backport-ai-resolve
Open

vcauesantos wants to merge 3 commits into
devops-1648/backport-fix-importsfrom
devops-1648/backport-ai-resolve

Conversation

@vcauesantos

@vcauesantos vcauesantos commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Parked until the script-only rollout (#329, #332) has run on real backports. Then this goes ahead with the AI pass.

Summary

Stacked on #329. Together they make backports to older release lines fix their own imports.

  • backport.yaml gets a fix-imports flag, off by default. When it's on, a job runs backport-fix-imports on every backport branch after the cherry-picks.
  • New AI pass in the action. It runs only when an anthropic-api-key is passed and Go conflicts are still left after the script. Its answers are committed only for packages that pass gofmt and go vet. Anything else is reverted and left for a person.
  • fix.py now skips quietly when a label has no backport branch, for example a legacy line.
  • Before an AI commit is pushed, the backport PR gets the backport-ai-resolved label, so auto-approve can keep it for a person. If the label can't be added, the AI commit isn't pushed. Skipping the label in auto-approve comes in a separate PR.
  • One sticky comment on the backport PR tells the assignee what the script fixed, what the AI wrote, and what's left. There's no comment when there's nothing to say.
  • go vet and gofmt run without the push token in their environment.

Flow

flowchart TD
    A[PR merged with backport-to labels] --> B[Bot cherry-picks and opens backport PRs]
    B --> C[One job per backport branch]
    C --> D[Fix imports with the script and push]
    D --> E{Go conflicts left and an API key set?}
    E -->|No| Done[Done: anything left is for a person]
    E -->|Yes| F[Send the conflicted Go files to the AI in one call]
    F --> G{Every conflict in the file answered, and the package passes go vet?}
    G -->|Yes| H[Commit and push the AI fix as its own commit]
    G -->|No| I[Revert: leave the conflict for a person]
Loading

If any step fails, nothing more is pushed, and the backport PR stays as it was.

How much AI this means

Replay of all 54 bot backports to 4.x since the v5 module change: the script alone leaves 43 with nothing to do. Of the 11 with conflicts left, about 4 or 5 have Go conflicts, so only those would call the AI. The rest are YAML conflicts, which are never sent.

Real run on loft-enterprise

The action ran from this branch on five temporary branches in loft-enterprise. Each holds the bot's original commit from a real backport. It used the repo's own token and Anthropic key. Nothing was merged, and the branches will be deleted after review.

Runs: first four, #8571

Backport What the action did Result
#8643: v5 imports only Fix commit b6f3a9f. No AI. Same as the human fix
#8719: import conflicts Fix commit 22cb3ca, 2 conflicts resolved. No AI. Same as the human fix
#8808: imports and a Go code conflict Fix commit b2c250f, 18 import conflicts resolved. The AI declined the code conflict: it needs pkg/tenancy, which 4.12 does not have. Correct: a person had to rewrite that code
#8654: Go code conflict on 4.9 Fix commit 63979b8. The AI declined: the change calls a function 4.9 does not have. Correct: a person had to rewrite that code
#8571: a Go test conflict, not import related Nothing to rewrite. The AI resolved the conflict, go vet passed, and it was committed: bee5777 Identical to the human fix

The script and AI call take 4 to 20 seconds per branch. When the AI answers, the go vet check adds about 4 minutes, mostly a cold Go build on the runner.

Test plan

  • make test-backport-fix-imports: 91 pytest tests. The AI-pass tests run real gofmt and go vet on a small made-up module and fake the model's answer:
    • a good answer is committed and pushed;
    • an answer that does not compile is reverted, and nothing is pushed;
    • a package with another conflicted file is not committed;
    • partial answers, answers containing conflict markers, and garbage results change nothing.
  • make lint (actionlint, zizmor) and make check-docs are clean.
  • Real model answers: see Real run on loft-enterprise above.

Merge #329 first, so this PR targets main when it merges.

Closes DEVOPS-1648
Closes DEVOPS-1671

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant