Skip to content

Update instructions for .expected files - #22856

Merged
geoffw0 merged 2 commits into
mainfrom
geoffw0-patch-2
Oct 21, 2026
Merged

geoffw0 merged 2 commits into
mainfrom
geoffw0-patch-2

Conversation

@geoffw0

@geoffw0 geoffw0 commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

This phrasing bothered me when I saw it. "ignore these files" suggests Copilot should neither write nor read .expected files, whereas I think we only mean to ban writing them ("manually"). Reading them should be a perfectly normal thing to do.

I haven't witnessed the existing wording drive Copilot astray, and I don't know of a reasonable way to test the change I propose.

This phrasing bothered me when I saw it.  "ignore these files" suggests Copilot should neither write nor *read* the files, but reading them is a perfectly normal, useful thing to do.

I don't know a reasonable way to test this proposed change.
Copilot AI max review requested due to automatic review settings October 20, 2026 15:07
@geoffw0 geoffw0 added the no-change-note-required This PR does not need a change note label Oct 20, 2026
@geoffw0
geoffw0 requested a review from a team as a code owner October 20, 2026 15:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The wording resolves the ambiguity without weakening the restriction on direct edits.

No open issues 🎉

What changed in this PR

Clarifies Copilot guidance for generated .expected files, allowing reading while preserving the restriction on direct edits.

Changes:

  • Replaces “Ignore these files” with an explicit instruction not to edit them directly.
File Description
.github/​instructions/​expected-files.instructions.md Clarifies guidance for generated test expectations.

Review effort: Max

owen-mc
owen-mc previously approved these changes Oct 20, 2026

@owen-mc owen-mc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another idea: tell it that for inline expectations tests, .expected files shouldn't be commited with a non-empty testFailures section.

@geoffw0

geoffw0 commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Another idea: tell it that for inline expectations tests, .expected files shouldn't be commited with a non-empty testFailures section.

Good idea. How about:

Do not edit these files directly. They are generated by running the tests. If a .expected file has a non-empty testFailures section it should not be committed - you may want to update the inline expectation comments in the corresponding test source file (e.g. marking expected results as MISSING / SPURIOUS) then re-run the test.

@owen-mc

owen-mc commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

That sounds good. I think there are some cases where it's very difficult to avoid a non-empty testFailures section, e.g. in files where we can't easily put a comment (yml? json?), or when the alerts change when you add a comment, so it's good that you didn't make it a hard requirement.

Comment thread .github/instructions/expected-files.instructions.md Outdated
@geoffw0
geoffw0 merged commit 6dc5ec9 into main Oct 21, 2026
7 checks passed
@geoffw0
geoffw0 deleted the geoffw0-patch-2 branch October 21, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants