Derive a coverage grid before writing scenarios, and sample it deliberately - #73
Draft
KarthikAvinashFI wants to merge 1 commit into
Draft
Derive a coverage grid before writing scenarios, and sample it deliberately#73KarthikAvinashFI wants to merge 1 commit into
KarthikAvinashFI wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Rewrites the write-scenarios skill so the scenarios stage derives a coverage grid from the agent's contract before writing anything, samples that grid deliberately, fans the writing out across the existing parallel writer sessions, and closes with a coverage report. Enables the parallel path in the hosted image and sizes its batches.
No code changes. The diff is one skill file and three ENV lines plus a revision bump in Dockerfile.hosted.
Why
Measured across 718 previously generated scenarios: 5 of the 12 canonical task operations were effectively untested (diagnose and navigate at 0%, compare at 1%), and only 5% carried any adversarial or safety overlay, with impersonation at zero. The suites collapse into create/execute because nothing tells the writing model what the space of situations is or which cells it must not skip.
The skill now instructs the model to:
Very large asks are handled the way the grid defines: the number is the space to cover, the written suite is the sample over it, and the report states what the sample covers.
Not verified yet
This is instruction-only and the behaviour is model-driven, so it needs the verification runs before leaving draft:
Steps to test