Skip to content

test: a rule learned on PRD-001 shapes the next drafts where it applies - #108

Merged
sloweyyy merged 1 commit into
mainfrom
test/lesson-transfer-across-sample-prds
Oct 8, 2026
Merged

sloweyyy merged 1 commit into
mainfrom
test/lesson-transfer-across-sample-prds

Conversation

@sloweyyy

@sloweyyy sloweyyy commented Oct 8, 2026

Copy link
Copy Markdown
Owner

Closes the offline half of TODO #6, lesson transfer across PRDs, using the real sample PRDs. There's no Jira and no model; generateText is stubbed.

evals/lesson-transfer.test.ts:

  • Global rule: feedback on PRD-001's draft is distilled into a rule for every audience and approved by a human. It is in the draft prompt for PRD-002 (workspace admins) and PRD-005 (integrators), and reported as applied.
  • Audience rule: a rule distilled for PRD-001's readers (AUDIENCE_LESSON, from feat: Scribe proposes a house rule for the doc's readers when the feedback holds only for them #106) reaches PRD-002, which has the same admins. It doesn't reach PRD-005.
  • Unapproved rule: a proposal that was never approved reaches no draft.

Gate: typecheck clean, 772 evals passed.

Co-Authored-By: Truong Le Vinh Phuc truonglevinhphuc2006@gmail.com

Offline and end to end on the sample PRDs:
- Feedback on PRD-001 is distilled, approved, and then drafted against.
- A rule for every audience reaches PRD-002 and PRD-005.
- A rule for PRD-001's readers (workspace admins) reaches PRD-002 but not PRD-005 (integrators).
- An unapproved rule reaches nothing.

Co-Authored-By: Truong Le Vinh Phuc <truonglevinhphuc2006@gmail.com>
@sloweyyy
sloweyyy merged commit 5c8ab46 into main Oct 8, 2026
1 check failed
@vercel

vercel Bot commented Oct 8, 2026

Copy link
Copy Markdown

Deployment failed for project scriptorium with the following error:

Resource is limited - try again in 24 hours (more than 100, code: "api-deployments-free-per-day").

Learn More: https://vercel.com/sloweyyy?upgradeToPro=build-rate-limit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant