Ask about example data, offer to add more before building, and plan new apps in 4-8 revisions

[pstrzalk]

Three prose changes to the chat and planner prompts. No Ruby logic, no schema, no migration.

What changes

The chat asks about example data before a first build. Most generated apps ship with an empty database, so the preview lands on blank index pages. The planner already knows how to write db/seeds.rb — nothing ever asked the user whether they wanted it. Confirmed end to end: answering yes produced a plan containing "Seed database with example books and loans", with no planner-prompt change needed.

The readiness ask offers a way out. "Ready to start?" becomes "Ready to start building, or would you like to add anything else?" — the last cheap moment to catch a missing feature. Creation only; modifications keep "Should I apply this?".

New-app plans are 4-8 revisions, preferring the smallest that fits. Widened from 3-6, then steered, because the model treats the upper bound as a target rather than a limit.

Two things worth a reviewer's attention

Scoping to the creation path is prose, not plumbing. Sections A and B of instructions.txt.erb split on whether a build is running, not on whether an application exists — section A covers both tools. A flat bullet would therefore also fire on "make the banner green". Since A already establishes that exactly one of create_application / modify_application is bound, keying the new rules to the offered tool is enough; no ERB local or second template needed.

"at most 2 clarifying questions" became "approximately 2". Under the hard cap the model spent one of its two slots on the example-data question instead of adding a third — the opposite of the intent. The cap was an arbitrary number to begin with.

Verification

Every prompt change was probed against the live LLM, not just asserted on.

Chat (bin/generate respond on scratch projects, since prompt wording can't be unit-tested for behaviour):

• Creation → two clarifying questions plus "Also, would you like the app to come pre-filled with some example books and lending records…", then "Ready to start building, or would you like to add anything else?"
• Modification (Gemfile in workspace, so ModifyApplication binds) → neither the example-data question nor the "start building" wording; "Should I apply this?" intact.
• Confirmation gate intact — zero tool-call rows at the readiness turn.

Planner (bin/inspect-plan-application-creation), before and after the low-end directive:

Intent	4-8 alone	+ directive
a todo list app	8	5
a recipe box, tagged by cuisine	—	6
personal finance tracker with expenses, budgets, dashboard	8	8

Both padding cases disappeared: the todo plan now folds the migration into the model revision and CRUD into the scaffold revision, where before they were separate. The large intent still reaches 8, so the directive steers rather than caps.

bin/rails test 742 runs / 3435 assertions / 0 failures · bin/rubocop clean.

Deliberately not done

• No seeds rule in the planner prompts. The clarification answer is the whole mechanism.
• No min_items/max_items on PlanSchema. They would emit real minItems/maxItems, turning an off-range plan into "Could not generate a plan. Ask the user to rephrase." — worse than a plan of three. The range stays a hint, as it always was; the new plan_schema_test assertion only keeps the two copies of it from drifting into a contradictory instruction.
• Modification planner's 1-6 untouched — deliberately divergent since the Bug A fix.
• Feature invention left open. The 8-revision finance plan adds scope the intent didn't ask for (category management, income tracking, filtering, export). That is a different problem from revision count.

Notes

Wall time per generation rises with revision count — roughly 20-27 min at 8 revisions against the ~9 min three-revision runs on record. Nothing gates this at runtime; WALL_TIME_BUDGET lives inside the E2E_GENERATE test, which uses a pinned 3-revision fixture and does not move with this change.

Unrelated, found while running the suite: the local test database held a stale ruby_llm_models row (anthropic/claude-sonnet-5). A non-empty table suppresses RubyLLM's fallback to the gem's bundled models.json, so every chat-touching test errored with ModelNotFoundError on anthropic/claude-haiku-4.5 — including files this branch never touches. bin/rails db:test:prepare clears it, but no fixture keeps that table empty, so it can recur.

Plan and research: thoughts/shared/plans/2026-09-12/chat-example-data-question-and-plan-size.md, thoughts/shared/research/2026-09-12/seed-data-question-and-plan-size.md.

🤖 Generated with Claude Code

https://claude.ai/code/session_01SW7desfNVvjcghfMuPHbT3