Skip to content

Image-aware generation: a photo set the planner and the code agent can use - #49

Merged
pstrzalk merged 8 commits into
mainfrom
image-aware-generation
Sep 11, 2026
Merged

pstrzalk merged 8 commits into
mainfrom
image-aware-generation

Conversation

@pstrzalk

Copy link
Copy Markdown
Owner

Generated apps rendered broken images. Production project 46 wrote image_tag speaker.photo_url in four views against a column whose only recorded value was https://example.com/photos/ada.jpg, and no workspace contained an image file at all. Nothing in any prompt — the two planner prompts, RevisionPrompt, the five frontend.md templates — had ever mentioned images.

This serves a curated photo set from hifumi.dev and tells the planners and the code agent it exists, as text. No image ever enters an LLM call.

What's here

A photo set — 43 images in public/photos/, 1600px long edge at q80, metadata stripped, slug-named. public/ is outside the Propshaft load path, so they're served as-is rather than digested and copied into public/assets by assets:precompile.

The set was rebuilt rather than re-encoded. Every photograph of an identifiable real person was dropped and replaced with generated portraits, so there is no model release, publicity right or privacy right in play anywhere in the set. Nine public-domain Pixnio stock photos remain, none showing a person. public/photos/CREDITS.md records origin and licence per file — it is the only provenance these have, since re-encoding strips EXIF, and it was captured before the strip.

lib/photos.rb — globs the directory, derives geometry per file, renders one text block. Adding a photo is copy-and-deploy: no manifest, no code change.

It is deliberately Rails-free. revision_workflow.rb requires it by path under bundle exec roast, where environment.rb never runs, so root comes from __dir__ and the base URL from ENV rather than an initializer. A Rails.root here would raise on every production revision while dev stayed green — so a test loads it in a bare Ruby process with Rails undefined.

Prompt wiring — the inventory is appended to both planner system prompts and to RevisionPrompt after the stack inventory, and PHOTOS_BASE_URL is forwarded into the agent sandbox.

The append sits in prompt assembly, not transport: both planner suites stub invoke_llm wholesale, so appending at with_instructions would have been invisible to every test. SYSTEM_PROMPT stays the frozen .md the refute_match guards assert on.

Per-template image treatment — the photo set is warm-neutral so the images cohere with each other; the templates are not. Each frontend.md gained an ## Images section in its own visual language: grayscale for cyber, sepia for earth, thick borders for kids, hairline for office.

Three new templates — launch (marketing pages; office is the inside of a tool, nothing covered the page in front of one), luxe (hotels, restaurants, property — flower is soft and sweet, not expensive and restrained), editorial (magazines and news — earth is deliberately quiet and low-contrast).

A font bug fixed along the way — every template's snippets used font-display and every fonts.html downloaded a body face, but nothing defined either token. Tailwind 4 resolves font-display from --font-display, and the skeleton ships a bare @import "tailwindcss". All eight templates rendered in system sans with their webfonts fetched and unused. Project 51 used font-display 18 times and its compiled CSS never mentioned Space Grotesk. Each template now carries a theme.css that Picker#apply writes into the workspace — deterministic rather than an instruction to the agent, because an instruction can be forgotten and this one demonstrably was.

What the measurements showed

The A/B changed the rationale. Blind, the planner doesn't write example.com/photos/ada.jpg — it reaches for Active Storage: has_one_attached :photo, a file_field upload form, image_tag with a placeholder. That is worse than a bad URL. libvips is absent from the preview base image and the skeleton gitignores storage/, so it passes W2.4 in the sandbox, shows empty pages in preview because nobody uploads anything, and drops the blobs on export. The photo block's real job is diverting the planner off that dead end.

The template snippets were A/B'd too, and the first draft made things worse. The hero snippet passed the URL as a bare photo_url variable and the model imitated the shape, emitting image_tag "photos/x.jpg" — a bare path that resolves through Propshaft and breaks. Fixed to an explicit absolute-URL placeholder and re-verified. The measured win is narrower than expected: unaided the agent got radius, aspect ratio and alt text mostly right, but never desaturated a warm photo for a dark template.

The font fix is verified end to end by compiling the real Tailwind binary against a workspace with the cyber template applied: .font-display rules 0 → 1, "Space Grotesk" occurrences 0 → 1.

Picker probe: 8/8 across all templates, including both confusable pairs — "a landing page to promote our new API product with a waitlist" → launch, "a boutique hotel site with rooms, rates and a booking enquiry form" → luxe.

Scope

The set covers page-level chrome — a hero, a section band, an about or team page — and will never cover per-record artwork. The prompt says so and points at CSS for anything that repeats per row. A gym-app probe confirmed no invented per-exercise URLs.

Phase 5 of the plan (seeding photos) was written as conditional and is not built: the probes showed the planner writes photos into seeds unprompted, so the gap never appeared.

Verification

  • Full suite green: 732 runs, 0 failures. rubocop clean. zeitwerk:check passes.
  • Planner probes run against both planners and a real project (46).
  • Photo serving verified on the dev server; the production equivalent is curl -sI https://hifumi.dev/photos/<slug>.jpg after deploy.

Follow-ups recorded

In docs/09-ideas/05-followups.md: exported apps hotlink hifumi.dev permanently (the zero-lock-in tension, deferred not solved); libvips still absent from the preview base image; app-shaped intents plan no homepage so they use no photos at all.

Two things worth knowing before merge: PHOTOS_BASE_URL only takes effect on deploy (dev falls back to http://localhost:3000), and existing projects keep their bare application.css because Picker#apply runs once at creation — the font fix reaches new projects only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GZD2tyMBmh9k3j1FsjbYoj

31 images at 1600px/q80, metadata stripped, slug-named. public/ is
outside the Propshaft load path, so these are served as-is rather than
digested and copied into public/assets by assets:precompile.

Rebuilt rather than merely re-encoded: every photograph of an
identifiable real person was dropped and replaced with generated
portraits, which also closed the product/food/interior/desk/texture
gaps. CREDITS.md records origin and licence per file — it is the only
provenance these have, since re-encoding strips EXIF.
Globs the directory, derives geometry per file, renders one text block.
Adding a photo is copy-and-deploy: no manifest, no code change.

Deliberately Rails-free. revision_workflow.rb requires this by path
under `bundle exec roast`, where environment.rb never runs, so root
comes from __dir__ and the base URL from ENV rather than an
initializer. A Rails.root here would raise on every production revision
while dev stayed green, so a test loads it in a bare Ruby process.

Tests run against fixtures, not the shipped set — pinning them to a
count would make adding a photo mean editing a test.
Appends the inventory to both planner system prompts and to
RevisionPrompt after the stack inventory, and forwards PHOTOS_BASE_URL
into the agent sandbox.

The append lives in prompt assembly, not transport: both planner suites
stub invoke_llm wholesale, so appending at with_instructions would have
been invisible to every test. SYSTEM_PROMPT stays the frozen .md the
refute_match guards assert on.

Probed against both planners. Blind, the planner reaches for Active
Storage — has_one_attached, an upload form — which passes verify in the
sandbox and then fails in preview and on export. Naming the available
photos diverts it off that path.
Conventions bullet, the seam written up in layer-integration, and four
follow-ups. Also drops the stale claim that CleanupIdlePreviewsJob
reaps idle previews — removed in Phase 4 per config/recurring.yml.
The photo set is deliberately warm-neutral so the images cohere with
each other; the templates are not. Dropping a cream-and-oak photo into
cyber's near-black terminal palette reads as stock imagery unless it is
desaturated first, and no palette description tells the agent that.
Each template now carries a hero band, portrait and image-card snippet
in its own visual language — grayscale for cyber, sepia for earth,
thick borders for kids, plain hairline for office.

A/B'd against the code agent's model, before and after, on office and
cyber. The measured win is the treatment: unaided the agent got radius,
aspect ratio and alt text mostly right, but never desaturated a warm
photo for a dark template.

The first draft of the hero snippet made things worse — it passed the
URL as a bare `photo_url` variable and the model imitated the shape,
emitting `image_tag "photos/x.jpg"`, which resolves through Propshaft
and breaks. The snippet now shows an explicit absolute-URL placeholder
and the section says why.

Line cap 100 -> 115: sized for five component snippets, images are a
sixth category. ~1000 chars, ~260 tokens per revision.
Three gaps the original five left open:

- launch    marketing pages. office is the inside of a tool; nothing
            covered the page in front of one, which is what most
            "promote X" intents actually ask for.
- luxe      anything premium. flower is soft and sweet, which is not
            the same as expensive and restrained. Hotels, restaurants,
            jewellery, property.
- editorial high-contrast publications. earth is deliberately quiet and
            low-contrast, so it only serves the personal end of writing.

Each ships the full definition — vibe, fonts, five component snippets,
its own Images section, density and voice — plus a Google Fonts link.

Picker descriptions are anchored on the kind of site rather than the
mood, because launch/office and luxe/flower are confusable on mood
alone; each of those lines names its pair explicitly. Probed across all
eight templates with one description each: 8/8, both pairs included.

Templates live outside the Zeitwerk load path, so the new directories
join the autoload_lib ignore list alongside the other five.
Every template's snippets use `font-display` and every fonts.html
downloads a body face, but nothing ever defined either token: Tailwind 4
resolves `font-display` from `--font-display`, and the skeleton ships a
bare `@import "tailwindcss"`. So all eight templates rendered in system
sans with their webfonts fetched and unused. Project 51 used
`font-display` 18 times and its compiled CSS never mentioned Space
Grotesk.

Each template now carries a theme.css, and Picker#apply writes it into
the workspace's tailwind entrypoint alongside the frontend.md and
fonts.html it already wrote. Deterministic rather than an instruction to
the agent, because an instruction can be forgotten and this one was.
lib/preview/Dockerfile runs tailwindcss:build, so it reaches preview.

Verified by compiling the real tailwind binary against a workspace with
the cyber template applied: `.font-display` rules 0 -> 1, "Space
Grotesk" occurrences 0 -> 1.

The write is idempotent via a marker comment, so re-picking a template
replaces the block instead of stacking a second @theme.
Chosen against real projects rather than guessed. Production project 48
is a broker's desk and the set held exactly one interior; project 46 is
a conference builder with one venue shot. Twelve additions:

- property    house exterior, apartment facade, kitchen, bedroom
- florist     shop interior, wrapped bouquet, single bloom
- children's  reading nook, blank-covered book stack, storybook scene
- conference  workshop room, event foyer

All people-free: the eight headshots cover faces, and more would only
add dedup work. Same pipeline as before — 1600px long edge, q80,
metadata stripped.

storybook-scene-illustration is a painted illustration, the only
non-photographic file in the set. A children's book app with a
stock-photo hero reads wrong. Its slug is the only thing disclosing
that; the rendered inventory reports geometry, not medium, and
CREDITS.md now says so.

Set: 43 files, 10.47 MB. Inventory 2608 chars, ~652 tokens per call.
@pstrzalk
pstrzalk merged commit 14c54fb into main Sep 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants