fix: send the fidelity report when it disagrees with the grade above it

.changepacks/changepack_log_fidelity_visibility.json

@@ -0,0 +1,8 @@
+{
+  "changes": {
+    "crates/devup-mcp/Cargo.toml": "Patch",
+    "crates/devup-mcp-devup-ui/Cargo.toml": "Patch"
+  },
+  "note": "Send the fidelity report whenever it disagrees with the grade above it, rather than whenever the grade is not exact. Keying it on quality.projection was wrong in a way that hid a real defect: projection_quality is computed from diagnostics alone, so a coverage shortfall that raises no diagnostic leaves the grade reading exact and took the report away with it. Measured on a real 50-node screen, fidelity.variables was 19 of 20 - a $primary binding on an ellipse whose parent was exported as an SVG, so the theme colour is frozen into the file and cannot follow the theme - while quality.projection read exact. Before this the response showed that shortfall; after the first attempt it hid it. The condition is now strict_compatible, which is the same predicate strict: true refuses on, so the two cannot disagree about whether anything was lost. On the ten real captured screens it stays silent for every screen that has neither a shortfall nor an approximation, which is what the change was for. And stop expecting a binding the platform cannot carry. An asset drawn in one colour becomes a Box masked to its shape with bg set, so a token inside it survives as a token; an asset drawn in more than one has no such form, because a CSS mask carries alpha only and an SVG loaded through img src renders in its own document where neither currentColor nor a CSS variable reaches it. It becomes an Image and every colour inside is baked into the file. Counting those bindings as expected-but-missing asked the generator for something no correct implementation could produce: on the measured screen two play buttons each hide a bound circle behind a white glyph, which alone put the whole report on the response with nothing in it anyone could act on. Bindings inside a non-mask asset are excluded from the variables expectation, which on that screen moves it from 19 of 20 to 17 of 17. Across the ten captured screens the expectation drops from 658 to 528 and is fully covered, while six of the ten still send the report for their absolute-layout approximations - so the exclusion narrows what is expected rather than silencing what was found. Also names the commit outright when CI builds a release, instead of letting build.rs infer it. It infers by asking git for HEAD and whether the tree is dirty, and on a release build the tree always is: changepacks update moves the version in Cargo.toml and leaves Cargo.lock behind, so the first cargo invocation rewrites the lock and dirties the checkout. Every published binary reported its build id as <sha>-dirty, which is exactly the state --version and --self-check exist to tell apart from a developer working tree. Committing the lock fixed that instance and the next Update Versions pull request brings it straight back, so the build job now passes DEVUP_MCP_BUILD_ID, which build.rs already prefers. A local build still infers, so a dirty working tree still says so.",
+  "date": "2026-09-08T17:20:00+09:00"
+}

.github/workflows/ci.yml

@@ -190,6 +190,20 @@ jobs:
           SUFFIX: ${{ matrix.suffix }}
           EXT: ${{ matrix.ext }}
           OS: ${{ matrix.os }}
+          # Name the commit outright instead of letting build.rs infer it.
+          #
+          # It infers by asking git for HEAD and whether the tree is dirty,
+          # and on a release build the tree always is: `changepacks update`
+          # moves the version in Cargo.toml and leaves Cargo.lock behind, so
+          # the first cargo invocation rewrites the lock and dirties the
+          # checkout. Every published binary then reported its build id as
+          # `<sha>-dirty`, which is precisely the state `--version` and
+          # `--self-check` exist to tell apart from a developer's working
+          # tree. Committing the lock fixed the instance; the next Update
+          # Versions pull request brings it straight back.
+          #
+          # CI knows exactly which commit it is building, so it says so.
+          DEVUP_MCP_BUILD_ID: ${{ github.sha }}
         run: |
           set -euo pipefail
           for target in $TARGETS; do

README.md

@@ -30,10 +30,13 @@ devup-mcp는 Figma Plugin API의 readable data property를 raw JSON으로 보존
 | `componentTsx` | `componentTsx` |
 | `responsiveTsx` | `responsiveTsx`, `responsiveSlots`, (표현 불가한 값이 있으면) `responsiveUnrepresented` |
 | `devupJson` | `devupJson`, `themeCounts`, `themeCompleteness`, `conflicts`, `unresolvedVariables` |
-| `sourceMap` / `rawSnapshot` / `rawPayload` / `assetManifest` / `referencePng` | 같은 이름의 키 |
+| `sourceMap` / `assetManifest` / `referencePng` | 같은 이름의 키 |
+| `rawSnapshot` / `rawPayload` | 같은 이름의 키 — **`debug: true` 필요** |
 
 그 밖에 항상 붙는 것은 `status`, `quality`, `completeness`, `cache`, `collection`, `source`, `targetKind`, `failures`, `outputPaths`뿐입니다. `fidelity`와 `completenessReport`는 결과가 exact/complete가 **아닐 때**, 또는 `includeDiagnostics: true`일 때만 나옵니다 — 깨끗한 결과에서는 `quality`가 이미 한 말을 되풀이할 뿐이라 빼두었고, 그만큼(측정값 797 B) 매 응답이 가벼워집니다.
 
+`rawSnapshot`과 `rawPayload`는 수집한 디자인을 raw로 담은 것이라 `debug: true` 없이는 거절됩니다. 화면을 구현하는 데는 필요 없습니다 — 실제 캡처 10개 화면에서 tsx가 node·text·typography·asset·layout 기대치를 100% 담고 있습니다. 쓰는 자리는 하나입니다: **화면이 이상해 보일 때 생성기 탓인지 디자인이 원래 그런지 판정하는 것.** 그때는 디자인을 코드 옆에 놓고 읽어야 하고, 그게 이 플래그입니다.
+
 에러는 호출 자체가 잘못된 경우(`DEVUP_INVALID_INPUT`, 없는 node/파일, 만료·부적합한 `artifactId` 등) JSON-RPC `-32602 INVALID_PARAMS`로, 그 밖의 실패는 `-32603 INTERNAL_ERROR`로 옵니다. 인자를 고쳐 다시 부를 일인지 멈추고 보고할 일인지를 메시지를 파싱하지 않고 구분할 수 있습니다. 정확한 `code`와 `retryable`은 예전처럼 `data`에 그대로 실립니다.
 
 devup-mcp는 Figma Remote MCP에 직접 붙습니다 — OAuth discovery, Dynamic Client Registration, PKCE S256, 일시적인 `127.0.0.1` callback을 구현합니다. Figma는 MCP Catalog에 승인된 client의 registration만 허용하므로 등록은 allowlist에 있는 `client_name`으로 이루어집니다(기본값 `Codex`). Figma PAT나 사용자가 만든 OAuth app은 필요하지 않습니다.

crates/devup-mcp-devup-ui/src/codegen/mod.rs

@@ -18,6 +18,6 @@ pub use component::{
     normalize_component_name, render_component_registration_snapshot, render_component_source,
 };
 pub(crate) use layout::{derived_padding, placed_by_a_free_layout};
-pub(crate) use style::asset_kind;
+pub(crate) use style::{AssetKind, asset_kind};
 pub use style::{asset_path, image_fill_path};
 pub(crate) use text::escape_jsx_text;

crates/devup-mcp-devup-ui/src/provenance.rs

@@ -4,7 +4,9 @@ use devup_mcp_figma::{DevupError, ErrorCode, FidelityImpact, Snapshot, discover_
 use serde::{Deserialize, Serialize};
 use serde_json::json;
 
-use crate::codegen::{CodegenOutput, asset_kind, derived_padding, placed_by_a_free_layout};
+use crate::codegen::{
+    AssetKind, CodegenOutput, asset_kind, derived_padding, placed_by_a_free_layout,
+};
 
 const START: &str = "\u{e000}DEVUP_PROVENANCE_START:";
 const END: &str = "\u{e000}DEVUP_PROVENANCE_END:";
@@ -342,7 +344,11 @@ pub fn validate_fidelity(
                 .is_some_and(|(index, _)| consumed_text_entries.insert(index))
         })
         .count();
-    let variables = variable_sources(snapshot, &semantic_nodes);
+    let baked = baked_into_assets(snapshot, root_id);
+    let variables = variable_sources(snapshot, &semantic_nodes)
+        .into_iter()
+        .filter(|(node_id, _)| !baked.contains(node_id))
+        .collect::<BTreeSet<_>>();
     let covered_variables = variables
         .iter()
         .filter(|(node_id, variable_id)| {
@@ -649,6 +655,44 @@ fn semantic_nodes<'a>(snapshot: &'a Snapshot, root_id: &str) -> BTreeSet<&'a str
     visible
 }
 
+/// Nodes whose variable bindings no correct generator could write out.
+///
+/// An asset drawn in one colour becomes a Box masked to its shape with `bg`
+/// set from that colour, so a token binding inside it survives as a token.
+/// An asset drawn in more than one has no such form - a CSS mask carries
+/// alpha only, and an SVG loaded through `<img src>` renders in its own
+/// document where neither `currentColor` nor a CSS variable reaches it - so
+/// it becomes an `<Image>` and every colour inside is baked into the file.
+///
+/// Counting those bindings as expected-but-missing asks the generator for
+/// something the platform does not offer, and it is not a one-off: measured
+/// on a real screen, two play buttons each hid a bound circle behind a white
+/// glyph, and the shortfall they produced put the whole fidelity report on
+/// the response with nothing in it anyone could act on.
+fn baked_into_assets(snapshot: &Snapshot, root_id: &str) -> BTreeSet<String> {
+    let mut baked = BTreeSet::new();
+    let mut pending = vec![root_id.to_owned()];
+    while let Some(node_id) = pending.pop() {
+        let Some(node) = snapshot.nodes.get(&node_id) else {
+            continue;
+        };
+        // A mask keeps its one colour in CSS, so keep looking inside it.
+        if !matches!(asset_kind(snapshot, node), None | Some(AssetKind::SvgMask)) {
+            let mut inside = vec![node_id];
+            while let Some(current) = inside.pop() {
+                let Some(node) = snapshot.nodes.get(&current) else {
+                    continue;
+                };
+                inside.extend(node.typed_view().child_ids().map(str::to_owned));
+                baked.insert(current);
+            }
+            continue;
+        }
+        pending.extend(node.typed_view().child_ids().map(str::to_owned));
+    }
+    baked
+}
+
 fn variable_sources(
     snapshot: &Snapshot,
     semantic_nodes: &BTreeSet<&str>,

crates/devup-mcp/src/server/mod.rs

@@ -613,7 +613,8 @@ impl DevupServer {
     #[tool(
         description = "Acquire a Figma design once and project any combination of outputs from that one collection; the Figma-to-code entry point. \
                        Ask only for what you will read: `tsx` is the deliverable, and the response always carries `status`, `quality`, `cache.artifactId`, `collection` and `source` beside it. \
-                       Each output adds its own keys and nothing else - tsx adds `tsx`; componentTsx adds `componentTsx`; responsiveTsx adds `responsiveTsx`, `responsiveSlots` and, where a width asked for something one tree cannot say, `responsiveUnrepresented`; devupJson adds `devupJson`, `themeCounts`, `themeCompleteness`, `conflicts` and `unresolvedVariables`; sourceMap, rawSnapshot, rawPayload, assetManifest and referencePng each add the key they name. \
+                       Each output adds its own keys and nothing else - tsx adds `tsx`; componentTsx adds `componentTsx`; responsiveTsx adds `responsiveTsx`, `responsiveSlots` and, where a width asked for something one tree cannot say, `responsiveUnrepresented`; devupJson adds `devupJson`, `themeCounts`, `themeCompleteness`, `conflicts` and `unresolvedVariables`; sourceMap, assetManifest and referencePng each add the key they name. \
+                       `rawSnapshot` and `rawPayload` are the collected design in raw form and need `debug: true`. Use them for one question only - a screen looks wrong and you must decide whether the generator is at fault or the design says so - never to implement, since the tsx already carries what they carry. \
                        `fidelity` and `completenessReport` appear only when the result is not exact or complete, or when includeDiagnostics is set. \
                        tsx expands every instance into primitives while componentTsx keeps them as <Name /> references, so requesting both gives the same screen twice and the difference between them is each component's body. responsiveTsx merges every width the capture carries into one module whose differing values are devup-ui responsive arrays, and is produced whenever there is more than one width. \
                        Reuse a previous acquisition with `artifactId` from `cache` to project further outputs without calling Figma again.",
@@ -623,7 +624,7 @@ impl DevupServer {
         &self,
         Parameters(input): Parameters<FigmaExportInput>,
     ) -> Result<CallToolResult, ErrorData> {
-        validate_outputs(&input.outputs).map_err(to_mcp_error)?;
+        validate_outputs(&input.outputs, input.debug).map_err(to_mcp_error)?;
         if !input.asset_requests.is_empty()
             && !input.outputs.iter().any(|output| output == "assetManifest")
         {

crates/devup-mcp/src/server/projection.rs

@@ -24,8 +24,8 @@ use super::{
     output::{OutputPolicy, OutputTransaction},
     parse_scope,
     quality::{
-        AcquisitionQuality, OutputQuality, ProjectionQuality, acquisition_quality, assets_quality,
-        projection_quality, theme_quality,
+        AcquisitionQuality, OutputQuality, acquisition_quality, assets_quality, projection_quality,
+        theme_quality,
     },
     section_candidate_as_explore, section_index_from_payload,
 };
@@ -412,26 +412,26 @@ pub(super) fn artifact_metadata(artifact: &ArtifactLookup) -> Value {
 /// response carried before this, none of it actionable. It is attached
 /// whenever the capture is not clean, and whenever the caller asked for
 /// diagnostics and therefore wants the detail regardless.
-/// Adds `fidelity` only when the conversion was not exact.
+/// Adds `fidelity` whenever the report has something `quality` does not say.
 ///
-/// `quality.projection` is the signal a caller acts on; `fidelity` is the
-/// drill-down beneath it, at 437 measured bytes, repeated once per screen on
-/// a Section export. Tying it to that same grade keeps the two from
-/// disagreeing: an exact conversion sends the grade alone, and anything less
-/// sends the axes that explain it.
+/// Keyed on the report itself, not on `quality.projection`. That was the
+/// first attempt and it hid a real defect: `projection_quality` is computed
+/// from diagnostics alone, so a coverage shortfall that raises no diagnostic
+/// leaves the grade reading `exact` and took the report away with it.
+/// Measured on a real 50-node screen, `variables` was 19 of 20 - a `$primary`
+/// binding frozen into an exported SVG - while `quality.projection` said
+/// `exact`, so the response went from showing the shortfall to hiding it.
 ///
-/// Deliberately not keyed on `strict_compatible`, which also fails on a
-/// coverage shortfall that changes nothing about the output - that would put
-/// the report back on almost every response while `quality` still read
-/// `exact`. `strict: true` keeps using `strict_compatible` to refuse, and
-/// returns the same report in the error.
+/// `strict_compatible` is the same predicate `strict: true` refuses on, which
+/// keeps the two from disagreeing about whether anything was lost. On the ten
+/// real captured screens it is true - nothing sent - for every screen with no
+/// shortfall and no approximation, which is what this is for.
 fn attach_fidelity(
     response: &mut Value,
     report: &devup_mcp_devup_ui::provenance::FidelityReport,
-    projection: ProjectionQuality,
     include_diagnostics: bool,
 ) {
-    if include_diagnostics || projection != ProjectionQuality::Exact {
+    if include_diagnostics || !report.strict_compatible() {
         response["fidelity"] = json!(report);
     }
 }
@@ -754,12 +754,7 @@ pub(super) async fn complete_operation(
                         "quality": frame_quality,
                         "tsx": output.tsx
                     });
-                    attach_fidelity(
-                        &mut frame,
-                        &output.fidelity_report,
-                        frame_quality.projection,
-                        include_diagnostics,
-                    );
+                    attach_fidelity(&mut frame, &output.fidelity_report, include_diagnostics);
                     attach_completeness_report(
                         &mut frame,
                         frame_quality,
@@ -1186,12 +1181,7 @@ pub(super) async fn complete_operation(
             // point; each Section frame already carries its own.
             if !section_tsx_projected && let Some(report) = fidelity_reports.first() {
                 let mut carrier = Value::Object(Map::new());
-                attach_fidelity(
-                    &mut carrier,
-                    report,
-                    quality.projection,
-                    include_diagnostics,
-                );
+                attach_fidelity(&mut carrier, report, include_diagnostics);
                 if let Some(fidelity) = carrier.get("fidelity") {
                     result.insert("fidelity".to_owned(), fidelity.clone());
                 }

crates/devup-mcp/src/server/tools.rs

@@ -56,6 +56,19 @@ pub struct FigmaExportInput {
     /// by default when driven as a library.
     #[serde(default = "default_true")]
     pub asset_names_per_node: bool,
+    /// Opens the two outputs that describe the design rather than the screen,
+    /// `rawSnapshot` and `rawPayload`.
+    ///
+    /// For one question only: a screen looks wrong, and it has to be decided
+    /// whether the generator is at fault or the design says so. Answering
+    /// that means reading the collected design beside the generated code.
+    ///
+    /// Nothing that implements a screen needs them. Measured across ten real
+    /// captured screens the tsx already carries 100% of the nodes, text,
+    /// typography, assets and layout the design expects, and on one screen
+    /// asking for them by habit spent about eight bytes for every one of code.
+    #[serde(default)]
+    pub debug: bool,
     #[serde(default)]
     pub strict: bool,
     #[serde(default)]

crates/devup-mcp/src/server/validation.rs

@@ -123,7 +123,25 @@ fn collection_scope_rank(scope: CollectionScope) -> u8 {
     }
 }
 
-pub(super) fn validate_outputs(outputs: &[String]) -> Result<(), DevupError> {
+/// The two outputs that answer a question about the generator rather than
+/// contributing to the screen.
+///
+/// They are the design as it was collected, in raw form. Nothing that
+/// implements a screen needs them - the tsx already carries what they carry,
+/// measured across ten real captured screens at 100% of nodes, text,
+/// typography, assets and layout. What they are for is the other question:
+/// the UI looks wrong, and someone has to decide whether the generator is at
+/// fault or the design says so. Answering that means reading the design
+/// beside the code, which is exactly this.
+///
+/// Left in the ordinary `outputs` list they were requested as a matter of
+/// course - the server's own instructions used to say to take `rawSnapshot`
+/// every time, which on one measured screen spent about eight bytes for
+/// every one of code. `debug` is the door: closed for implementation, open
+/// when a defect is being adjudicated.
+pub(crate) const DIAGNOSIS_OUTPUTS: [&str; 2] = ["rawSnapshot", "rawPayload"];
+
+pub(super) fn validate_outputs(outputs: &[String], debug: bool) -> Result<(), DevupError> {
     if outputs.is_empty() {
         return Err(DevupError::new(
             ErrorCode::DevupInvalidInput,
@@ -142,6 +160,19 @@ pub(super) fn validate_outputs(outputs: &[String]) -> Result<(), DevupError> {
                 false,
             ));
         }
+        if !debug && DIAGNOSIS_OUTPUTS.contains(&output.as_str()) {
+            return Err(DevupError::new(
+                ErrorCode::DevupInvalidInput,
+                format!(
+                    "{output} is the collected design in raw form, for deciding whether a \
+                     screen that looks wrong is the generator's fault or the design's. It is \
+                     not needed to implement anything - the tsx already carries what it \
+                     carries - and requesting it by habit is most of the response. Pass \
+                     debug: true to read it."
+                ),
+                false,
+            ));
+        }
     }
     Ok(())
 }