Skip to content

feat(catalog): show field-level CSV diffs and drop CanonicalType column - #149

Merged
Jared Holgate (jaredfholgate) merged 1 commit into
mainfrom
jaredfholgate-csv-diff-column-level
Sep 19, 2026
Merged

Jared Holgate (jaredfholgate) merged 1 commit into
mainfrom
jaredfholgate-csv-diff-column-level

Conversation

@jaredfholgate

Copy link
Copy Markdown
Member

Why

Reviewing a real plan-only run (https://github.com/Azure/azure-verified-modules-tools/actions/runs/35462875427) showed two problems:

  1. The CSV diff added in feat(catalog): show full CSV diffs in plan-only runs #148 is line-based, so a single changed column rewrites the entire ~800-character row. Real diffs show massive churn even when only 3-6 of 19 columns actually changed.
  2. All six generated CSVs carry a CanonicalType column. This was traced to PR feat(metadata): centralize schemas, catalog sync and backfill #113 (2026-09-16) — a deliberate, documented decision, but unrelated to diff review and something we can't do right now since the CSV structure can't change.

What changed

  • Field-level CSV diff (New-ModuleCatalogCsvDiff.ps1): rows are matched by ModuleName (falling back to RepoURL), and the summary now shows, per changed file, added/removed row counts plus a Module | Field | Before | After table of only the columns that actually changed. The full unified line diff is still generated and still available — it's nested inside a collapsed "Raw line diff" section, and still written to .diff/all-csv.diff artifacts. A new per-file .fields.md artifact is added alongside the existing .diff file. Inline-size gating (-MaxInlineDiffBytes) now measures the rendered summary block rather than just the raw diff.
  • CanonicalType removed from CSV output (ModuleCatalog.ps1): the column is no longer force-added to legacy CSV tables or populated in generated rows. The internal canonicalType metadata field and JSON catalog module grouping (v1/modules.json) are untouched — this is a CSV-column-only change.
  • Updated component tests (ModuleCatalog.Component.Tests.ps1, ModuleCatalog.Publication.Tests.ps1, MetadataBackfill.Component.Tests.ps1, ModuleCatalog.CsvDiff.Tests.ps1) and catalog docs (repository-management/module-catalog/README.md, docs/metadata-rollout.md) to match.

Validation

./build.ps1 pre-commit — build succeeded with warnings only (expected negative-path test log noise): Unit 1659 passed / 0 failed / 8 skipped; Component 831 passed / 0 failed / 1 skipped.

Plan-only module catalog CSV diffs previously rewrote entire ~800-char
rows for a single changed column, and the six generated CSVs carried a
CanonicalType column added in an earlier PR unrelated to diff review.

- Add field-level (column-aware) diffing to New-ModuleCatalogCsvDiff.ps1:
  match rows by ModuleName/RepoURL, render added/removed rows and a
  Field | Before | After table as the default summary view, with the
  full unified line diff nested in a collapsed "Raw line diff" section.
- Remove CanonicalType from CSV generation in ModuleCatalog.ps1; the
  internal canonicalType metadata field and JSON catalog grouping are
  unaffected.
- Update component tests for both changes and refresh catalog docs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant