Skip to content

feat(data): scope conversion to a model-driven app - #192

Draft
david-hudec-networg wants to merge 9 commits into
TALXIS:masterfrom
david-hudec-networg:feat/data-model-convert-app-scope
Draft

feat(data): scope conversion to a model-driven app#192
david-hudec-networg wants to merge 9 commits into
TALXIS:masterfrom
david-hudec-networg:feat/data-model-convert-app-scope

Conversation

@david-hudec-networg

Copy link
Copy Markdown

Stacked on #190#191. Only the last commit belongs to this PR; the earlier ones are those two. GitHub cannot express a cross-fork stack, so the base has to be master. Review in order.

What it does

Lets the tool answer "what is this app built on", not only "what does this solution declare".

txc data model convert --root . --root ../TALXIS --app ntg_projectmanagement --target dbml

No environment or profile — an app names its own tables in its AppModule file, and everything it names is in source.

Three things about how apps sit on disk

These drove the implementation and are each covered by a test:

  • The search is anchored on the AppModules folder, not on Declarations. Some modules keep their declarations under CDS instead, and a search anchored on either name silently misses the other.
  • Identity is the UniqueName inside the file, never the folder name. The two differ in case in the wild — invisible on Windows, wrong on a case-sensitive filesystem.
  • One app can be declared across several files — a base declaration plus fragments from other areas whose components carry solutionaction="Added". Its component set is the union of all of them.

Only type="1" components carry a table name; views, forms, charts and workflows reference their owner by id alone. Sitemap entities are included too, from both the Entity attribute and the etn parameter inside a Url — both forms occur, sometimes in the same file.

--root

Apps and entity schema live in different modules (Apps.Home vs Model), so scoping to an app structurally needs to reach past a single declarations folder. --root expands to every declarations folder beneath it, found by looking for the entity declarations themselves rather than a folder name. Pass the product repository as a second root when the base model lives there.

Ordering, which turned out to matter

Scoping runs before relationships are built. Filtering afterwards lets a relationship between two dropped tables synthesise both of them straight back as stubs — measured at 261 tables reappearing before this was ordered correctly.

A relationship is kept when its referencing side is in scope, so a lookup out of the app still terminates somewhere visible rather than dangling. Option sets belonging to dropped tables are pruned as well, or the output declares more enums than it has columns using them (907 enums for 883 columns, before).

Measured

Project root plus product root, one invocation each:

app tables refs columns enums
ntg_projectmanagement 70 124 883 88
ntg_administration 36 59 388 29
ntg_easementmanagement 79 183 1055 130
ntg_hiltipartnerportal 25 34 216 21

Four apps across three unrelated solutions, all parsed with @dbml/core. Same repositories, different app, radically different diagram — which is the point.

An unknown app fails with the apps it did find (Apps found: ntg_administration, ntg_operation, …), so a typo is a question rather than an empty diagram.

Back-compat

Without --app, nothing is filtered. Single-input conversion across three solutions × five targets is unchanged and idempotent by hash. ParseModules(List<Module>) and ParseRelationships(modules, tables) keep their existing signatures; the scope is an added optional parameter with one code path, not a parallel implementation.

Tests

tests/TALXIS.CLI.Tests/Data/DataModelConverter/AppScopeTests.cs — ten tests over temp folders and in-memory modules: type-1 only; CDS folder found; several files union; identity from content not folder; sitemap via attribute and via URL; unknown app lists what it found; out-of-scope tables dropped and not resurrected as stubs; a lookup out of the app still terminates; and nothing filtered without a scope.

🤖 Generated with Claude Code

david-hudec-networg and others added 9 commits September 1, 2026 15:15
A table with several lookups to one target rendered a single edge, while all
its lookup columns still appeared - understating the model without looking
broken.

The duplicate guard keyed on (LeftSideTable, RighSideTable), ignoring which
column the relationship ran through.

The key now includes LeftSideRow. Genuine duplicates still collapse, which is
what the guard is for.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Required, form-visible columns disappeared from the output with no warning, and
a module whose only contribution was such an attribute read as contributing
nothing.

Rows of an optionset kind were deleted outright when their OptionSetName did not
resolve. Three causes seen in real solutions: the global option set declares
<options />, it is declared in a different module, or it is platform-owned.

The row is kept and only OptionSetName is cleared. That is what
ToDbDiagramNotation prefers over RowType, so leaving it set would reference an
Enum that was never emitted; RowType is left alone so sql and edmx keep their
own handling for the kind.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The same unchanged solution converted to a different file on every run, at
identical length - so a generated diagram could not be committed, diffed, or
compared across a model change.

Two causes. Module seeded Colorhex from new Random(). And three file
enumerations used Directory.GetFiles, which guarantees no ordering, so table,
relationship and enum order followed the filesystem.

Colour now derives from the module name with FNV-1a - not string.GetHashCode,
which is randomised per process on .NET Core - and all three enumerations are
ordered ordinally. Everyone's colours change; nothing could have depended on
the old values.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Where an entity has a many-to-many with itself, the intersect table carried the
same column twice, the same Ref twice, and the same EDMX navigation property
twice. A DBML parser rejects the first two outright.

Both sides resolved to <entity>id, and both legs carried the relationship name.

The second column and the second leg's name are suffixed positionally. The real
per-side names live in metadata (Entity1/Entity2IntersectAttribute) and are
author-chosen - the platform's own example pairs connectionroleid with
associatedconnectionroleid - so they cannot be derived from solution XML and are
not guessed at here.

Known limit: on the entity side EDMX still names the navigation property after
the primary key row, so one duplicate remains there. The intersect side is clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…rejected

Every plainsql conversion failed, and the error listed the formats it did
support - contradicting the option's own help.

The format is declared in the option's AllowedValues and fully implemented in
the conversion switch, but was missing from the service's SupportedFormats
guard three lines earlier.

Added. A test now asserts every value the option advertises actually converts,
so the two lists cannot drift apart again.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…once

A table declared by two modules ended up with the same column listed twice.

ParseMultipleRowsFromXml appended every parsed row without checking whether the
table already carried one of that name. Harmless while only one input could be
given; routine as soon as several can.

Rows are matched case-insensitively. Where two declarations disagree the first
input wins, so the result is deterministic in the order the caller gave; a
differing type warns rather than aborting, because several modules extending one
shared table is normal for a layered product; and text lengths widen but never
narrow, since a consumer breaks on too little room, not too much.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…onstructor

Every module would come out the same colour once modules carry distinct names.

Colorhex was assigned in the constructor, which runs before an object
initializer sets ModuleName - so the colour was derived from an empty name.
Invisible while there was only ever one module.

Colorhex is now a computed property, so it always reflects the name in effect.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A delivery project's model is spread across the modules a product ships plus the
project's own layer, and several of them declare part of the same table.
Converting each separately and concatenating the output keeps only the first
declaration of each table, so the merge had to be done by hand.

--input accepted a single path, and only zip inputs were ever built into more
than one Module.

--input is now repeatable. Folder and zip inputs both resolve to a Module and go
through the ParseModules seam that already existed for zips, so the two can be
mixed in one invocation. Modules are named after the folders that own their
declarations, so a merged diagram attributes each table to its source instead of
rendering an empty comment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds --app and --root, so the question the tool answers can be "what is this app
built on" rather than only "what does this solution declare".

An app names its tables in its own AppModule file, so this needs no environment.
Three things about how apps sit on disk drove the implementation:

- The search is anchored on the AppModules folder, not on "Declarations". Older
  modules keep their declarations under "CDS", and a search anchored on either
  name silently misses the other.
- Identity is read from the UniqueName inside the file, never the folder name --
  the two differ in case in the wild, which is invisible on Windows and wrong on
  a case-sensitive filesystem.
- One logical app can be declared across several files, a base declaration plus
  fragments from other areas carrying solutionaction="Added". Its component set
  is the union of all of them.

Only type="1" components carry a table name; views, forms, charts and workflows
reference their owner by id alone. Sitemap entities are picked up as well, from
both the Entity attribute and the etn parameter inside a Url -- both forms occur,
sometimes in the same file.

--root exists because apps and entity schema live in different modules, so
scoping to an app structurally needs to reach past a single declarations folder.
It expands to every declarations folder beneath it. Pass the product repository
as a second root when the base model lives there.

Scoping runs before relationships are built. Filtering afterwards would let a
relationship between two dropped tables synthesise both of them straight back as
stubs -- measured at 261 tables reappearing before this was ordered correctly. A
relationship is kept when its referencing side is in scope, so a lookup out of
the app still terminates somewhere visible rather than dangling. Option sets
belonging to dropped tables are pruned too, or the output declares more enums
than it has columns using them.

Measured, project root plus product root, one invocation each:

  ntg_projectmanagement    70 tables  124 refs   883 cols   88 enums
  ntg_administration       36         59         388        29
  ntg_easementmanagement   79        183        1055       130
  ntg_hiltipartnerportal   25         34         216        21

All parse with @dbml/core. Unknown app names fail listing the apps that were
found. Without --app nothing is filtered, and single-input conversion across
three solutions and all five targets is unchanged and idempotent.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant