Repository navigation
Support non-primitive column types across SQL Server, Postgres and MySQL (supersedes #72) - #73
Merged
Merged
Conversation
…inference `SqlServerSchemaConfig.ToDotNetTypeName` threw for `geography`, `geometry`, `hierarchyid`, `xml` and `vector`, and because the schema inference walks every table of the database, a single column of one of these types anywhere failed the `Data`, `Reset` and `Inspect` commands as a whole. They now map to `string`: SQL Server converts the string representation (WKT, path, XML, JSON array) implicitly on insert, so YAML/JSON data files can seed these columns as text. Adds the `[Test].[ClrTypes]` table to the test console and asserts the mapping in `SqlServerSelectSchema` (`vector` is left out of the table because the CI image is SQL Server 2019).
Builds on the intent of #72 (Wesley Teixeira) to evolve it into a more complete solution: - DbColumnSchema: DotNetType is now the true (unqualified) EF type; new DataParserType for text-based data input, IsDotNetTypeAClass, IsDataComparable, UdtName and NativeSqlType. - SQL Server: geography/geometry, hierarchyid, vector, xml, json, sql_variant; merge skips EXCEPT comparison for non-comparable columns. - Postgres: network, text-search, geometric, bit-string, range/multirange, array, enum, hstore, ltree, citext, PostGIS and pgvector types. - MySQL: mediumint, year, bit(n), vector and spatial types; WKT/EWKT values use axis-order=long-lat so coordinates match PostGIS and SQL Server. - EF model template emits '= default!' for non-nullable class-typed properties. - Provider catalog queries moved to embedded resources. - Tests: same-shape extra-types table and round-trip tests for all providers; Inspect tests extended and made line-ending agnostic; code-gen output added to DbEx.Test.Empty. - Test infrastructure: docker-compose (SQL Server 2025, MySQL 9, Postgres+PostGIS+pgvector), CI uses docker compose and newer actions. - Docs: README data types section, CONTRIBUTING test setup, CHANGELOG v3.3.0. Supersedes #72. Co-authored-by: Wesley Teixeira <wesley@nuvem.net> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
PostgreSQL metadata ordering and array handling, MySQL BIT mapping, and CI readiness contain unresolved correctness and reliability issues.
5 open findings
What changed in this PR
Extends schema inference, data loading, and EF model generation to support non-primitive types across SQL Server, PostgreSQL, and MySQL.
Changes:
- Adds provider-specific type mappings and textual data parsing.
- Handles SQL Server merge comparisons for non-comparable types.
- Adds integration coverage, updated database containers, documentation, and version 3.3.0.
| File | Description |
|---|---|
tests/DbEx.Test/SqlServerMigrationTest.cs |
Tests SQL Server type round trips. |
tests/DbEx.Test/Resources/SqlServerInspect.md |
Adds SQL Server inspect baseline. |
tests/DbEx.Test/Resources/PostgresInspect.md |
Adds PostgreSQL inspect baseline. |
tests/DbEx.Test/Resources/MySqlInspect.md |
Adds MySQL inspect baseline. |
tests/DbEx.Test/PostgresMigrationTest.cs |
Tests PostgreSQL type round trips. |
tests/DbEx.Test/MySqlMigrationTest.cs |
Tests MySQL type round trips. |
tests/DbEx.Test/MarkdownAssert.cs |
Normalizes markdown comparisons. |
tests/DbEx.Test/ExtraTypesReader.cs |
Shares round-trip test assertions. |
tests/DbEx.Test/DatabaseSchemaTest.cs |
Verifies inferred type metadata. |
tests/DbEx.Test.PostgresConsole/Migrations/006-create-test-extratypes-table.pgsql |
Creates PostgreSQL fixture types. |
tests/DbEx.Test.PostgresConsole/DbEx.Test.PostgresConsole.csproj |
Embeds PostgreSQL migration. |
tests/DbEx.Test.PostgresConsole/Data/Data.yaml |
Seeds PostgreSQL extra types. |
tests/DbEx.Test.MySqlConsole/Migrations/006-create-test-extratypes-table.mysql |
Creates MySQL fixture types. |
tests/DbEx.Test.MySqlConsole/DbEx.Test.MySqlConsole.csproj |
Embeds MySQL migration. |
tests/DbEx.Test.MySqlConsole/Data/Data.yaml |
Seeds MySQL extra types. |
tests/DbEx.Test.Empty/Repositories/TestDbContext.g.cs |
Adds generated EF configuration. |
tests/DbEx.Test.Empty/Persistence/ExtraTypes.g.cs |
Adds generated SQL Server model. |
tests/DbEx.Test.Empty/GlobalUsing.cs |
Imports generated model types. |
tests/DbEx.Test.Empty/DbEx.Test.Empty.csproj |
Adds EF type dependencies. |
tests/DbEx.Test.Console/Migrations/009-create-test-extratypes-table.sql |
Creates SQL Server fixture types. |
tests/DbEx.Test.Console/dbex.yaml |
Enables extra-type code generation. |
tests/DbEx.Test.Console/Data/Data2.yaml |
Exercises SQL Server merge loading. |
tests/DbEx.Test.Console/Data/Data.yaml |
Seeds SQL Server extra types. |
src/DbEx/Templates/EfModel_cs.hbs |
Initializes non-nullable reference properties. |
src/DbEx/Migration/Data/DataTable.cs |
Detects non-comparable merge columns. |
src/DbEx/Migration/Data/DataRow.cs |
Uses parser-specific CLR types. |
src/DbEx/DbSchema/DbColumnSchema.cs |
Exposes additional type metadata. |
src/DbEx/DatabaseSchemaConfig.cs |
Adds provider type extension points. |
src/DbEx.SqlServer/SqlServerSchemaConfig.cs |
Maps and formats SQL Server types. |
src/DbEx.SqlServer/Resources/SelectTableVectorColumns.sql |
Reads vector dimensions. |
src/DbEx.SqlServer/Resources/DatabaseData_sql.hbs |
Avoids invalid merge comparisons. |
src/DbEx.Postgres/Resources/SelectTableNativeTypes.pgsql |
Reads PostgreSQL native metadata. |
src/DbEx.Postgres/PostgresSchemaConfig.cs |
Maps PostgreSQL advanced types. |
src/DbEx.MySql/MySqlSchemaConfig.cs |
Maps MySQL advanced types. |
README.md |
Documents supported type behavior. |
docker-compose.yml |
Updates integration databases. |
CONTRIBUTING.md |
Documents live-database setup. |
Common.targets |
Bumps version to 3.3.0. |
CHANGELOG.md |
Records release changes. |
.github/workflows/CI.yml |
Starts databases via Compose. |
.github/postgres/Dockerfile |
Builds PostgreSQL extension image. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…tive types ordering - MySQL: multi-bit BIT(n) maps to ulong (BIT/BIT(1) remain bool) - Compose/CI: add healthchecks and use 'docker compose up --wait' instead of fixed sleep - Postgres: strip only a single array underscore prefix from UDT names - Add DatabaseSchemaConfig.LoadNativeTypesSchema hook, invoked before ref-data inference, so Postgres enum names are loaded before DotNetType is cached - Postgres: native types query includes views, materialized views and foreign tables Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: Wesley Teixeira <wesley@nuvem.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #72. That PR's commit (by @wesleyteixeira) is preserved as the first commit here, so credit is retained. Thanks Wesley for raising and starting this.
Why
Mapping non-primitive types (CLR/spatial/etc.) to
stringfixes schema inference, but the code-gen targets EF, so the generated models need the real .NET types. This evolves #72 to do that and extends it to Postgres and MySQL.What
DbColumnSchema:DotNetTypeis now the true, unqualified EF type (consumers need the relevantglobal using). NewDataParserType(text input for YAML/JSON data),IsDotNetTypeAClass,IsDataComparable,UdtName,NativeSqlType.geography/geometry,hierarchyid,vector,xml,json,sql_variant. Merge skips theEXCEPTcomparison for non-comparable columns.hstore,ltree,citext, PostGIS, pgvector.mediumint,year,bit(n),vector, spatial types. Spatial WKT/EWKT is parsed withaxis-order=long-lat; otherwise MySQL defaults to lat/long for geographic SRIDs and silently swaps coordinates vs PostGIS/SQL Server.= default!for non-nullable class-typed properties (e.g.byte[]). This changes generated output for such columns.Tests
DbEx.Test.Empty(builds on net8.0/9.0/10.0).docker compose.Infra / docs
docker-compose.yml: SQL Server 2025, MySQL 9, Postgres 17 + PostGIS + pgvector (built from.github/postgres).docker compose up -d --buildand newercheckout@v7/setup-dotnet@v6.Not covered / follow-ups
Each needs its own design decision, so deliberately left out: MySQL unsigned ints,
TINYINT(bytevssbytedepends on the EF provider),byte[]literals in YAML data,DATETIME2(n)precision,vector(n, float16).Caveats
SQL_VARIANTmaps toobject, which EF Core can't map; it is in the test model only to exercise the mapping.