Repository navigation
feat(rust): add opt-in enableExtensionData to keep undeclared JSON properties - #318
NopeCharon wants to merge 1 commit into
Conversation
…operties
Generated Rust structs only declare the properties of their Concerto class,
so serde silently drops every other JSON property when deserializing. A
ConceptDeclaration read through the metamodel's Declaration struct loses
isAbstract, superType and properties, and data written by a newer version
of a model loses the fields the older generated code does not know about.
Add an enableExtensionData parameter to RustVisitor, mirroring the C#
visitor option of the same name. When it is set, every generated struct
gets a catch-all field:
#[serde(flatten)]
pub extension_data: std::collections::HashMap<String, serde_json::Value>,
serde collects the undeclared properties into it when deserializing and
writes them back when serializing, so they survive a round trip. Rust has
no struct inheritance, so the field is added to every struct rather than
only to root classes as in C#. If a class property already maps to
extension_data, underscores are appended until the name is unique. The
field uses fully qualified paths, so the generated imports do not change;
the generated code does need the serde_json crate when the option is on.
The option defaults to false, so existing output and snapshots are
unchanged. It is documented in the RustVisitor class JSDoc.
Tests:
- unit tests for the emitted field, the default (no field), subclasses,
name clashes and imports
- a verification test that generates the metamodel plus a user model,
then builds and runs them with cargo to check that a ConceptDeclaration
read as a Declaration, and a Person with extra nested fields, round-trip
unchanged (skipped when cargo is not installed)
Closes accordproject#206
Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
|
Thanks for the PR. It adds an opt-in Drafted by an Accord Project agent, reviewed by a maintainer. |
There was a problem hiding this comment.
🟡 Changes recommended
The regression test does not exercise the nested deserialization behind #206’s stack overflow and needs Model and typed MapDeclaration round trips that invoke the generated MapKeyType and MapValueType deserializers.
1 open issue
What changed in this PR
Adds opt-in preservation of undeclared JSON properties to generated Rust structs, addressing #206’s data-loss use case while leaving default output unchanged.
Changes:
- Adds
enableExtensionDatato emit a flattened catch-all map on every struct. - Avoids field-name collisions and documents the required
serde_jsondependency. - Adds unit coverage and Cargo-based round-trip verification for metamodel declarations and nested user data.
| File | Description |
|---|---|
test/verification/rust.compile.test.js |
Adds runtime checks for unknown-property preservation. |
test/codegen/fromcto/rust/rustvisitor.js |
Tests opt-in output, defaults, subclasses, naming collisions, and imports. |
lib/codegen/fromcto/rust/rustvisitor.js |
Implements and documents extension-data generation. |
Review effort: Balanced
| '}', | ||
| '', | ||
| 'fn main() {', | ||
| ' let declaration: Declaration = round_trip(r#"{"$class":"concerto.metamodel@1.0.0.ConceptDeclaration","name":"Dog","isAbstract":false,"properties":[]}"#);', |
There was a problem hiding this comment.
Thank you for this change. The opt-in enableExtensionData option is small and well scoped. It defaults to off, so existing output and snapshots don't change. The name-clash helper is sensible, and the PR description follows the repo's guidance: commands run, numbered design questions, and the choices explained. The unit tests pin the exact output and the default. One gap remains in the verification test: it doesn't exercise the nested case that issue #206 warns about. If the round-trip covers a Model with non-empty declarations, including a typed MapDeclaration, this is ready for a maintainer to decide on the two design questions (option name, and every struct versus Declaration only).
Nothing blocking; some suggestions below. Findings: 1 should-fix.
Automated review by Mira, an Accord Project agent, at 1837e66. Findings are labelled blocking, should-fix or nit; a maintainer makes the final call.
| '}', | ||
| '', | ||
| 'fn main() {', | ||
| ' let declaration: Declaration = round_trip(r#"{"$class":"concerto.metamodel@1.0.0.ConceptDeclaration","name":"Dog","isAbstract":false,"properties":[]}"#);', |
There was a problem hiding this comment.
should-fix · tests
The round-trip test only reads a ConceptDeclaration with properties: [] as a Declaration. Declaration doesn't declare properties or isAbstract, so those land in extension_data as raw serde_json::Value and the generated nested deserializers never run. Issue #206 says flatten on all base types overflowed the stack on MapKeyType/MapValueType, and the PR description says nested Model → Declaration data deserializes fine. The committed test doesn't pin that claim. Please extend the Rust program to round-trip a Model with non-empty declarations, including a MapDeclaration with a typed key and value. Assert the typed fields as well as extension_data, so the MapKeyType and MapValueType deserializers are really exercised. This also answers the open review thread on the same line.

Closes #206
Generated Rust structs only declare their own Concerto properties, so serde drops every other JSON property when it deserializes. This PR adds an opt-in
enableExtensionDataparameter toRustVisitor, named after the C# visitor option from #275. With it on, every generated struct gets a#[serde(flatten)]catch-all field. Undeclared properties land there when deserializing and are written back when serializing. For example, aConceptDeclarationread through the metamodel'sDeclarationstruct now keepsisAbstract,superTypeandproperties, which is the case #206 describes.Changes
RustVisitor.visitClassDeclarationwrites the field whenparameters.enableExtensionDatais set. Rust has no struct inheritance, so every struct gets one; in C# only the root classes do.getExtensionDataFieldNamehelper: if a class property already maps toextension_data, it appends underscores until the name is unique.uselines don't change. When the option is on, the generated crate needsserde_json.RustVisitorclass JSDoc. It defaults tofalse, so existing output and snapshots are unchanged.test/codegen/fromcto/rust/rustvisitor.jscover the emitted field, the default, subclasses, name clashes and imports.test/verification/rust.compile.test.jsgenerates the metamodel plus a small user model and builds and runs them withcargo run. It checks that aConceptDeclarationread as aDeclaration, and aPersonwith extra top-level and nested fields, round-trip unchanged. It is skipped when cargo is not installed, like the other Rust verification tests.Flags
Design questions for a maintainer:
enableExtensionDatafrom the C# visitor so the same parameter works for both targets, and named the Rust fieldextension_datato match.preserveUnknownFieldsis an alternative if you'd rather have a Rust-specific name.Declarationonly. I made it an opt-in for every struct so it works for any model, including newer model versions read by older generated code. feat(rust): add serde flatten to Declaration in generated output #206 mentioned a stack overflow with fix(rust): add serde flatten for subclass field preservation #205's approach. I couldn't reproduce one: the whole metamodel generated with the option builds, and deserializes nestedModel→Declarationdata, with serde 1 / serde_json 1. It also builds together withflattenSubclassesToUnion.What I ran:
npm test: lint, licence check and JSDoc pass. 702 tests pass, 59 pending, 0 snapshot changes. The 8 Avro verification failures are from my machine's Java 8, which is too old for avro-tools; they are unrelated to this change.npx mocha test/codegen/fromcto/rust/rustvisitor.js test/verification/rust.compile.test.jswith cargo 1.100 installed: every Rust verification case passes, including the new round-trip test. With the visitor change reverted, the 3 new unit tests that expect the field and the round-trip test fail.Screenshots or Video
N/A
Related Issues
Author Checklist
--signoffoption of git commit.mainfromfork:branchname