Skip to content

feat(generator): add type modules and optional formatting - #318

Merged
Bergmann89 merged 2 commits into
Bergmann89:masterfrom
sundermann:feat/type-output-layout
Nov 12, 2026
Merged

Bergmann89 merged 2 commits into
Bergmann89:masterfrom
sundermann:feat/type-output-layout

Conversation

@sundermann

@sundermann sundermann commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Adds selectable per-type output files and optional prettyplease formatting, so callers can generate readable bindings as smaller Rust files without custom splitting or formatting code.

Changes

  • Add GeneratorFlags::USE_TYPE_MODULES to assign each generated type its own module before building reference and inheritance trait paths. Public namespace and schema paths remain available through explicit re-exports.
  • Allocate module names only for selected types and their dependencies. Reject dynamic-trait name collisions, and let custom render steps register additional public items with Module::export. Store each type's owning module in DataType::module_path and exported items in Module::exports.
  • Register imports only when the generated code needs them, including enum fallback and defaultable-content paths. Cover simple types, unions, closed enums, and enums with a native fallback in a compiled fixture with unused_imports denied. The split benchmark also builds with -Dunused-imports, without the original 20 warnings.
  • Keep the existing benchmark and bpmn, read_write, and vsme examples in their previous layout by subtracting USE_TYPE_MODULES from their GeneratorFlags::all() configurations.
  • Add optional formatting with the pretty Cargo feature (prettyplease and syn/full). CodeFormat::Plain remains the default. CodeFormat::Pretty and Error::FormattingError are available with or without the feature; selecting Pretty without it returns a formatting error. Both new formatting entry points return the crate's Error, including IO errors. The existing write_to_files keeps its io::Error API.
  • Add compiled fixtures for dynamic types, substitution groups, namespace/schema grouping, and all three Serde renderers, with XML read tests and stable-name, collision, and formatting regressions. Cover type modules combined with the newly merged native aliases, including absolute, crate-qualified, and generated-root-relative paths. Regenerate the binary-facet snapshot to reflect the naming changes inherited from current master.
  • Add the examples/type_modules crate to demonstrate separate files and explicit formatting. Leave the README and release changelog unchanged.

Breaking changes

  • GeneratorFlags::all() now includes USE_TYPE_MODULES, changing the generated layout for callers using all(). Subtract that flag to retain the previous layout. Ordinary generation remains unchanged unless the flag is enabled.
  • Public DataType and Module struct initializers must supply the new module_path and exports fields, respectively. Custom render steps should call Module::export for additional public items that need re-exporting from the namespace/schema module.
  • In the type-module layout, quick_xml_serialize and quick_xml_deserialize helper modules are private. Use the serialization/deserialization traits' associated types instead of public helper paths. The unsplit layout keeps its existing helper visibility.
  • Error gains the unconditional FormattingError variant at the end of the enum, so exhaustive downstream matches must account for it.

Checklist

  • Tests for the change (a bug fix has a test that fails without the fix)
  • Changed snapshots are regenerated with update-expectations and reviewed
  • cargo fmt, cargo clippy and cargo test pass (see CONTRIBUTING.md)
  • Breaking changes of the public API or the generated code are described above

@sundermann sundermann changed the title Feat/type output layout (feat): Output layout Nov 10, 2026
@sundermann
sundermann force-pushed the feat/type-output-layout branch from 87608af to 6b6b0cc Compare November 10, 2026 20:49

@Bergmann89 Bergmann89 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sundermann,

Thanks for this, I'd love to get this merged, but there is some rework needed before that:

Blocking

  • Split output does not compile for dynamic types and substitution groups. E.g. for the dynamic_type and substitution_group feature schemas you get cannot find trait AnimalTypeTrait in this scope inside the type modules. make_trait_impls produces a bare FooTrait for traits in the same namespace, and render_trait_impls (steps/mod.rs:459-485) only rebases paths that start with super.
  • No test that pins the generated code. The tests only check to_string().contains(...), which is why the issue above slipped through. Please add a feature test the usual way (generate_test with expected/*.rs reference files that get compiled via include!). One file per renderer with the type modules as inline mod { ... } blocks is fine, no need for one file per module. Please cover at least a dynamic type / substitution group, the combination with namespace modules, and quick_xml with a read test.

Design / API

  • Make it a generator flag: GeneratorFlags::USE_TYPE_MODULES. That's in line with USE_NAMESPACE_MODULES and USE_SCHEMA_MODULES, which also define the module layout. The generator then already knows the type module when it builds paths (traits, references), so the generated paths are correct from the start and the token patching in render_trait_impls can go away. It also keeps RendererFlags::all() unchanged: right now it silently switches everyone who uses it to the new layout (mixed_choice_with_any had to be changed for exactly that reason).
  • Rename to_source to to_code_with_format to match the existing to_code.
  • New dependencies: prettyplease and syn with full are unconditional now. Please put them behind an optional feature (e.g. pretty), so build-script users who don't need formatting don't pay the compile time.
  • syn::Error in the public API makes syn a public dependency. Please return the crate's own error type instead.

Documentation

  • Please remove the README section. If you want to show how to use it, an example crate under examples/ is the better place.

Smaller things

  • module.rs: import ErrorKind instead of the inline std::io::ErrorKind::InvalidData path.

@sundermann
sundermann force-pushed the feat/type-output-layout branch from 6b6b0cc to dffab99 Compare November 11, 2026 18:54
@sundermann sundermann changed the title (feat): Output layout feat(generator): add type modules and optional formatting Nov 11, 2026
@sundermann

Copy link
Copy Markdown
Contributor Author

@Bergmann89 thanks for the review! Really appreciate it. Initially, I had stripped tests because I didn't want to make the PR huge. I've kept the current tests as is: GeneratorFlags::all() - GeneratorFlags::USE_TYPE_MODULES but that can be changed.

Especially for large schematas this would be an substantial improvement. It fixes IDE / rust analyzer limitations for huge files. I also have a PoC that takes the generated split output and distributes it into multiple crates with a public facing crate that reexports the API to speed up compilation times (as mentioned in #232).

@sundermann
sundermann force-pushed the feat/type-output-layout branch from dffab99 to bffabe6 Compare November 11, 2026 19:47

@Bergmann89 Bergmann89 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update and the context! The type modules look really good now: the generator computes the paths, and the new feature tests cover exactly what I was hoping for. And I can see the use case for large schemas, so let's get this in.

I added the small things as suggestions. Apart from that, two things are still open:

  • Unused imports in type modules. With USE_TYPE_MODULES, some type modules get usings they don't need (e.g. ErrorKind and RawByteStr in simple and union types). You can see it when building the benchmark with the flag (20 warnings). The snapshot tests don't show it because the modules have #![allow(unused_imports)]. Please only add a using when the code that needs it is actually emitted.
  • Benchmark and examples: benchmark/build.rs and the bpmn, read_write and vsme examples use GeneratorFlags::all() and silently switch to type modules now. Please subtract the flag there like in the tests, so their output stays as it is.

Nit: please add Error::FormattingError at the end of the enum instead of the beginning.

And I'd be very interested in your PoC for #232 as well! Splitting the generated code into multiple crates sounds like a great next step on top of this. Feel free to open an issue or a draft PR for it once this one is in.

Comment thread examples/type_modules/src/main.rs Resolved
Comment thread examples/type_modules/src/main.rs Resolved
Comment thread README.md Resolved
@sundermann
sundermann force-pushed the feat/type-output-layout branch from bffabe6 to 9bdcf74 Compare November 11, 2026 22:48
Assign type modules before building references and inheritance trait paths.
Add GeneratorFlags::USE_TYPE_MODULES and expose the owning module path in
DataType. Preserve namespace re-exports, inline content, and private XML
helpers while allocating collision-free file names.

Add compiled reference fixtures for dynamic types and substitution groups,
including namespace modules and quick-xml reads.
Add explicit Plain and Pretty output modes, with prettyplease and syn
enabled only by the optional pretty feature. Keep the public enum variants
available in both configurations and report a formatting error when Pretty
is selected without that feature.

Return crate-owned errors from both formatting entry points while preserving
the existing IO-only writer API. Demonstrate separate files and explicit
formatting in the type_modules example.
@sundermann
sundermann force-pushed the feat/type-output-layout branch from 9bdcf74 to 9a1de5b Compare November 11, 2026 23:00
@Bergmann89
Bergmann89 merged commit 10edca9 into Bergmann89:master Nov 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants