Repository navigation
fix(rust): deserialize DateTime values in generated code - #320
NopeCharon wants to merge 1 commit into
Conversation
Rust generated by RustVisitor could not deserialize any Concerto JSON that
contains a DateTime, including the $timestamp of every transaction and
event. There were two separate problems.
Every parse call in the generated utils.rs used
DateTime::parse_from_str(s, "%Y-%m-%dT%H:%M:%S%.3f%Z"). chrono cannot
parse %Z, so each call failed with "input is not enough for unique date
and time", even for the strings the generated serialize_datetime ("%+")
writes. All 11 calls (single values, arrays and the HashMap helpers) now
use DateTime::parse_from_rfc3339. It accepts the ISO 8601 strings that
Concerto uses, with a Z or numeric offset and any number of fractional
digits.
Optional DateTime, DateTime[] and DateTime-keyed or DateTime-valued map
fields use deserialize_with. serde only treats a missing Option field as
None when it deserializes the field itself, so JSON that left out such a
property failed with "missing field". Concerto omits optional properties
that have no value, so this hit ordinary data. visitField now adds
`default` to the serde attributes of those fields.
The cargo check verification tests only compile the generated code, so
they did not catch either problem. A new verification test generates a
transaction with required, optional, array and map DateTime properties,
then builds and runs it with cargo to deserialize Concerto JSON with and
without the optional properties, and to read back what it serializes. It
is skipped when cargo is not installed. Unit tests cover the new
`default` attribute, and the snapshots are updated for the new parse
calls.
Closes accordproject#319
Signed-off-by: NopeCharon <115794254+NopeCharon@users.noreply.github.com>
|
Thanks for the fix for DateTime deserialisation in the generated Rust code. It appears to address #319. The maintainers requested as reviewers (jamieshorten, sanketshevkar) will look at it and comment here. We don't have a date for that yet. Drafted by an Accord Project agent, reviewed by a maintainer. |
There was a problem hiding this comment.
🟢 Approval recommended
There are no active review issues for the Rust DateTime deserialization and optional-field handling changes.
No open issues 🎉
What changed in this PR
Fixes #319 in the Rust code generator so generated code can deserialize Concerto DateTime values and accept omitted optional DateTime properties.
Changes:
- Uses RFC 3339 parsing across all generated DateTime deserializers.
- Adds serde defaults for optional DateTime fields, arrays, and maps.
- Updates snapshots and adds unit tests plus a Cargo runtime test for deserialization and round-trip behavior.
| File | Description |
|---|---|
test/verification/rust.compile.test.js |
Adds runtime verification for DateTime parsing, omitted fields, and round trips. |
test/codegen/fromcto/rust/rustvisitor.js |
Tests optional-field defaults and updates parser expectations. |
test/codegen/__snapshots__/codegen.js.snap |
Updates the 11 generated DateTime parsing calls. |
lib/codegen/fromcto/rust/rustvisitor.js |
Generates RFC 3339 parsing and defaults for optional DateTime fields. |
Review effort: Balanced
There was a problem hiding this comment.
Thanks for this fix, and for the clear description. It resolves both causes in #319. The generated utils.rs now parses all 11 DateTime sites with parse_from_rfc3339, which accepts Concerto's ISO 8601 strings and matches what serialize_datetime writes. Optional DateTime, DateTime-array and DateTime-keyed or -valued map fields now get default, and not the fields that serde already defaults. The unit tests check the attribute order, the snapshot changes follow from the 11 parse calls, and the new cargo run test exercises required, optional, array and map values with Z and +01:00 offsets, omitted optionals and a round trip. I found nothing that needs changing. The overlap with #309 and the test-file conflict with #318 are noted in your description and are sensible follow-ups. It is ready from my side once a maintainer approves.
No problems found.
Automated review by Mira, an Accord Project agent, at dcff02e. Findings are labelled blocking, should-fix or nit; a maintainer makes the final call.
Closes #319
Generated Rust couldn't deserialize any Concerto JSON containing a
DateTime, including the$timestampthat every transaction and event has. This PR fixes both causes reported in #319.Changes
addUtilsModelFile: all 11 parse calls in the generatedutils.rsnow useDateTime::parse_from_rfc3339(s)instead ofDateTime::parse_from_str(s, "%Y-%m-%dT%H:%M:%S%.3f%Z"). chrono can't parse%Z, so every call used to fail withinput is not enough for unique date and time. RFC 3339 accepts the ISO 8601 strings Concerto uses (Zor a numeric offset, any number of fractional digits) and the generatedserialize_datetimeoutput (%+).visitField: optionalDateTime,DateTime[]and DateTime-keyed or DateTime-valued map fields getdefaultin their serde attributes. These fields usedeserialize_with, so withoutdefaultserde rejects JSON that leaves them out (missing field `shipped`). Concerto omits optional properties that have no value.defaulton optional DateTime and DateTime-array fields, and on optional DateTime maps.test/verification/rust.compile.test.jsgenerates a transaction with required, optional, array and map DateTime properties, then builds and runs it withcargo run. It deserializes Concerto JSON with and without the optional properties (includingZand+01:00offsets), checks the parsed values, and reads back what the generated code serializes. Like the other Rust verification tests, it is skipped when cargo is not installed. The existingcargo checkcases only compile the output, which is why neither problem was caught.Flags
%+(for example2024-01-02T03:04:05.678+00:00), which is valid ISO 8601.visitFieldto take the optional flag of scalar properties. If it merges first, the two newfield.isOptional?.()checks here should use that flag, so optionalscalar ... extends DateTimeproperties getdefaulttoo. I'm happy to rebase either PR.test/verification/rust.compile.test.js, so the merge conflict there is trivial.What I ran:
npm test: lint, licence check and JSDoc pass. 699 tests pass, 59 pending. The 8 Avro verification failures are from my machine's Java 8, which is too old for avro-tools; they are unrelated to this change.npx mocha test/codegen/fromcto/rust/rustvisitor.js test/verification/rust.compile.test.js test/codegen/codegen.jswith cargo 1.100 installed: all pass, including the new cargo test. With the visitor change reverted, the new unit tests fail and the cargo test fails withinput is not enough for unique date and time.Screenshots or Video
N/A
Related Issues
Author Checklist
--signoffoption of git commit.mainfromfork:branchname