Repository navigation
*: add text annotations and their go and ruby codecs - #17
Draft
AlexisMontagne wants to merge 21 commits into
Draft
AlexisMontagne wants to merge 21 commits into
AlexisMontagne wants to merge 21 commits into
Conversation
Recent libc++ compares iterators with == when building a map from a range, so the compiler failed to build on macOS with only != defined.
…tations to plugins Structured annotations were always serialized as an empty list, so plugins could not read them. Their values are now converted to types.value.Value, following the type of the annotation.
Enum values only accepted legacy annotations, so plugins could not attach typed metadata to them, e.g. the text form of a value.
Runtime consumers of the annotations, such as the text codec, need the annotations of the enum values and whether a struct field is optional.
…initions Every enum gets an EnumDefinition method holding the annotations of the enum and of its values, and the field definitions of the structs hold their requiredness, so libraries can read them without code generation.
…es and annotations Enum modules now hold their name, namespace, annotations and the annotations of their values. THRIFT_NAMES keeps the names as declared since VALUE_MAP capitalizes them, so the ruby and go libraries derive the same texts from them.
Describes the canonical text form of structs and enum values in the IDL, e.g. "company:42" for an ownership, so it is defined once instead of being hand written in every language.
Encodes the structs annotated with Format and the enums as their text form, with wrappers for JSON, SQL and flags. The test cases are shared with the ruby library so both accept and produce exactly the same texts.
Mirrors the go text codec: Thrift::Types::Annotation::Text dumps and loads the structs annotated with Format and the enums, checked against the test cases shared with the go library.
Generates the test types in the go and ruby test pipelines, resolving the annotations as standard types like the code generated by consumers does.
…alue annotations Tools consuming the JSON descriptors, such as tcurl, need the annotations of the enum values, e.g. text.Name, to encode them like the libraries do.
Every consumer of the structured annotations looped over them with a type assertion, the text codec and the naming extractor now share the helper.
gocodegen and the text codec each kept a copy of the revive list, in different shapes. The canonical spelling also removes the IDs special case of gocodegen.
…tting words The acronym rule split UserIDs into "User I Ds", so the snake case of an enum value named after IDs was "user_i_ds". Go and ruby now keep the initialisms ending in lower case whole, the caser always applying the Go initialisms.
Plugins receive the structured annotations as values, they can now decode them into the generated annotation types instead of reading every field by hand.
Gives plugins the definitions the libraries read at runtime, with the structured annotations decoded into their generated types, so they can validate them with the same code, e.g. text.CompileEnum.
Thrift structs render as an object with ActiveSupport and as their inspection with Oj in compat mode, as used by userializer. Including the module renders a struct as its text form with both, without changing the JSON of the structs that did not opt in.
…tional fields
A struct field annotated with Format can be referenced by a pattern, its
pattern being inlined, so composite identifiers such as crm-server's entity
segments ("production:list:42:selected") reuse the text form of the entity.
Optional fields can be left out of the pattern, for structs carrying hints
such as the handle of a media.
…t optional fields Mirrors the go codec, dump and load building the nested structs, checked against the shared test cases.
Callers building the field types of CompileStruct, such as the plugins, only need to resolve the referenced fields, skipping the optional fields left out of the pattern, which may reference the struct itself.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Typed values like ownerships (
company:42) and credential references (shopify:7) are turned into strings and back by code written by hand in each language:baseutilin Go,identity-thrift-utilsandbase-thrift-utilsin Ruby. Those copies have drifted apart. This PR describes the text form once, in the IDL, with structured annotations, and adds Go and Ruby codecs that read them at runtime. Both codecs are checked against the same test cases, so they accept and produce exactly the same texts.Annotations:
types/annotation/text.thriftFormattakes a python-like f-string pattern made of{field}placeholders and literal separators, with{{and}}as escaped braces. Format specs and conversions are not supported.Nameoverrides the text of a single enum value.EnumFormatsets the defaults for an enum: aCase,trim_prefixes/trim_suffixes(the first match in list order is trimmed), andaccept_thrift_names, which also accepts the thrift names such asType_Company.IDswhole. Camel and Pascal case render the Go initialisms.Compiler:
thrifty.yy).t_serializer.h).func (T) EnumDefinition() thrift.EnumDefinitionholding the annotations of the enum and of its values.Requiredness.NAME,NAMESPACE,THRIFT_NAMES(the names as declared, sinceVALUE_MAPcapitalizes them), the enum annotations, andVALUE_LEGACY_ANNOTATIONS/VALUE_STRUCTURED_ANNOTATIONS.TEnumIteratorgainsoperator==, which fixes the build with recent libc++ on macOS.Go:
lib/go/thrift/types/annotation/textMarshalStruct,UnmarshalStruct[T],MarshalEnumandUnmarshalEnum[E].Struct[T]andEnum[E]wrappers for JSON, SQL and flags.Validatereports invalid annotations, for use in tests.CompileStructandCompileEnumwork from definitions only, for dynamic callers such as tcurl or an MCP gateway.internal/goinitialismis now the single Go initialisms list, shared withgocodegen, whose output doesn't change.Ruby:
Thrift::Types::Annotation::Text, loaded byrequire 'thrift'dump/load,dump_enum/load_enumandvalidate.ArgumentError.Plugin helpers:
thrift.FindAnnotations[T].value.DecodeValue, which decodes a structured annotation value into its generated type.Runtime()onAnnotationDefinition,EnumDefinitionandStructDefinition, which convert plugin definitions into the runtime ones. With these, a plugin can validate annotations withtext.CompileEnum/text.CompileStruct.Tests:
test/TextTestCases.jsonholds the cases shared by the Go tests and the Ruby spec.test/TextTest.thriftis generated by thelib/go/testandlib/rbtest pipelines. Both useTHRIFT_TYPES_PATH, so the generated code imports the annotations the way consumer code does.Did you create an Apache Jira ticket? (not required for trivial changes)
If a ticket exists: Does your pull request title follow the pattern "THRIFT-NNNN: describe my issue"?
Did you squash your changes to a single commit? (not required, but preferred)
Did you do your best to avoid breaking changes? If one was needed, did you label the Jira ticket with "Breaking-Change"?
If your change does not involve any code, add
[skip ci]at the end of your pull request to free up build resources.Notes:
EnumDefinition(), Go struct fields getRequiredness, Ruby enums get the new constants, and JSON enum values get annotation keys.NAME,NAMESPACEorTHRIFT_NAMESwould collide with the new constants.textstubs are new; the other stubs underlib/(rpc,program_definition…) will gainEnumDefinition()on their next regeneration.master:json_protocol_spec.rb"pretty print union" and "pretty print union with a container field". They fail the same way with code from the released v2.7.14 compiler.types/annotation/text.thriftand drops itstext-codecplugin.baseutil,identity-thrift-utilsandbase-thrift-utilsserializers delegate to the codecs.