Repository navigation
Conversation
igbinary decodes into the PHP serialized tree, numbered as serialize() numbers it. Avro decodes with a schema file chosen per column, stored by a hash of its text.
…formats readable by 0.83 Fixes the review of TableProApp#3962: igbinary r:/R: numbering, copies, caps and float text; Avro framing, decimal width, named logical types and zero-byte caps; a second storage key for formats 0.83 lacks; schema files instead of defaults.
… whose schema comes back
…inary Serializable payloads
…tore and an open viewer
…n, keep byte property names
…ema shows from the next load
…hemas from a Schema Registry Display As > Protobuf Schema… reads a .proto file or descriptor set and shows field names and declared types. Display As > Schema Registry fetches the Avro, Protobuf or JSON schema each Kafka value names by its wire-format prefix, from the registry set on the connection.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Builds on #3962 (igbinary and Avro), which is still open: this branch starts from it because it needs that PR's Avro decoder and schema file storage. Until #3962 merges, its commits show in this diff too; the new work is the last commits.
.protofile or a descriptor set (protoc --descriptor_set_out), then asks which message when the file has several. The tree names each field and badges it with its declared type. Text matchesprotoc --decode; fields the schema does not name, or names with another wire type, show by number. A singular message sent twice is merged, as a parser does.GET /schemas/ids/{id}, with its references from/subjects/{subject}/versions/{version}(Avro and Protobuf), and kept for the session. Protobuf picks the message by the frame's message index. JSON Schema values open in the JSON tree.AvroSchemaStore's file handling moved to a genericSchemaFileCache;ProtobufSchemaStorekeepsProtobufSchemas/<hash>.pbschemathe same way, swept with the Avro ones..protofile is read alone (its imports are not opened), so a field of an imported type shows by number; a descriptor set built with--include_importscovers that case. The registry is reached directly, not through the connection's SSH tunnel.Test plan
ProtobufSchemaDecoderTests: fixtures written byprotoc --encode(libprotoc 36.2), compared withprotoc --decodeoutput, for a.protofile and the descriptor set protoc built from it: every scalar type, enums, packed and unpacked repeats, maps, oneof, nesting, a proto2 group, unknown fields, merged messages, the packed-field budget, broken values and bad schema files.SchemaRegistryTests: responses recorded fromconfluentinc/cp-schema-registry7.7.1, values written by confluent-kafka 2.6.1'sAvroSerializerandProtobufSerializeragainst it. Avro, Protobuf with a reference and message index [0] and [1], basic auth, the 404 reason, Avro references, broken values, the cache's single fetch, its notification and kept failure.ValueDisplayFormatTests,BinaryDisplayFormatTests,ValueDisplayFormatStorageTestsfor the new stored formats, the cell and the schema sweep.protoc --decodeline for line. Hosted tests compile here (build-for-testing), and run in CI.Docs: TableProApp/docs#155 (draft, merges after this one).
Part of #3607