Skip to content

fix: canonicalize type OIDs embedded in binary array and composite values - #1515

Draft
rlittlefield wants to merge 1 commit into
pgdogdev:mainfrom
rlittlefield:oid-payload-rewrite
Draft

fix: canonicalize type OIDs embedded in binary array and composite values#1515
rlittlefield wants to merge 1 commit into
pgdogdev:mainfrom
rlittlefield:oid-payload-rewrite

Conversation

@rlittlefield

@rlittlefield rlittlefield commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Closes a gap in canonicalize_type_information: rewriting RowDescription / ParameterDescription / Parse is not enough for arrays and composites. Binary array values embed the element type OID and binary composite (and anonymous record) values embed the OID of every field. Postgrex decodes arrays in binary format and checks the element OID against its type cache, so an array of a custom enum coming from a shard with different OIDs fails to decode even though the RowDescription was canonicalized.

Independent of #1514 (which handles types created after PgDog loaded its mappings); both apply cleanly on main.

Behavior:

  • Type kinds (typtype, typcategory, typelem, typbasetype) are loaded from pg_type alongside the OIDs, for both the shard and the canonical set.
  • The backend PreparedStatements tracks bound portals per server connection (Bind → Execute, matched by portal name; suspended portals are kept for re-execution). For the executing portal, DataRow columns requested in binary format whose type is an array/composite/record have the embedded OIDs rewritten shard → canonical, recursing through nested values and looking through domains. Column types come from the portal's own Describe response (shard OIDs) or the cached RowDescription of the statement (canonical, mapped back).
  • Bind parameters in binary format of array/composite/record types are rewritten canonical → shard. Parameter types come from a per-statement ParameterDescription cache (new, mirrors the RowDescription cache) or the types the client declared in Parse.
  • Text-format values, builtin types, and shards with no OID drift are untouched; malformed payloads are logged and left as-is. The per-row work is a precomputed plan per portal, so statements without such columns pay a couple of branches per row.

Testing: unit tests cover the payload rewriter (arrays, empty arrays, composites with nested arrays, arrays of composites, anonymous records, domains over arrays, malformed input), type kind classification, ParameterDescription caching, Bind rewrites (from ParameterDescription and from Parse data types), DataRow rewrites via both the portal Describe and the cached RowDescription, suspended portals, and text format being left alone. integration/rust gains test_oid_drift_arrays in cross_shard_oid_drift.rs: an enum array round-trips through every shard with the element OID inside the binary payload checked against shard 0's. Ran locally against a two-shard setup; cargo fmt and cargo clippy are clean.

@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.08108% with 29 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pgdog/src/backend/prepared_statements.rs 94.14% 23 Missing ⚠️
pgdog/src/backend/pool/shard/oids/payload.rs 97.50% 6 Missing ⚠️

📢 Thoughts on this report? Let us know!

@rlittlefield
rlittlefield force-pushed the oid-payload-rewrite branch 3 times, most recently from 37d1f52 to 7a25da1 Compare September 10, 2026 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant