Repository navigation
feat(http): sign text-transport requests and send AWS JSON media types - #2301
Merged
Merged
Conversation
The text transport takes the byte transport's RequestSigner as an opt-in: checked when built, called once per request just before the write with the exact method, Host, path, query, headers and body, Host set explicitly so the Host signed is the Host sent. SigningInput gains POST and body. Signer header values may be up to 8192 bytes on both transports so a session token fits; a request's own headers keep 1024 (the transport half of D262). jsonContentType sends a JSON body as application/x-amz-json-1.0 or 1.1, and any other value is refused. A connection header in two spellings is now refused on the text transport, as on the byte transport.
Address review round 1: keep a __proto__ connection header on the text transport, pin the Snowflake partition read with a long access token, and correct the jsonContentType, SigningInput and S3 key-length wording.
Add the docs/DB_AUTH.md section on the transport facts a request signer relies on (plan section 4, A2 row).
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



The shared text transport (
createNodeTransport) can now sign each request, and can send a JSON body under AWS's JSON media types.Both are opt-in. The one shipped provider whose requests change is Snowflake, through the raised signer cap below.
This is the transport groundwork an AWS JSON or Query API client needs: a SigV4 signer must see the exact Host, path, query, headers and body that go on the wire, and DynamoDB and Athena expect
application/x-amz-json-1.0and1.1.What changes in
src/lib/db/http/node-transport.ts:NodeTransportOptions.signertakes the byte transport'sRequestSigner, checked when the transport is built the same way. It is called once per request, just before the write, inside its own try: a signer's ownErrorreaches the caller unchanged and nothing is sent. With a signer the transport sets Host itself, so the Host signed is the Host sent.SigningInput.methodadds"POST", andSigningInput.bodyis the exact payload text written,""for none and always""on the byte transport.docs/BACKLOG.md.NodeTransportOptions.jsonContentType:application/json(the default when absent),application/x-amz-json-1.0orapplication/x-amz-json-1.1. Any other value is refused when the transport is built. Form fields keep their own type.{ "X-Api": "a", "x-api": "b" }) with the byte transport's sentence,Invalid headers: x-api is named twice. It used to keep whichever came last. Its header record has no prototype, so a header named__proto__is still sent as before rather than dropped.User-visible change: a Snowflake programmatic access token longer than 1,017 characters (Studio reads up to 4,096) now reads result partitions.
Before, it signed in and submitted on the text lane, but every partition read on the byte lane failed with
Invalid signature headers: the value of authorization must be visible ASCII or space, at most 1024 bytes, because the bearer goes through the byte transport's signer.No other shipped provider changes what it sends.
The one new refusal reaches no current caller:
git grep -l createNodeTransport -- src/lib/db/providerslists 11 files (couchbase, firestore, mtconnect, kubernetes, bigquery, databend, snowflake, rabbitmq, influxdb, qdrant, weaviate), and each one builds its connection headers from fixed lower-case names.The only existing message that changes is the signer value refusal, which now reads "at most 8192 bytes" where it read "at most 1024 bytes".
Docs: the transport docblock,
docs/DB_AUTH.md(a new section, "Transport facts the signer relies on"),docs/ADDING_A_PROVIDER.md(shared REST transport section),docs/ARCHITECTURE.md(thehttp/line), D262 indocs/BACKLOG.md, and theS3_ACCESS_KEY_ID_MAX_CHARSrationale insrc/lib/db/providers/objectstore/s3/constants.ts, which cited the old 1,024-byte rule.Row 0.6 of
docs/SECURITY.mdstays accurate as written, so it is not changed.Test plan
tests/unit/db/http/node-transport-text-signer.test.ts, run red first. It checks that the signer receives the method, Host, path, query, headers and body exactly as the local server received them; that the signed Host is the Host sent, exactly once; and that a signer'sErroris passed through with no socket opened. It checks that an 8192-bytex-amz-security-tokenpasses on both transports and 8193 bytes is refused, and that a request header over 1024 bytes is still refused. It also coversjsonContentType(sent, defaulted, unknown values refused), the duplicate-spelling refusal, and a transport without the new options sending exactly the head lines it sent before, captured byte for byte from a raw listener.tests/unit/db/snowflake/transport.test.tsreads a partition with a 4,096-character programmatic access token through the real transports against a raw listener; it fails at the old 1024-byte cap.SigningInputgainbody: "". The byte transport and S3 wire tests move the signer cap case from 1025 to 8193 bytes.bun tests/run-tests.ts tests/unit/db/http tests/unit/db/s3 tests/unit/db/snowflake tests/unit/db/influxdb tests/unit/db/firestore: 194 files pass.format,lint,typecheck,knip,chart:check,channels:showcase:check,readme:check,security:check,test,build;test:coverageandcoverage:check;build:lib,attw,lib:closure; the VS Code extension build.