Skip to content

feat(http): sign text-transport requests and send AWS JSON media types - #2301

Merged
cevheri merged 3 commits into
mainfrom
feat/text-transport-signer
Nov 15, 2026
Merged

cevheri merged 3 commits into
mainfrom
feat/text-transport-signer

Conversation

@cevheri

@cevheri cevheri commented Nov 15, 2026

Copy link
Copy Markdown
Member

The shared text transport (createNodeTransport) can now sign each request, and can send a JSON body under AWS's JSON media types.
Both are opt-in. The one shipped provider whose requests change is Snowflake, through the raised signer cap below.
This is the transport groundwork an AWS JSON or Query API client needs: a SigV4 signer must see the exact Host, path, query, headers and body that go on the wire, and DynamoDB and Athena expect application/x-amz-json-1.0 and 1.1.

What changes in src/lib/db/http/node-transport.ts:

  • NodeTransportOptions.signer takes the byte transport's RequestSigner, checked when the transport is built the same way. It is called once per request, just before the write, inside its own try: a signer's own Error reaches the caller unchanged and nothing is sent. With a signer the transport sets Host itself, so the Host signed is the Host sent.
  • SigningInput.method adds "POST", and SigningInput.body is the exact payload text written, "" for none and always "" on the byte transport.
  • A signer's header values may be up to 8192 bytes on both transports, so an STS session token fits. A request's own headers stay at 1024. This is the transport half of D262, noted in docs/BACKLOG.md.
  • NodeTransportOptions.jsonContentType: application/json (the default when absent), application/x-amz-json-1.0 or application/x-amz-json-1.1. Any other value is refused when the transport is built. Form fields keep their own type.
  • The text transport now refuses a connection header given in two spellings ({ "X-Api": "a", "x-api": "b" }) with the byte transport's sentence, Invalid headers: x-api is named twice. It used to keep whichever came last. Its header record has no prototype, so a header named __proto__ is still sent as before rather than dropped.

User-visible change: a Snowflake programmatic access token longer than 1,017 characters (Studio reads up to 4,096) now reads result partitions.
Before, it signed in and submitted on the text lane, but every partition read on the byte lane failed with Invalid signature headers: the value of authorization must be visible ASCII or space, at most 1024 bytes, because the bearer goes through the byte transport's signer.
No other shipped provider changes what it sends.
The one new refusal reaches no current caller: git grep -l createNodeTransport -- src/lib/db/providers lists 11 files (couchbase, firestore, mtconnect, kubernetes, bigquery, databend, snowflake, rabbitmq, influxdb, qdrant, weaviate), and each one builds its connection headers from fixed lower-case names.
The only existing message that changes is the signer value refusal, which now reads "at most 8192 bytes" where it read "at most 1024 bytes".

Docs: the transport docblock, docs/DB_AUTH.md (a new section, "Transport facts the signer relies on"), docs/ADDING_A_PROVIDER.md (shared REST transport section), docs/ARCHITECTURE.md (the http/ line), D262 in docs/BACKLOG.md, and the S3_ACCESS_KEY_ID_MAX_CHARS rationale in src/lib/db/providers/objectstore/s3/constants.ts, which cited the old 1,024-byte rule.
Row 0.6 of docs/SECURITY.md stays accurate as written, so it is not changed.

Test plan

  • New tests/unit/db/http/node-transport-text-signer.test.ts, run red first. It checks that the signer receives the method, Host, path, query, headers and body exactly as the local server received them; that the signed Host is the Host sent, exactly once; and that a signer's Error is passed through with no socket opened. It checks that an 8192-byte x-amz-security-token passes on both transports and 8193 bytes is refused, and that a request header over 1024 bytes is still refused. It also covers jsonContentType (sent, defaulted, unknown values refused), the duplicate-spelling refusal, and a transport without the new options sending exactly the head lines it sent before, captured byte for byte from a raw listener.
  • tests/unit/db/snowflake/transport.test.ts reads a partition with a 4,096-character programmatic access token through the real transports against a raw listener; it fails at the old 1024-byte cap.
  • Existing tests that build a SigningInput gain body: "". The byte transport and S3 wire tests move the signer cap case from 1025 to 8193 bytes.
  • bun tests/run-tests.ts tests/unit/db/http tests/unit/db/s3 tests/unit/db/snowflake tests/unit/db/influxdb tests/unit/db/firestore: 194 files pass.
  • Gate: format, lint, typecheck, knip, chart:check, channels:showcase:check, readme:check, security:check, test, build; test:coverage and coverage:check; build:lib, attw, lib:closure; the VS Code extension build.

The text transport takes the byte transport's RequestSigner as an opt-in:
checked when built, called once per request just before the write with the
exact method, Host, path, query, headers and body, Host set explicitly so the
Host signed is the Host sent. SigningInput gains POST and body.

Signer header values may be up to 8192 bytes on both transports so a session
token fits; a request's own headers keep 1024 (the transport half of D262).
jsonContentType sends a JSON body as application/x-amz-json-1.0 or 1.1, and
any other value is refused. A connection header in two spellings is now
refused on the text transport, as on the byte transport.
Address review round 1: keep a __proto__ connection header on the text
transport, pin the Snowflake partition read with a long access token,
and correct the jsonContentType, SigningInput and S3 key-length wording.
Add the docs/DB_AUTH.md section on the transport facts a request signer relies on (plan section 4, A2 row).
@codecov

codecov Bot commented Nov 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@sonarqubecloud

Copy link
Copy Markdown

@cevheri
cevheri merged commit 49fa037 into main Nov 15, 2026
39 checks passed
@cevheri
cevheri deleted the feat/text-transport-signer branch November 15, 2026 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant