Skip to content

fix(client): redact the api key from error bodies - #37

Open
guyposka wants to merge 1 commit into
ScrapingBee:mainfrom
guyposka:fix/redact-key-in-error-bodies
Open

guyposka wants to merge 1 commit into
ScrapingBee:mainfrom
guyposka:fix/redact-key-in-error-bodies

Conversation

@guyposka

Copy link
Copy Markdown

a wrong or revoked key gets printed in full by most commands

SCRAPINGBEE_API_KEY=<wrong key> scrapingbee usage
API returned status 401: {"message":"Invalid api key: <the full key>"}

same for scrape and the other commands (stderr), auth, and batch .err files. the api echoes the key in its 401 body and the cli passes the body through. a typo'd key is usually one char off the real one, and this ends up in terminals, ci logs and agent transcripts (the skill itself says never print the key)

fix: Client._get and Client._request replace the key with [redacted] in any 4xx/5xx body, so every command that goes through Client gets it without touching each error path. success bodies are left alone. the key is stripped first since the api drops trailing whitespace when it echoes it. crawl goes through scrapy (key in the url) and isn't touched here

  • 4 tests against a local fake api (get + post path, trailing space, success untouched), the 401 ones fail without the change
  • pytest -m "not integration" and ruff pass
  • checked live with a fake key: usage and scrape both print Invalid api key: [redacted]

probably worth dropping the key from the 401 message on the api side too, this just covers the cli

the api answers a wrong key with 401 "Invalid api key: <key>" and every
command prints error bodies to stderr (batch writes them to .err files),
so the full key ended up in terminals, logs and agent transcripts

the client now replaces the key with [redacted] in any 4xx/5xx body, in
both _get and _request, so every command gets it for free
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant