Add hand-written baseten.sandbox surface

[marius-baseten]

🚀 What

• Adds baseten.sandbox, a hand-written SDK surface for Baseten sandboxes over the generated clients from 💥 Add sandbox API client and split query params from request bodies #21, covering the control plane (/v1/sandboxes) and each sandbox's execution API
• Sync and async twin clients throughout, on httpx

💻 How

• SandboxClient / AsyncSandboxClient: create, get, get_info, sandbox_from_url, list, delete, library_images
• Sandbox / AsyncSandbox: fs (text read/write) and process (exec, list, logs); api reaches the generated execution client for anything unwrapped
• API key exchanged at /v1/token for a cached, refreshed token used on both planes; token_provider overrides
• Generated clients reachable as api / sandbox_api on the client

🔬 Testing

• Unit: a routing transport serving both planes, covering token minting and revocation, retries, pagination, and wire shapes
• Parity: every translated generated type is pinned, so a new server field fails until the surface forwards it or lists it under an explicit exception
• E2E (env-gated): lifecycle and exec against a live environment

Important review notes and deviations (running list of decisions)

1. API shape
   1. Matches the JS and Go SDKs' behavior with Python conventions: keyword-only options, sync and async twins, Optional meaning "leave unset" on wire types. The Go and JS SDK PRs are the source of what to build; this list tracks the same decisions here
   2. create/get return a Sandbox handle (name, URL); get_info/list return data only. create also carries info, the record as of creation
   3. No public Sandbox constructor: sandbox_from_url builds one without a call, so a sandbox always shares its client's authentication, connections, and retry budgets, and closes with it
   4. get_info takes show_secrets; the server masks values for callers without the workspace administrator role even then
   5. create sends only what the caller set; the server owns defaults and conflicts (create_if_not_exists returns the existing live sandbox or recreates a dying one). create fails loudly when the served record carries no URL, which the served spec says is always present
   6. SandboxInfo has no state and no display_name: the served spec drops both. create takes create_if_not_exists as a flag, not a tri-state
   7. list is a lazy generator; a repeated server cursor errors rather than paging forever
   8. Env values default to secret=True, matching the server's default; secret=False opts out and unmasks the value in responses. The previous draft's False default would have unmasked everything
   9. Execution-plane timestamps parse an explicit layout list (ISO 8601, RFC 1123) and fail clearly on anything else
   10. library_images drops nothing client-side: the served endpoint already excludes hidden and coming-soon entries. Creation options are flattened to creation_extra_args and creation_volumes, documented as suggestions since create cannot pass volumes or extra args yet
   11. Not implemented yet: update, the image client (push, wait, logs, tags, cleanup), process breadth (exec_stream, wait, stop, kill, stdin), filesystem breadth (bytes, multipart, list, mkdir, find, grep, copy, write_tree, watch), drives, codegen, system, network. api and sandbox_api reach them
2. Errors and retries
   1. SandboxApiError on both planes, SandboxGatewayError for edge 502/503/504 on the execution API
   2. Control-plane calls are not retried. Execution-plane calls retry idempotent ones only, with the same budgets as Go and JS: reads 5, gateway 2, uploads 3, settable via SandboxRetryOptions
   3. A revoked token (401 plus x-blaxel-error-code: TOKEN_REVOKED) is re-minted and the request resent; later resends wait 1s past the revocation's rounded-up cutoff second, since a token minted right after the revoking event can still fall before it. Past two resends the rejection is returned
   4. One token exchange is bounded by 30s, so a stalled mint is not what every caller waits on
3. Transport
   1. httpx, HTTP/2 when the optional h2 package is installed (http2=None auto-detects). The default timeout bounds the connect time only: exec with wait_for_completion holds the response open until the command exits, so a response-header timeout would fail every long command

[CLAassistant]

All committers have signed the CLA.

[Copilot]

Copilot review overview

🟡 Changes recommended

Authentication exhaustion, gateway retries, error conversion, resource cleanup, and forward-compatible statuses have unresolved correctness issues.

Review effort: Balanced
Findings: 13

Open (13)

• Lock the complete token get-or-mint sequence · New
• Preserve the final revoked-token response · New
• Omit unset name instead of sending null · New
• Handle created sandboxes without an execution URL · New
• Apply configured timeout to async token minting · New
• Extract execution error code from body_error · New
• Dump ErrorResponse before constructing SandboxApiError · New
• Accept unknown sandbox status and state values · New
• Accept unknown execution status values · New
• Close the owned synchronous transport pool · New
• Close the owned asynchronous transport pool · New
• Convert gateway errors inside retry attempts · New
• Convert async gateway errors before retry handling · New

What changed in this PR

Adds a hand-written synchronous and asynchronous sandbox SDK over the generated control- and execution-plane clients.

Changes:

• Adds sandbox lifecycle, authentication, retries, errors, and curated models.
• Implements filesystem read/write and process execution.
• Adds API documentation and routing-based tests.

File	Description
baseten/​sandbox/​__init__.py	Exports the public sandbox API.
baseten/​sandbox/​_auth.py	Implements token caching and revocation handling.
baseten/​sandbox/​_client.py	Adds sync and async control-plane clients.
baseten/​sandbox/​_errors.py	Adds curated API error types.
baseten/​sandbox/​_info.py	Defines curated sandbox records.
baseten/​sandbox/​_retry.py	Adds transient retry policies.
baseten/​sandbox/​_sandbox.py	Implements execution-plane wrappers.
scripts/​docgen/​conf.py	Configures sandbox API documentation.
scripts/​docgen/​index.rst	Adds sandbox documentation navigation.
tests/​sandbox/​test_surface.py	Tests the new public surface.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[cretz]

I am holding off reviewing this because I think it's a bit premature. basetenlabs/baseten-js#23 I think needs to get further along first, and then can be used as inspiration for this PR.

[Copilot]

Copilot review overview

🟡 Changes recommended

Authentication retry exhaustion, gateway retries, error conversion, request omission, and host routing contain correctness issues.

Review effort: Balanced
Findings: 16 · 1

Open (17)

• Pass configured timeout to token-mint client · New
• Expose sandbox API when a base URL override is configured · New
• Retry generated gateway ResponseError responses · New
• Convert async gateway errors before retry handling
• Convert gateway errors inside retry attempts
• Close the owned asynchronous transport pool
• Close the owned synchronous transport pool
• Accept unknown execution status values
• Accept unknown sandbox status and state values
• Dump ErrorResponse before constructing SandboxApiError
• Extract execution error code from body_error
• Apply configured timeout to async token minting
• Handle created sandboxes without an execution URL
• Omit unset name instead of sending null
• Preserve the final revoked-token response
• Lock the complete token get-or-mint sequence
• Annotate async constructor with async token providers · New

[Copilot]

Copilot review overview

🔵 Needs a closer look

The advertised HTTP/2 installation path is unavailable, and list handling has input and pagination correctness issues.

Review effort: Balanced
Findings: None

Resolved since last review (17)

• Retry generated gateway ResponseError responses
• Expose sandbox API when a base URL override is configured
• Pass configured timeout to token-mint client
• Convert async gateway errors before retry handling
• Convert gateway errors inside retry attempts
• Close the owned asynchronous transport pool
• Close the owned synchronous transport pool
• Accept unknown execution status values
• Accept unknown sandbox status and state values
• Dump ErrorResponse before constructing SandboxApiError
• Extract execution error code from body_error
• Apply configured timeout to async token minting
• Handle created sandboxes without an execution URL
• Omit unset name instead of sending null
• Preserve the final revoked-token response
• Lock the complete token get-or-mint sequence
• Annotate async constructor with async token providers

Previously missed (2)

In code that hasn't changed since last review

Add the missing http2 package extra or document installing h2

baseten/​sandbox/​_auth.py:50

The installation command in this error does not work: pyproject.toml defines no http2 optional dependency, so pip install baseten[http2] will not install h2. Add the advertised package extra (or direct users to install h2) so http2=True has an actionable failure path.

Reject bare status strings in synchronous API

baseten/​sandbox/​_client.py:327

str satisfies Sequence[str], so the public annotation accepts statuses="DEPLOYED"; the comprehension then sends one status per character and the API rejects the request. Reject a bare string before constructing the query (and keep accepting actual sequences).

This issue also appears in the following locations of the same file:

• line 349
• line 598
• line 626

[Copilot]

Copilot review overview

🟡 Changes recommended

Async token minting can remain unbounded, and several advertised retry budgets are not applied.

Review effort: Balanced
Findings: 1 · 2 · 1

Open (4)

• Bound async token exchange with the 30-second mint timeout · New
• Retry synchronous process list and logs requests · New
• Apply upload retry budget to synchronous file replacement · New
• Export LibraryImageVolume from the public sandbox API · New

[Copilot]

Note

Copilot was unable to run its full agentic suite in this review.

Copilot review overview

Review effort: Lite
Findings: 3 · 4 · 1

Open (8)

• In the async transport, calling request.read() can break when the request body is backed by an… · New
• SandboxApiError.code extraction is incorrect: the fallback to body_error is gated on… · New
• datetime.fromisoformat() does not accept a trailing Z in Python versions prior to 3.11, but the… · New
• Sandbox advertises context-manager support but __exit__ is a no-op. This is misleading because… · New
• The async filesystem surface (AsyncSandboxFileSystem.read/write) introduces… · New
• Apply upload retry budget to synchronous file replacement
• Retry synchronous process list and logs requests
• Export LibraryImageVolume from the public sandbox API

Resolved since last review (1)

• Bound async token exchange with the 30-second mint timeout

[Copilot]

Copilot review overview

🟡 Changes recommended

Authentication eagerly buffers potentially large uploads, and live tests race sandbox deployment readiness.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• Eagerly buffers synchronous request bodies in memory · New
• The async filesystem surface (AsyncSandboxFileSystem.read/write) introduces…

Resolved since last review (7)

• datetime.fromisoformat() does not accept a trailing Z in Python versions prior to 3.11, but the…
• SandboxApiError.code extraction is incorrect: the fallback to body_error is gated on…
• In the async transport, calling request.read() can break when the request body is backed by an…
• Sandbox advertises context-manager support but __exit__ is a no-op. This is misleading because…
• Apply upload retry budget to synchronous file replacement
• Retry synchronous process list and logs requests
• Export LibraryImageVolume from the public sandbox API

Previously missed (7)

In code that hasn't changed since last review

Advertised http2 extra does not install the h2 dependency

baseten/​sandbox/​_auth.py:59

The suggested installation command cannot work because this package defines no http2 optional dependency in pyproject.toml; pip will not install h2 for baseten[http2]. Point users to an installation command that actually provides h2, or add the advertised extra.

Second-resolution sandbox names collide across concurrent runs

tests/​sandbox/​test_sandbox_e2e.py:36

Rounding wall-clock time to one second does not produce a unique sandbox name. Concurrent E2E runs can create the same resource, and each test's unconditional cleanup may then delete another run's sandbox. Use a UUID or equivalent random suffix, as the existing E2E suite does in tests/test_e2e.py:144.

Test assumes deployment completes before execution calls

tests/​sandbox/​test_sandbox_e2e.py:54

The served create contract explicitly allows deployment to continue after the response (scripts/apigen/specs/management.json:10632), so create() may return DEPLOYING. This strict assertion and the immediate execution calls make the live test flaky; poll get_info() with a bounded timeout until DEPLOYED before using the execution API.

Test does not tolerate transitional DEPLOYING status

tests/​sandbox/​test_sandbox_e2e.py:98

This test also assumes the first follow-up read must already report DEPLOYED, although the create endpoint is allowed to return while deployment continues. Poll to readiness, or assert a contract that permits the transitional status, so a valid DEPLOYING response does not fail the E2E suite.

Process starts before sandbox deployment is ready

tests/​sandbox/​test_sandbox_e2e.py:142

The process API is called immediately after creation even though creation may return before deployment is ready. A gateway failure here is especially misleading because this test is marked xfail and will be accepted as the expected server bug without exercising stdin/log behavior. Wait for DEPLOYED before starting the process.

Multipart execution starts before sandbox deployment completes

tests/​sandbox/​test_sandbox_e2e.py:177

Multipart initiation is attempted immediately after create(), but the create response may still be DEPLOYING under the served API contract. Wait for the sandbox to reach DEPLOYED before calling its execution API to avoid intermittent gateway failures.

Execution write races sandbox deployment readiness

tests/​sandbox/​test_sandbox_e2e.py:217

This execution-plane write can race sandbox deployment because create() is allowed to return while status is still DEPLOYING. Poll with a bounded timeout until DEPLOYED before writing/exporting so the E2E test does not intermittently fail at the gateway.

[Copilot]

Copilot review overview

🟡 Changes recommended

Public handles remain directly constructible in an invalid state, readiness guidance is incomplete, and HTTP/2 installation instructions reference a nonexistent extra.

Review effort: Balanced
Findings: 1 · 1

Open (2)

• Reject direct construction of uninitialized Sandbox objects · New
• The async filesystem surface (AsyncSandboxFileSystem.read/write) introduces…

Resolved since last review (1)

• Eagerly buffers synchronous request bodies in memory

Previously missed (4)

In code that hasn't changed since last review

Documented baseten[http2] extra is missing

baseten/​sandbox/​_auth.py:59

The documented baseten[http2] extra does not exist in pyproject.toml, so following this recovery instruction does not install h2 and http2=True continues to fail. Point users to the actual package (or add the advertised extra).

Document that sandbox creation may return before deployment completes

baseten/​sandbox/​_client.py:280

The served create contract says deployment may continue after the response and only DEPLOYED sandboxes accept execution calls (scripts/apigen/specs/management.json:10632, baseten/sandbox/_info.py:10). The current docstring omits this, making the natural client.create(...).process.exec(...) flow race deployment. Document polling get_info() (the handle's info is only the creation snapshot), or make create wait.

Document async sandbox creation readiness and polling

baseten/​sandbox/​_client.py:586

The async create call has the same readiness contract: the response may still be DEPLOYING, but this docstring does not warn callers before they await an execution operation. Document polling get_info() (because AsyncSandbox.info is a snapshot), or make creation wait consistently.

Add parity checks for all nested translated model types

tests/​sandbox/​test_parity.py:151

This only pins the top-level library-image model. The converter also translates SandboxPort, SandboxLibraryImageCreationOptions, and SandboxLibraryImageVolume, so new fields on any of those nested generated types would still be silently dropped despite the parity suite's stated guarantee. Add field-set parity assertions for each nested translated type (and SandboxEnv, which is translated in create/info paths).

[Copilot]

Copilot review overview

🟡 Changes recommended

Revoked-token caching, request serialization, and readiness handling leave important sandbox workflows unreliable.

Review effort: Balanced
Findings: 2 · 1

Open (3)

• Streaming revocation responses fail to invalidate cached tokens · New
• Running process responses reject null logs · New
• The async filesystem surface (AsyncSandboxFileSystem.read/write) introduces…

Resolved since last review (1)

• Reject direct construction of uninitialized Sandbox objects

Previously missed (5)

In code that hasn't changed since last review

Synchronous create returns handles before deployment is ready

baseten/​sandbox/​_client.py:302

The generated control-plane contract says creation may return while status is still DEPLOYING, with the URL assigned before readiness, but this immediately hands out an execution handle. Since process.exec() is deliberately not retried, the common create-then-exec flow can fail against an allowed successful create response. Poll to DEPLOYED before returning, or expose a documented readiness/wait operation so callers can safely use the handle.

This issue also appears on line 605 of the same file.

Status filters incorrectly expand bare strings

baseten/​sandbox/​_client.py:350

A plain string satisfies Sequence[str], so client.list(statuses="DEPLOYED") is accepted by type checkers but this implementation emits one status per character. Reject string input explicitly (or use an input type that excludes strings) rather than sending a malformed filter.

This issue also appears on line 651 of the same file.

Synchronous exec serializes omitted options as null

baseten/​sandbox/​_sandbox.py:397

All optional process fields are passed explicitly even when None. The generated serializer preserves explicitly set nulls, so exec("cmd") sends workingDir, env, name, timeout, and waitForCompletion as null instead of leaving them unset, contrary to this PR's stated wire contract and potentially overriding server defaults. Build the request from only the arguments the caller supplied, as _sandbox_create_request already does.

This issue also appears on line 473 of the same file.

CI omits the sandbox domain secret

tests/​sandbox/​test_sandbox_e2e.py:27

These new E2E tests cannot run in the repository CI configuration: .github/workflows/ci.yml:41-47 injects the API key and management domain but never BASETEN_E2E_TEST_SANDBOXES_DOMAIN, so this condition always skips the entire sandbox suite even on main. Wire the sandbox-domain secret into the workflow so the claimed live coverage executes.

Recovery instructions reference an undefined http2 extra

baseten/​sandbox/​_auth.py:59

This recovery instruction names a package extra that the project does not define, so pip install baseten[http2] will not install h2. Point users to an installation command that actually provides the imported package.