Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/content/docs/containers/guides/snapshots.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ Snapshots are only supported by Container applications that use the [`durable_ob

## Create a container snapshot

Use `snapshotContainer()` to capture the full container filesystem. Snapshots are immutable. If you restore a snapshot and then change files, create a new snapshot to persist those changes.
Use `snapshotContainer()` to capture the writable root filesystem of the container. Snapshots are immutable. If you restore a snapshot and then change files, create a new snapshot to persist those changes.

:::caution[Snapshot image compatibility]
Snapshots capture the Container's complete filesystem state, but not its memory or running processes. A snapshot is tied to the Container image version it was created from and is not portable to a different image. After updating the image, create a new snapshot from a Container running that image.
Snapshots capture the writable root filesystem of the Container. They do not capture separately mounted filesystems, files in `/run`, memory, or running processes. A snapshot is tied to the Container image version it was created from and is not portable to a different image. After updating the image, create a new snapshot from a Container running that image.
:::

The returned snapshot handle is a plain data object. You can store it and restore it later, including from another Durable Object:
Expand Down
39 changes: 39 additions & 0 deletions src/content/docs/sandbox/files/back-up-a-directory-to-r2.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,45 @@ An [object lifecycle rule](/r2/buckets/object-lifecycles/) that expires objects

A backup that fails does not leave an object. If the Durable Object restarts during a backup, the incomplete multipart upload stays in the bucket until an [object lifecycle rule](/r2/buckets/object-lifecycles/) of the bucket aborts it.

## Send other requests to a catch-all intercept

If your Durable Object sends the other HTTP requests from the container to an entrypoint with [`interceptAllOutboundHttp()`](/containers/api/durable-object-container/#interceptalloutboundhttp), register the backup route first. A catch-all that a container registers first receives its backup traffic until a new container starts, and `backup()` and `restore()` fail with `BACKUP_TRANSFER`.

Replace `startSandbox()` with a version that registers both routes on each new container. `Outbound` is a `WorkerEntrypoint` that your Worker exports:

```ts title="src/index.ts"
export class MyContainer extends DurableObject<Env> {
// ...

// The setup that requests in progress share
private starting?: Promise<void>;

private startSandbox(): Promise<void> {
if (!this.container.running) {
this.starting ??= this.setUpSandbox().finally(() => {
this.starting = undefined;
});
}
return this.starting ?? Promise.resolve();
}

private async setUpSandbox(): Promise<void> {
this.container.start({
image: this.container.images.workspace,
enableInternet: false,
});
await this.container.setInactivityTimeout(INACTIVITY_TIMEOUT_MS);
// Register the backup route before the catch-all
await this.backups.intercept();
await this.container.interceptAllOutboundHttp(
this.ctx.exports.Outbound({ props: {} }),
);
}
}
```

Requests that arrive during setup wait for it, so no backup starts before both routes exist, and no `intercept()` call replaces the access of a running backup. The routes belong to the container and stay in place when the Durable Object restarts, so a restarted Durable Object that finds the container running registers nothing.

## Related resources

- [DirectoryBackup API](/sandbox/reference/directory-backups/): what a backup keeps, one operation at a time per container, cancellation, and every error.
Expand Down
67 changes: 65 additions & 2 deletions src/content/docs/sandbox/files/manage-files.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,70 @@ The `Files` class from `@cloudflare/sandbox` reads and writes files in a running

Compressed files stay in `/workspace` until the container stops. To delete a file sooner, call `remove()` after the caller has read the response.

For other file operations, such as listing a directory or renaming a file, refer to [Files API](/sandbox/reference/files/).
For other file operations, refer to [Files API](/sandbox/reference/files/).

## Replace a file in one step

A process that reads a file while `writeFile()` writes it can read part of the file. To replace a file in one step, write the new content to a temporary file in the same directory, then rename the temporary file onto the original. `rename()` replaces the destination in one step, so a reader gets either the old file or the new one. The temporary file must be on the same filesystem, because `rename()` rejects with `EXDEV` across filesystems.

This example adds a `replace()` method:

```ts title="src/index.ts"
export class MyContainer extends DurableObject<Env> {
// ...

async replace(path: string, content: string): Promise<void> {
await this.startSandbox();
const temporary = `${path}.${crypto.randomUUID()}.partial`;

try {
await this.files.writeFile(temporary, content, { cwd: workspace });
await this.files.rename(temporary, path, { cwd: workspace });
} catch (error) {
await this.files.remove(temporary, { cwd: workspace, force: true });
throw error;
}
}
}
```

If a caller chooses `path`, check it first, as [Accept paths from callers](#accept-paths-from-callers) describes.

## List a directory tree

`readDirectory()` returns the entries of one directory. To list a tree, read each subdirectory in turn. This example adds a `listTree()` method that returns every path under a directory. It does not follow symbolic links to directories, because their entries have the type `symlink`:

```ts title="src/index.ts"
export class MyContainer extends DurableObject<Env> {
// ...

async listTree(directory: string): Promise<string[]> {
await this.startSandbox();
const paths: string[] = [];
const pending = [directory];
let current: string | undefined;

while ((current = pending.pop()) !== undefined) {
const entries = await this.files.readDirectory(current, {
cwd: workspace,
});

for (const entry of entries) {
const path = `${current}/${entry.name}`;
paths.push(path);

if (entry.type === "directory") {
pending.push(path);
}
}
}

return paths;
}
}
```

Each call to `readDirectory()` starts one process in the container. For a large tree, one `find` command through [`exec()`](/containers/api/durable-object-container/#exec) is faster, and its `signal` option can cancel it. The output of `find` counts toward the [memory limit](/workers/platform/limits/#memory) when you read it with `output()`.

## Accept paths from callers

Expand Down Expand Up @@ -283,6 +346,6 @@ For more information, refer to [Sandbox security](/sandbox/concepts/security/#ev

## Related resources

- [Files API](/sandbox/reference/files/): every method, including `readDirectory()`, `rename()`, and `remove()`.
- [Files API](/sandbox/reference/files/): every method, with its errors and options.
- [Save and restore a sandbox with snapshots](/sandbox/files/save-and-restore-a-workspace/): keep the files after the container stops.
- [Mount an R2 bucket](/sandbox/files/mount-an-r2-bucket/): write files that other systems read.
43 changes: 38 additions & 5 deletions src/content/docs/sandbox/network/clone-a-private-repository.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -114,13 +114,39 @@ Clone a private repository into a sandbox while the access token stays in your W

`git-upload-pack` serves clones and fetches. The entrypoint allows it for one repository and rejects every other request. Code in the sandbox cannot push through `git-receive-pack` or read other repositories, even when the token allows it.

5. Add a method to your Durable Object that intercepts `github.com` and clones the repository:
5. Add a method to your Durable Object that intercepts `github.com` and clones the repository, and a constructor that keeps the inactivity timeout after a restart:

```ts title="src/index.ts"
type CloneResult = { exitCode: number; stderr: string };

const INACTIVITY_TIMEOUT_MS = 10 * 60 * 1000;

export class MyContainer extends DurableObject<Env> {
// ...

async clone(): Promise<{ exitCode: number; stderr: string }> {
// The clone that requests in progress share
private cloning?: Promise<CloneResult>;

constructor(ctx: DurableObjectState, env: Env) {
super(ctx, env);
const container = ctx.container;

// A restarted Durable Object sets the timeout again
if (container?.running) {
void ctx.blockConcurrencyWhile(() =>
container.setInactivityTimeout(INACTIVITY_TIMEOUT_MS),
);
}
}

clone(): Promise<CloneResult> {
this.cloning ??= this.cloneOnce().finally(() => {
this.cloning = undefined;
});
return this.cloning;
}

private async cloneOnce(): Promise<CloneResult> {
const container = this.ctx.container;

if (!container) {
Expand All @@ -139,7 +165,14 @@ Clone a private repository into a sandbox while the access token stays in your W
enableInternet: false,
});
// Keep the clone for later requests
await container.setInactivityTimeout(10 * 60 * 1000);
await container.setInactivityTimeout(INACTIVITY_TIMEOUT_MS);
}

// A running container keeps the clone from an earlier request
const cloned = await container.exec(["test", "-d", "/workspace/.git"]);

if ((await cloned.output()).exitCode === 0) {
return { exitCode: 0, stderr: "" };
}

const process = await container.exec(
Expand Down Expand Up @@ -182,7 +215,7 @@ Clone a private repository into a sandbox while the access token stays in your W

The route uses a sandbox name of its own. A container that is already running keeps the image and Internet access it started with, so it would have no Git.

When Git fails, the route responds with `500` and the error from Git. While the sandbox keeps running, a second request fails this way, because `/workspace` already holds the clone.
When Git fails, the route responds with `500` and the error from Git. Requests that arrive during a clone wait for it, and a later request returns without cloning again while the container keeps running.

Authenticate callers first, so other people cannot read the repository through the sandbox. For more information, refer to [Sandbox security](/sandbox/concepts/security/#every-opening-is-also-a-way-out).

Expand All @@ -200,7 +233,7 @@ Clone a private repository into a sandbox while the access token stays in your W
Cloning into '/workspace'...
```

The repository is in `/workspace`, and `git remote get-url origin` prints the URL without a token. A `git push` from the sandbox fails with `403`. The clone lasts until the container stops. For more information, refer to [Sandbox lifetime](/sandbox/concepts/lifetime/#what-stops-an-instance).
The repository is in `/workspace`, and `git remote get-url origin` prints the URL without a token. A `git push` from the sandbox fails with `403`. The clone lasts until the container stops. The constructor sets the inactivity timeout again when a restarted Durable Object finds the container running, because the timeout does not survive a restart. For more information, refer to [Sandbox lifetime](/sandbox/concepts/lifetime/#what-stops-an-instance).

</Steps>

Expand Down
38 changes: 4 additions & 34 deletions src/content/docs/sandbox/reference/directory-backups.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -144,43 +144,13 @@ Routes backup traffic from the container to `DirectoryBackupGateway`, which refu

{/* Remove the second sentence and the last list item when a hostname intercept registered after interceptAllOutboundHttp() receives its requests. */}

`backup()` and `restore()` add this route themselves. If the Durable Object also calls [`interceptAllOutboundHttp()`](/containers/api/durable-object-container/#interceptalloutboundhttp), `intercept()` must run before it. Otherwise the catch-all receives the backup traffic from the container, and `backup()` and `restore()` fail with `BACKUP_TRANSFER`.
`backup()` and `restore()` add this route themselves. If the Durable Object also calls [`interceptAllOutboundHttp()`](/containers/api/durable-object-container/#interceptalloutboundhttp), `intercept()` must run before it on each new container. Otherwise the catch-all receives the backup traffic from the container, and `backup()` and `restore()` fail with `BACKUP_TRANSFER`.

- Call `intercept()` in the setup that runs after you start the container, and again when a restarted Durable Object finds the container running. Backups and restores in that container then keep their route.
- The routes belong to the container. They stay in place when the Durable Object restarts, so a restarted Durable Object that finds the container running does not need to call `intercept()` again.
- Do not call `intercept()` while a backup or restore can be running. It replaces the access of the running operation, and the operation fails.
- A container that registered the catch-all first keeps sending backup traffic to it until the container restarts.
- The order of the routes is fixed when each is first registered on a container. A container that registered the catch-all first keeps sending backup traffic to it until a new container starts.

```ts
#setup: Promise<void> | null = null;

// Call before each use of the container.
#ensureRunning(container: Container): Promise<void> {
// Set up each new container, and a running container
// after this Durable Object restarts.
if (this.#setup === null || !container.running) {
this.#setup = this.#setUp(container).catch((error) => {
this.#setup = null;
throw error;
});
}
return this.#setup;
}

async #setUp(container: Container): Promise<void> {
// `sandbox` is an image in the Wrangler configuration, and `Outbound`
// is a WorkerEntrypoint that the main module of the Worker exports.
if (!container.running) {
container.start({
image: container.images.sandbox,
enableInternet: false,
});
}
await this.#backups.intercept();
await container.interceptAllOutboundHttp(
this.ctx.exports.Outbound({ props: {} }),
);
}
```
For setup code that registers both routes, refer to [Send other requests to a catch-all intercept](/sandbox/files/back-up-a-directory-to-r2/#send-other-requests-to-a-catch-all-intercept).

## Types

Expand Down
26 changes: 2 additions & 24 deletions src/content/docs/sandbox/reference/files.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -129,12 +129,7 @@ Creates a file, or truncates an existing file, then streams `content` into it.
- The file opens before a `ReadableStream` is consumed. A later failure can leave the file created, truncated, or partially written.
- If the `content` stream errors, the call rejects with the error from the stream instead of a `SandboxFileError`.

`rename()` replaces an existing destination in one step. Writing a file to a temporary path and then renaming it keeps other processes from reading a partially written file:

```ts
await files.writeFile(`${path}.partial`, content, { cwd });
await files.rename(`${path}.partial`, path, { cwd });
```
A process that reads the file during the write can read part of it. To replace a file in one step, write a temporary file and rename it, as [Replace a file in one step](/sandbox/files/manage-files/#replace-a-file-in-one-step) shows.

### `stat`

Expand Down Expand Up @@ -169,24 +164,7 @@ Returns the immediate entries of a directory as [`SandboxDirectoryEntry`](#sandb
- A file at `path` rejects with `ENOTDIR`.
- An entry name that is not valid UTF-8 rejects the whole call with `EILSEQ`.

A recursive walk with `readDirectory()` lists a directory tree. This walk does not follow symbolic links to directories:

```ts
import type { Files } from "@cloudflare/sandbox";

async function* walk(files: Files, directory: string): AsyncGenerator<string> {
for (const entry of await files.readDirectory(directory)) {
const path = `${directory}/${entry.name}`;
yield path;

if (entry.type === "directory") {
yield* walk(files, path);
}
}
}
```

Each call to `readDirectory()` starts one process in the container. For a large tree, one `container.exec(["find", directory])` call is faster. The `signal` option of `exec()` cancels it.
Each call starts one process in the container. To list a directory tree, refer to [List a directory tree](/sandbox/files/manage-files/#list-a-directory-tree).

### `mkdir`

Expand Down
24 changes: 22 additions & 2 deletions src/content/docs/sandbox/sdk/migrate/tunnels.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -80,10 +80,30 @@ This page replaces `tunnels.get()`, `tunnels.list()`, and `tunnels.destroy()`. R
export class MySandbox extends DurableObject<Env> {
// ...

private async startTunnel(
// The tunnel starts that requests in progress share, by port
private readonly tunnelStarts = new Map<number, Promise<string>>();

private startTunnel(
port: number,
args: string[],
env: Record<string, string> = {},
): Promise<string> {
let start = this.tunnelStarts.get(port);

if (!start) {
start = this.runTunnel(port, args, env).finally(() => {
this.tunnelStarts.delete(port);
});
this.tunnelStarts.set(port, start);
}

return start;
}

private async runTunnel(
port: number,
args: string[],
env: Record<string, string>,
): Promise<string> {
const dir = `${TUNNEL_ROOT}/${port}`;
const argv = ["cloudflared", "tunnel", "--no-autoupdate", ...args];
Expand Down Expand Up @@ -128,7 +148,7 @@ This page replaces `tunnels.get()`, `tunnels.list()`, and `tunnels.destroy()`. R

`openTunnel()` replaces `tunnels.get(port)`, and returns the `trycloudflare.com` URL. The URL can take several seconds to resolve after the method returns. A quick tunnel ends with its container, as in 0.12.

Each call stops any `cloudflared` that already serves the port, and returns a new URL. 0.12 returned the tunnel that already served the port. To keep a URL, look it up with `listTunnels()` from the next step before you call `openTunnel()`.
Requests that arrive while a tunnel for the port is starting wait for the same start and get the same URL. Without the shared start, two requests can each start `cloudflared` in the same directory, and the recorded process ID can belong to the one that exits. A later call stops any `cloudflared` that already serves the port, and returns a new URL. 0.12 returned the tunnel that already served the port. To keep a URL, look it up with `listTunnels()` from the next step before you call `openTunnel()`.

4. Add methods that list and close quick tunnels:

Expand Down
Loading