Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/plugins/apps/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,11 +103,11 @@ When Datadog rejects a bundle as longer than the `jsFunctionWithActions` `script
- **Fresh state**: module-level variables and globals start over on every call.
- **Concurrency**: calls run in parallel, with no local limit, so a large burst starts that many processes. Production's concurrency limits vary by org, and `dev:verify` runs against them. Calls that start together share one Datadog query for their runtime context (`$.Source`).
- **Timeout**: a call that runs past its time budget is killed, even inside a synchronous loop.
- **Sandbox**: Node's [permission model](https://nodejs.org/api/permissions.html) lets a call read the project, its `node_modules` folders and the packages linked from them, Vite and Vite's [`server.fs.allow`](https://vite.dev/config/server-options#server-fs-allow) folders, and write only to its own temp dir (`os.tmpdir()`), which is also its working directory, and `/tmp`. Modules loaded through Vite, though not `fs` reads or a native `require`, refuse files matching Vite's [`server.fs.deny`](https://vite.dev/config/server-options#server-fs-deny), such as `.env`, and, unless `server.fs.strict` is `false`, files outside those folders. It refuses subprocesses, worker threads and native addons, allows the network, and caps the heap at 256 MB. On a Node without [`module.registerHooks`](https://nodejs.org/api/module.html#moduleregisterhooksoptions), added in Node 22.15, modules get no `import.meta.resolve`, which Vite resolves on a worker thread there, and the dev server warns once. A known gap, from Node's [documented limitations](https://nodejs.org/api/permissions.html#limitations-and-known-issues): "Symbolic links will be followed even to locations outside of the set of paths that access has been granted to." Here, a symlink in a writable folder (the call's temp dir or `/tmp`) lets a call read or write wherever it points, project files included. Local execution is a parity aid, not a security boundary.
- **Sandbox**: Node's [permission model](https://nodejs.org/api/permissions.html) lets a call read the project, its `node_modules` folders and the packages linked from them, Vite and Vite's [`server.fs.allow`](https://vite.dev/config/server-options#server-fs-allow) folders, and write only to its own temp dir (`os.tmpdir()`), which is also its working directory, and `/tmp`. Modules loaded through Vite, though not `fs` reads or a native `require`, refuse files matching Vite's [`server.fs.deny`](https://vite.dev/config/server-options#server-fs-deny), such as `.env`, and, unless `server.fs.strict` is `false`, files outside those folders. It refuses subprocesses, worker threads and native addons, allows the network, and caps the heap at 256 MB. A refusal the function lets reach its result fails the call with a message that points to `dev:verify`, since production may allow it, and names the path for a file refusal. On a Node without [`module.registerHooks`](https://nodejs.org/api/module.html#moduleregisterhooksoptions), added in Node 22.15, modules get no `import.meta.resolve`, which Vite resolves on a worker thread there, and the dev server warns once. A known gap, from Node's [documented limitations](https://nodejs.org/api/permissions.html#limitations-and-known-issues): "Symbolic links will be followed even to locations outside of the set of paths that access has been granted to." Here, a symlink in a writable folder (the call's temp dir or `/tmp`) lets a call read or write wherever it points, project files included. Local execution is a parity aid, not a security boundary.
- **Environment**: a call sees no variables from the dev server's environment, keys included, except Node's TLS and time zone settings (`NODE_EXTRA_CA_CERTS`, `SSL_CERT_FILE`, `SSL_CERT_DIR`, `TZ`), and no Node options from `NODE_OPTIONS`. Vite still inlines `import.meta.env` and `define` values into the code it serves. `$.Actions` calls go back to the dev server, which holds the credentials and checks each call's connection against the function's allowed connections.
- **Logs**: a call's `stdout` and `stderr` reach the dev server's terminal line by line, each prefixed with its call, for example `[getGreeting#3]`.
- **Debugging**: when the dev server runs under a debugger (`--inspect`), each call's process listens on a random local port and prints its `Debugger listening on ws://…` URL with its prefix. Attach to that URL to debug the call; calls don't time out while the dev server is under a debugger.
- **Older Node**: on a Node without the permission model, or without `--allow-inspector` while debugging, a call's process runs under in-process guards instead. They refuse writes outside the temp dirs, subprocesses and worker threads, from the module's top-level code on.
- **Older Node**: on a Node without the permission model, or without `--allow-inspector` while debugging, a call's process runs unrestricted, and the dev server warns once that production's limits aren't enforced locally.

Local execution doesn't support Yarn Plug'n'Play, whose packages load through hooks a call's process doesn't get: each call fails with a hint to set `nodeLinker: node-modules` or use `dev:verify`.

Expand Down
1 change: 0 additions & 1 deletion packages/plugins/apps/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,6 @@
"@types/estree": "1.0.8",
"@types/node": "^20",
"@types/picomatch": "4.0.2",
"graceful-fs": "4.2.11",
"rollup": "4.60.4",
"typescript": "5.4.3",
"vite": "6.4.3"
Expand Down
2 changes: 0 additions & 2 deletions packages/plugins/apps/src/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ import type { AppsOptionsWithDefaults } from './types';
import { BACKEND_OUT_DIR_PREFIX } from './vite/build-backend-functions';
import { buildAppPackage, MANIFEST_DIR_PREFIX } from './vite/build-package';

// Another test file in this Jest worker can install the guards, which makes fs.promises.mkdtemp
// non-configurable for jest.spyOn, so the mock comes from the module registry instead.
jest.mock('fs/promises', () => {
const actual = jest.requireActual<typeof import('fs/promises')>('fs/promises');
return { ...actual, mkdtemp: jest.fn(actual.mkdtemp) };
Expand Down
5 changes: 0 additions & 5 deletions packages/plugins/apps/src/vite/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,6 @@ import {
import * as buildPackage from './build-package';
import * as devServer from './dev-server';
import * as localExecutionProcess from './local-execution-process';
import * as networkGuard from './network-guard';
import { MIN_VITE_VERSION, unsupportedViteMessage } from './vite-version';

type TransformHandler = (code: string, id: string, transformOptions?: { ssr?: boolean }) => unknown;
Expand Down Expand Up @@ -1479,7 +1478,6 @@ describe('Backend Functions - getVitePlugin', () => {
'Should run backend functions in child processes and check local Node only when local execution can run ($scenario)',
({ mode, withAuth, runsLocally }) => {
const plugin = getVitePlugin(defaultOptions);
const installGuards = jest.spyOn(networkGuard, 'installGuards');
const createMiddleware = jest.spyOn(devServer, 'createDevServerMiddleware');
const createExecutor = jest.spyOn(localExecutionProcess, 'createChildProcessExecutor');
const runnerPath = '/build/node_modules/vite/dist/node/module-runner.js';
Expand Down Expand Up @@ -1515,14 +1513,11 @@ describe('Backend Functions - getVitePlugin', () => {
}));
const expectedOptions = { invokeChannel: hot, moduleRunnerPath: runnerPath };
expect(executorOptions).toEqual(runsLocally ? [expectedOptions] : []);
// The dev server's own fs and subprocess APIs stay unpatched: calls run elsewhere.
expect(installGuards).not.toHaveBeenCalled();
const warning = localNodeWarning(NODE_WITHOUT_WEBSOCKET);
const warnings = mockLogFn.mock.calls.filter(([message]) => message === warning);
expect(warnings).toEqual(runsLocally ? [[warning, 'warn']] : []);
} finally {
restoreEnv();
installGuards.mockRestore();
createMiddleware.mockRestore();
createExecutor.mockRestore();
resolveRunner.mockRestore();
Expand Down
28 changes: 13 additions & 15 deletions packages/plugins/apps/src/vite/local-execution-child.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import nodeModule from 'node:module';
import type { Writable } from 'node:stream';

import { withPermissionHint } from './local-execution-limits';
import type { ChildMessage, ChildStartConfig, ParentMessage } from './local-execution-protocol';
import { isParentMessage } from './local-execution-protocol';
import type { ActionCallArgs } from './local-execution-shared';
Expand All @@ -25,7 +26,6 @@ import {
serializeActionInputs,
validateActionCallShape,
} from './local-execution-shared';
import { installGuards, runBlocked } from './network-guard';

type ModuleRunnerInstance = { import(url: string): Promise<Record<string, unknown>> };
type CreateImportMeta = (modulePath: string) => unknown;
Expand Down Expand Up @@ -172,7 +172,7 @@ async function load(config: ChildStartConfig): Promise<unknown> {
Reflect.set(globalThis, '$', undefined);
const evaluator = new ESModulesEvaluator();
// Without `module.registerHooks`, Vite resolves it on a hooks thread, which the permission model refuses.
const resolvesImportMeta = config.installGuards || 'registerHooks' in nodeModule;
const resolvesImportMeta = !('permission' in process) || 'registerHooks' in nodeModule;
// Without it, Vite's runner gives modules an import.meta.resolve that always throws.
const runnerOptions: ModuleRunnerOptions = {
transport: { invoke },
Expand All @@ -195,30 +195,28 @@ async function load(config: ChildStartConfig): Promise<unknown> {
Reflect.set(globalThis, '$', $);
await registerSdkAdapters(runner, config, $);
await sendMessage({ type: 'loaded' });
const returned = await callBackendFunction(() => fn(...config.args));
const returned = await callBackendFunction(async () => {
try {
return await fn(...config.args);
} catch (error) {
throw withPermissionHint(error);
}
});
const subject = returnValueSubject(config.functionName);
return assertJsonRoundTrippable(returned, subject);
}

// Without the permission model, the guards cover the module's top-level code as well as the call.
function execute(config: ChildStartConfig): Promise<unknown> {
if (!config.installGuards) {
return load(config);
}
installGuards();
return runBlocked(() => load(config));
}

async function run(config: ChildStartConfig): Promise<void> {
let outcome: ChildMessage;
try {
const data = await execute(config);
const data = await load(config);
outcome = { type: 'result', data };
} catch (error) {
const failure = withPermissionHint(error);
outcome = {
type: 'error',
message: describeError(error),
...(error instanceof FunctionThrewError && { detail: error.detail }),
message: describeError(failure),
...(failure instanceof FunctionThrewError && { detail: failure.detail }),
};
}
const stdoutFlushed = flush(process.stdout, writeStdout, config.flushTimeoutMs);
Expand Down
116 changes: 116 additions & 0 deletions packages/plugins/apps/src/vite/local-execution-limits.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
// Unless explicitly stated otherwise all files in this repository are licensed under the MIT License.
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2019-Present Datadog, Inc.

import {
NATIVE_ADDON_BLOCKED_MESSAGE,
OUTSIDE_READ_BLOCKED_MESSAGE,
OUTSIDE_TEMP_WRITE_BLOCKED_MESSAGE,
PERMISSION_BLOCKED_MESSAGE,
SUBPROCESS_BLOCKED_MESSAGE,
WORKER_THREAD_BLOCKED_MESSAGE,
withPermissionHint,
} from './local-execution-limits';

const refusal = (fields: Record<string, unknown>) =>
Object.assign(new Error('Access to this API has been restricted'), fields);

describe('withPermissionHint', () => {
const realPermission = Object.getOwnPropertyDescriptor(process, 'permission');

beforeEach(() => {
Object.defineProperty(process, 'permission', {
value: { has: () => false },
configurable: true,
});
});

afterEach(() => {
if (realPermission) {
Object.defineProperty(process, 'permission', realPermission);
} else {
Reflect.deleteProperty(process, 'permission');
}
});

test.each([
{
refused: 'a subprocess',
fields: { code: 'ERR_ACCESS_DENIED', permission: 'ChildProcess', resource: 'echo' },
message: SUBPROCESS_BLOCKED_MESSAGE,
},
{
refused: 'a worker thread, whose resource is empty',
fields: { code: 'ERR_ACCESS_DENIED', permission: 'WorkerThreads', resource: '' },
message: WORKER_THREAD_BLOCKED_MESSAGE,
},
{
refused: 'a write, naming its path',
fields: { code: 'ERR_ACCESS_DENIED', permission: 'FileSystemWrite', resource: '/x' },
message: `${OUTSIDE_TEMP_WRITE_BLOCKED_MESSAGE} Refused: /x`,
},
{
refused: 'a read, naming its path',
fields: { code: 'ERR_ACCESS_DENIED', permission: 'FileSystemRead', resource: '/y' },
message: `${OUTSIDE_READ_BLOCKED_MESSAGE} Refused: /y`,
},
{
refused: 'another permission',
fields: {
code: 'ERR_ACCESS_DENIED',
permission: 'Inspector',
resource: 'StartIoThread',
},
message: PERMISSION_BLOCKED_MESSAGE,
},
{
refused: 'a native addon',
fields: { code: 'ERR_DLOPEN_DISABLED' },
message: NATIVE_ADDON_BLOCKED_MESSAGE,
},
])('Should hint at dev:verify for $refused', ({ fields, message }) => {
const original = refusal(fields);
const hinted = withPermissionHint(original);

expect(hinted).toBeInstanceOf(Error);
expect(hinted).toMatchObject({ message });
expect(hinted).toHaveProperty(
'stack',
expect.stringContaining(`Error: ${message}\n at `),
);
});

const throwingCode = new Error('custom');
Object.defineProperty(throwingCode, 'code', {
get() {
throw new Error('getter');
},
});

test.each([
{
kind: 'an ERR_ACCESS_DENIED without a permission',
error: refusal({ code: 'ERR_ACCESS_DENIED' }),
},
{ kind: 'an unrelated error', error: new Error('boom') },
{
kind: "an ERR_ACCESS_DENIED with a scope that isn't Node's",
error: refusal({ code: 'ERR_ACCESS_DENIED', permission: 'read:users' }),
},
{ kind: 'a thrown string', error: 'boom' },
{ kind: 'an error whose code getter throws', error: throwingCode },
])('Should keep $kind as it is', ({ error }) => {
const result = withPermissionHint(error);

expect(result).toBe(error);
});

test('Should keep a refusal as it is without the permission model', () => {
Reflect.deleteProperty(process, 'permission');
const original = refusal({ code: 'ERR_ACCESS_DENIED', permission: 'ChildProcess' });

const result = withPermissionHint(original);

expect(result).toBe(original);
});
});
76 changes: 76 additions & 0 deletions packages/plugins/apps/src/vite/local-execution-limits.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
// Unless explicitly stated otherwise all files in this repository are licensed under the MIT License.
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2019-Present Datadog, Inc.

const DEV_VERIFY = '`dev:verify` (`npx datadog-apps dev --verify`)';
const LIMITS_DOC =
'Rules: https://github.com/DataDog/build-plugins/tree/master/packages/plugins/apps#local-execution';
const LOCALLY = 'is blocked when running backend functions locally.';
const SUFFIX = `It may work in production; run ${DEV_VERIFY} to check it on the real runtime. ${LIMITS_DOC}`;
export const SUBPROCESS_BLOCKED_MESSAGE = `Spawning a subprocess ${LOCALLY} ${SUFFIX}`;
export const WORKER_THREAD_BLOCKED_MESSAGE = `Starting a worker thread ${LOCALLY} ${SUFFIX}`;
export const OUTSIDE_TEMP_WRITE_BLOCKED_MESSAGE = `Writing outside the call's temp dirs ${LOCALLY} ${SUFFIX}`;
export const OUTSIDE_READ_BLOCKED_MESSAGE = `Reading outside the folders a call can read ${LOCALLY} ${SUFFIX}`;
export const NATIVE_ADDON_BLOCKED_MESSAGE = `Loading a native addon ${LOCALLY} ${SUFFIX}`;
export const PERMISSION_BLOCKED_MESSAGE = `This operation ${LOCALLY} ${SUFFIX}`;
export const UNRESTRICTED_LOCAL_EXECUTION_WARNING = `Backend functions run locally without Node's permission model, so production's limits aren't enforced; ${DEV_VERIFY} runs them on the real runtime. ${LIMITS_DOC}`;

const NODE_PERMISSION_SCOPES = new Set([
'FileSystemRead',
'FileSystemWrite',
'ChildProcess',
'WorkerThreads',
'Inspector',
'WASI',
'Net',
'Addons',
]);
const PERMISSION_REFUSAL_MESSAGES = new Map([
['ChildProcess', SUBPROCESS_BLOCKED_MESSAGE],
['WorkerThreads', WORKER_THREAD_BLOCKED_MESSAGE],
['FileSystemWrite', OUTSIDE_TEMP_WRITE_BLOCKED_MESSAGE],
['FileSystemRead', OUTSIDE_READ_BLOCKED_MESSAGE],
]);

function permissionRefusalMessage(error: Error): string | undefined {
const code = 'code' in error ? error.code : undefined;
if (code === 'ERR_DLOPEN_DISABLED') {
return NATIVE_ADDON_BLOCKED_MESSAGE;
}
// Node's refusals always carry one of its permission scopes; any other error is the function's own.
if (code !== 'ERR_ACCESS_DENIED' || !('permission' in error)) {
return undefined;
}
const permission = String(error.permission);
if (!NODE_PERMISSION_SCOPES.has(permission)) {
return undefined;
}
const message = PERMISSION_REFUSAL_MESSAGES.get(permission) ?? PERMISSION_BLOCKED_MESSAGE;
const resource = 'resource' in error ? error.resource : undefined;
const isPath = permission.startsWith('FileSystem');
return isPath && typeof resource === 'string' && resource
? `${message} Refused: ${resource}`
: message;
}

// Node's own refusal suggests a permission flag, which the developer can't pass to a call's process.
// The refused call's frames stay, so the reported stack still points at the code that made it.
export function withPermissionHint(error: unknown): unknown {
// Only the permission model raises these codes; without it, a matching error is the function's own.
if (!process.permission) {
return error;
}
try {
const hint = error instanceof Error ? permissionRefusalMessage(error) : undefined;
if (!(error instanceof Error) || hint === undefined) {
return error;
}
const hinted = new Error(hint);
const lines = typeof error.stack === 'string' ? error.stack.split('\n') : [];
const frames = lines.filter((line) => line.startsWith(' at '));
hinted.stack = [`Error: ${hint}`, ...frames].join('\n');
return hinted;
} catch {
return error;
}
}
Loading
Loading