diff --git a/docs/maintainers.md b/docs/maintainers.md index d67aaa308..63ee61bdb 100644 --- a/docs/maintainers.md +++ b/docs/maintainers.md @@ -6,7 +6,7 @@ This guide is for maintainers who build and publish OpenAgentCore. To install Co ## Build a distribution -A distribution is a matched set of release assets built from one commit: the control archive (the installer, the `oac` command, and the Core, Web, ingress and PostgreSQL images), the Runtime image and node artifacts as separate files, and the native installers. +A distribution is a matched set of release assets built from one commit: the Runtime image and node artifacts as separate files, and the native installers. The build also writes a control archive per architecture (the installer, the `oac` command, and the Core, Web, ingress and PostgreSQL images). The publisher reads the images and the native catalog from it; nothing installs from it, so it is not a release asset. Core, Web and ingress images are published as verified Linux amd64/arm64 indexes. The arm64 control archive contains those three images; Node, hosted Runtime and offline payloads use Linux amd64. Release builders use QEMU for ARM image steps, including the E2B helper. Host `oac` binaries are built from the same command for Linux amd64/arm64, macOS amd64/arm64 and Windows amd64; launchers only select, verify and invoke them. Each version index is checked against its platform archives before floating tags move. @@ -53,7 +53,7 @@ node scripts/build-native-catalog.mjs INPUT_DIR OUTPUT_DIR export OAC_NATIVE_INSTALLER_BUILD_DIR=OUTPUT_DIR ``` -Each platform has a small base archive `-.tar.gz` (`oac-daemon` and `bundle.json`, which lists every component with the SHA-256 of each file) and one archive per component, `--node` and `--` (MiniMax Code is not built for Windows). `build-native-installer.mjs` writes them. Core serves these artifacts by name and knows nothing of their content; `oac-daemon install` downloads the components it needs. +Each platform has a small base archive `-.tar.gz` (`oac-daemon` and `bundle.json`, which lists every component with the SHA-256 of each file) and one archive per component, `--node` and `--` (MiniMax Code is not built for Windows). `build-native-installer.mjs` writes them and leaves out files that only build or install tooling: the CI runner's global pnpm in `node`, and the compiler inputs of `better-sqlite3` in `minimax`. Core serves these artifacts by name and knows nothing of their content; `oac-daemon install` downloads the components it needs. The catalog records the commit, the Runtime protocol version, each artifact's SHA-256 and, with a release base, its versioned URL. The control archive and Core image carry only `native-installers/catalog.json`; the archives become separate `oac-native--.tar.gz` release assets, and the offline archive holds a copy of every Linux amd64 archive outside the Core image, which its `install.py` copies into the installation for Core to serve. Core redirects the other platforms to their release assets; a catalog without a release base has no such URL, so its offline archive holds every archive. Without a catalog, Sessions report the install command as unavailable, and the release workflow refuses to publish. diff --git a/docs/zh/maintainers.md b/docs/zh/maintainers.md index 24908cbf9..32e882a5a 100644 --- a/docs/zh/maintainers.md +++ b/docs/zh/maintainers.md @@ -1,14 +1,14 @@ --- title: "构建并发布 OpenAgentCore" source: docs/maintainers.md -source_hash: 6f1b4534b4b473c47175560c24ca6e1d1fe2ff4c5e764f204d1022939e4178b3 +source_hash: 0b605c6f5df332a2eb9831d3026b4bcde4ae6d7a2e2cb7bd8b3ce3a21704a7db --- 本指南面向负责构建和发布 OpenAgentCore 的维护者。要安装 Core 和 Web，请按 [安装 Core 和 Web](getting-started/install.md) 操作。安装程序代码遵循的规则见 [部署](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/deploy/README.md) 和 [节点安装程序](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/deploy/node/README.md)；必需检查见 [CONTRIBUTING](https://github.com/MiniMax-AI/OpenAgentCore/blob/main/CONTRIBUTING.md#required-checks)。 ## 构建分发包 {#build-a-distribution} -分发包是从同一个提交构建的一组相互匹配的发布资源：控制归档（安装程序、`oac` 命令，以及 Core、Web、ingress 和 PostgreSQL 镜像）、作为独立文件的 Runtime 镜像和节点构件，以及原生安装程序。 +分发包是从同一个提交构建的一组相互匹配的发布资源：作为独立文件的 Runtime 镜像和节点构件，以及原生安装程序。构建还会为每种架构生成一个控制归档（安装程序、`oac` 命令，以及 Core、Web、ingress 和 PostgreSQL 镜像）。发布器从中读取镜像和原生目录；没有任何安装流程使用它，因此它不是发布资源。 Core、Web 和 ingress 镜像发布为经过校验的 Linux amd64/arm64 多架构索引。arm64 控制归档包含这三个镜像；Node、托管 Runtime 和离线包使用 Linux amd64。发行构建使用 QEMU 执行 ARM 镜像步骤，包括 E2B helper。主机 `oac` 从同一份实现构建为 Linux amd64/arm64、macOS amd64/arm64 和 Windows amd64 二进制；启动脚本只选择、校验并运行它们。所有版本索引校验通过后才更新浮动标签。 @@ -55,7 +55,7 @@ node scripts/build-native-catalog.mjs INPUT_DIR OUTPUT_DIR export OAC_NATIVE_INSTALLER_BUILD_DIR=OUTPUT_DIR ``` -每个平台有一个小型基础归档 `-.tar.gz`（`oac-daemon` 和 `bundle.json`，后者列出每个组件及其每个文件的 SHA-256），以及每个组件一个归档，即 `--node` 和 `--`（MiniMax Code 不为 Windows 构建）。`build-native-installer.mjs` 生成它们。Core 按名称提供这些制品，不了解其内容；`oac-daemon install` 下载所需的组件。 +每个平台有一个小型基础归档 `-.tar.gz`（`oac-daemon` 和 `bundle.json`，后者列出每个组件及其每个文件的 SHA-256），以及每个组件一个归档，即 `--node` 和 `--`（MiniMax Code 不为 Windows 构建）。`build-native-installer.mjs` 生成它们，并略去仅用于构建或安装的文件：`node` 中 CI 运行器全局安装的 pnpm，以及 `minimax` 中 `better-sqlite3` 的编译输入。Core 按名称提供这些制品，不了解其内容；`oac-daemon install` 下载所需的组件。 该目录会记录提交、Runtime 协议版本、每个制品的 SHA-256，并在提供发布基址时记录其带版本号的 URL。控制归档和 Core 镜像仅携带 `native-installers/catalog.json`；这些归档会成为独立的 `oac-native--.tar.gz` 发布资源，离线归档则会在 Core 镜像之外保留所有 Linux amd64 归档的一份副本，由其 `install.py` 复制到安装目录中供 Core 提供下载。其他平台由 Core 重定向到对应的发布资源；没有发布基址的目录没有这类 URL，因此其离线归档会保留所有归档。如果没有目录，Sessions 会将安装命令报告为不可用，并且发布工作流会拒绝发布。 diff --git a/scripts/build-native-installer.mjs b/scripts/build-native-installer.mjs index 474d73a34..168278947 100644 --- a/scripts/build-native-installer.mjs +++ b/scripts/build-native-installer.mjs @@ -27,11 +27,12 @@ async function outputRealPath(path) { // Flatten contained links, including directory links used by pnpm exports. Every // traversal checks its resolved target; active ancestors detect directory cycles. -export async function copyComponent(source, destination) { +export async function copyComponent(source, destination, omit = []) { const root = await realpath(source); if (!(await stat(root)).isDirectory()) throw new Error('Component source must be a directory'); const files = {}; async function visit(path, target, ancestors) { + if (omit.includes(relative(destination, target).split(sep).join('/'))) return; const resolved = await realpath(path); if (!inside(root, resolved)) throw new Error('Component link escapes its source'); const info = await stat(resolved); @@ -66,6 +67,14 @@ function probe(command, args, cwd, env) { if (result.error || result.status !== 0) throw new Error('Native component compatibility probe failed'); return result.stdout.trim(); } +// Files that only build or install tooling and that no Harness or the daemon loads at run time. +// pnpm is the CI runner's global install, not part of the Node distribution. better-sqlite3 loads only +// build/Release/better_sqlite3.node, with SQLite compiled in; deps/ and src/ are its compiler inputs. +// Names cover the Unix and Windows layouts. +const buildOnly = { + node: ['bin/pnpm', 'bin/pnpx', 'lib/node_modules/pnpm', 'node_modules/pnpm', 'pnpm', 'pnpm.cmd', 'pnpm.ps1', 'pnpx', 'pnpx.cmd', 'pnpx.ps1'], + minimax: ['native/node_modules/better-sqlite3/deps', 'native/node_modules/better-sqlite3/src', 'native/node_modules/better-sqlite3/binding.gyp'], +}; const json = async path => JSON.parse(await readFile(path, 'utf8')); const requireFile = async path => { if (!(await stat(path)).isFile()) throw new Error('Required component file is missing'); }; @@ -160,7 +169,7 @@ export async function buildBundle(options) { await copyFile(options.daemon, daemon); await chmod(daemon, 0o755); await mkdir(join(staging, 'components')); const components = {}; - for (const name of ['node', ...names]) components[name] = { version: pins[name], files: await copyComponent(options[name], join(staging, 'components', name)) }; + for (const name of ['node', ...names]) components[name] = { version: pins[name], files: await copyComponent(options[name], join(staging, 'components', name), buildOnly[name]) }; await prepareNodeEntrypoints(join(staging, 'components', 'node'), components.node.files); if (components.minimax) { const component = join(staging, 'components', 'minimax'); diff --git a/scripts/build-native-installer.test.mjs b/scripts/build-native-installer.test.mjs index 700ea462c..e9131568f 100644 --- a/scripts/build-native-installer.test.mjs +++ b/scripts/build-native-installer.test.mjs @@ -28,6 +28,18 @@ test('component preserves bytes and executable metadata with contained links fla assert.deepEqual(Object.keys(files).sort(), ['lib/native', 'linked/native']); }); +test('component leaves out the named build-only paths', async t => { + const { source, output } = await fixture(t); + await mkdir(join(source, 'pkg/deps'), { recursive: true }); + await mkdir(join(source, 'pkg/build'), { recursive: true }); + await writeFile(join(source, 'pkg/deps/sqlite3.c'), 'source'); + await writeFile(join(source, 'pkg/binding.gyp'), '{}'); + await writeFile(join(source, 'pkg/build/addon.node'), 'binary'); + const files = await copyComponent(source, output, ['pkg/deps', 'pkg/binding.gyp']); + assert.deepEqual(Object.keys(files), ['pkg/build/addon.node']); + await assert.rejects(lstat(join(output, 'pkg/deps'))); +}); + test('component rejects escaping directory links', async t => { const { root, source, output } = await fixture(t); await mkdir(join(root, 'outside')); await writeFile(join(root, 'outside/secret'), 'not bundled'); diff --git a/scripts/publish-core-release.py b/scripts/publish-core-release.py index 8f6308995..cb568d4b7 100644 --- a/scripts/publish-core-release.py +++ b/scripts/publish-core-release.py @@ -250,6 +250,9 @@ def publish(assets, repository, revision, tag, mode): files = sorted(assets.iterdir()) if not files or any(p.is_symlink() or not p.is_file() for p in files): raise ValueError("Expected a nonempty directory containing only regular asset files") + # The control archives carry the Core, Web and ingress images and the native + # catalog from the build to this script. Nothing installs from them, so they are not release assets. + control = {f"oac-{revision}-linux-{arch}.tar.gz{suffix}" for arch in ("amd64", "arm64") for suffix in ("", ".sha256")} # The builder validates the manifest and Runtime assets. Verify archives again # after the Actions artifact transfer between jobs. stem = "oac-" + revision + "-linux-amd64" @@ -296,6 +299,7 @@ def publish(assets, repository, revision, tag, mode): endpoint = "releases/" + str(release_id) # Keep every operation bound to the ID returned by creation. No tag lookup, # overwrite, deletion or automatic retry can select another release. + files = [p for p in files if p.name not in control] expected = {p.name: p.stat().st_size for p in files} def upload(path): print(f"Uploading {path.name} ({expected[path.name]} bytes)", flush=True) diff --git a/scripts/publish-core-release.test.py b/scripts/publish-core-release.test.py index 584b2e4b8..d2cd4b67e 100644 --- a/scripts/publish-core-release.test.py +++ b/scripts/publish-core-release.test.py @@ -68,7 +68,7 @@ def test_draft_publishes_images_and_stays_unpublished(self): self.publish(tag="build-" + self.revision, mode="draft") self.images.assert_called_once() self.assertTrue(self.release["draft"]) - self.assertEqual(len(self.release["assets"]), 28) + self.assertEqual(len(self.release["assets"]), 24) def test_missing_native_asset_refuses_release_creation(self): (self.assets / f"oac-native-{self.revision}-windows-amd64-codex.tar.gz").unlink() @@ -142,7 +142,7 @@ def response(repo, endpoint, *args): return result if endpoint == "releases/7": self.assertEqual(active, 0) - self.assertIn(len(self.release["assets"]), (26, 28)) + self.assertIn(len(self.release["assets"]), (22, 24)) return self.response(repo, endpoint, *args) self.api.side_effect = response self.publish() @@ -153,8 +153,10 @@ def test_version_tag_publishes_complete_fixed_id(self): self.publish() self.assertFalse(self.release["draft"]) self.assertFalse(self.release["prerelease"]) - self.assertEqual(len(self.release["assets"]), 28) + self.assertEqual(len(self.release["assets"]), 24) self.assertEqual({a["name"] for a in self.release["assets"] if a["name"].endswith(".yaml")}, {"compose.yaml"}) + self.assertFalse([a for a in self.release["assets"] if "-linux-amd64.tar.gz" in a["name"] and a["name"].startswith("oac-" + self.revision) + or "-linux-arm64.tar.gz" in a["name"] and a["name"].startswith("oac-" + self.revision)]) self.assertEqual(self.api.call_args.args[1:], ("releases/7", "--method", "PATCH", "-F", "draft=false"))