Repository navigation
fix(bootstrap): install openclaw/acpx from a committed lockfile - #239
Merged
Merged
Conversation
`npm install -g openclaw@2026.3.13 acpx@0.3.0` pinned only the two direct
packages, so every bootstrap re-resolved ~570 transitive packages from live
registry metadata. That broke on stale cached packuments (ETARGET
protobufjs@^7.5.5) and on a non-atomic AWS SDK publish (ETARGET
@aws-sdk/credential-provider-http@^3.972.86).
- Commit platform/node-tools/{package.json,package-lock.json} (Node 22.22.1,
npm 10.9.4), mirrored into the packaged assets. Versions are unchanged.
- The manifest is the single source of truth: DEFAULT_OPENCLAW_VERSION and
DEFAULT_ACPX_VERSION derive from it; the varlock default and memory-plugin
helper reuse the constant; a contract test keeps docs/env/workflow quotes
in sync.
- Bootstrap runs `npm ci --omit=dev --no-audit --no-fund` into a staging dir
under <npm prefix -g>/lib/strongclaw-node-tools, smoke-tests both CLIs,
then atomically switches a `current` symlink. Bins stay in
<npm prefix -g>/bin with the same sudo rule on Linux, so PATH resolution is
unchanged. Legacy global packages are deleted after the links are swapped,
so `npm uninstall -g` can no longer remove them. A re-run with the same
lock is a no-op.
- Retry only transient network/tarball failures with bounded backoff; drop
the `npm cache clean --force` retry. ETARGET and other resolution errors
fail immediately.
- Add tests/scripts/update_node_tools_lock.py for manual, quarantined bumps.
…allbacks - Drop `npm_config_prefer_offline`: it skipped packument revalidation, so a months-old cached packument produced ETARGET protobufjs@^7.5.5. Nothing else needs it; every other CI npm path is `npm ci` against a lock. - Key the fresh-host package cache on uv.lock plus the node-tools and memory-plugin locks, and remove the broad OS/arch restore-keys fallback. - Cache warm: probe the exact key with `lookup-only` and warm/save only on a miss, from an empty cache, so old fallback content is never re-saved under a new key. The warmer now prefetches the locked node-tools tarballs. - Dependabot: add an npm entry for platform/node-tools and its packaged mirror, grouped by dependency so both copies move together.
Deleting <prefix>/lib/node_modules/{openclaw,acpx} after migration broke
absolute-path consumers of the old `npm install -g` layout, notably the
memory plugin's openclaw/dist/extensionAPI.js fallbacks.
- After the bin links switch, replace each legacy package path with a
symlink to ../strongclaw-node-tools/current/node_modules/<pkg>, so
absolute-path readers follow every release switch. A real legacy dir is
moved aside into the tool root and pruned after success.
- npm treats the compat links as linked globals: `npm ls -g` lists them,
`npm uninstall -g <pkg>` removes only that package's links (never the
release), and `npm install -g` swaps in a floating copy; the next
bootstrap run reconciles both back to the lock.
- memory-lancedb-pro: probe the managed extensionAPI.js path ahead of each
legacy path, and add the hoisted managed node_modules to the test
NODE_PATH lists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause
clawops bootstraprannpm install -g openclaw@2026.3.13 acpx@0.3.0(sudoon Linux). Only the two direct packages were pinned. Neither tarball ships a shrinkwrap, so every bootstrap re-resolved about 570 transitive packages from live registry metadata. Run 42883800324, attempt 1, failed twice:ETARGET protobufjs@^7.5.5, although 7.5.5 was published 2026-04-15. The cause was stale cached metadata.fresh-host-core.ymlsetnpm_config_prefer_offline=true, which skips packument revalidation, and restored a broadfresh-host-packages-<os>-<arch>-fallback cache.fresh-host-cache-warm.ymlthen re-saved that fallback content under new exact keys, so stale packuments carried forward indefinitely.ETARGET @aws-sdk/credential-provider-http@^3.972.86, after thenpm cache clean --forceretry. This was a live, non-atomic AWS SDK publish.credential-provider-node@3.972.96(19:40:41Z) required the http package 29s before it was published (19:41:10Z). Cleaning the cache cannot fix this: a time-dependent resolver can always land inside such a window.Design
platform/node-tools/{package.json,package-lock.json}is committed and mirrored tosrc/clawops/assets/platform/node-toolsviasync_runtime_assets.py. It was generated with Node 22.22.1 / npm 10.9.4, and the versions are unchanged (openclaw 2026.3.13, acpx 0.3.0). 571 of 572 entries carry a registryresolvedURL and sha512integrity. The one exception islibsignal, a git dependency pinned to a 40-hex commit: npm records no integrity for git deps. The contract test allow-lists it explicitly.DEFAULT_OPENCLAW_VERSIONandDEFAULT_ACPX_VERSIONare read from the packaged manifest.platform_compat.pystays stdlib-only so the Makefile can still run it as a script. The varlock default and the memory-plugin helper reuse the constant. A contract test finds every trackedopenclaw@X/OpenClaw `X`/OPENCLAW_VERSION=X/acpx@X/ACPX `X`mention and requires it to equal the manifest.clawops/strongclaw_node_tools.py):npm ci --omit=dev --no-audit --no-fundinto<npm prefix -g>/lib/strongclaw-node-tools/.staging-<lock-digest>-<pid>.openclaw --version(printsOpenClaw 2026.3.13 (61d171a)) andacpx --version(0.3.0).<digest>, re-smokes it, then atomically switches thecurrentsymlink.<npm prefix -g>/bin/<bin> -> ../lib/strongclaw-node-tools/current/node_modules/.bin/<bin>. This is exactly wherenpm install -gput them, and Linux keeps the samesudorule. A foreign file or symlink at a bin path is a hard error, asnpm install -gwould also refuse.<prefix>/lib/node_modules/<pkg>becomes a compat symlink to../strongclaw-node-tools/current/node_modules/<pkg>.extensionAPI.jsprobe also lists the managed path directly, so it does not depend only on the compat link.npm ls -glistsopenclaw@2026.3.13 -> ./strongclaw-node-tools/<digest>/node_modules/openclaw(and acpx) as linked packages.npm uninstall -g openclawremoves only that package's compat link and bin link. It never touches the managed release (it does not follow the link). This is an explicit uninstall, with the same semantics as today.clawops bootstrapdetects the missing links and restores them (changed=True).doctor-hostalready fails with "Runclawops bootstrap" ifopenclaw/acpxis missing.npm install -g openclaw@Xover a compat link swaps in a floating install. The next bootstrap classifies it as replaceable and migrates back to the lock.npm cache cleanstep is removed.npm_config_prefer_offlineis removed. The fresh-host cache is keyed onuv.lock+ the node-tools lock + the memory-plugin lock, with no restore-keys.lookup-onlyand warms/saves only on a miss, starting from an empty cache. It now also prefetches the locked node-tools tarballs.npm ciagainst locks, except QMD's pinnednpm install, which now just revalidates.tests/scripts/update_node_tools_lock.pyhandles manual bumps: it checks for Node 22.22.1/npm 10.9.4, applies a 3-day--beforerelease quarantine, validates, smoke-installs withnpm ci, rewrites quoted versions, and syncs the mirror.Legacy global-layout consumers (audited)
Runtime, absolute
lib/node_modulespath:platform/plugins/memory-lancedb-pro/index.ts:385-393(getExtensionApiImportSpecifiers): legacy/usr,/usr/local,/opt/homebrewlib/node_modules/openclaw/dist/extensionAPI.jsfallbacks. These are served by the compat link, and the managedlib/strongclaw-node-tools/current/...path is now probed first.npm ci(strongclaw_bootstrap.py:402; the--omit=devat:485is lossless-claw). Its bare import fails becauseopenclawis not a dependency of the plugin at all.Plugin tests,
NODE_PATH. These now also list the hoisted<prefix>/lib/strongclaw-node-tools/current/node_modules, becausenpm cihoists OpenClaw's deps instead of nesting them under the package:test/cli-smoke.mjs:12-20,test/context-support-e2e.mjs:19-21,test/memory-update-supersede.test.mjs:18-20,test/memory-upgrader-diagnostics.test.mjs:8-10,test/plugin-manifest-regression.mjs:13-15,test/preference-slots.test.mjs:7-9,test/smart-extractor-branches.mjs:12-14,test/smart-metadata-v2.mjs:12-14,test/temporal-facts.test.mjs:12-14,test/vector-search-cosine.test.mjs:15-17(all underplatform/plugins/memory-lancedb-pro/)Service units (
PATHonly, bin dirs unchanged):platform/launchd/ai.openclaw.{backup-create,backup-verify,maintenance}.plist.template:34andai.openclaw.{browserlab,gateway,sidecars}.plist.template:39(/usr/local/bin:/opt/homebrew/bin:/usr/bin)platform/systemd/openclaw-{backup-create,backup-verify,gateway,maintenance}.service:15andopenclaw-{browserlab,sidecars}.service:16(/usr/local/bin:/usr/bin)tests/utils/helpers/_fresh_host/shell.py:77(fresh-host phasePATH)Doctor and other bare-name callers (resolve via
PATH, unchanged):src/clawops/setup_cli.py:129-131,153-156strongclaw_runtime.py:858,881,strongclaw_ops.py:940,strongclaw_model_auth.py:393,acp_runner.py:258,strongclaw_recovery.py:209-211,recovery/backends.py:42-49,memory_tools.py:428Other:
npm root -g/npm prefix -glookups: none outside the newstrongclaw_node_tools.py.platform/docs/HOST_PLATFORMS.md:23-24,platform/workers/acpx/README.md:20, andmemory-lancedb-pro/STRONGCLAW_VENDOR.md(local-patch notes)./usr/local/bin/openclawor/usr/bin/openclaw(tests/suites/unit/ci/test_security_workflow.py:303,338-340,tests/suites/unit/clawops/recovery/test_backends.py:14,tests/suites/unit/clawops/test_strongclaw_recovery.py:66,330) only stubshutil.which; no change needed.PATH compatibility proof
Every
src/clawopscall site resolves the CLIs by bare name on PATH:setup_clidoctor,strongclaw_runtime.run_openclaw_command,strongclaw_opsgateway,model_auth,recovery,acp_runner, andmemory_tools. The launchd/systemd units hardcodePATH=...:/usr/local/bin:/opt/homebrew/bin:/usr/bin:.... Keeping the bins in$(npm prefix -g)/bintherefore keeps every one of them working.Each OS run below did: baseline with today's
npm install -g→ new installer (migration) → legacy absolute-path import →npm ls -g→npm uninstall -g openclaw→ bootstrap re-run (restore) → re-run (no-op).macOS (Homebrew node 26.6.0 / npm 11.18.0, prefix
/usr/local), probed withenv -i HOME=$HOME /bin/zsh -land/bin/bash -l:Linux (
ubuntu:24.04container, NodeSource Node 22 / npm 10.9.9 as bootstrap installs it, non-root sudo user,sudo npm prefix -g=/usr), probed withenv -i HOME=... /bin/bash -l:Test evidence
test_strongclaw_node_tools.py, 36 tests). A real-filesystem fakenpmcovers:npm ciis used and neverinstall -gremoveof a bin path)lib/node_modules/openclaw/dist/extensionAPI.jspath follows the upgrade switchresolved/integrity, unpinned git deps, links, and drifttest_strongclaw_node_tools_npm.py, real npm + node, passes on npm 10.9.4 and 11.x):nodeimportsextensionAPI.jsthrough the legacy path, andnpm ls -glists the linksnpm uninstall -gonly unlinks and bootstrap restoresnpm install -gover a compat link is reconciled backnpm cismoke).make lint imports typecheck(ruff, isort, black, mypy, pyright: clean),make test-contracts(134 passed),make test(1352 passed;test_hosted_docker_pull_clientprocess-group test flaked once under load, passes 3/3 in isolation, untouched here), pre-commit actionlint/check-yaml/shellcheck (pass).Notes
gitstays a host prerequisite (thelibsignalgit dep), exactly as it already was fornpm install -g.memory-plugin-verification.ymlstill does a floatingnpm install openclaw@2026.3.13into a temp dir. Its version quote is now contract-checked, but the install itself is not yet lock-based.