Skip to content

fix: unify native package preparation for pip installs - #3970

Draft
TomCC7 wants to merge 1 commit into
mainfrom
cc/fix/rust-recorder-chore
Draft

fix: unify native package preparation for pip installs#3970
TomCC7 wants to merge 1 commit into
mainfrom
cc/fix/rust-recorder-chore

Conversation

@TomCC7

@TomCC7 TomCC7 commented Sep 8, 2026

Copy link
Copy Markdown
Member

Contribution path

Follow-up to #3924 and the reported pip-install, MCAP dependency, and first-use build issues.

Problem

The Rust recorder launches from a checkout-relative directory that is absent from pip installations. The other Cachix-published native modules also carry their own checkout build paths. Core installations additionally lack MCAP support, and the recorder does not inherit a transport's custom LCM URL.

Solution

  • Share one native package catalog and preparation path across the Rust recorder, Livox Mid360, PointLIO, and FastLIO2. Nix reuses local outputs, substitutes from dimensionalos Cachix, or builds with one job and two cores.
  • Embed an immutable source revision in wheels and source distributions; evaluate local native inputs for editable checkouts. Add dimos native prepare <package-id> and use the same catalog in the cache publisher.
  • Remove the selected modules' checkout-relative launch paths. Preserve their existing process lifecycle.
  • Include MCAP in core dependencies and decode built-in DimOS message channels for dimos mem rerun. Pass the recorder's selected LCM URL and reject incompatible connections before launch.
  • Defer optional hardware imports so the core-only installed CLI loads. Document the preparation contract and architectural decision.

Nix remains a runtime requirement. Other native build workflows are outside this change.

How to Test

dimos native prepare dimos-memory-recorder

For installed-wheel recording and replay, run the included smoke script from outside the checkout using a core-only installed environment:

/path/to/installed-env/bin/python /path/to/repo/.github/scripts/native_wheel_smoke.py

Passed locally on Linux:

  • 97 focused Python tests, 5 real recorder end-to-end tests, and 11 Rust tests.
  • Real preparation of all four packages; lidar outputs downloaded from Cachix.
  • Core-only wheel installation recorded SQLite and MCAP and generated a Rerun file from MCAP.
  • Git-free sdist-to-wheel build preserved the source revision and package catalog.
  • Python-only edits reused the Nix derivation; Rust edits selected a new derivation.
  • Mypy on 10 changed runtime files and all applicable pre-commit hooks.

No robot hardware validation. Local packaging used the supported Cockpit-assets opt-out; full release CI remains to run.

AI assistance

Codex (GPT-6) assisted with investigation, implementation, documentation, and validation under maintainer direction.

Checklist

  • I have read and approved the CLA.

@codecov

codecov Bot commented Sep 8, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
4910 1 4909 182
View the top 1 failed test(s) by shortest run time
dimos.codebase_checks.test_docs_branding::test_docs_use_current_branding
Stack Traces | 0.014s run time
def test_docs_use_current_branding() -> None:
        """Fail if any file under docs/ spells the brand "DimOS" instead of "dimOS"."""
        hits = find_old_branding()
        if hits:
            listing = "\n".join(
                f"  - {p.relative_to(DIMOS_PROJECT_ROOT)}:{lineno}: {line.strip()}"
                for p, lineno, line in hits
            )
>           raise AssertionError(f'Found "DimOS" in docs/:\n{listing}\n\nThe brand is spelled "dimOS".')
E           AssertionError: Found "DimOS" in docs/:
E             - docs/adr/0001-native-package-preparation.md:3: DimOS native launchers depended on build directories beside Python modules,
E             - docs/usage/native_modules.md:285: Nix with flakes enabled is required. A pip-installed DimOS uses the immutable
E             - docs/usage/native_modules.md:303: links under the DimOS cache root. Runtime processes do not need a source-tree
E             - docs/usage/native_modules.md:319: DimOS requests the same cache CI publishes to on each Nix invocation. For a
E             - docs/usage/native_modules.md:328: Follow Nix's installation instructions when configuring daemon trust. DimOS
E             - docs/usage/recording.md:42: MCAP reading is included in the core Python dependencies. Built-in DimOS
E           
E           The brand is spelled "dimOS".

hits       = [(PosixPath('.../docs/adr/0001-native-package-preparation.md'), 3, 'DimOS native launchers d....../docs/usage/recording.md'), 42, 'MCAP reading is included in the core Python dependencies. Built-in DimOS')]
listing    = '  - docs/adr/0001-native-package-preparation.md:3: DimOS native launchers depended on build directories beside Python...trust. DimOS\n  - docs/usage/recording.md:42: MCAP reading is included in the core Python dependencies. Built-in DimOS'

dimos/codebase_checks/test_docs_branding.py:43: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant