Skip to content

ci: skip the self-hosted macOS tests in the merge queue - #4036

Closed
mustafab0 wants to merge 2 commits into
mainfrom
mb/ci-skip-macos-in-merge-queue
Closed

ci: skip the self-hosted macOS tests in the merge queue#4036
mustafab0 wants to merge 2 commits into
mainfrom
mb/ci-skip-macos-in-merge-queue

Conversation

@mustafab0

@mustafab0 mustafab0 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

One macOS runner serves the whole org, so a merge_group entry queues behind every other consumer of it.

Solution is to not run the macOS test in merge queue

One macOS runner serves the whole org, so a merge_group entry queues behind
every other consumer of it. Waiting long enough gets the leg cancelled, and
cancelled is neither success nor skipped, so alls-green fails ci-complete and
the queue ejects the PR. It requeues and does it again: 9 of the last 50
merge-queue ci runs were cancelled, across three authors.

continue-on-error covers a macOS failure, which is why the red X on a PR does
not block anything. It does nothing for a cancellation.

self-hosted-tests is a matrix job and job-level `if` cannot read matrix, so the
include list moves into compute-ros-pin, which already computes the container
digest the Linux leg needs. On merge_group it emits Linux only. Every pull
request still runs both legs, so nothing stops being tested before merge.
@greptile-apps

greptile-apps Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

Safe to merge: the remaining concern is non-blocking and does not change current test behavior.

Findings

  1. P2 Remove stale digest output

Summary

  • The self-hosted test matrix keeps macOS coverage for pull requests while limiting merge-queue runs to Linux. One unused job output remains and should be removed to keep the workflow contract clear.

Reviews (1) · Last reviewed commit: "ci: skip the self-hosted macOS tests in ..."

Comment thread .github/workflows/ci.yml Outdated
@codecov

codecov Bot commented Sep 10, 2026

Copy link
Copy Markdown

❌ 7 Tests Failed:

Tests completed Failed Passed Skipped
5426 7 5419 103
View the full list of 7 ❄️ flaky test(s)
dimos.experimental.memory.test_rust_recorder_e2e::test_cli_recording_uses_existing_binary_for_both_formats[mcap]

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0s run time
@pytest.fixture(scope="module")
    def rust_recorder_executable() -> Path:
>       subprocess.run(
            [
                "nix",
                "--extra-experimental-features",
                "nix-command flakes",
                "build",
                "-L",
                ".#dimos-memory-recorder",
                "--no-write-lock-file",
            ],
            cwd=_RUST_PACKAGE,
            check=True,
        )


.../experimental/memory/test_rust_recorder_e2e.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = None
        kwargs     = {'cwd': PosixPath('.../experimental/memory/rust')}
        popenargs  = (['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...],)
        timeout    = None
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
        bufsize    = -1
        c2pread    = -1
        c2pwrite   = -1
        close_fds  = True
        creationflags = 0
        cwd        = PosixPath('.../experimental/memory/rust')
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        gid        = None
        gids       = None
        group      = None
        p2cread    = -1
        p2cwrite   = -1
        pass_fds   = ()
        pipesize   = -1
        preexec_fn = None
        process_group = -1
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = None
        stdout     = None
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
args = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
executable = b'nix', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = PosixPath('.../experimental/memory/rust')
env = None, startupinfo = None, creationflags = 0, shell = False, p2cread = -1
p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1, errwrite = -1
restore_signals = True, gid = None, gids = None, uid = None, umask = -1
start_new_session = False, process_group = -1

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session, process_group):
        """Execute program (POSIX version)"""
    
        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)
    
        if shell:
            # On Android the default shell is at '....../system/bin/sh'.
            unix_shell = ('....../system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable
    
        if executable is None:
            executable = args[0]
    
        sys.audit("subprocess.Popen", executable, args, cwd, env)
    
        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and process_group == -1
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return
    
        orig_executable = executable
    
        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()
    
                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        process_group, gid, gids, uid, umask,
                        preexec_fn, _USE_VFORK)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)
    
            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)
    
            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)
    
        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass
    
            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                if err_msg == "noexec:chdir":
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                elif err_msg == "noexec":
                    err_msg = ""
                    err_filename = None
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
                if err_filename is not None:
>                   raise child_exception_type(errno_num, err_msg, err_filename)
E                   FileNotFoundError: [Errno 2] No such file or directory: 'nix'

args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
c2pread    = -1
c2pwrite   = -1
child_exception_type = <class 'OSError'>
close_fds  = True
creationflags = 0
cwd        = PosixPath('.../experimental/memory/rust')
env        = None
env_list   = None
err_filename = 'nix'
err_msg    = 'No such file or directory'
errno_num  = 2
errpipe_data = bytearray(b'OSError:2:')
errpipe_read = 299
errpipe_write = 300
errread    = -1
errwrite   = -1
exception_name = bytearray(b'OSError')
executable = b'nix'
executable_list = (b'.../dimos/dimos/.venv/bin/nix', b'....../Users/ec2-user/.local.../uv/python/nix', b'/...l/uv/0.12.12/aarch64/nix', b'....../Users/ec2-user/.local/bin/nix', b'.../homebrew/bin/nix', b'.../homebrew/sbin/nix', ...)
fds_to_keep = {300}
gid        = None
gids       = None
hex_errno  = bytearray(b'2')
low_fds_to_close = []
orig_executable = 'nix'
p2cread    = -1
p2cwrite   = -1
part       = b''
pass_fds   = ()
pid        = 14106
preexec_fn = None
process_group = -1
restore_signals = True
self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
shell      = False
start_new_session = False
startupinfo = None
sts        = 65280
uid        = None
umask      = -1

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1955: FileNotFoundError
dimos.experimental.memory.test_rust_recorder_e2e::test_cli_recording_uses_existing_binary_for_both_formats[sqlite]

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0s run time
@pytest.fixture(scope="module")
    def rust_recorder_executable() -> Path:
>       subprocess.run(
            [
                "nix",
                "--extra-experimental-features",
                "nix-command flakes",
                "build",
                "-L",
                ".#dimos-memory-recorder",
                "--no-write-lock-file",
            ],
            cwd=_RUST_PACKAGE,
            check=True,
        )


.../experimental/memory/test_rust_recorder_e2e.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = None
        kwargs     = {'cwd': PosixPath('.../experimental/memory/rust')}
        popenargs  = (['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...],)
        timeout    = None
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
        bufsize    = -1
        c2pread    = -1
        c2pwrite   = -1
        close_fds  = True
        creationflags = 0
        cwd        = PosixPath('.../experimental/memory/rust')
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        gid        = None
        gids       = None
        group      = None
        p2cread    = -1
        p2cwrite   = -1
        pass_fds   = ()
        pipesize   = -1
        preexec_fn = None
        process_group = -1
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = None
        stdout     = None
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
args = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
executable = b'nix', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = PosixPath('.../experimental/memory/rust')
env = None, startupinfo = None, creationflags = 0, shell = False, p2cread = -1
p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1, errwrite = -1
restore_signals = True, gid = None, gids = None, uid = None, umask = -1
start_new_session = False, process_group = -1

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session, process_group):
        """Execute program (POSIX version)"""
    
        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)
    
        if shell:
            # On Android the default shell is at '....../system/bin/sh'.
            unix_shell = ('....../system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable
    
        if executable is None:
            executable = args[0]
    
        sys.audit("subprocess.Popen", executable, args, cwd, env)
    
        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and process_group == -1
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return
    
        orig_executable = executable
    
        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()
    
                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        process_group, gid, gids, uid, umask,
                        preexec_fn, _USE_VFORK)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)
    
            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)
    
            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)
    
        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass
    
            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                if err_msg == "noexec:chdir":
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                elif err_msg == "noexec":
                    err_msg = ""
                    err_filename = None
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
                if err_filename is not None:
>                   raise child_exception_type(errno_num, err_msg, err_filename)
E                   FileNotFoundError: [Errno 2] No such file or directory: 'nix'

args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
c2pread    = -1
c2pwrite   = -1
child_exception_type = <class 'OSError'>
close_fds  = True
creationflags = 0
cwd        = PosixPath('.../experimental/memory/rust')
env        = None
env_list   = None
err_filename = 'nix'
err_msg    = 'No such file or directory'
errno_num  = 2
errpipe_data = bytearray(b'OSError:2:')
errpipe_read = 299
errpipe_write = 300
errread    = -1
errwrite   = -1
exception_name = bytearray(b'OSError')
executable = b'nix'
executable_list = (b'.../dimos/dimos/.venv/bin/nix', b'....../Users/ec2-user/.local.../uv/python/nix', b'/...l/uv/0.12.12/aarch64/nix', b'....../Users/ec2-user/.local/bin/nix', b'.../homebrew/bin/nix', b'.../homebrew/sbin/nix', ...)
fds_to_keep = {300}
gid        = None
gids       = None
hex_errno  = bytearray(b'2')
low_fds_to_close = []
orig_executable = 'nix'
p2cread    = -1
p2cwrite   = -1
part       = b''
pass_fds   = ()
pid        = 14106
preexec_fn = None
process_group = -1
restore_signals = True
self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
shell      = False
start_new_session = False
startupinfo = None
sts        = 65280
uid        = None
umask      = -1

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1955: FileNotFoundError
dimos.experimental.memory.test_rust_recorder_e2e::test_rust_artifact_is_readable_by_python_memory2[mcap]

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0s run time
@pytest.fixture(scope="module")
    def rust_recorder_executable() -> Path:
>       subprocess.run(
            [
                "nix",
                "--extra-experimental-features",
                "nix-command flakes",
                "build",
                "-L",
                ".#dimos-memory-recorder",
                "--no-write-lock-file",
            ],
            cwd=_RUST_PACKAGE,
            check=True,
        )


.../experimental/memory/test_rust_recorder_e2e.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = None
        kwargs     = {'cwd': PosixPath('.../experimental/memory/rust')}
        popenargs  = (['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...],)
        timeout    = None
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
        bufsize    = -1
        c2pread    = -1
        c2pwrite   = -1
        close_fds  = True
        creationflags = 0
        cwd        = PosixPath('.../experimental/memory/rust')
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        gid        = None
        gids       = None
        group      = None
        p2cread    = -1
        p2cwrite   = -1
        pass_fds   = ()
        pipesize   = -1
        preexec_fn = None
        process_group = -1
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = None
        stdout     = None
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
args = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
executable = b'nix', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = PosixPath('.../experimental/memory/rust')
env = None, startupinfo = None, creationflags = 0, shell = False, p2cread = -1
p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1, errwrite = -1
restore_signals = True, gid = None, gids = None, uid = None, umask = -1
start_new_session = False, process_group = -1

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session, process_group):
        """Execute program (POSIX version)"""
    
        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)
    
        if shell:
            # On Android the default shell is at '....../system/bin/sh'.
            unix_shell = ('....../system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable
    
        if executable is None:
            executable = args[0]
    
        sys.audit("subprocess.Popen", executable, args, cwd, env)
    
        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and process_group == -1
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return
    
        orig_executable = executable
    
        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()
    
                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        process_group, gid, gids, uid, umask,
                        preexec_fn, _USE_VFORK)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)
    
            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)
    
            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)
    
        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass
    
            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                if err_msg == "noexec:chdir":
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                elif err_msg == "noexec":
                    err_msg = ""
                    err_filename = None
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
                if err_filename is not None:
>                   raise child_exception_type(errno_num, err_msg, err_filename)
E                   FileNotFoundError: [Errno 2] No such file or directory: 'nix'

args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
c2pread    = -1
c2pwrite   = -1
child_exception_type = <class 'OSError'>
close_fds  = True
creationflags = 0
cwd        = PosixPath('.../experimental/memory/rust')
env        = None
env_list   = None
err_filename = 'nix'
err_msg    = 'No such file or directory'
errno_num  = 2
errpipe_data = bytearray(b'OSError:2:')
errpipe_read = 299
errpipe_write = 300
errread    = -1
errwrite   = -1
exception_name = bytearray(b'OSError')
executable = b'nix'
executable_list = (b'.../dimos/dimos/.venv/bin/nix', b'....../Users/ec2-user/.local.../uv/python/nix', b'/...l/uv/0.12.12/aarch64/nix', b'....../Users/ec2-user/.local/bin/nix', b'.../homebrew/bin/nix', b'.../homebrew/sbin/nix', ...)
fds_to_keep = {300}
gid        = None
gids       = None
hex_errno  = bytearray(b'2')
low_fds_to_close = []
orig_executable = 'nix'
p2cread    = -1
p2cwrite   = -1
part       = b''
pass_fds   = ()
pid        = 14106
preexec_fn = None
process_group = -1
restore_signals = True
self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
shell      = False
start_new_session = False
startupinfo = None
sts        = 65280
uid        = None
umask      = -1

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1955: FileNotFoundError
dimos.experimental.memory.test_rust_recorder_e2e::test_rust_artifact_is_readable_by_python_memory2[sqlite]

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0.018s run time
@pytest.fixture(scope="module")
    def rust_recorder_executable() -> Path:
>       subprocess.run(
            [
                "nix",
                "--extra-experimental-features",
                "nix-command flakes",
                "build",
                "-L",
                ".#dimos-memory-recorder",
                "--no-write-lock-file",
            ],
            cwd=_RUST_PACKAGE,
            check=True,
        )


.../experimental/memory/test_rust_recorder_e2e.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = None
        kwargs     = {'cwd': PosixPath('.../experimental/memory/rust')}
        popenargs  = (['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...],)
        timeout    = None
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
        bufsize    = -1
        c2pread    = -1
        c2pwrite   = -1
        close_fds  = True
        creationflags = 0
        cwd        = PosixPath('.../experimental/memory/rust')
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        gid        = None
        gids       = None
        group      = None
        p2cread    = -1
        p2cwrite   = -1
        pass_fds   = ()
        pipesize   = -1
        preexec_fn = None
        process_group = -1
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = None
        stdout     = None
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
args = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
executable = b'nix', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = PosixPath('.../experimental/memory/rust')
env = None, startupinfo = None, creationflags = 0, shell = False, p2cread = -1
p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1, errwrite = -1
restore_signals = True, gid = None, gids = None, uid = None, umask = -1
start_new_session = False, process_group = -1

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session, process_group):
        """Execute program (POSIX version)"""
    
        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)
    
        if shell:
            # On Android the default shell is at '....../system/bin/sh'.
            unix_shell = ('....../system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable
    
        if executable is None:
            executable = args[0]
    
        sys.audit("subprocess.Popen", executable, args, cwd, env)
    
        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and process_group == -1
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return
    
        orig_executable = executable
    
        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()
    
                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        process_group, gid, gids, uid, umask,
                        preexec_fn, _USE_VFORK)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)
    
            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)
    
            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)
    
        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass
    
            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                if err_msg == "noexec:chdir":
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                elif err_msg == "noexec":
                    err_msg = ""
                    err_filename = None
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
                if err_filename is not None:
>                   raise child_exception_type(errno_num, err_msg, err_filename)
E                   FileNotFoundError: [Errno 2] No such file or directory: 'nix'

args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
c2pread    = -1
c2pwrite   = -1
child_exception_type = <class 'OSError'>
close_fds  = True
creationflags = 0
cwd        = PosixPath('.../experimental/memory/rust')
env        = None
env_list   = None
err_filename = 'nix'
err_msg    = 'No such file or directory'
errno_num  = 2
errpipe_data = bytearray(b'OSError:2:')
errpipe_read = 299
errpipe_write = 300
errread    = -1
errwrite   = -1
exception_name = bytearray(b'OSError')
executable = b'nix'
executable_list = (b'.../dimos/dimos/.venv/bin/nix', b'....../Users/ec2-user/.local.../uv/python/nix', b'/...l/uv/0.12.12/aarch64/nix', b'....../Users/ec2-user/.local/bin/nix', b'.../homebrew/bin/nix', b'.../homebrew/sbin/nix', ...)
fds_to_keep = {300}
gid        = None
gids       = None
hex_errno  = bytearray(b'2')
low_fds_to_close = []
orig_executable = 'nix'
p2cread    = -1
p2cwrite   = -1
part       = b''
pass_fds   = ()
pid        = 14106
preexec_fn = None
process_group = -1
restore_signals = True
self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
shell      = False
start_new_session = False
startupinfo = None
sts        = 65280
uid        = None
umask      = -1

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1955: FileNotFoundError
dimos.experimental.memory.test_rust_recorder_e2e::test_tf_records_over_zenoh_and_replays_through_python

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0s run time
@pytest.fixture(scope="module")
    def rust_recorder_executable() -> Path:
>       subprocess.run(
            [
                "nix",
                "--extra-experimental-features",
                "nix-command flakes",
                "build",
                "-L",
                ".#dimos-memory-recorder",
                "--no-write-lock-file",
            ],
            cwd=_RUST_PACKAGE,
            check=True,
        )


.../experimental/memory/test_rust_recorder_e2e.py:78: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:548: in run
    with Popen(*popenargs, **kwargs) as process:
        capture_output = False
        check      = True
        input      = None
        kwargs     = {'cwd': PosixPath('.../experimental/memory/rust')}
        popenargs  = (['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...],)
        timeout    = None
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1026: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
        args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
        bufsize    = -1
        c2pread    = -1
        c2pwrite   = -1
        close_fds  = True
        creationflags = 0
        cwd        = PosixPath('.../experimental/memory/rust')
        encoding   = None
        env        = None
        errors     = None
        errread    = -1
        errwrite   = -1
        executable = None
        extra_groups = None
        gid        = None
        gids       = None
        group      = None
        p2cread    = -1
        p2cwrite   = -1
        pass_fds   = ()
        pipesize   = -1
        preexec_fn = None
        process_group = -1
        restore_signals = True
        self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
        shell      = False
        start_new_session = False
        startupinfo = None
        stderr     = None
        stdin      = None
        stdout     = None
        text       = None
        uid        = None
        umask      = -1
        universal_newlines = None
        user       = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
args = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
executable = b'nix', preexec_fn = None, close_fds = True, pass_fds = ()
cwd = PosixPath('.../experimental/memory/rust')
env = None, startupinfo = None, creationflags = 0, shell = False, p2cread = -1
p2cwrite = -1, c2pread = -1, c2pwrite = -1, errread = -1, errwrite = -1
restore_signals = True, gid = None, gids = None, uid = None, umask = -1
start_new_session = False, process_group = -1

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session, process_group):
        """Execute program (POSIX version)"""
    
        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)
    
        if shell:
            # On Android the default shell is at '....../system/bin/sh'.
            unix_shell = ('....../system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable
    
        if executable is None:
            executable = args[0]
    
        sys.audit("subprocess.Popen", executable, args, cwd, env)
    
        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and process_group == -1
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return
    
        orig_executable = executable
    
        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()
    
                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
                self.pid = _fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        process_group, gid, gids, uid, umask,
                        preexec_fn, _USE_VFORK)
                self._child_created = True
            finally:
                # be sure the FD is closed no matter what
                os.close(errpipe_write)
    
            self._close_pipe_fds(p2cread, p2cwrite,
                                 c2pread, c2pwrite,
                                 errread, errwrite)
    
            # Wait for exec to fail or succeed; possibly raising an
            # exception (limited in size)
            errpipe_data = bytearray()
            while True:
                part = os.read(errpipe_read, 50000)
                errpipe_data += part
                if not part or len(errpipe_data) > 50000:
                    break
        finally:
            # be sure the FD is closed no matter what
            os.close(errpipe_read)
    
        if errpipe_data:
            try:
                pid, sts = os.waitpid(self.pid, 0)
                if pid == self.pid:
                    self._handle_exitstatus(sts)
                else:
                    self.returncode = sys.maxsize
            except ChildProcessError:
                pass
    
            try:
                exception_name, hex_errno, err_msg = (
                        errpipe_data.split(b':', 2))
                # The encoding here should match the encoding
                # written in by the subprocess implementations
                # like _posixsubprocess
                err_msg = err_msg.decode()
            except ValueError:
                exception_name = b'SubprocessError'
                hex_errno = b'0'
                err_msg = 'Bad exception data from child: {!r}'.format(
                              bytes(errpipe_data))
            child_exception_type = getattr(
                    builtins, exception_name.decode('ascii'),
                    SubprocessError)
            if issubclass(child_exception_type, OSError) and hex_errno:
                errno_num = int(hex_errno, 16)
                if err_msg == "noexec:chdir":
                    err_msg = ""
                    # The error must be from chdir(cwd).
                    err_filename = cwd
                elif err_msg == "noexec":
                    err_msg = ""
                    err_filename = None
                else:
                    err_filename = orig_executable
                if errno_num != 0:
                    err_msg = os.strerror(errno_num)
                if err_filename is not None:
>                   raise child_exception_type(errno_num, err_msg, err_filename)
E                   FileNotFoundError: [Errno 2] No such file or directory: 'nix'

args       = ['nix', '--extra-experimental-features', 'nix-command flakes', 'build', '-L', '.#dimos-memory-recorder', ...]
c2pread    = -1
c2pwrite   = -1
child_exception_type = <class 'OSError'>
close_fds  = True
creationflags = 0
cwd        = PosixPath('.../experimental/memory/rust')
env        = None
env_list   = None
err_filename = 'nix'
err_msg    = 'No such file or directory'
errno_num  = 2
errpipe_data = bytearray(b'OSError:2:')
errpipe_read = 299
errpipe_write = 300
errread    = -1
errwrite   = -1
exception_name = bytearray(b'OSError')
executable = b'nix'
executable_list = (b'.../dimos/dimos/.venv/bin/nix', b'....../Users/ec2-user/.local.../uv/python/nix', b'/...l/uv/0.12.12/aarch64/nix', b'....../Users/ec2-user/.local/bin/nix', b'.../homebrew/bin/nix', b'.../homebrew/sbin/nix', ...)
fds_to_keep = {300}
gid        = None
gids       = None
hex_errno  = bytearray(b'2')
low_fds_to_close = []
orig_executable = 'nix'
p2cread    = -1
p2cwrite   = -1
part       = b''
pass_fds   = ()
pid        = 14106
preexec_fn = None
process_group = -1
restore_signals = True
self       = <Popen: returncode: 255 args: ['nix', '--extra-experimental-features', 'nix-...>
shell      = False
start_new_session = False
startupinfo = None
sts        = 65280
uid        = None
umask      = -1

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/subprocess.py:1955: FileNotFoundError
dimos.hardware.whole_body.dual_openyam_damiao.test_adapter::test_adapter_connects_complete_dual_yam_topology

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0.001s run time
mocker = <pytest_mock.plugin.MockerFixture object at 0x3b1a152b0>

    @pytest.fixture
    def adapter(mocker: MockerFixture) -> Iterator[DualOpenYamDamiaoAdapter]:
>       mocker.patch.object(can_motor_control, "SocketCanBus", can_motor_control.MockCanBus)

mocker     = <pytest_mock.plugin.MockerFixture object at 0x3b1a152b0>

.../whole_body/dual_openyam_damiao/test_adapter.py:32: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.12....../site-packages/pytest_mock/plugin.py:294: in object
    return self._start_patch(
        attribute  = 'SocketCanBus'
        autospec   = None
        create     = False
        kwargs     = {}
        new        = <class 'can_motor_control.MockCanBus'>
        new_callable = None
        self       = <pytest_mock.plugin.MockerFixture._Patcher object at 0x3b1a15310>
        spec       = None
        spec_set   = None
        target     = <module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'>
.venv/lib/python3.12....../site-packages/pytest_mock/plugin.py:263: in _start_patch
    mocked: MockType = p.start()
        args       = (<module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'>, 'SocketCanBus')
        kwargs     = {'autospec': None, 'create': False, 'new': <class 'can_motor_control.MockCanBus'>, 'new_callable': None, ...}
        mock_func  = <function _patch_object at 0x103c7f2e0>
        p          = <unittest.mock._patch object at 0x3b1a15340>
        self       = <pytest_mock.plugin.MockerFixture._Patcher object at 0x3b1a15310>
        warn_on_mock_enter = True
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/unittest/mock.py:1624: in start
    result = self.__enter__()
        self       = <unittest.mock._patch object at 0x3b1a15340>
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/unittest/mock.py:1467: in __enter__
    original, local = self.get_original()
        autospec   = None
        kwargs     = {}
        new        = <class 'can_motor_control.MockCanBus'>
        new_callable = None
        self       = <unittest.mock._patch object at 0x3b1a15340>
        spec       = None
        spec_set   = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <unittest.mock._patch object at 0x3b1a15340>

    def get_original(self):
        target = self.getter()
        name = self.attribute
    
        original = DEFAULT
        local = False
    
        try:
            original = target.__dict__[name]
        except (AttributeError, KeyError):
            original = getattr(target, name, DEFAULT)
        else:
            local = True
    
        if name in _builtins and isinstance(target, ModuleType):
            self.create = True
    
        if not self.create and original is DEFAULT:
>           raise AttributeError(
                "%s does not have the attribute %r" % (target, name)
            )
E           AttributeError: <module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'> does not have the attribute 'SocketCanBus'

local      = False
name       = 'SocketCanBus'
original   = sentinel.DEFAULT
self       = <unittest.mock._patch object at 0x3b1a15340>
target     = <module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'>

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/unittest/mock.py:1437: AttributeError
dimos.hardware.whole_body.openarm_damiao.test_adapter::test_openarm_feedback_limits_match_urdf_joint_limits

Flake rate in main: 33.33% (Passed 6 times, Failed 3 times)

Stack Traces | 0.001s run time
mocker = <pytest_mock.plugin.MockerFixture object at 0x3b1a47c50>

    @pytest.fixture
    def openarm_adapter(mocker: MockerFixture) -> Iterator[OpenArmDamiaoAdapter]:
>       mocker.patch.object(can_motor_control, "SocketCanBus", can_motor_control.MockCanBus)

mocker     = <pytest_mock.plugin.MockerFixture object at 0x3b1a47c50>

.../whole_body/openarm_damiao/test_adapter.py:33: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.venv/lib/python3.12....../site-packages/pytest_mock/plugin.py:294: in object
    return self._start_patch(
        attribute  = 'SocketCanBus'
        autospec   = None
        create     = False
        kwargs     = {}
        new        = <class 'can_motor_control.MockCanBus'>
        new_callable = None
        self       = <pytest_mock.plugin.MockerFixture._Patcher object at 0x3b1a47ce0>
        spec       = None
        spec_set   = None
        target     = <module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'>
.venv/lib/python3.12....../site-packages/pytest_mock/plugin.py:263: in _start_patch
    mocked: MockType = p.start()
        args       = (<module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'>, 'SocketCanBus')
        kwargs     = {'autospec': None, 'create': False, 'new': <class 'can_motor_control.MockCanBus'>, 'new_callable': None, ...}
        mock_func  = <function _patch_object at 0x103c7f2e0>
        p          = <unittest.mock._patch object at 0x3b1a47d10>
        self       = <pytest_mock.plugin.MockerFixture._Patcher object at 0x3b1a47ce0>
        warn_on_mock_enter = True
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/unittest/mock.py:1624: in start
    result = self.__enter__()
        self       = <unittest.mock._patch object at 0x3b1a47d10>
../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/unittest/mock.py:1467: in __enter__
    original, local = self.get_original()
        autospec   = None
        kwargs     = {}
        new        = <class 'can_motor_control.MockCanBus'>
        new_callable = None
        self       = <unittest.mock._patch object at 0x3b1a47d10>
        spec       = None
        spec_set   = None
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <unittest.mock._patch object at 0x3b1a47d10>

    def get_original(self):
        target = self.getter()
        name = self.attribute
    
        original = DEFAULT
        local = False
    
        try:
            original = target.__dict__[name]
        except (AttributeError, KeyError):
            original = getattr(target, name, DEFAULT)
        else:
            local = True
    
        if name in _builtins and isinstance(target, ModuleType):
            self.create = True
    
        if not self.create and original is DEFAULT:
>           raise AttributeError(
                "%s does not have the attribute %r" % (target, name)
            )
E           AttributeError: <module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'> does not have the attribute 'SocketCanBus'

local      = False
name       = 'SocketCanBus'
original   = sentinel.DEFAULT
self       = <unittest.mock._patch object at 0x3b1a47d10>
target     = <module 'can_motor_control' from '.../dimos/dimos/.venv/lib/python3.12............/site-packages/can_motor_control/__init__.py'>

../../../../..../uv/python/cpython-3.12.13-macos-aarch64-none/lib/python3.12/unittest/mock.py:1437: AttributeError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Sep 10, 2026
@mustafab0 mustafab0 closed this Sep 10, 2026
@mustafab0
mustafab0 deleted the mb/ci-skip-macos-in-merge-queue branch September 10, 2026 14:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant