Stats collection crashes on Python 3.11 when a trampolined __del__ runs during GC (ContextVar re-entry) #596

Summary

On CPython 3.11, mutmut run crashes during stats collection when a trampolined __del__ runs during cyclic garbage collection. mutmut 3.7.0 and 3.8.0 are affected; 3.6.0 is not, and neither is CPython 3.12.

Versions

• mutmut 3.7.0 and 3.8.0 crash; 3.6.0 works.
• CPython 3.11.15 on Linux x86_64 crashes; CPython 3.12.13 works.

Minimal reproduction

pyproject.toml

[project]
name = "repro812"
version = "0.0.0"
requires-python = ">=3.10"

[tool.mutmut]
source_paths = ["repro812"]
pytest_add_cli_args_test_selection = ["tests"]

repro812/__init__.py: empty.

repro812/core.py

class Node:
    def __del__(self):
        self.closed = 1 + 1  # anything mutmut can mutate, so __del__ gets a trampoline


def make_garbage(n):
    a, b = Node(), Node()
    a.other, b.other = b, a  # reference cycle: freed only by the cyclic GC
    a.junk = [[] for _ in range(n)]  # vary allocations so GC fires at varying points
    return 1 + 1

tests/test_core.py

from repro812.core import make_garbage


def test_make_garbage():
    for i in range(200_000):
        assert make_garbage(i % 13) == 2

python3.11 -m pip install mutmut==3.8.0 pytest
python3.11 -m mutmut run

Expected

Stats collection succeeds and the mutants are tested. That is what happens on CPython 3.12, and with mutmut 3.6.0 on 3.11.

Actual

Running stats Fatal Python error: Segmentation fault, exit code 139, every run. The top frames are mutmut/stats.py:173 record_trampoline_hit ← trampoline.py:97 trampoline ← trampoline.py:172 _trampoline_wrapper.

In a larger project (pycubrid), which way it fails depends on how the freed memory is reused:

• trampoline.py:98 token = MutmutCallStack.set((orig_qual_name, depth + 1)) TypeError: can't concat int to bytes, leading to failed to collect stats. runner returned 1. At that point MutmutCallStack.get() returned an unrelated tuple.
• Child exited with code -11.
• On a debug build of CPython 3.11, an abort: Python/hamt.c:2368: _PyHamt_Without: Assertion 'new_o->h_count >= 0' failed.

Cause

1. Since feat: Mutation caching and transitive dependency tracking #509, released in 3.7.0, the stats-mode trampoline calls MutmutCallStack.set() / reset() (a ContextVar, defined at mutmut/core.py:9) on every trampolined call, and __del__ methods are trampolined too.
2. On CPython 3.11 and earlier, the cyclic GC can run inside ContextVar.set(), while the context's HAMT is being allocated. A trampolined __del__ then re-enters ContextVar.set() on the same variable. That is a use-after-free in CPython, tracked as Crash in HAMT during garbage collection python/cpython#148891, with fix PR gh-148891: prevent crash in HAMT python/cpython#148928. CPython 3.12 and later run the GC only at eval-breaker points, so they are not affected.

A plain-CPython script with the same set/reset pattern on a __del__ also crashes on 3.11 (exit 139), which supports this cause.

Possible fixes

• Disable GC around the call (verified): on Python < 3.12, disable the GC around the MutmutCallStack set / reset calls in mutmut/core.py and restore its previous state afterwards. With that change, the reproduction above exits 0 on 3.11.
• Skip __del__ (untested): don't trampoline __del__, or skip the ContextVar tracking inside it. Other finalizers that call trampolined functions could still trigger the crash.

Found while running mutation tests for pycubrid, which now requires Python 3.12+ for make mutation as a workaround.