[FabianIsensee]

NDArray.get_oselection_numpy(), set_oselection_numpy() and NDArray.oindex[...] passed their index lists to c-blosc2 unchanged, and c-blosc2's orthogonal_selection() only rejects indices greater than the shape. On a = blosc2.asarray(np.arange(100, dtype=np.int32).reshape(10, 10), chunks=(4, 4), blocks=(2, 2)):

• a.oindex[[-1], [0, 1]] returned [[81, 0]] (NumPy: [[90, 91]]), and a.oindex[[0, 1], [-3]] returned values that change between runs: a negative index addresses memory before c-blosc2's buffers. Under ASan the get writes before the block mask and the set writes before the decompressed chunk (heap-buffer-overflow).
• a.oindex[[10], [0, 1]] returned [[0, 0]] (the chunk padding) instead of raising IndexError. On an (8, 8) array with chunks (4, 4), [[0, 1], [8]] returned [[32], [40]] (items of the next row of chunks) and [[8], [0, 1]] raised RuntimeError: Error while getting orthogonal selection.

a[[...]] and take() were not affected: they normalize their indices first (_normalize_take_indices()).

The first commit adds NDArray._normalize_oselection(), called by get_oselection_numpy() and set_oselection_numpy() (which oindex uses): negative indices count from the end of their axis like in NumPy, and an index outside [-n, n) raises IndexError: index 10 is out of bounds for axis 0 with size 10 (the first such index, as NumPy reports it) before anything is read or written. The caller's lists are not changed. It works on Python lists with min()/max() rather than converting them to NumPy arrays, since it runs on every call; NumPy index arrays are turned into lists first (the extension copies list items faster than NumPy scalars). Index types are still not checked: float lists are normalized and range-checked like integer ones and then truncated by the extension as before, and boolean lists are still taken as indices 0/1 (NumPy rejects floats and treats booleans as masks). I left that alone here.

The second commit checks the value of set_oselection_numpy() / oindex[...] = value. The extension hands the value's buffer to c-blosc2 with a size computed from the selection, not from the value, so (same array as above, a.oindex[[0, 1], [0, 1]] = ...):

• 2 int32 items for the 4 of the selection were read past their end: [[-1, -2], [1129385424, 24270]] (NumPy broadcasts the row: [[-1, -2], [-1, -2]]);
• a float64 value was copied as raw bytes into the int32 array (1.5 became [0, 1073217536]), and a value of the right size but another shape was taken in C order;
• scalars and lists raised TypeError: a bytes-like object is required.
  Now any value that is not already a C-contiguous array of the selection's shape and dtype goes through NumPy's assignment into such an array: it is converted to the array's dtype and broadcast as NumPy does for assignments (leading length-1 dimensions ignored; lists via np.asarray(), so nested lists broadcast like arrays, as with NumPy's index arrays), and a shape that does not broadcast raises NumPy's ValueError: could not broadcast input array from shape (3,2) into shape (2,3) before anything is written. NDArray values are decompressed first. A value that already fits is passed on without a copy. It matches the value shape check of NDArray.__setitem__ in Check the value's shape in NDArray.__setitem__ and broadcast it as NumPy does #784.

A companion c-blosc2 PR (Blosc/c-blosc2#864) makes orthogonal_selection() itself reject indices outside [0, shape). This PR does not depend on it; the extension methods get_oindex_numpy()/set_oindex_numpy(), when called directly, rely on that C check (they get a RuntimeError instead of corrupted memory once it is in).

Tests in tests/ndarray/test_ndarray.py, all against NumPy (x[np.ix_(*key)], get and set through oindex): test_oindex_normalizes_like_numpy (negative indices, -n and n - 1, repeated indices also through negative aliases, empty lists, NumPy int64/int8/uint64 arrays, a range; the caller's indices stay unchanged), test_oindex_rejects_bad_indices (n, -n - 1, 2**63, a uint64 array, which index the message names; the array stays unchanged), and test_oindex_random_negative_indices (1-D to 3-D, random indices in [-n, n) with repeats). On main all of them fail except the two empty-key cases. For the values: test_oindex_set_value_like_numpy (13 values: the selection's shape and dtype, float, big-endian, non-contiguous, Fortran-ordered, row and column broadcasts, a leading length-1 dimension, a scalar, flat/nested/deeper-nested lists, an NDArray) and test_oindex_set_value_wrong_shape (6 values that NumPy rejects too, including int32 C-contiguous ones that reach the no-copy check; the array stays unchanged). 18 of the 19 fail on main and with the first commit alone (the same-shape, same-dtype value passes); dropping any one of the three no-copy conditions (shape, dtype, C-contiguity) fails 2 of them. Full suite: same results as main (only the 2 b2view tests that need aiohttp fail locally).

Cost (EPYC 9554, 1 thread, alternated rounds, main built twice for the noise floor, which stayed within 1.1% for these calls):

• Index check (first commit): about 1 us per call plus ~12 ns per index, for the min()/max() passes. A 2-D get of a 3x2 selection from a (10, 10) array goes from 2.51 to 3.46 us; reading 1000 indices given as a list from a 1-D array of 10**6 items from 345 to 361 us (+4.5%); the same indices as a NumPy array get faster, 411 -> 372 us, because the extension copies list items faster than NumPy scalars. Selections that take milliseconds (images, volumes, row reads; 0.4-60 ms) stay within ±2% (the ~800-1000-index ones show the +1-2% of the index passes; a noisy 20 MB box read varies by ±5-8% between rounds). The same on the previous base (abed944) and on a second machine.
• Value check (second commit): about 0.3-0.5 us per write of a value that already fits (a 3x2 int32 set: 3.81 -> 4.33 us, 8 rounds; in one process on another machine 2.74 -> 3.08 us), for the shape, dtype and contiguity tests; reads and large selections are unchanged.
• Doing the index check inside the extension's per-item copy loop, and the value check with NumPy's C API there (plus passing the buffer's real length to c-blosc2), would make both free, but that code is being reworked in another PR (Free b2nd contexts, arrays and buffers on error paths #770, error-path leaks), so I kept these changes in ndarray.py.

I used Claude Code (Anthropic) to trace the bug, write the fix and the tests, and run the measurements; I reviewed the change and the numbers.

🤖 Generated with Claude Code