Pandas version checks
Reproducible Example
import numpy as np
n, k = 6, 10
# Current test code: fresh generator instantiated per iteration
codes = [np.random.default_rng(2).choice(n, k * n) for _ in range(3)]
# The arrays have identical values
print(all(np.array_equal(codes[0], code) for code in codes[1:])) # True
# The MultiIndex built from them has only 6 distinct label tuples
print(len(set(zip(*codes)))) # 6
# Proposed version: one generator, successive draws
rng = np.random.default_rng(2)
codes = [rng.choice(n, k * n) for _ in range(3)]
print(len(set(zip(*codes)))) # 53
Issue Description
In test_duplicated_hashtable_impl, the codes for each level are generated like this:
codes = [np.random.default_rng(2).choice(n, k * n) for _ in levels]
Because default_rng(2) is instantiated inside the comprehension, a new generator with the same seed is created on every iteration, and each iteration draws the identical sequence. As a result, the three levels receive equal-valued code arrays (i.e. they are separate objects, but element-wise they are identical).
Before PR #54209, the test called the shared NumPy RNG once per level. The current behavior deviates from that. Every level's codes match position-by-position, so the MultiIndex collapses to only 6 distinct label tuples. Hence, the current test exercises duplicate detection over a much narrower input than what seems to be intended.
AI Assistance Disclosure
I found this bug while researching a prospective Ruff rule for detecting repeated RNG resets inside comprehensions. For that work, I'm assisted by OpenAI Codex gpt-5.6-sol xhigh. However, I personally verified the bug and wrote all of the code here myself.
Expected Behavior
Each level's codes should be a distinct draw from the RNG. We can do this by generating the codes as successive samples from one advancing generator:
rng = np.random.default_rng(2)
codes = [rng.choice(n, k * n) for _ in levels]
With the fix, we obtain 53 distinct tuples for the same seed, instead of the 6 tuples returned by the current behavior.
Installed Versions
Details
INSTALLED VERSIONS
commit : a3cbd38
python : 3.14.3
python-bits : 64
OS : Darwin
OS-release : 25.3.0
Version : Darwin Kernel Version 25.3.0: Wed Jan 28 20:53:15 PST 2026;
root:xnu-12377.81.4~5/RELEASE_ARM64_T6000
machine : arm64
processor : arm
byteorder : little
LC_ALL : C.UTF-8
LANG : C.UTF-8
LOCALE : C.UTF-8
pandas : 3.1.0.dev0+1573.ga3cbd3804c.dirty
numpy : 2.5.2
dateutil : 2.9.0.post0
pip : 26.0.1
Cython : None
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
psycopg2 : None
pymysql : None
pyarrow : None
pyiceberg : None
pyreadstat : None
pytest : None
python-calamine : None
pytz : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
qtpy : None
pyqt5 : None
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
In
test_duplicated_hashtable_impl, the codes for each level are generated like this:Because
default_rng(2)is instantiated inside the comprehension, a new generator with the same seed is created on every iteration, and each iteration draws the identical sequence. As a result, the three levels receive equal-valued code arrays (i.e. they are separate objects, but element-wise they are identical).Before PR #54209, the test called the shared NumPy RNG once per level. The current behavior deviates from that. Every level's codes match position-by-position, so the MultiIndex collapses to only 6 distinct label tuples. Hence, the current test exercises duplicate detection over a much narrower input than what seems to be intended.
AI Assistance Disclosure
I found this bug while researching a prospective Ruff rule for detecting repeated RNG resets inside comprehensions. For that work, I'm assisted by OpenAI Codex gpt-5.6-sol xhigh. However, I personally verified the bug and wrote all of the code here myself.
Expected Behavior
Each level's codes should be a distinct draw from the RNG. We can do this by generating the codes as successive samples from one advancing generator:
With the fix, we obtain 53 distinct tuples for the same seed, instead of the 6 tuples returned by the current behavior.
Installed Versions
Details
INSTALLED VERSIONS
commit : a3cbd38
python : 3.14.3
python-bits : 64
OS : Darwin
OS-release : 25.3.0
Version : Darwin Kernel Version 25.3.0: Wed Jan 28 20:53:15 PST 2026;
root:xnu-12377.81.4~5/RELEASE_ARM64_T6000
machine : arm64
processor : arm
byteorder : little
LC_ALL : C.UTF-8
LANG : C.UTF-8
LOCALE : C.UTF-8
pandas : 3.1.0.dev0+1573.ga3cbd3804c.dirty
numpy : 2.5.2
dateutil : 2.9.0.post0
pip : 26.0.1
Cython : None
sphinx : None
IPython : None
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : None
bottleneck : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : None
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
psycopg2 : None
pymysql : None
pyarrow : None
pyiceberg : None
pyreadstat : None
pytest : None
python-calamine : None
pytz : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
qtpy : None
pyqt5 : None