Where: COMPAS/README.md and the root README.md's Results-at-a-Glance table, row 01 ("Reduction" column).
The gap: every other audit's "Reduction" column is a relative reduction, (before - after) / before. COMPAS's row alone uses the raw point difference instead.
Repro (pure arithmetic, no model re-run needed):
$ grep -n "86.77\|15.69\|71%" COMPAS/README.md
| Race | 86.77% | 15.69% | 71% |
$ python3 -c "print((86.77-15.69)/86.77*100)"
81.91771349544774
71.08 is the raw point gap (86.77 - 15.69 ≈ 71.08 ≈ "71%") - not the relative reduction, which is actually 81.9%. Every other row in the same table checks out against the relative-reduction formula:
$ python3 -c "
print('AI Fair Recruitment:', (4.51-0.12)/4.51*100) # 97.3 - matches stated 97.3%
print('German Credit:', (7.16-1.89)/7.16*100) # 73.6 - matches stated 73.6%
print('Insurance Age:', (7.93-3.18)/7.93*100) # 59.9 - matches stated ~60%
print('Insurance Gender:', (5.44-1.54)/5.44*100) # 71.7 - matches stated ~72%
print('Benefits Sex:', (18.00-8.52)/18.00*100) # 52.7 - matches stated ~53%
"
Only COMPAS's row breaks the pattern the other 6 audits (14 rows including multi-attribute ones) all follow correctly, and it appears identically in both COMPAS/README.md and the root README.md, so it isn't a one-off transcription slip.
Why it matters: the table's own convention (relative % reduction, used everywhere else) is broken for the flagship first audit, understating COMPAS's actual mitigation effectiveness (real reduction is ~82%, not 71%).
Suggested fix: change "71%" to "82%" (or "81.9%") in both COMPAS/README.md and the root README.md Results-at-a-Glance table.
Where:
COMPAS/README.mdand the rootREADME.md's Results-at-a-Glance table, row 01 ("Reduction" column).The gap: every other audit's "Reduction" column is a relative reduction,
(before - after) / before. COMPAS's row alone uses the raw point difference instead.Repro (pure arithmetic, no model re-run needed):
71.08 is the raw point gap (86.77 - 15.69 ≈ 71.08 ≈ "71%") - not the relative reduction, which is actually 81.9%. Every other row in the same table checks out against the relative-reduction formula:
Only COMPAS's row breaks the pattern the other 6 audits (14 rows including multi-attribute ones) all follow correctly, and it appears identically in both
COMPAS/README.mdand the rootREADME.md, so it isn't a one-off transcription slip.Why it matters: the table's own convention (relative % reduction, used everywhere else) is broken for the flagship first audit, understating COMPAS's actual mitigation effectiveness (real reduction is ~82%, not 71%).
Suggested fix: change "71%" to "82%" (or "81.9%") in both
COMPAS/README.mdand the rootREADME.mdResults-at-a-Glance table.