fix(data): merge 18 duplicate Intel Atom CPU records by Seungpyo1007 · Pull Request #321 · GetTechAPI/TechAPI

Seungpyo1007 · 2026-09-28T18:38:05Z

What & why

TechEngine's integrity_check.py --strict gate has been hard-blocking weekly-refresh for at least the last 5-6 runs (all failures/cancellations since 2026-08-31), previously misdiagnosed as CI infra flakiness. The real cause: 18 Intel Atom CPUs were each imported twice under two slugs — once without an intel- prefix (Wikipedia-sourced, older) and once with it (a later Kaggle-dataset import) — and the strict gate treats same-name CPU records as a hard anomaly.

These were not harmless exact duplicates; the two sides disagreed on release_date, threads, and other fields for most pairs.

How each pair was resolved

Cross-checked every pair against https://en.wikipedia.org/wiki/List_of_Intel_Atom_processors:

release_date: the atom-X (Wikipedia-cited) record matches the table almost exactly, often to the exact day. The intel-atom-X record's dates are frequently wrong — in two cases (Z3530, Z3560) off by a full 8 years (claiming 2022 releases for 2014 chips).
threads: Wikipedia lists Hyper-Threading support for 17 of these 18 SKUs (only D2500 does not support it). The intel-atom-X record already correctly encoded threads = 2× cores; the atom-X record wrongly had threads = cores for 16 of them (missing HT).
clock/tdp/cache/socket/process_node: spot-checked the same way; kept whichever side matched Wikipedia — mostly the atom-X side (e.g. the intel-atom-z3560 record had base_clock_ghz=1.0 vs Wikipedia's stated 1.83 GHz).

Each merged record keeps the union of both sides' source_urls, survives under the shorter atom-X slug/file path, and stays verified: true. The corresponding stale dump pages for the 18 deleted intel-atom-X slugs are removed too (the dump generator doesn't prune records that no longer exist, so those pages would otherwise have stayed orphaned).

Verification

python integrity_check.py <data> --strict (the exact command weekly-refresh runs) now reports 0 hard anomalies, down from 18.

Closes #296

The strict integrity gate (integrity_check.py --strict) has been hard-blocking weekly-refresh for at least the last 5 runs: 18 Intel Atom CPUs were each imported twice under two slugs, once without an "intel-" prefix (Wikipedia- sourced) and once with it (a later Kaggle-dataset import). These were not harmless duplicates - the two sides disagreed on release_date, threads and other fields for most pairs. Cross-checked every pair against https://en.wikipedia.org/wiki/List_of_Intel_Atom_processors: - release_date: the "atom-X" (Wikipedia) record matches the table almost exactly, often to the day. The "intel-atom-X" record's dates are frequently wrong, in two cases (Z3530, Z3560) off by a full 8 years. - threads: Wikipedia lists Hyper-Threading support for 17 of these 18 SKUs (only D2500 does not). The "intel-atom-X" record already encoded threads = 2x cores; the "atom-X" record wrongly had threads = cores for 16 of them. - clock/tdp/cache/socket/process_node: spot-checked the same way; kept whichever side matched Wikipedia (mostly the "atom-X" side, e.g. Z3560's "intel-atom-z3560" record had clock_ghz=1.0 vs Wikipedia's stated 1.83). Each merged record keeps the union of both sides' source_urls and survives under the shorter "atom-X" slug/path. `integrity_check.py --strict` now reports 0 hard anomalies. Closes #296

TechEngineBot · 2026-09-28T18:42:21Z

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

12/12 categories ? 191,819 records ? diff base develop@c317ed9

Changed records in this PR

18 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 18
    "Yellow" : 0
    "Red" : 0

Category	Distribution	Total	🟢	🟡	🔴	🟢 %
cpu	🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩	18	18	0	0	100.0%
All	🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩	18	18	0	0	100.0%

Full-dataset baseline

191819 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 32772
    "Yellow" : 157992
    "Red" : 1055

Category	Distribution	Total	🟢	🟡	🔴	🟢 %
brand	🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨	207	122	85	0	58.9%
soc	🟩🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥	2117	284	837	996	13.4%
smartphone	🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥	93407	18003	75350	54	19.3%
tablet	🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥	3459	292	3166	1	8.4%
watch	🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥	447	94	350	3	21.0%
pda	🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨	140	83	57	0	59.3%
gpu	🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨	2076	1529	547	0	73.7%
cpu	🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥	4512	4219	292	1	93.5%
laptop	🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨	1995	107	1888	0	5.4%
monitor	🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨	882	0	882	0	0.0%
software	🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨	42493	366	42127	0	0.9%
website	🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨	40084	7673	32411	0	19.1%
All	🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥	191819	32772	157992	1055	17.1%

Hard violations (forced red):

Count	Check
1	`!boost_ge_base`

_{green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.}

TechEngineBot · 2026-09-28T18:48:04Z

TechEngine change review: PASS

12/12 categories ? 191,819 records ? diff base develop@c317ed9

PR: fix(data): merge 18 duplicate Intel Atom CPU records #321
Ref: fix/atom-cpu-dedup
Commit: 6a26317
Requested by: @Seungpyo1007
Run: https://github.com/GetTechAPI/TechEngine/actions/runs/36466602507

Check	Result
`python -m app.validate`	PASS
Changed dump JSON and manifest/index counts	PASS
New hard integrity anomalies vs PR base	PASS

Changed data

Category	Modified	Deleted
brand	0	0
soc	0	0
smartphone	0	0
tablet	0	0
watch	0	0
pda	0	0
gpu	0	0
cpu	18	18
laptop	0	0
monitor	0	0
software	0	0
website	0	0

Changed record examples

cpu modified

cpu/intel/2008/consumer/atom-230.json - Intel Atom 230
cpu/intel/2008/consumer/atom-330.json - Intel Atom 330
cpu/intel/2008/consumer/atom-z530.json - Intel Atom Z530
cpu/intel/2009/consumer/atom-n280.json - Intel Atom N280
cpu/intel/2010/consumer/atom-d410.json - Intel Atom D410
cpu/intel/2010/consumer/atom-d425.json - Intel Atom D425
cpu/intel/2010/consumer/atom-n455.json - Intel Atom N455
cpu/intel/2010/consumer/atom-n470.json - Intel Atom N470
cpu/intel/2010/consumer/atom-n475.json - Intel Atom N475
cpu/intel/2010/consumer/atom-n550.json - Intel Atom N550
cpu/intel/2011/consumer/atom-d2500.json - Intel Atom D2500
cpu/intel/2011/consumer/atom-d2700.json - Intel Atom D2700
cpu/intel/2011/consumer/atom-n2800.json - Intel Atom N2800
cpu/intel/2011/consumer/atom-n570.json - Intel Atom N570
cpu/intel/2012/consumer/atom-d2550.json - Intel Atom D2550
... 3 more

cpu deleted

cpu/intel/2009/consumer/intel-atom-230.json - Intel Atom 230
cpu/intel/2009/consumer/intel-atom-330.json - Intel Atom 330
cpu/intel/2009/consumer/intel-atom-n280.json - Intel Atom N280
cpu/intel/2009/consumer/intel-atom-z530.json - Intel Atom Z530
cpu/intel/2010/consumer/intel-atom-d410.json - Intel Atom D410
cpu/intel/2010/consumer/intel-atom-n455.json - Intel Atom N455
cpu/intel/2010/consumer/intel-atom-n470.json - Intel Atom N470
cpu/intel/2010/consumer/intel-atom-n475.json - Intel Atom N475
cpu/intel/2010/consumer/intel-atom-n550.json - Intel Atom N550
cpu/intel/2011/consumer/intel-atom-d2700.json - Intel Atom D2700
cpu/intel/2011/consumer/intel-atom-d425.json - Intel Atom D425
cpu/intel/2011/consumer/intel-atom-n2800.json - Intel Atom N2800
cpu/intel/2011/consumer/intel-atom-n570.json - Intel Atom N570
cpu/intel/2012/consumer/intel-atom-d2500.json - Intel Atom D2500
cpu/intel/2012/consumer/intel-atom-d2550.json - Intel Atom D2550
... 3 more

Heuristic review

Heuristic warnings: none found.

TechEngineBot · 2026-09-28T18:48:06Z

TechEngine validation stats: PASS

12/12 categories ? 191,819 records ? diff base develop@c317ed9

PR: fix(data): merge 18 duplicate Intel Atom CPU records #321
Ref: fix/atom-cpu-dedup
Commit: 6a26317
Run: https://github.com/GetTechAPI/TechEngine/actions/runs/36466602507

Data summary

Category	Total	Verified	Unverified	Tracked	Verified % of tracked
brand	207	123	84	207	59.4%
soc	2117	284	1833	2117	13.4%
smartphone	93407	17581	75826	93407	18.8%
tablet	3459	288	3171	3459	8.3%
watch	447	94	353	447	21.0%
pda	140	74	66	140	52.9%
gpu	2076	1496	580	2076	72.1%
cpu	4512	4219	293	4512	93.5%
laptop	1995	105	1890	1995	5.3%
monitor	882	0	882	882	0.0%
software	42493	358	42135	42493	0.8%
website	40084	7635	32449	40084	19.0%
all	191819	32257	159562	191819	16.8%

Warning

Tracked verified coverage is below 50% for monitor 0.0% (0/882), software 0.8% (358/42493), laptop 5.3% (105/1995), tablet 8.3% (288/3459), soc 13.4% (284/2117), all 16.8% (32257/191819), smartphone 18.8% (17581/93407), website 19.0% (7635/40084), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4512 GPU=2076

Integrity section	Flagged lines
structural	0
CPU name/tier consistency (desktop mainstream only)	0
CPU single>multi (cinebench/geekbench — should be multi>=single)	0
CPU era-vs-score outliers	8
CPU cross-source ratio outliers (possible wrong-variant)	132
GPU cross-source ratio outliers + sanity	51

Seungpyo1007 added 2 commits September 29, 2026 03:35

chore(dump): refresh static API for the 18 merged Atom CPU records

6a26317

Seungpyo1007 added bug Something isn't working data Dataset changes labels Sep 28, 2026

Seungpyo1007 assigned Seungpyo1007 and TechEngineBot Sep 28, 2026

Seungpyo1007 added this to the Massive dataset rebuild (1989-2026) milestone Sep 28, 2026

github-actions Bot added the enhancement New feature or request label Sep 28, 2026

Seungpyo1007 mentioned this pull request Sep 28, 2026

Verification and TechEngineBot (workstream) GetTechAPI/TechEngine#98

Open

4 tasks

Seungpyo1007 added this to TechAPI-Project Sep 28, 2026

github-project-automation Bot moved this to Todo in TechAPI-Project Sep 28, 2026

Seungpyo1007 merged commit 01d6565 into develop Sep 28, 2026
5 checks passed

github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Sep 28, 2026

Seungpyo1007 deleted the fix/atom-cpu-dedup branch September 28, 2026 18:44

This was referenced Sep 28, 2026

Dump, API and CI (workstream) GetTechAPI/TechEngine#100

Open

chore(data): weekly refresh 2026-09-28 #325

Closed

chore(site): refresh public dump (scoring v2.0.0) #331

Closed

fix(data): resolve 8 GPU near-duplicate and spec-accuracy findings #343

Open

Navigation Menu

Repository navigation

fix(data): merge 18 duplicate Intel Atom CPU records - #321

fix(data): merge 18 duplicate Intel Atom CPU records#321
Seungpyo1007 merged 2 commits into
developfrom
fix/atom-cpu-dedup

Seungpyo1007 commented Sep 28, 2026

Uh oh!

TechEngineBot commented Sep 28, 2026

Uh oh!

Uh oh!

TechEngineBot commented Sep 28, 2026

Uh oh!

TechEngineBot commented Sep 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Seungpyo1007 commented Sep 28, 2026

What & why

How each pair was resolved

Verification

Uh oh!

TechEngineBot commented Sep 28, 2026

🔎 Data verification — Tier 0 (offline existence/trust)

Changed records in this PR

Full-dataset baseline

Uh oh!

Uh oh!

TechEngineBot commented Sep 28, 2026

TechEngine change review: PASS

Changed data

Changed record examples

cpu modified

cpu deleted

Heuristic review

Uh oh!

TechEngineBot commented Sep 28, 2026

TechEngine validation stats: PASS

Data summary

Validation notes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants