Commit e1cdb2f
committed
fix(data): resolve 364 smartphone brand-prefix conflicts (same-chip duplicates)
Follow-up to #329, which merged the 50 identical-core brand-prefix pairs and left
717 pairs that disagreed on a core field (soc / release_date / ram_gb / battery_mah
/ weight_g / os) documented but unresolved. This resolves the largest, highest-
confidence slice of those.
Classification of the 717 (scripted, then spot-verified against sources):
- The soc "conflict" in 572 pairs is two DIFFERENT slugs that both resolve to a
real soc record. For 414 of those the two slugs are the SAME physical chip with
a different spelling (e.g. "qualcomm-msm7227-snapdragon-s1" vs "qualcomm-msm7227",
"qualcomm-msm8260a-snapdragon-s4-plus" vs "qualcomm-snapdragon-s4-msm8260a") — not
a real spec difference.
- The other conflicts on those same-chip pairs are the fabricated import placeholders
(weight_g=150.0, battery_mah=1500, and the generic YYYY-01-01 launch date) that
#329 already flagged as a systemic bulk-import bug.
This commit merges the 364 same-chip pairs whose two sides are a GSMArena scrape
(bare slug, Kaggle `gsmarena-mobile-devices`, carries a `raw_gsmarena` block) and a
phonedb.net twin (brand-prefixed slug). Keeper = the GSMArena record:
- Its specs are the real ones. Spot-verified: Acer DX650 weighs 133 g
(GSMArena, imei.org, softpedia) — the phonedb twin's 150.0 g is the stamped
placeholder; Acer CloudMobile S500 is 122 g (GSMArena) vs phonedb's 150.0.
- soc kept as the keeper's slug (confirmed FK-valid for all 364; same physical chip).
- name upgraded to the brand-inclusive form ("Acer DX650" over "DX650").
- source_urls unioned so the phonedb.net reference is retained.
- verified left unchanged — NOT flipped to true, because the phonedb side's scalar
values proved unreliable (fabricated weights), so it is not a trustworthy source.
- release_date left as the keeper's value and NOT resolved here: the phonedb twin's
specific dates are unreliable too (phonedb says Acer DX650 = 2009-03 but
GSMArena/imei say 2009-06), so importing them would be guessing. Dates are
reported as still-unresolved rather than changed.
The redundant phonedb twin file is deleted. No scalar value was invented; in every
one of these 364 pairs the keeper already held the real GSMArena value and only the
deleted twin carried the placeholder.
Left for follow-up (documented on #296): 37 same-chip pairs sourced against vendor
/Wikipedia pages rather than phonedb, 157 pairs with a genuine ram/weight/battery
question needing per-record sourcing, and 158 pairs whose soc slugs are two
different chips — several of which are genuinely different devices that share a
name (e.g. Honor Magic V2 SD8Gen2 vs Kirin 9000S, Huawei P50 Pro SD888 vs Kirin
9000, HTC One M9 Helio X10 vs Snapdragon 810) and will be un-paired, not merged.
Both gates pass: `python -m app.validate` and `integrity_check.py <data> --strict`
(0 hard anomalies).
Closes #2961 parent 2360ec8 commit e1cdb2f
728 files changed
Lines changed: 1126 additions & 10219 deletions
File tree
- data/smartphone
- acer
- 2009
- 2010
- 2012
- 2013
- alcatel/2013
- allview
- 2013
- 2014
- archos/2021
- asus
- 2007
- 2008
- 2012
- 2014
- 2016
- blackview/2019
- blu
- 2014
- 2019
- 2020
- caterpillar
- 2019
- 2020
- 2021
- coolpad/2021
- doogee
- 2018
- 2019
- 2020
- fujitsu
- 2019
- 2020
- 2022
- gigabyte
- 2007
- 2008
- 2009
- 2011
- 2012
- 2013
- 2014
- hisense
- 2018
- 2019
- htc
- 2011
- 2014
- 2015
- 2019
- 2020
- huawei
- 2009
- 2013
- 2014
- 2021
- i-mate
- 2006
- 2007
- infinix
- 2018
- 2019
- 2020
- karbonn
- 2013
- 2014
- kyocera
- 2012
- 2013
- 2019
- 2020
- 2021
- 2022
- lava
- 2014
- 2018
- 2019
- 2020
- lenovo
- 2011
- 2012
- 2014
- 2019
- lg
- 2007
- 2020
- micromax
- 2012
- 2013
- 2014
- microsoft
- 2015
- 2020
- motorola
- 2007
- 2010
- 2011
- 2012
- 2013
- mwg/2008
- nokia
- 2002
- 2004
- 2005
- 2008
- 2009
- 2010
- 2011
- 2012
- 2013
- 2020
- o2
- 2006
- 2007
- oneplus/2021
- oppo
- 2013
- 2014
- 2019
- 2020
- 2021
- orange/2012
- palm/2008
- panasonic
- 2012
- 2013
- 2014
- 2019
- samsung/2018
- sharp
- 2018
- 2019
- 2020
- siemens/2003
- smartisan/2020
- sony
- 2012
- 2021
- spice
- 2011
- 2012
- 2013
- tcl/2019
- ulefone/2019
- unihertz
- 2018
- 2020
- vodafone
- 2013
- 2014
- 2015
- wiko
- 2014
- 2020
- 2022
- xiaomi/2020
- zte
- 2011
- 2012
- 2013
- 2014
- 2017
- 2018
- 2019
- 2020
- 2021
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
14 | | - | |
| 14 | + | |
| 15 | + | |
15 | 16 | | |
16 | 17 | | |
17 | 18 | | |
| |||
0 commit comments