data: import GSMArena cigarplug batch — 2,498 SoC-linked phones + tablets/watches/pdas - #98
Merged
Merged
Conversation
GSMArena device dump (cigarplug/scrape-gsma) for Samsung Galaxy, Huawei and Honor lines, with SoC linked to the known SoC catalog. Bulk import, verified:false. Refs #1
GSMArena device dump (cigarplug/scrape-gsma) for LG, Xiaomi/Redmi/Poco, Oppo and Vivo lines. SoC-linked, verified:false. Refs #1
GSMArena device dump (cigarplug/scrape-gsma) for Motorola, HTC, ZTE and Sony Xperia lines. SoC-linked, verified:false. Refs #1
…, Realme GSMArena device dump (cigarplug/scrape-gsma) for Asus, Lenovo, Nokia, Meizu and Realme lines. SoC-linked, verified:false. Refs #1
GSMArena device dump (cigarplug/scrape-gsma) for the remaining brands (Infinix, BLU, Alcatel, Gionee, Allview, Wiko, Tecno, Microsoft, Micromax, Panasonic, BlackBerry, Archos, BQ and others). SoC-linked, verified:false. Refs #1
Additional GSMArena tablet records from the cigarplug/scrape-gsma dump not covered by the earlier batch. verified:false. Refs #1
…g dump) Remaining GSMArena smartwatch and PDA records from the cigarplug/scrape-gsma dump. verified:false. Refs #1
Add the 2,498 new GSMArena (cigarplug/scrape-gsma) records to the static v1 dump — smartphone/tablet/watch/pda detail + smartphone score entries — and bump the category index counts and manifest. Regenerated via TechEngine app.dump at pinned submodule c0928f9 (scoring v2.0.0); only the new records and affected indexes are committed. Refs #1
Member
Author
🤖 Review — PASS
Data-change summary
Example records
Heuristic warnings
Site build
|
Member
Author
📊 Dataset stats (post-merge projection)
Warning Verified ratio is 9.57% (down slightly from 9.80% as 2,498 new Campaign progress toward +50k: cigarplug/scrape-gsma is now largely exhausted (~2,871 total imported across PR #97 + #98). Remaining headroom needs additional datasets (Kaggle phone dumps, PhoneDB, AllPhones SKU variants) — tracked in #1. |
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Second GSMArena batch from the cigarplug/scrape-gsma dataset (10,679-row GSMArena scrape). PR #97 imported the tablet/watch/pda subset; this brings in 2,498 new records — mostly smartphones — with each phone linked to a real SoC in the catalog.
All records land
verified: false. Smartphones satisfy the stricter phone schema: integerram_gb, ranges (battery 500–12000 mAh, weight 50–500 g), and asocFK resolved to an existing SoC via a precise die-number + marketing-name matcher (unmatched chipsets are skipped, not guessed — avoids the fuzzy-match trap).Commits (each
Refs #1): 5 smartphone brand-group commits (Samsung/Huawei/Honor · LG/Xiaomi/Oppo/Vivo · Motorola/HTC/ZTE/Sony · Asus/Lenovo/Nokia/Meizu/Realme · remaining), tablet, watch+pda, and a surgical dump refresh.Closes #1
Source
GSMArena via
cigarplug/scrape-gsma(https://github.com/cigarplug/scrape-gsma),*-scrapegsma-<row>slugs.raw_sourcepreserves the original chipset/CPU/GPU/memory strings;variant.source_rowcites the CSV row.Dump
Regenerated via TechEngine
app.dumpat pinned submodulec0928f9(scoring v2.0.0). Only the 2,498 new records' detail+score files, the four category indexes and the manifest counts are committed (smartphones 90118→92497, tablets 3354→3455, watches 424→433, pdas 131→140) — no churn on unchanged records.Checklist
python -m app.validate— ✅ passed (full dataset)python TechEngine/integrity_check.py data --strict— ✅ no hard anomaliesgit diff --check— ✅ no whitespace errorsdata/<category>/<brand>/<year>/<base>/<slug>.jsonpaths, kebab-case unique slugssource_urlspresent on every record