Skip to content

fix(data): drop RAM sizes from storage lists of 16 multi-edition tablets - #302

Merged
Seungpyo1007 merged 1 commit into
developfrom
fix/multi-edition-storage
Sep 26, 2026
Merged

Seungpyo1007 merged 1 commit into
developfrom
fix/multi-edition-storage

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

What & why

Follow-up to #301. Fixes 16 tablets whose memory_internal lists several editions ("16 GB, 2 GB RAM or 32 GB, 3 GB RAM"): the import had merged each edition's RAM into storage_options_gb. Re-parsed per edition, so [2, 3, 16, 32] becomes [16, 32]. Closes #296.

Skipped: 2 records where the raw text is malformed ("3 RAM" without a unit) or lists more editions than the parse recovered.

Source

Each record's own memory_internal text; no new sources.

Checklist

  • The PR targets develop
  • python -m app.validate passes locally
  • No slugs or paths changed

The import merged each edition's RAM into storage_options_gb (e.g. [2, 3, 16, 32]
for "16 GB, 2 GB RAM or 32 GB, 3 GB RAM"). Re-parsed per edition from the
record's memory_internal text.

Refs #296
@github-actions github-actions Bot added bug Something isn't working data Dataset changes enhancement New feature or request labels Sep 26, 2026
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

12/12 categories ? 191,814 records ? diff base develop@a658758

Changed records in this PR

16 record(s) assessed.

Software and website assess required fields and sources only; domain consistency rules are unavailable and these categories cannot earn green.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 0
    "Yellow" : 16
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
tablet 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 16 0 16 0 0.0%
All 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 16 0 16 0 0.0%

Full-dataset baseline

191814 record(s) assessed.

Software and website assess required fields and sources only; domain consistency rules are unavailable and these categories cannot earn green.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 24725
    "Yellow" : 166034
    "Red" : 1055
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 207 122 85 0 58.9%
soc 🟩🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥 2117 284 837 996 13.4%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93407 18003 75350 54 19.3%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3459 292 3166 1 8.4%
watch 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 447 94 350 3 21.0%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 2076 1529 547 0 73.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4507 4211 295 1 93.4%
laptop 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1995 107 1888 0 5.4%
monitor 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 882 0 882 0 0.0%
software 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 42493 0 42493 0 0.0%
website 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 40084 0 40084 0 0.0%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 191814 24725 166034 1055 12.9%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit 4704a54 into develop Sep 26, 2026
5 checks passed
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

12/12 categories ? 191,814 records ? diff base develop@a658758

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 16 0 0 0 0
watch 0 0 0 0 0 0
pda 0 0 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0

Changed record examples

tablet modified

  • tablet/honor/2019/tab-5/honor-tab-5-scrapegsma-3734.json - Honor Tab 5
  • tablet/huawei/2015/mediapad-m2-8-0/huawei-mediapad-m2-8-0-scrapegsma-4186.json - Huawei MediaPad M2 8.0
  • tablet/huawei/2015/mediapad-x2/huawei-mediapad-x2-scrapegsma-4194.json - Huawei MediaPad X2
  • tablet/huawei/2016/mediapad-m2-10-0/huawei-mediapad-m2-10-0-scrapegsma-4176.json - Huawei MediaPad M2 10.0
  • tablet/huawei/2017/mediapad-t3-10/huawei-mediapad-t3-10-scrapegsma-4139.json - Huawei MediaPad T3 10
  • tablet/huawei/2017/mediapad-t3-7-0/huawei-mediapad-t3-7-0-scrapegsma-4145.json - Huawei MediaPad T3 7.0
  • tablet/huawei/2017/mediapad-t3-8-0/huawei-mediapad-t3-8-0-scrapegsma-4144.json - Huawei MediaPad T3 8.0
  • tablet/huawei/2018/mediapad-m5-lite/huawei-mediapad-m5-lite-scrapegsma-4107.json - Huawei MediaPad M5 lite
  • tablet/huawei/2019/mediapad-m5-lite-8/huawei-mediapad-m5-lite-8-scrapegsma-4090.json - Huawei MediaPad M5 Lite 8
  • tablet/lenovo/2017/tab-4-10-plus/lenovo-tab-4-10-plus-scrapegsma-4757.json - Lenovo Tab 4 10 Plus
  • tablet/lenovo/2017/tab-4-8-plus/lenovo-tab-4-8-plus-scrapegsma-4759.json - Lenovo Tab 4 8 Plus
  • tablet/samsung/2015/galaxy-tab-a-8-0/samsung-galaxy-tab-a-8-0-scrapegsma-7997.json - Samsung Galaxy Tab A 8.0
  • tablet/samsung/2015/galaxy-tab-a-9-7/samsung-galaxy-tab-a-9-7-scrapegsma-7995.json - Samsung Galaxy Tab A 9.7
  • tablet/samsung/2019/galaxy-tab-s5e/samsung-galaxy-tab-s5e-scrapegsma-7855.json - Samsung Galaxy Tab S5e
  • tablet/xiaomi/2018/mi-pad-4-plus/xiaomi-mi-pad-4-plus-scrapegsma-10175.json - Xiaomi Mi Pad 4 Plus
  • ... 1 more

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

12/12 categories ? 191,814 records ? diff base develop@a658758

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 122 85 0 207 58.9%
soc 2117 193 1924 0 2117 9.1%
smartphone 93407 17501 75906 0 93407 18.7%
tablet 3459 273 3186 0 3459 7.9%
watch 447 39 408 0 447 8.7%
pda 140 74 66 0 140 52.9%
gpu 2076 1026 1050 0 2076 49.4%
cpu 4507 2869 1638 0 4507 63.7%
laptop 1995 0 1995 0 1995 0.0%
monitor 882 0 882 0 882 0.0%
software 42493 0 42493 0 42493 0.0%
website 40084 0 40084 0 40084 0.0%
all 191814 22097 169717 0 191814 11.5%

Warning

Tracked verified coverage is below 50% for laptop 0.0% (0/1995), monitor 0.0% (0/882), software 0.0% (0/42493), website 0.0% (0/40084), tablet 7.9% (273/3459), watch 8.7% (39/447), soc 9.1% (193/2117), all 11.5% (22097/191814), and 2 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4507 GPU=2076
Integrity section Flagged lines
structural 18
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 8
CPU cross-source ratio outliers (possible wrong-variant) 131
GPU cross-source ratio outliers + sanity 51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Data accuracy: corrections, duplicates and dates (workstream)

2 participants