Skip to content

fix(data): self-consistency correction for 1 PDA record - #340

Merged
Seungpyo1007 merged 1 commit into
developfrom
Seungpyo1007/pda-derivation
Sep 29, 2026
Merged

Seungpyo1007 merged 1 commit into
developfrom
Seungpyo1007/pda-derivation

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Summary

Within-record self-consistency pass over data/pda/ (140 records), following the established pattern from #301 (derive only from a record's own fields; never invent or fetch external sources).

Found and fixed one unambiguous within-record inconsistency:

  • data/pda/palm/2018/palm/palm.json had storage_options_gb: [3, 32]. The 3 is the RAM value (
    am_gb: 3, �ariant.memory.ram_gb: 3) that leaked into the storage list. The record's own �ariant.memory.storage_gb: 32 confirms storage is 32 GB only. Removed the stray 3 -> [32].

Reviewed but intentionally left (ambiguous, per task rules)

  • Rounding artifacts (6 records: i-mate PDAL, Mitac A701 x2/A501/A502/A702):
    aw_source.memory_internal says '64 MB RAM' (= 0.0625 GB) while
    am_gb is stored as

Closes #296

The 2018 Palm record listed storage_options_gb as [3, 32], but the 3 is the RAM value (ram_gb=3, variant.memory.ram_gb=3) leaked into the storage list. The record's own variant.memory.storage_gb=32 confirms storage is 32 GB only. Removed the stray 3 -> [32], derived solely from the record's own fields.

Refs #296
@TechEngineBot

Copy link
Copy Markdown
Member

🔎 Data verification — Tier 0 (offline existence/trust)

Scored by app.verify; posted by TechEngineBot. Informational only — the structural gate (app.validate) is separate and authoritative for merge.

12/12 categories ? 191,819 records ? diff base develop@2b4639b

Changed records in this PR

1 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 0
    "Yellow" : 1
    "Red" : 0
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
pda 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1 0 1 0 0.0%
All 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1 0 1 0 0.0%

Full-dataset baseline

191819 record(s) assessed.

%%{init: {"theme":"base","themeVariables":{"pie1":"#3fb950","pie2":"#d29922","pie3":"#f85149","pieStrokeWidth":"0px","pieOpacity":"1"}}}%%
pie showData
    title Verification bands — all records
    "Green" : 32772
    "Yellow" : 157992
    "Red" : 1055
Loading
Category Distribution Total 🟢 🟡 🔴 🟢 %
brand 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 207 122 85 0 58.9%
soc 🟩🟩🟨🟨🟨🟨🟨🟥🟥🟥🟥🟥 2117 284 837 996 13.4%
smartphone 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 93407 18003 75350 54 19.3%
tablet 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 3459 292 3166 1 8.4%
watch 🟩🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟥 447 94 350 3 21.0%
pda 🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨🟨🟨 140 83 57 0 59.3%
gpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟨🟨 2076 1529 547 0 73.7%
cpu 🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟨🟥 4512 4219 292 1 93.5%
laptop 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 1995 107 1888 0 5.4%
monitor 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 882 0 882 0 0.0%
software 🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 42493 366 42127 0 0.9%
website 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 40084 7673 32411 0 19.1%
All 🟩🟩🟨🟨🟨🟨🟨🟨🟨🟨🟨🟥 191819 32772 157992 1055 17.1%

Hard violations (forced red):

Count Check
1 !boost_ge_base

green = authoritative source + complete + consistent · yellow = plausible, needs confirmation · red = sparse/weak source or a hard contradiction. Promotion to verified runs in the scheduled verify-network workflow.

@Seungpyo1007
Seungpyo1007 merged commit e174584 into develop Sep 29, 2026
5 checks passed
@Seungpyo1007
Seungpyo1007 deleted the Seungpyo1007/pda-derivation branch September 29, 2026 15:06
@Seungpyo1007 Seungpyo1007 removed the enhancement New feature or request label Sep 29, 2026
@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine change review: PASS

12/12 categories ? 191,819 records ? diff base develop@2b4639b

Check Result
python -m app.validate PASS
Changed dump JSON and manifest/index counts PASS
New hard integrity anomalies vs PR base PASS

Changed data

Category Added Modified Deleted Added verified Added unverified Added Kaggle-sourced
brand 0 0 0 0 0 0
soc 0 0 0 0 0 0
smartphone 0 0 0 0 0 0
tablet 0 0 0 0 0 0
watch 0 0 0 0 0 0
pda 0 1 0 0 0 0
gpu 0 0 0 0 0 0
cpu 0 0 0 0 0 0
laptop 0 0 0 0 0 0
monitor 0 0 0 0 0 0
software 0 0 0 0 0 0
website 0 0 0 0 0 0

Changed record examples

pda modified

  • pda/palm/2018/palm/palm.json - Palm

Heuristic review

  • Heuristic warnings: none found.

@TechEngineBot

Copy link
Copy Markdown
Member

TechEngine validation stats: PASS

12/12 categories ? 191,819 records ? diff base develop@2b4639b

Data summary

Category Total Verified Unverified Missing verified Tracked Verified % of tracked
brand 207 123 84 0 207 59.4%
soc 2117 284 1833 0 2117 13.4%
smartphone 93407 17581 75826 0 93407 18.8%
tablet 3459 288 3171 0 3459 8.3%
watch 447 94 353 0 447 21.0%
pda 140 74 66 0 140 52.9%
gpu 2076 1496 580 0 2076 72.1%
cpu 4512 4219 293 0 4512 93.5%
laptop 1995 105 1890 0 1995 5.3%
monitor 882 0 882 0 882 0.0%
software 42493 358 42135 0 42493 0.8%
website 40084 7635 32449 0 40084 19.0%
all 191819 32257 159562 0 191819 16.8%

Warning

Tracked verified coverage is below 50% for monitor 0.0% (0/882), software 0.8% (358/42493), laptop 5.3% (105/1995), tablet 8.3% (288/3459), soc 13.4% (284/2117), all 16.8% (32257/191819), smartphone 18.8% (17581/93407), website 19.0% (7635/40084), and 1 more.
Tracked coverage excludes records missing the verified field; see the Missing verified column for those records.
This does not fail validation. Keep imported records verified: false until manual audit, but treat this as follow-up verification work before relying on the affected categories as curated data.

Validation notes

  • The integrity gate blocks only hard anomalies introduced by this PR relative to its merge base.
  • Full advisory outlier listings are suppressed on successful runs because they are dataset-wide and mostly stable between PRs.
  • Failure runs still include a detailed log excerpt for debugging.

Key output:

## app.validate
## integrity_check.py (PR head compared with PR base)
loaded CPU=4512 GPU=2076
Integrity section Flagged lines
structural 0
CPU name/tier consistency (desktop mainstream only) 0
CPU single>multi (cinebench/geekbench — should be multi>=single) 0
CPU era-vs-score outliers 0
CPU cross-source ratio outliers (possible wrong-variant) 53
GPU cross-source ratio outliers + sanity 51

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data Dataset changes

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Data accuracy: corrections, duplicates and dates (workstream)

2 participants