Skip to content

data(smartphone): derive os_version from os field (48,462 records) - #111

Merged
Seungpyo1007 merged 2 commits into
mainfrom
data/import-staging
Jul 9, 2026
Merged

Seungpyo1007 merged 2 commits into
mainfrom
data/import-staging

Conversation

@Seungpyo1007

@Seungpyo1007 Seungpyo1007 commented Jul 9, 2026 •

Copy link
Copy Markdown
Member

Summary

Normalizes the launch OS version that is already present in each smartphone's os string into the structured os_version field — a deterministic parse of existing data, no new/external data and no fabrication.

  • 48,462 smartphones filled (coverage os_version ~6% → ~58%).
  • Examples: "Android 4.0.3 (Ice Cream Sandwich)" → 4.0.3; "Microsoft Windows Mobile 6.5 Professional" → 6.5; "iOS v26" → 26.
  • Takes the shipped version, not "upgradeable to X" follow-ons.
  • Symbian excluded — its Series 60/S60 rel. 3.2/Symbian^3 strings mix platform/UI numbers with OS versions and can't be parsed unambiguously (would have produced wrong values like 60/90).
  • Guards reject 4-digit years and implausible majors (>100).

Records already having os_version (6,104) untouched; records whose os has no parseable version (38,830 — feature phones, "Proprietary", Symbian) left empty.

All verified: false. Validation passes. Dump regenerated — 48,462 changed smartphone detail pages.

Closes #1

Normalizes the launch OS version already present in each record's `os`
string into the structured `os_version` field (e.g. "Android 4.0.3 (Ice
Cream Sandwich)" -> "4.0.3"). Deterministic parse of existing data — no
new/external data. Takes the shipped version, not "upgradeable to" ones.
Symbian excluded (Series/UI numbers are ambiguous). os_version coverage
~6% -> ~58%.

Refs #1
@Seungpyo1007

Copy link
Copy Markdown
Member Author

🤖 TechEngine · data validation

category      changed   verified
smartphone     48,462     false
  • python -m app.validate → ✅ passed (0 errors), 108,575 records / 10 categories.
  • os_version coverage 6,104 → 54,566 (~6% → ~58% of smartphones).
  • Extracted value range: major 1–26 (Android 1.x … iOS 26). 0 anomalous values after Symbian exclusion.
  • Dump regenerated: 48,462 smartphone detail pages. List summaries carry no os_version, so only detail pages changed.

No schema change — normalizes a field the schema already defines.

@Seungpyo1007

Copy link
Copy Markdown
Member Author

🤖 TechEngine · verify (Tier 0)

Not a data import — a derivation from data already in the record:

  • The version string lives inside each record's os (sourced originally from GSMArena et al.); this only lifts it into the typed os_version field. Nothing invented.
  • Adversarial pass flagged two failure modes and both are handled: (1) "upgradeable to Android X" → we keep the shipped version; (2) Symbian Series 60/S60 → excluded so the platform/UI number is never mistaken for a version.
  • Year-like (\d{4}) and major>100 tokens rejected.

verified stays false; promotion remains the verify-network job's call. Safe to rebase-merge.

@Seungpyo1007 Seungpyo1007 moved this from Todo to In Progress in TechAPI-Project Jul 9, 2026
@Seungpyo1007
Seungpyo1007 merged commit 7998f68 into main Jul 9, 2026
5 checks passed
@github-project-automation github-project-automation Bot moved this from In Progress to Done in TechAPI-Project Jul 9, 2026
Seungpyo1007 added a commit that referenced this pull request Jul 9, 2026
… fields

Applies the same two derivations used for smartphones (#111, #112) to the
tablet/watch/pda catalogues: parse launch os_version from the `os` string, and
compute display.ppi = round(sqrt(w^2+h^2)/size_inch). Deterministic, no new data.
tablet +2656 os_version / +2080 ppi; watch +202 / +279; pda +16 / +54.

Refs #1
Seungpyo1007 added a commit that referenced this pull request Jul 9, 2026
… fields

Applies the same two derivations used for smartphones (#111, #112) to the
tablet/watch/pda catalogues: parse launch os_version from the `os` string, and
compute display.ppi = round(sqrt(w^2+h^2)/size_inch). Deterministic, no new data.
tablet +2656 os_version / +2080 ppi; watch +202 / +279; pda +16 / +54.

Refs #1
@Seungpyo1007 Seungpyo1007 added the bug Something isn't working label Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working data Dataset changes enhancement New feature or request

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

TechAPI dataset roadmap and status: all categories (1989-2026)

1 participant