Skip to content

feat(verify): cross-reference smartphone records against Wikipedia - #75

Merged
Seungpyo1007 merged 5 commits into
mainfrom
Seungpyo1007/feat-smartphone-wikipedia-backfill
Sep 22, 2026
Merged

Seungpyo1007 merged 5 commits into
mainfrom
Seungpyo1007/feat-smartphone-wikipedia-backfill

Conversation

@Seungpyo1007

Copy link
Copy Markdown
Member

Summary

Cross-reference tool to harvest verified Wikipedia URLs for unverified smartphone records (gsmarena-kaggle dataset).

  • Reuses strict safeguards from crossref.py:
    • Exact heading matching (_heading_matches)
    • Requirement of >= 2 independent agreeing specifications (launch year alone is blocked)
    • Hard-gated variant confusion filters (Pro, Plus/+, Max, Ultra, Mini, Lite, Fold/Flip, 4G vs 5G, Tablet/Watch)
    • Rejection of multiple competing candidates (multiple-rows)
    • Brand isolation on WikiRow preventing cross-brand matching
  • Supports fast sampling across brand directories without full filesystem scans
  • Respects Wikipedia robot policy and rate limits
  • Includes 11 comprehensive unit tests covering variant conflict, year-only block, spec rank, spec conflict, and HTML table extraction

@Seungpyo1007
Seungpyo1007 merged commit 8c25e5d into main Sep 22, 2026
1 check passed
@Seungpyo1007
Seungpyo1007 deleted the Seungpyo1007/feat-smartphone-wikipedia-backfill branch September 23, 2026 12:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant