Skip to content

Selectable dictionary packs via generated manifest (/data/packs.json) - #107

Closed
abdulazizalmalki-gh wants to merge 2 commits into
wtetsu:masterfrom
abdulazizalmalki-gh:feat/pack-selector
Closed

abdulazizalmalki-gh wants to merge 2 commits into
wtetsu:masterfrom
abdulazizalmalki-gh:feat/pack-selector

Conversation

@abdulazizalmalki-gh

@abdulazizalmalki-gh abdulazizalmalki-gh commented Oct 3, 2026 •

Copy link
Copy Markdown

What

Adds a language-neutral dictionary pack mechanism: tools/make_dict.ts emits a /data/packs.json manifest listing each pack (id, metaFile, optional label), the options page renders a checkbox panel from it, and storage is synced on apply (unregistering dropped packs). Headwords owned by multiple packs get merged descriptions instead of last-write-wins.

Why — relates to #63 / PR #63

I read the design requirements from #63 (existing users unaffected, no friction for new users, no unacceptable speed overhead, per-dictionary management). This implementation targets all four:

  • Existing users unaffected: without a manifest the code falls back to the historical single-dict.json layout, and default settings keep dictionaryPacks: ["en-ja"] — first-run and upgrade behavior is identical to today. Old stored data keeps working; nothing re-registers until the user acts.
  • Low friction: the pack panel is one checkbox row + one apply button under the existing dictionary section; new users never see it.
  • No lookup overhead: registration happens once on apply; runtime storage layout is unchanged (flat head -> desc). Merging happens at registration time, not lookup time.
  • Per-pack management: syncInstalledPacks removes exactly the keys of unselected packs and rebuilds selected ones — pack-level add/remove without a full reload.

The registry is data, not code: adding a language pair means shipping pack shards + a manifest entry, no extension-code change (labels come from the manifest). Happy to align this with whatever redesign direction you prefer — including folding the merge behavior into a bigger storage rework if that's the plan; the code is small (one settings field, one module, ~500 lines with tests).

RTL note: the output template adds unicode-bidi:plaintext so right-to-left descriptions render correctly without per-pack layout code — relevant for any future Arabic/Hebrew/Farsi packs.

Tests: registry load, manifest fallback (non-ok response and fetch rejecting), unknown ids, stale-pack sync, prototype-key regression (real bug: constructor is an actual headword in ejdict-hand). 300 tests pass; lint and typecheck clean.

Rebased onto master (tools TS migration + SweetAlert2 swap); behavior unchanged.

@abdulazizalmalki-gh
abdulazizalmalki-gh force-pushed the feat/pack-selector branch 2 times, most recently from e8322e0 to 9980d76 Compare October 3, 2026 08:49
@codecov

codecov Bot commented Oct 3, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.76%. Comparing base (1fe59fd) to head (39a4cd0).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #107      +/-   ##
==========================================
+ Coverage   98.74%   98.76%   +0.01%     
==========================================
  Files          51       52       +1     
  Lines        2155     2187      +32     
  Branches      415      428      +13     
==========================================
+ Hits         2128     2160      +32     
  Misses         27       27              
Flag Coverage Δ
unittests 98.76% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@wtetsu

wtetsu commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Thank you for taking the time to read through #63 and for putting this together. I do recognize that there is real interest in handling multiple dictionaries.

That said, this touches a fundamental part of the user experience, and I'm afraid this project can't accept a PR for it without prior discussion. Before thinking about implementation, we first need to carefully consider things like the impact on existing users, whether the benefit justifies making the UI and workflow more complex, and what the UI should look like. How to implement it only comes after that.

Honestly, a user seeing an "Apply dictionary packs" button would have no idea what pressing it actually does.

Suggestions are always welcome, but this is something that requires a very deep understanding of the application to design and implement, and on top of that, careful thought about long-term maintainability. So please understand that an unsolicited PR on this is unlikely to be accepted as-is. 🙏

@abdulazizalmalki-gh

Copy link
Copy Markdown
Author

If you have a specific concern please state it, otherwise I don't see any issue with having a button to apply the other languages packs, it's pretty obvious what it does, it's simply enabling other languages to be hovered over to get the dictionary entries for, it's that simple. You can load the unpacked extension as I did and test it yourself for the new languages I added. I took a portion off my time to implement this as it is a useful tool for me as a Japanese language learner. Again, if there is a specific issue or concern please state that to clarify whether we need to do any further modification to satisfy you as the maintainer and to progress the merging in order to deploy this to the store for other learners like me to download.

abdulazizalmalki-gh pushed a commit to abdulazizalmalki-gh/mouse-dictionary that referenced this pull request Oct 3, 2026
The English-Gloss bridge returned the WHOLE en-ar page entry for a
gloss, which merges every homograph Wiktionary put on that English
page: ノー (loanword no) carried the archaic cattle noun نَعَم and a
duplicated interjection chunk, and arwiktionary definitions attached
as full grammar essays.

Bridging now keeps only the dominant (first) sense chunk and caps the
parenthetical definition at 80 chars in both bridge sites (JMdict
render_sense + kaikki fill_from_jawikt). Applied across the pack:
92k shard lines shortened, 0 headwords changed, en-ar pack untouched.

Code paths (src/, PRs wtetsu#106/wtetsu#107) unchanged — data layer only.
@abdulazizalmalki-gh

Copy link
Copy Markdown
Author

@wtetsu How can we move this forward?

i would like to share this extension with my fellow learners.

Abdulaziz Al Malki added 2 commits October 4, 2026 11:55
Redesign of the dictionary-pack idea from issue wtetsu#63, rebuilt against the
current TypeScript/Vite sources and structured around the concerns raised
then:

- No impact on existing users: builds without /data/packs.json keep the
  historical single-dictionary layout (built-in fallback registry), so an
  old or hand-assembled data bundle still registers en-ja exactly as
  before. Default settings select ["en-ja"] only.
- Low friction for new users: the options page shows a checkbox per pack
  bundled in the build plus "Apply dictionary packs"; first-run
  registration is unchanged.
- No runtime speed overhead: lookups still hit one flat key-value store.
  Packs are merged at registration time; hover lookup does zero extra
  work regardless of how many packs are enabled.
- Per-dictionary management: enabling/disabling a pack removes exactly
  the keys that pack owns (shard-aware unregister) and re-registers the
  rest, so storage converges to the selected set.
- Language-neutral: the registry is generated data (tools/make_dict.js
  writes /data/packs.json from its pack list), not a hardcoded table in
  the UI. Adding a language pair = one entry in DICTIONARY_PACKS.

Also includes:
- storage.local.remove wrapper (main/lib/storage.ts)
- description merge helper tolerant of non-string values (keys named
  "constructor" exist in real dictionary data)
- RTL-safe output template (unicode-bidi:plaintext on description spans)
  so Arabic/Hebrew definitions render in correct direction
- tests for the registry, merge logic and pack sync against a fake
  extension runtime
@wtetsu

wtetsu commented Oct 4, 2026

Copy link
Copy Markdown
Owner

@abdulazizalmalki-gh

Thank you for the effort you've put into this, and for explaining how you'd like to use it as a Japanese learner.

That said, I'm afraid this PR isn't something I can merge as it is.
Before any implementation, could you open an Issue that explains what problem you're trying to solve? The solution and implementation come after that.

You asked about specific concerns. Here are some I noticed from a quick look:

  • Basic Settings is meant for essential settings that are easy to understand. A button whose purpose isn't clear could confuse users.
  • To be honest, even I wasn't sure what would happen when pressing the new button. What exactly is a "dictionary pack"? Is the operation reversible?
  • Looking at src/options/logic/dict.ts, there's quite a lot of ad hoc handling (e.g. joining descriptions as strings when headwords collide). I don't think this can be brought in with a few small fixes.

As you may recall from #106, even a fairly self-evident fix like that took several rounds of review and testing on my side. A change like this #107 needs much more discussion and verification. Since I work on this project in my spare time, going back and forth on each point in this PR isn't something I can realistically take on.

I can't promise the idea will be adopted, but I'm happy to discuss it there. Thanks for your understanding.

@abdulazizalmalki-gh

Copy link
Copy Markdown
Author

Closed as requested. Problem statement opened separately as #108, per your guidance — no implementation attached.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants