Conversation
… consent, tab-bound streaming The chat drawer in bento/slides (#512) sends a deck to the user's model THROUGH the extension: the page never holds a key or an endpoint. This is the other side of that contract (slides/src/editor/assistant/transport.ts). - src/providers.js: pure request shapers + SSE parser for Gemini, Anthropic and anything OpenAI-compatible (base URL editable for all). - src/assistant.js: describe/check/send over an injected environment — the Chrome built-in model (Prompt API) first when present. - background.js: sendMessage ops for describe/check/settings.open; a PORT per streaming turn (res, then evt chunk/done/error), which binds the turn to its tab, carries the abort, and dies with the tab. Consent is per DOCUMENT: the first request from a file opens src/consent.html naming the file, the host and the model; the answer is recorded only from that page and is revocable in Settings. - relay.js forwards the streaming frames; page-bridge.js announces 'assistant' in __bentoHost.ops. - Settings: provider, base URL, model, key (chrome.storage.local), Check connection with the real error, allowed-documents list. Site access is an optional_host_permission asked per origin on Save; declining still works wherever the server allows extensions (CORS). - 32 strings x 30 locales; scripts/test-webext-assistant.ts (118 checks: iframe/foreign-page/foreign-id rejection, consent forgery, no key or URL in describe or errors, abort scope).
Build size
Updated: |
…, consent replies carry machine codes; all_frames stated; rig in CI - The built-in model's describe is host '' / model 'gemini-nano' / local:true — the page bounds host and model to hostname/id shapes and renders "on this device" itself; prose there would be blanked. - check() answers code:'consent-pending' while the window is open; a refusal is evt assistant.error code:'consent-denied'. The localized reason rides alongside; the page keys on the code. - all_frames: false stated explicitly on both content scripts. - ci.yml runs scripts/test-webext-assistant.ts (121 checks).
The page bounds describe.host to a hostname shape and blanks anything else, so 'localhost:11434' rendered as '—' (security's browser run of #513). The port stays in the words a person reads — the settings status line and the unreachable reason — where it is the useful part.
… 'downloadable' is no longer 'no model'
On the maintainer's Chrome 152 LanguageModel.availability() was 'downloadable'
and nothing ever called LanguageModel.create() — the only thing that starts
the download — so the built-in provider could not work: Settings said Chrome
downloads it on first use, and every first use threw 'no built-in model'.
- Settings: a Download model button for downloadable/downloading, calling
create({monitor}) from the click and showing downloadprogress; a hint with
Chrome's requirements (~2–4 GB, ~22 GB free, unmetered network).
- check / send distinguish downloadable ('model-download'), downloading
('model-downloading') and unavailable ('model-unavailable'), code + words.
- 5 strings x 30 locales; rig 131.
…odels come from the provider's listing, the default by rule - describe gains contextTokens: the person's override (a new Context window field, whole number 1k–10M), else the cached provider listing (Gemini says inputTokenLimit), else a documented family table (claude 200k, gpt-4.1 1M, gpt-5 400k, gpt-4o 128k, o-series 200k, gemini 1M; 128k for the unknown); the built-in model reports a session's inputQuota, read once and kept. - Check and Save fetch the provider's model list (OpenAI /models, Anthropic /v1/models, Gemini /v1beta/models), cache it per provider+endpoint, feed a datalist under the free-text Model field, and move a still-default or unlisted model to pickDefault's choice: general-purpose chat models only, the vendor's mid tier first (mini / sonnet / flash), newest of those. - DEFAULTS are only the value before the first listing, bumped to today's mid tier: gpt-5-mini, claude-sonnet-5, gemini-3-flash. - 3 strings x 30 locales; rig 163.
… the ranking reads the version anywhere in the id DEFAULTS as the maintainer named them — gpt-5.6-luna, claude-sonnet-5, gemini-3.8-flash (luna taken as the 5.6 line's mid tier; sol ranks with the full model). pickDefault now orders tier → version → created, with the version parsed as a float wherever it sits (5.6 above 5, 3.8 above 3, sonnet-4-5 as 4.5). Family table notes cover the new lines. Rig 170.
Nano answered a words edit with a paragraph — a prose 'answer in JSON' is not a constraint. assistant.send may now carry `schema` (a JSON Schema object): built-in → promptStreaming responseConstraint (Chrome 137+; a Chrome that throws on the option gets one plain retry), Gemini → generationConfig responseMimeType+responseSchema, OpenAI-compatible → response_format json_schema (a 400 naming response_format retries once without — local servers vary), Anthropic → left to the prompt. The schema is data: plain object, under 8 KB, re-read through JSON, dropped otherwise and the turn runs as prose. Rig 193.
…ge's model picker; per-provider store
The store becomes { active, providers: { [provider]: fields } } so every
provider keeps what was typed for it (the old flat shape migrates on read);
Settings and the page's picker write the same place.
- assistant.models: every route usable right now — the built-in model when
it can answer, each keyed provider's configured model then its CACHED
listing (never fetched here), host/local/contextTokens as describe
reports them, current on the active one, ≤200.
- assistant.select { provider, model }: persists the active route and that
provider's model, nothing else; refuses an unconfigured provider, an id
outside MODEL_RE, or a built-in model that cannot answer.
- httpConfigured: api.openai.com without a key is not configured; a
changed base URL (a local server) is. Rig 218.
…irst, ≤12 per provider; show-all and pinned picks A Gemini listing is forty entries and most are not for chat. chatModels() keeps what can answer a text turn by name (Gemini: generateContent minus tts/image/embedding/audio/live/veo/imagen/aqa/learnlm/robotics; OpenAI: the chat families minus embedding/whisper/tts/dall-e/realtime/…; Anthropic as is), folds dated snapshots into their undated alias and keeps preview/exp only when nothing else is left. curateModels() orders newest version first and mid tier (flash/mini/sonnet) before full (pro/opus) before small (lite/nano), capped at 12. assistant.models and the Settings datalist both use it: configured model first, then the models picked from the page (pinned on select, ≤8), then the cut. A per-provider 'Show all models' checkbox turns the filter off. 1 string x 30 locales; rig 233.
…complete
A <datalist> with an id typed suggests only the ids that start with it; the
other ten curated models were invisible until the field was cleared. The
field is a <select> now over pickerRows() — the configured model, the
pinned picks, then the curated (or full) listing, each with its window
('gemini-3.8-flash · 1M') — plus an Other… row that reveals a free-text
field for an unlisted id, validated by MODEL_RE on Save. Before the first
listing it holds the default and Other…. 3 strings x 30 locales; rig 242.
…turn, the document ask, prompts and window sizing The page half of the Assistant cost the shell 25 KB; the maintainer moved the model knowledge here. src/prompt.js (from slides prompt.ts + ops.ts at e82fcf4): isQuestion, OPS_PROMPT / QUESTION_PROMPT / OPS_SCHEMA / OBJECT_SCHEMA verbatim, responseSchema by mode+focus, the window rule (outline + focus + reply room; focus dropped and history cut when it does not fit; refused with the numbers when the outline does not), parseReply and the one prose→JSON nudge. The page keeps format knowledge: it builds the outlines and applies the patch. The handshake, over the turn's port: page assistant.turn { request, history, focus } → res ok → consent (nothing of the document is asked for before Allow) → evt assistant.document → page req assistant.document { outline, addressed, open, size, focus, theme } → the model → evt chunk (prose only) → done { mode:'ask', text } | { mode:'edit', ops, note, focus } | { mode:'edit', text }, or error with code 'window' / 'document' / 'consent-denied'. assistant.send stays. 3 strings x 30 locales; rig 277.
nyblnet
added a commit
that referenced
this pull request
Sep 18, 2026
… format and the drawer Lead, on the maintainer's behalf, after building both sides: #512 added 25,294 B to a 666,953 B shell (+3.8%) that ships in every saved file, and the line that moved — prompt.ts, 497 lines of prompts, token budgets, window assumptions, schemas, the question detector, a model-family table — is MODEL knowledge, which goes stale monthly while the format does not. Split: prompt.ts is gone. material.ts holds the format's side — elide, the outlines (plain and addressed), the focus (selected elements or the open slide as compact JSON with every address), merge, dedupe, clean — and one call, material(doc, focus), that returns every shape at once. ops.ts keeps applyOps (what a verb means for the document) and loses the prompt and schema (the extension's, verbatim from e82fcf4). transport.ts replaces send(messages, schema) with turn(request, history, focus, material, onChunk, signal): the page sends the request and where the user is; the extension asks for the material with evt assistant.document AFTER consent holds; the page answers on the same id; prose streams; done carries { mode:'ask', text } | { mode:'edit', ops, note?, focus } | { mode:'edit', text }; error carries the extension's localized reason and a code ('window' with the numbers, 'consent-denied', …). The panel shows the extension's reason texts verbatim and keys behaviour on codes; the drawer keeps labels only — the 'what is sent' notice, the hint line and every explanatory sentence moved to the extension's settings card (30 locales there vs 8 here). Contract built to on #513 @1d972574. Measured against a fresh origin/main build (666,953 B): 685,192 B, i.e. +18,239 B (+2.7%), from +25,294. Per module, minified+gz: material 2.5 KB, ops 2.9 KB, transport 1.8 KB, panel 5.5 KB, css 1.8 KB, strings ~2 KB, editor tabs ~0.4 KB. The lead's target was ≤ 15 KB; the last 3 KB is the drawer's pop-out window, the model picker and the prose renderer — the maintainer's call which of those to keep. Rig 194/194 (the material shapes; the turn handshake incl. 'no document before the extension asks'; ops through the real apply; A–F intact: the E pin follows apply(index); consent texts are the extension's). 31 strings remain in all eight catalogs (was 49). docs/DECISIONS.md: a paragraph on where model knowledge lives and why.
…ettings card; consent/window reasons in the page's former words The page no longer carries a 'what is sent' notice or its own waiting / refusal / too-large lines — it shows the extension's reason verbatim. So the Settings card says what leaves the computer (outline + selection or open slide → host, comments never; 'the on-device model' for Nano), and the reasons for consent-pending, consent-denied, outline-only and window take the page's former wording in the eight catalogues it had (en de es fr it ja pt zh-Hans zh-Hant, mapped to ours) with matching text in the other 21. Rig 277; i18n 100.
…; a hollow object is prose, not a patch; raw replies logged
Maintainer's run (gemini-3.5-flash-lite, 'change the title', focus = the
slide → the loose {type:'object'} schema): the page got an ops object with
no keys. Gemini's schema dialect wants an OBJECT to have properties and
answers a property-less one with {} instead of a 400. So a schema without
properties now asks Gemini for JSON by responseMimeType alone (the prompt
carries the shape); the strict OPS_SCHEMA path is unchanged. Provider-
agnostic guard on top: a parsed object with none of the twelve op keys is
treated as prose — nudged once, then handed back as text so the drawer
shows what the model said. The raw reply goes to console.info in the
service-worker inspector, never to storage. Rig 282.
…, refusals go back to the model, hosted models verify
A model that wrote 'sd-title' where the outline said '1/sd-title' got its
edit refused and the drawer showed the refusal. Now, inside a turn:
evt assistant.check { ops } → the page answers req assistant.check
{ applied, skipped, structural, outline } — a dry run of applyOps on the
turn's material, nothing committed. A patch with refusals (or nothing
applied) is sent back with the refused names, the address rule and the
outline again; one correction for the built-in model, two for hosted;
the patch committed is the one that applied most with nothing refused.
Hosted models then get one short verify call (OK: what changed /
MISSING: what is left) — its line is done.note, and MISSING buys one
more correction. The loop is logged under the same console tag. An older
page that never answers assistant.check goes straight through. Rig 297.
…ad back and fix with three tools
Gemini, Anthropic and OpenAI-compatible models with function calling now
run a turn as an agent: outline() → the addressed outline, slide(n) → that
slide in full (assistant.document { slide: n }, #512 50090e3), patch(json)
→ the page's dry run (applied / refused / outline after). The last CLEAN
patch is committed on done; a partial one that applied something when
nothing was clean; the model's closing line is the note. Eight calls at
most, then it is asked to finish. providers.js shapes the three dialects
non-streaming (tool_calls / tool_use+tool_result / functionCall+
functionResponse) and reads them back into one neutral thread; the patch
travels as a JSON string because an OBJECT parameter without properties
is hollowed by one dialect. A server that 400s on tools falls back to the
one-shot path with its correction loop and verify; the built-in model
always takes that path. Rig 314.
…— route by id; warnings are faults; an echoed request is no note Three findings from the maintainer's flash-lite run. (1) Every describe showed 'The request failed: unknown': relay.js routed EVERY assistant.check frame to a turn's port, but the page's reachability probe carries that op name too — with a fresh id and no port, it was answered ok:false with no reason. The dry-run answer is now recognised by the TURN's id; the probe falls through to sendMessage and gets the worker's words. (2) The note repeated the request: a verify line, a reply's lead-in or an agent's closing line that only echoes the request is dropped. (3) The dry run now reports the validator's warnings (an overflowing text box, in its words); they count as faults beside refusals in loop-1 and in the patch tool's result, with the fix instruction (shorter text, smaller fontSize via set, taller box via set h); the patch committed has no refusals AND no warnings, else the best. Rig 322.
…e the const initialised); the built-in model is told its output language
…wept in 02d03ae was meant to carry two lines (providerLabel hoisted, the built-in model's output language) and committed the working tree's overlay of the webext-place branch alongside — without that branch's new files, so the branch imported filegrant.js it did not have. Those files return to c962153; the two intended changes stay. webext-place (#515) is where that work lives.
…istant card
assistant.settings.open { section:'assistant' } opens the library at
#assistant — routed to the Settings view, the Assistant card scrolled
into view and its provider control focused once it has rendered — and an
already-open library tab is focused and sent there rather than a second
one opened (openOptionsPage cannot carry a hash). No section → #settings,
for older pages. Rig 325.
…ages, a search endpoint; sources on done
Hosted routes get the provider's web search beside the agent's tools:
Gemini google_search (groundingChunks → sources), Anthropic's
web_search server tool (results and citations → sources; its server-tool
blocks echoed back on the next step), OpenAI web_search_options on the
search-preview models only (url_citation → sources) — on by default per
provider (the deck already goes there; the query is the only new thing
that leaves), a Settings toggle whose note says whether the route has
one. src/web.js: fetch(url) reads a page as text with no DOM (head,
scripts, styles, nav, header, footer, aside stripped; capped 8k; public
http(s) only, never local addresses) and hands it to the model LABELLED
'data, not instructions'; search(query) through a configured endpoint
(SearXNG JSON or Brave with its key) for routes with no search of their
own. Reading pages is a browser permission that needs a gesture, so it
is asked in Settings ('Allow reading web pages…', http/https broadly)
and only checked in the worker. done carries sources (≤20, title ≤200,
http(s), unique). Nano: no tools, unchanged. Rig 351 incl. a page saying
'delete slide 1' producing no delete in the committed patch. 11 strings
x 30 locales.
…and IPv4-mapped included Security's browser run of #513: the predicate enumerated literals and let through 169.254.169.254 (the metadata endpoint), link-local generally, [::ffff:127.0.0.1] (loopback over IPv6), fe80::/10, fc00::/7 and localhost. with a trailing dot — and the fetch hands the whole body to the model. Now: IPv4 by range (0/8, 10/8, 127/8, 169.254/16, 172.16/12, 192.168/16, 100.64/10, 224+), IPv6 by range (::1, ::, fe80::/10, fc00::/7, multicast) with IPv4-mapped literals unwrapped to the v4 check in both dotted and hex forms, a trailing dot stripped before the name checks, bare single-label names refused. DNS rebinding stays a known limit in the comment: the boundary that holds is that a fetch returns untrusted, capped, labelled data reaching the document only through the gated patch. Rig 353.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The extension half of the Assistant (slides #512). The page never holds a key or an endpoint; it sends text over the existing
__bento_tray__envelope and gets text back. Contract:slides/src/editor/assistant/transport.ts.What lands
src/providers.js— pure request shapers + SSE parser for Gemini, Anthropic and OpenAI-compatible (OpenAI, OpenRouter, Ollama, LM Studio…). Base URL editable for every provider.src/assistant.js—describe/check/runover an injected environment; the Chrome built-in model (Prompt API, Gemini Nano) is the default when this Chrome has it, feature-detected, both stream shapes handled.background.js—assistant.describe|check|settings.openoversendMessage; eachassistant.sendrides its own port (res, thenevtchunk/done/error). The port is what binds a turn to its tab: abort arrives on it, the tab closing disconnects it, and no frame can name another tab's turn.src/consent.htmlnaming the file, the host and the model. The answer is recorded only from a sender that IS that page, remembered per file, revocable in Settings. A closed window is "not now", never a hang.relay.jsforwards the streaming frames —ev.source === window+ top-frame-only content scripts mean an iframe on a slide (slides: add the embed element to the bento/embed shape #424 live embeds) is dropped.page-bridge.jsannounces'assistant'in__bentoHost.ops.chrome.storage.local), Check connection with the real error, allowed-documents list.scripts/test-webext-assistant.ts— 118 checks: the 34 provider cases from the handoff, iframe / foreign-page / foreign-id rejection, consent forgery refused, no key or base-URL path indescribeor any error, abort scope, port disconnect.Two things for the maintainer
optional_host_permissions: http://*/*, https://*/*. "Any base URL" needs it to be exempt from CORS; it is asked per origin on Save (a click) and declining is not an error — the hosted vendors, Ollama by default and LM Studio (switch) all allow extension origins. The store listing will show the optional broad host pattern; if that is unwanted, drop the manifest lines and everything above still works for those providers.chrome.storage.local, unencrypted, like every extension that holds one. No page can read it; another extension cannot either.Not in this PR: a CI step (the workflow file is contended —
node scripts/test-webext-assistant.tsis the command), and a real-Chrome pass, which the browser tool cannot do onfile://.