[lalalune]

Text consumers could admit a prompt against a guessed context window while the fused runtime selected a different environment default or GGUF training-context clamp. Add eliza_inference_llm_context_capacity and use the same resolver in stream creation, so callers can obtain and explicitly pass the effective window before token admission. The resolver loads the resident model with the requested GPU placement and does not allocate a decode context.

Rebased (2026-10-13): this is now a single commit directly on eliza/develop-pin-ea8b3f2 (c6e523b6, which includes #59's libelizainference ABI v17) and no longer depends on #52. Because #59 took v17, this additive capacity contract is renumbered to ABI v18: ELIZA_INFERENCE_ABI_VERSION is 18, the header banner and changelog carry a v18 entry above #59's v17 entry, and eliza-inference-test-abi / eliza-inference-test-paradee-asr expect "18". eliza-inference-test-abi also now checks the new entry's NULL-argument and missing-text-GGUF errors.

Parent integration note: eliza's packages/app-core/src/services/local-inference/voice/ffi-bindings.ts must bump its expected ABI to 18 in the same gitlink bump that picks up this commit; a loader still expecting 17 will refuse to bind. Parent integration: elizaOS/eliza#28432.

Local validation of the rebased commit (macOS arm64, CPU build, -DLLAMA_BUILD_TESTS=ON): ctest -R eliza-inference-test 2/2 passed (eliza-inference-test-abi 25 checks, eliza-inference-test-paradee-asr 518 checks, 0 failures). The validation list below was recorded by the earlier stacked revision 76dd0ca5, whose capacity code is unchanged here.

Validation on 76dd0ca5b189f1f1b5bd2f4014d06f8caec0540b:

• Private macOS arm64 Metal build, one CMake job, clean native source; required fused symbol verification passed.
• Real BGE GGUF resolves its 512-token training clamp. Gemma resolves its 32768-token training clamp and 8192-token default. Separate process startup with ELIZA_LLM_N_CTX=256 resolves 256; explicit 128 resolves 128 for both models.
• Real Gemma native prefill/decode/close completed with the full prompt The capital of France is. Four bounded output tokens produced Paris.\n\nThe.
• Actual desktop and AOSP JavaScript consumers, running against the private macOS library, reject budget-limited output with MODEL_OUTPUT_INCOMPLETE and a complete oversized prompt with MODEL_PROMPT_TOO_LARGE. This also exposed and repaired a parent-side use of speculative acceptance statistics as output-token counts.
• Parent consumer suites: 2926 desktop tests plus 60 harness tests, 144 AOSP tests; package lint/typechecks and native contract gate pass. Parent 26aff10fab6d48bd11d145f6fd1a2a78651df2df root verification passes all 381 tasks (0 cached, 8m9s), with source hashes unchanged. Scalar-reference self-test also passes: finite kernel outputs, fused-attention, TBQ V-cache and split-K online-softmax merge parity.

The decode checks used CPU model placement in the source-built Metal-capable library. These are macOS host results, including the AOSP loader exercised on macOS; no Android device is attached. They do not qualify installed Android, CUDA, or #52's outstanding hosted GPU checks. No installed global library was replaced.

Exact artifact identities

• Native source: 76dd0ca5b189f1f1b5bd2f4014d06f8caec0540b; dirty fingerprint empty.
• Fused library SHA-256: 500dcabd043f20576365c460eacd8e6924eabc1a8a9b642aaa2c580bf1ae3662.
• bge-small-en-v1.5-f16.gguf: SHA-256 f0b2fef971e8366438bfd2d9aefea1b0115919389448806d290237f638bae999 (67308128 bytes).
• gemma-3-270m-it-Q8_0.gguf: SHA-256 d156a5159f2f79c1b1d53c7c1cc20f1ff28ab8d00f17a292620aad13399b9698 (291546144 bytes).

Prepared with Codex assistance under the repository owner's direction.

🤖 Generated with Claude Code