fix(context): honour an explicit context_size of 512

[andinux]

Fixes the "context_size=512 is silently ignored" part of #28.

The bug

llm_context_create_with_options() decided whether the caller had asked for a context size by comparing the parsed value against llama's default:

struct llama_context_params defaults = llama_context_default_params();
if (ai->model && ctx_params.n_ctx == defaults.n_ctx) {
    ctx_params.n_ctx = 0;   // 0 = use the model's training window
}

That default is 512 (llama-context.cpp:2772) — a value a caller can perfectly well pass. So an explicit context_size=512 was indistinguishable from unset:

requested	actual llm_context_size()
64	256
512	32768 — asked for 512, got 64× that
513	768
1000	1024

n_ctx=512 hit it too, since the check looked at the resolved value rather than at which key was written.

The fix

The intent was right; only the detection was wrong. llama.cpp already defines n_ctx = 0 as "use the training window", so start from that sentinel rather than inferring it after the fact:

struct llama_context_params ctx_params = llama_context_default_params();

// n_ctx = 0 tells llama.cpp to use the model's training window. Start from it so
// that "the caller did not ask for a context size" is an explicit sentinel rather
// than something inferred after the fact from the value [...]
ctx_params.n_ctx = 0;

and delete the comparison block. The caller's value now always survives, and 0 keeps its documented meaning. Net +1/−4 lines — no new struct, no signature changes, and no churn through the four llm_context_options_callback call sites (two of which pass xdata = NULL).

Also stops context_size=0 driving n_batch to 0, which llama will not accept.

Verified against the built extension:

option	before	after
context_size=256	256	256
context_size=512	32768	512
context_size=1024	1024	1024
n_ctx=512	32768	512
context_size=0	32768	32768
(omitted)	32768	32768

Behaviour change

Anyone passing context_size=512 or n_ctx=512 was getting the model's whole window and now gets 512. Nothing errors — the context simply becomes what was asked for, so a conversation that used to fit may now reach the limit.

That is the intended outcome: silently ignoring the configuration is the bug. It does make the other half of #28 easier to hit, where a full context returns an error and leaves the chat unusable for every subsequent turn. That issue stays open.

Patch version bump, 1.0.5 → 1.0.6.

Docs

API.md claimed llm_context_create_chat() and llm_context_create_textgen() were "equivalent to SELECT llm_context_create('context_size=4096')". Their presets are empty strings, so both inherit the model's training window. Corrected, and 0 is now documented on context_size and n_ctx.

Testing

test_context_size_is_honoured covers 256/512/1024 exactly — llama pads n_ctx up to a multiple of 256, so only exact multiples compare equal — plus both spellings, and that omitting the key or passing 0 still auto-sizes. It asserts the model trains above 1024 first, so it can actually tell an honoured size from an auto-sized one.

Verified to fail against a pre-fix build:

[context_size_is_honoured] context_size=512 produced n_ctx=32768
1 C test(s) failed. First failure: context_size_is_honoured

make test: 44/44 pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_012KRojTrn4Q4SaZQpqcwpAt