Skip to content

[Feat] Make the decision deadline configurable for slower local System One servers - #36

Open
chaimaerachdi wants to merge 2 commits into
ThinkFlowLab:mainfrom
chaimaerachdi:decision-timeout
Open

chaimaerachdi wants to merge 2 commits into
ThinkFlowLab:mainfrom
chaimaerachdi:decision-timeout

Conversation

@chaimaerachdi

@chaimaerachdi chaimaerachdi commented Oct 1, 2026 •

Copy link
Copy Markdown

Why

JevModel cuts every decision at 5 s (DECISIONS_TIMEOUT_S), a deadline sized for Jev, which answers in about
0.3 s. A local System One server reached through TYPESAFE_API_URL can be slower. On Google Flights, OneJev-27B
(OmniJev's new 27B model, which speaks the same API) on an A100 takes about 3.7 s a decision and more on the calendar
page, which offers 66 more elements. Every run stopped at the eighth step, right after the calendar opened.

How

  • s1a/decision_models/wire.py: decisions_timeout_from_env() reads S1A_DECISION_TIMEOUT_S; unset, it returns
    DECISIONS_TIMEOUT_S (5 s). A value that is not a finite, positive number (nan, inf and 1e9999 included) is
    MODEL_SERVICE_CONFIG_ERROR, naming the variable.
  • s1a/decision_models/jev.py: JevModel.from_env(timeout_s=None) uses it; an explicit timeout_s still wins.

What

  • New setting S1A_DECISION_TIMEOUT_S (docs/configuration.md, .env.example, CHANGELOG.md).
  • Nothing changes when it is unset: Jev keeps 5 s.

Measured (Google Flights, Zurich to London, one way; A100 80 GB; headless Chromium)

Success is judged by a vision model (Gemini 2.5 Flash) on the final screenshot. Time is split with the repo's
profiler (--profile-out), as in docs/benchmarks.md.

Model Deadline Success Model time per task Per decision Page Browser
Jev 1.13 (OpenRouter) 5 s 5 / 5 3.3 s 0.29 s 12.3 s 9.6 s
OneJev-27B (local, qev serve) 5 s 0 / 5, all stopped at step 8 - 3.0 s - -
OneJev-27B (local, qev serve) 30 s 7 / 7, the same 11 actions as Jev in every run 61.2 s 3.7 s 11.4 s 9.7 s

Times are medians over the runs. At 30 s, ten runs were started and three were interrupted by hand before the end;
those three are not counted.

Verification

  • uv run ruff format --check . && uv run ruff check .: clean.
  • uv run ty check: 1 diagnostic in s1a/decision_models/cua.py, the same on main.
  • uv run pytest -q --ignore=tests/test_browser_policy.py: 394 passed, 16 failed; main on this Windows machine
    fails the same 16 (environment), so none is new. 4 new tests in tests/test_decision_models_jev.py
    (TestDecisionTimeout): default 5 s, the variable, an explicit value winning, invalid values (abc, 0, -3, nan, inf, -inf, 1e9999).

JevModel cut every decision at 5 s, a deadline sized for Jev (0.3 s). A local
System One server behind TYPESAFE_API_URL can be slower: on Google Flights,
OneJev-27B on an A100 takes about 3.7 s a decision and more on the calendar
page, so every run stopped at the eighth step. With 30 s it completed the
task with the same 11 actions as Jev.

S1A_DECISION_TIMEOUT_S sets the deadline; unset, it stays 5 s, so Jev is
unchanged. A value that is not a positive number is a config error. An
explicit timeout_s passed to JevModel.from_env still wins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread s1a/decision_models/wire.py Outdated
value = float(raw)
except ValueError:
value = 0.0
if value <= 0:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

float() accepts nan, inf, and overflow such as 1e9999; neither nan nor positive infinity satisfies value <= 0, so they pass this guard. nan then makes the computed deadline and remaining timeout nan, while inf removes the intended bound. Please also reject not math.isfinite(value) and extend the config-error regression with nan, inf, and 1e9999.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you @IRONICBo ; fixed in 847db86: the guard now rejects non-finite values (not math.isfinite(value) or value <= 0), and the config-error test covers nan, inf, -inf and 1e9999.

float() reads nan, inf and 1e9999; nan passed the `<= 0` guard and made the deadline nan, inf removed the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MrDongsls

MrDongsls commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Non-blocking:

  • docs/browser-front.md still documents DECISIONS_TIMEOUT_S as a fixed 5-second value, while this PR introduces S1A_DECISION_TIMEOUT_S. The setting is already documented in docs/configuration.md, so this does not affect runtime behavior, but updating the browser-front reference would keep the documentation consistent.

FYI, out of scope for this PR: an explicit timeout_s passed to JevModel.from_env() can still bypass the new environment-value validation with float("nan") or float("inf"). The environment-variable path introduced here correctly rejects them; this can be addressed separately if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants