Repository navigation
test(workflows): pin untested workflow expression behavior - #5130
Merged
mnriem merged 2 commits intoOct 28, 2026
Merged
Conversation
Add positive and negative tests for evaluator behavior that was implemented
and documented but never asserted, so a replacement engine can be compared
against what the evaluator actually promises:
- >= / <= for numbers, numeric strings, plain strings and mixed operands
- case-insensitive boolean literals and none/null literals
- map('a.b'), default on an empty string, from_json for every JSON type,
filter arguments resolved from the namespace
- and/or precedence without parentheses
- the item and fan_in roots, interpolation with steps and item
- the empty block, a missing intermediate path, whitespace around tokens
- gate message interpolation
Tests only; no behavior change. 830 passed in tests/test_workflows.py
(12845 passed, 435 skipped across the full suite).
Assisted-by: OpenCode (model: claude-opus-5.5, autonomous)
Assisted-by: Google Antigravity (model: Gemini 4 Argon Medium, supervised)
mnriem
added
the
triage-can-wait
Collaborator
|
Please address Copilot feedback |
… passthroughs
- Assert _collect_leaves(literal) == [] for null-literal forms.
- Assert default preserves documented falsy values (0, False, [], {}).
Assisted-by: Google Antigravity (model: Gemini 3.8 Flash, supervised)
Contributor
Author
|
Addressed in
AI disclosure: Posted on behalf of @markuswondrak by Google Antigravity (model: Gemini 3.8 Flash, supervised); the change was reviewed and approved before commit. |
Collaborator
|
Thank you! |
3 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Preparation for #5060, PR 2 of 3 (follows #5114). Adds unit tests in
tests/test_workflows.pyfor workflow expression evaluator behavior that is already implemented and documented onmainbut was not directly asserted. Tests only; no production code or behavior change.Why
Before PR 3 replaces the hand-written parser in
engine_current.pywith the Lark LALR grammar (engine_lark_a2.py), the existing suite should pin every documented and implemented behavior of the current evaluator so that the engine swap is verified against the full behavioral surface onmain.Coverage added
test_comparison_greater_equal_and_less_equal>=and<=across numbers, numeric strings ("10" >= "9"), plain strings (lexicographic), and incomparable number-vs-string operands (False)test_boolean_literals_are_case_insensitiveTrue,TRUE,False,FALSEtest_null_literalsnoneandnullin any casing resolve toNonerather than a namespace pathtest_filter_map_dot_path_attributemap('result.status')walks nested dicts and yieldsNonefor missing intermediate branchestest_filter_default_on_empty_stringdefaultfalls back on""as well asNone, while passing non-empty strings throughtest_filter_from_json_returns_non_object_typesfrom_jsonfor JSON arrays, integers, floats, booleans, andnulltest_filter_argument_resolved_from_namespaceinputs.*paths (join(inputs.sep),default(inputs.sep))test_boolean_precedence_without_parenthesesandbinds tighter thanor(a or b and cvs(a or b) and c)test_item_and_fan_in_namespaceitem,item.name, list-indexeditem[0]/item[-1], andfan_in.resultstest_interpolation_with_steps_and_itemsteps.<id>.output.*anditemtest_empty_expression_block_is_empty_string{{ }}evaluates to""in expressions andFalsein conditionstest_missing_intermediate_path_resolves_to_noneinputs.a.bwhena={}) resolve toNonetest_whitespace_inside_and_around_block{{ ... }}and around filter pipes/argumentsTestGateStep.test_message_expression_is_interpolatedGateStepinterpolates{{ ... }}inmessagebefore persisting step outputVerification
uv sync --extra test && .venv/bin/python -m pytest tests/test_workflows.py -q→ 830 passed.venv/bin/python -m pytest tests -n auto -q→ 12845 passed, 435 skipped, 0 failedruff check src tests→ cleanNot in this PR
PR 3 replaces the hand-written parser with the Lark LALR grammar and adds regression tests for the hand-written-parser bugs reported in #5071 and #5073.
AI Disclosure
Tests originally authored with OpenCode (model: claude-opus-5.5, autonomous) under human direction; cherry-picked onto
main, verified, and PR description drafted with Google Antigravity (model: Gemini 4 Argon Medium, supervised).