feat: API executor runs DataMixin prompts, reads python stages and reports token usage - #157

Open

timzsu wants to merge 78 commits into

mainfrom

zsu/api-executor-dataframe

Commits on Sep 27, 2026

feat: add user-specified retries to the API executor

timzsu
and
claude
authored
22b222e
fix(worker): reset stale API executor cancellations and honor cancel during backoff

timzsu
and
claude
committed
ad4752f
fix(worker): shorten stale cancellation comment to a single line

timzsu
and
claude
committed
275ca45
fix(worker): reject cancellations addressed to a different active task

timzsu
and
claude
committed
67b795b
fix(worker): shorten stale cancellation comment to a single line

timzsu
and
claude
committed
38a2a07
fix(worker): keep pending cancellations by task id in the API executor

timzsu
and
claude
committed
096a58a
feat: batch multiple HTTP requests in the API executor

timzsu
and
claude
committed
0fb3d26
fix(sdk): mirror APIResult.items and APIItem in the SDK models

timzsu
and
claude
committed
d2fe09c
refactor: single parallel path for the API executor

timzsu
and
claude
committed
b5583ea
docs: fold batching into the two-stage api template

timzsu
and
claude
committed
630d55a
fix: let APIItem round-trip its own result output

timzsu
and
claude
committed
aff4e63
fix: size the API HTTP pool from the capped concurrency

timzsu
and
claude
committed
da6a2df
fix: resolve API stage dependencies from the first row's text

timzsu
and
claude
committed
d3e31b8
docs: correct the APIResult docstring and guard SDK by-name construction

timzsu
and
claude
committed
4358a0b
refactor: fold API dependency and cache-key notes into docstrings

timzsu
and
claude
committed
dc9da05
fix: make the GPU test import-safe and clean up test style

timzsu
and
claude
committed
da2aef1
docs: fold the concurrency cap note into the API executor docstring

timzsu
and
claude
committed
0b4f12c
test: cover row-specific fields, effective concurrency, and NVML deferral

timzsu
and
claude
committed
7dff652
fix: classify API HTTP errors before strict payload parsing

timzsu
and
claude
committed
9524d1c
feat: add task-scoped cancellation to the API executor

timzsu
and
claude
committed
0341004
test: correct the SDK APIItem docstring and narrow the NVML skip

timzsu
and
claude
committed
3f9e5c7
pin pre-start and in-flight cancellation guards separately
timzsu
committed
40bf0a2
fix: recheck cancellation during in-flight requests and before run
timzsu
committed
9691501
fix: make API executor cancel check-and-reset atomic under a lock
timzsu
committed
76e218c
fix: correct cancellation docs and rename collection-guard test
timzsu
committed
d4ced9f
test: merge API executor batch tests into test_api_executor.py

timzsu
and
claude
committed
b6c01a6
refactor: revert gitignore and GPU test, condense added docstrings
timzsu
committed
de41f50
fix: use flowmesh/v1 apiVersion in API executor tests
timzsu
committed
e4991dd
test: adapt API executor retry tests to the merged batch executor

timzsu
and
claude
committed
2109405
fix(worker): substitute a structured prompt object into the API body

timzsu
and
claude
committed
7d2ddb9
fix(schema): serialize APIItem's json field under its wire alias

timzsu
and
claude
committed
4a05001
test: assert completed requests before collection in cancellation test

timzsu
and
claude
committed
d11996d
test: bound the submission wait in the collection-guard cancellation test

timzsu
and
claude
committed
fc8d5c2
feat: API executor runs DataMixin row-wise and aggregate prompts

timzsu
and
claude
committed
821b2d0
feat: echo task runs a function over whole upstream lists

timzsu
and
claude
committed
7ab51b6
fix: echo function arguments and sandbox sources fail closed

timzsu
and
claude
committed
b4cf53f
fix(worker): declare tabulate, which graph-template DataFrame rendering needs

timzsu
and
claude
committed
1141a99
fix(sdk): mirror APIGroupItem and grouped APIResult.items

timzsu
and
claude
committed
d1c0e3e
fix(worker): a dataframe over zero rows runs zero rows

timzsu
and
claude
committed
13a42cf
fix(worker): expose ValueError to sandboxed functions

timzsu
and
claude
committed
f86ee96
fix(worker): report status from first row across grouped results

timzsu
and
claude
committed
657ba7b
fix(worker): decide dataframe grouping from upstream structure, not cell shape

timzsu
and
claude
committed
bb949c8
fix(worker): group dataframe columns over list-mode lambda output

timzsu
and
claude
committed
5ad3ce1
test(worker): assert APIResult dumps grouped rows under the json alias

timzsu
and
claude
committed
fc4a960
test(sdk): assert grouped APIResult rows dump under the json alias

timzsu
and
claude
committed
ae0be15
feat(worker): log per-call statistics in the API executor

timzsu
and
claude
committed
16676b6
fix(worker): harden API executor prompt and telemetry handling

timzsu
and
claude
committed
5db7f13
docs: document grouped API results and echo function data type

timzsu
and
claude
committed
77e4bec
refactor(worker): pass summary snapshot as named locals

timzsu
and
claude
committed
608e834
fix(worker): cancel a zero-row grouped API task after the pool drains

timzsu
and
claude
committed
6bf1192
fix(worker): log the real attempt count for an exhausted connection retry

timzsu
and
claude
committed
5ef1e96
Merge main (PR 155) into zsu/api-executor-batch

timzsu
and
claude
committed
c16c3c4
Merge zsu/api-executor-batch (main with PR 155) into zsu/api-executor-dataframe

timzsu
and
claude
committed
037d99b

Commits on Sep 28, 2026

Address PR comments
timzsu
committed
d8ac38f
Merge zsu/api-executor-batch (PR 141 review round 2) into zsu/api-executor-dataframe

timzsu
and
claude
committed
477a003
Fix PR comments.
timzsu
committed
b1fe866

Commits on Sep 29, 2026

Merge origin/main into zsu/api-executor-batch

timzsu
and
claude
committed
64e1f92
Merge zsu/api-executor-batch (PR 141 final) into zsu/api-executor-dataframe

timzsu
and
claude
committed
2cbe3d5
Merge origin/main (PR 141 squash) into zsu/api-executor-dataframe

timzsu
and
claude
committed
7a493be

Commits on Sep 30, 2026

Merge remote-tracking branch 'origin/main' into zsu/pr157-rescope

timzsu
and
claude
committed
a48f21f
refactor: remove echo data.type function mode

timzsu
and
claude
committed
5fee751
test: read a python stage's output in dataframe API tasks

timzsu
and
claude
committed
33f2f9e
docs: document how a dataframe API task reads a python stage

timzsu
and
claude
committed
a2be9e5
fix: issue one aggregate prompt per group in graph_template

timzsu
and
claude
committed
6e64b31
test: graph_template direct grouped column issues one request per group

timzsu
and
claude
committed
631c857
fix: record api calls that fail parsing or are cancelled

timzsu
and
claude
committed
73fc8bc
fix: reject echo data.type function with a pointer to python tasks

timzsu
and
claude
committed
ef976c8
style: make APIGroupItem docstrings one paragraph

timzsu
and
claude
committed
2460ac5
docs: dataframe api results are always grouped into APIGroupItems

timzsu
and
claude
committed
fb29688
fix: keep per-row expansion for ungrouped graph-template columns

timzsu
and
claude
committed
5217718

Commits on Oct 1, 2026

fix: address the review round on grouping, echo and API usage

timzsu
and
claude
committed
8257947
feat: report token usage per API task and per workflow

timzsu
and
claude
committed
2bce1a1
fix: serve tasks contribute no usage to the workflow sum

timzsu
and
claude
committed
d7af140
docs: shorten the grouped results section

timzsu
and
claude
committed
3ea5ac7
docs: restore the embedded {{prompt}} substitution rule

timzsu
and
claude
committed
c33e6b2
fix: a condition-skipped task records no usage instead of nulling the sum

timzsu
and
claude
committed
4353593
test: guard APIUsage against server and SDK drift

timzsu
and
claude
committed
01ee51c

Commits on Oct 2, 2026

refactor: drop workflow-level usage; clients sum per-task usage

timzsu
and
claude
committed
349dfae