Skip to content

Retry the model manifest fetch in stage-model.sh - #285

Merged
mehfuzh merged 1 commit into
mainfrom
stage-model-manifest-retry
Nov 5, 2026
Merged

mehfuzh merged 1 commit into
mainfrom
stage-model-manifest-retry

Conversation

@mehfuzh

@mehfuzh mehfuzh commented Nov 5, 2026

Copy link
Copy Markdown
Member

Why

The 1.0.69 release build (run 42085166947) failed on every platform at Stage bundled resources. The request for the model download manifest (/v1/models/slp-orchestra-mini/download) hung for the full 60s, and the script gave up after that single attempt. The log only said "Could not reach the download manifest", with no reason.

The API side of this is fixed in smartloop-ai/slp-api#161 (stuck DB calls now time out and return 503). This makes the build tolerate a transient failure too.

What

scripts/stage-model.sh:

  • curl --retry 5 --retry-delay 10 --retry-connrefused: retries timeouts, refused connections and 408/429/5xx. That includes a 503 from the API or a 429 from its 5/minute download limit.
  • --max-time 30 per attempt (was 60, single try): about 4 minutes worst case.
  • -S: the log now shows why the last attempt failed (timeout vs HTTP status).

Testing

  • bash -n scripts/stage-model.sh
  • Ran the same curl invocation against the live API: it returned the manifest.

On Nov 5 the release build failed on every platform at "Stage bundled
resources": the manifest request to api.smartloop.ai hung for the full 60s
and the script gave up after one attempt, with no reason logged.

- Retry up to 5 more times, 10s apart, on timeouts, refused connections
  and 408/429/5xx; each attempt gets 30s (about 4 minutes worst case)
- -S so the log shows why the last attempt failed
@mehfuzh
mehfuzh merged commit dc35a9f into main Nov 5, 2026
2 checks passed
@mehfuzh
mehfuzh deleted the stage-model-manifest-retry branch November 5, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant