Deploy LFS assets; stop bengio_infer from wedging the worker - #4
Merged
Conversation
After the cold-start fix, staging surfaced two model failures that were previously masked by the timeout: - shannon_words_ngram -> "MDB_INVALID: File is not an LMDB file" - bengio_infer -> torch.load on a 133-byte file, unhandled exception, which killed the stdio RPC worker and 503'd every request for ~30s until shimmy respawned it. Both are Git LFS: *.mdb / *.pt are LFS-tracked but the deploy workflows checked out with lfs:false, shipping pointer stubs. basic_nn.pt predates the LFS filter (stored inline) so it was unaffected. - staging-deploy.yml / production-deploy.yml: lfs: true - bengio_infer.run(): wrap load+infer in try/except -> return a failing Result instead of raising, so a bad asset degrades one request rather than taking down the worker - bengio_infer.predict_next(): clamp the re-encoded context to exactly N tokens (it could exceed N and hit "mat1 and mat2 shapes cannot be multiplied"), so the model actually produces output once assets load Verified against an image with real LFS assets: all five models (basic_nn, shannon_letters_single, shannon_letters_ngram, shannon_words_ngram, bengio_infer) return is_correct results. pytest passes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UsgxJUsq1vG176rbznHRqz
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After the cold-start fix (#3), staging surfaced two model failures that the 30s timeout had been masking:
shannon_words_ngram→MDB_INVALID: File is not an LMDB filebengio_infer→torch.loadon a 133-byte file, unhandled exception, which killed the stdio RPC worker and 503'd every request for ~30s until shimmy respawned itBoth are Git LFS:
*.mdb/*.ptare LFS-tracked but the deploy workflows checked out withlfs: false, shipping pointer stubs.basic_nn.ptpredates the LFS filter (stored inline) so it was unaffected.Changes
staging-deploy.yml/production-deploy.yml:lfs: truebengio_infer.run(): wrap load+infer in try/except → return a failingResultinstead of raising, so a bad asset degrades one request rather than taking down the workerbengio_infer.predict_next(): clamp the re-encoded context to exactlyNtokens (could exceedN→mat1 and mat2 shapes cannot be multiplied), so the model actually produces output once assets loadVerified against an image with real LFS assets: all five models (
basic_nn,shannon_letters_single,shannon_letters_ngram,shannon_words_ngram,bengio_infer) returnis_correctresults.pytestpasses.🤖 Generated with Claude Code