Repository navigation
feat(cli): --profile, per-entry check times - #1639
Closed
ScottMoore0 wants to merge 1 commit into
Closed
ScottMoore0 wants to merge 1 commit into
ScottMoore0 wants to merge 1 commit into
Conversation
bend <file.bend> --profile checks the file as --check-only does, one order entry at a time: book_valid(book, i) takes the entries before i as validated, so each call times one entry on the book the full check builds. Bodies stay hidden until their last entry, as in the full check. It lists the 20 slowest entries with file and line, then gives --check-only's verdict and exit code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
|
Thanks for this, Scott, and for the careful testing (the full tests/ sweep against --check-only and the timing sanity checks especially). We're going to close it, though. We're trying to keep the CLI small, and per-entry profiling is the kind of thing that works better outside the language: an agent can put together a one-off loop over book_valid quickly when someone needs it, and if we want a proper tool, a standalone Bend profiler (like your bend-prof) is a better home than a bend flag. Your repo already covers that well, so linking to it is probably the best way to get it to people. Thanks again, and please keep the audit findings coming. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
bend <file.bend> --profile: it checks the file as--check-onlydoes, one order entry at a time, lists the 20 slowest entries with file and line, then gives--check-only's verdict and exit code. It adds 77 lines tomain.ts(+958 tokens, 12,489 of the 16,000 cap) and doesn't touchbend.ts. It's independent of #1631: the two merge cleanly, and together leavemain.tsat about 12,900.How it works.
book_valid(book, done)takes the firstdoneentries as validated.cli_profilecuts the order after entryiand callsbook_valid(book, i), so each call checks one entry against the entries before it, on the same book. Checked bodies and template instances stay in the book, so the times are the full check's, split by entry.book_validhides the bodies of the defs in the order it's given until their last entry. Because the order is cut at each step,cli_profilehides every body up front, as the full check does. Without that, a later def's body would be visible early and a forward reference would check. A law's statement entry keeps its body hidden, so its proof is timed once, at its fill, and rows are labelled(statement)or(proof). After the last entry it counts TODOs and callscli_verdict, as--check-onlydoes.This is the loop from a standalone tool I published at https://github.com/ScottMoore0/bend-prof, offered here so it ships with
bend.Tested on
mainat e5527d5 (this branch'smain.tsis byte-identical to the tested copy):.bendfiles undertests/:--profile's output, with the table removed, is identical to--check-only's, and so is the exit code. This includes forward references, unfilled laws, TODOs and@unsafedependants.--check-onlyoutput is unchanged by the patch (first 400 test files compared).aes/inverse.bend(583 entries), and 1.2 s against 0.9 s on adynamic_arrayproof (1,851 entries; each call has a small fixed cost).--profilewith any other option fails with "--profile takes no other option".gates/repo.ts: PASS 54 / 54.tsc --noEmitreports the same errors as onmain, none inmain.ts.Not run:
gates/test.tsandgates/safe.tson the minis. There's no test file for it, since its output is timings.Happy to change the flag name, the row count or the output format, or to drop the verdict at the end.
Found and tested with Claude Code, running Claude Opus 5.5, during an audit of
--verdict.