Repository navigation
[AUTOMATED] Add --progress: live progress over a reproducible project export - #1290
Merged
Merged
Conversation
…port `decompile-project --stream --stream-layout address` runs the non-stream export unchanged (same serial or pooled decompile, plan, structure naming, merge and artifacts) and reports on it while it runs: `.streaming` on its clock, a running README, and `index.jsonl` as a per-function feed in completion order with null `.c` spans. At the end the four artifacts are the non-stream export's byte for byte, `index.jsonl` is replaced by the address-ordered index of that `.c`, and `.streaming` is removed, so the folder does not depend on --jobs N wherever the non-stream export does not. The default layout (`decompile`) is unchanged. Hooks: jobs::PoolWatch on the pool's planned first pass, a `landed` callback on the serial and callee-first loops, and decompile_project::ExportWatch around the export. Measured on /usr/bin/ssh (1,090 functions): address-layout runs at --jobs 2 and 4 and a non-stream --jobs 4 run leave identical folders apart from index.jsonl; CPU time of the two --jobs 4 arms is within noise. [AUTOMATED] Authored by an agent.
Lukas-Dresel
marked this pull request as draft
October 27, 2026 21:22
Progress reporting and incremental artifacts are two separate things; `--stream` bundles both, and only the incremental `.c` costs reproducibility. `--progress` is now its own flag: the status file and the index.jsonl feed over an ordinary export written at the end. With both flags the export is the streamed one. `--progress` accepts `--assert` under the non-stream export's rules. `--stream-layout` is removed. [AUTOMATED] Authored by an agent.
Lukas-Dresel
marked this pull request as ready for review
October 27, 2026 22:01
mahaloz
approved these changes
Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
decompile-project --stream --jobs Nis the only way to get live progress (.streaming,index.jsonl), but it also writes the.cas functions finish, in worker completion order, so two runs of the same command differ. A pipeline that stores the export content-addressed has to choose between progress and reproducible output.--progressgives the progress over an ordinary export:While it runs,
.streamingreports as it does under--stream, from the load on:--jobs N--progress--streamThe change:
--progressruns the non-stream export unchanged (same plan, structure naming, merge and artifacts) and reports on it..c/.h/.asm/README.mdare byte-identical to the export without the flag.index.jsonlis a feed while the run lasts: one line per function's first result, in completion order, withc_offset/c_lennull. At the end it is replaced by the address-ordered index of the final.c, then.streamingis removed..c/.h/.asmare removed once the load succeeds, so a stale body never sits next to a new feed.--streamis unchanged and wins when both flags are given.--progressaccepts--assertunder the non-stream rules.jobs::PoolWatchon the pool's first pass, alandedcallback on the serial and callee-first loops,ExportWatcharound the export.Tests:
a_progress_export_is_the_non_stream_export_plus_its_index(byte-identity at--jobs 1/2/3),a_progress_export_reports_progress_before_it_writes_the_export(reads the feed mid-run during an injected worker stall),stream_wins_over_progress, and a README render test. Datatests and stage tests unchanged.One limit, inherited from the non-stream export: a function that hits
--max-fn-secondsgets a wall-clock-dependent error string (the pass it was in), so near-budget functions can still differ run to run.