Conversation
The upstream .gitignore had been deleted locally, leaving 23 GB of results/ and a 5.8 GB conda_envs/ visible to git. Restored it and added the two local dirs. results/ stays ignored except top-level *.csv: the per-problem allocation files are the interface between the probe and LLaDA_ProbeLen (problem_id -> canvas length), so a run cannot be reproduced without them, and all ten together are 133 KB. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…EDAL LLaDA_ProbeLen: denoise at a per-problem canvas read from a CSV (problem_id -> length), with no expansion. This is how a length PREDICTOR is evaluated against DAEDAL's own Stage-1 without changing anything else -- same decoder, same grader, same prompt; only the initial canvas differs. LLaDA_SmartCrop: the EOS inverse-survival allocator, as a second published baseline. LLaDA_DAEDAL gains alloc_out, which captures the canvas actually chosen, before the +eos_check_tokens/2 padding and before the clamp to max_gen_length. That makes Stage-1's prediction readable per problem, so it can be scored as a predictor and not only as an end-to-end policy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
capture_task_features.py rebuilds the prompt exactly as DAEDAL builds it (task prompt fn -> chat template -> assistant prefill -> tokenize with add_special_tokens=False) and reads the dataset from the task's own YAML, so doc_id here IS the harness doc_id. That is the whole point: features captured under a different prompt cannot be joined to labels measured under this one, and nothing has to be aligned by hand. canvas_probe.py fits one binary probe per canvas, P(solve @ c | features), and walks the ladder low->high to the first rung clearing tau. Per-canvas fits rather than one pooled model with canvas as a feature -- pooling shares weights across rungs and dilutes the per-rung structure that carries the signal. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every arm of the allocation comparison runs the SAME decoder and differs only in the initial canvas: uniform (both stage thresholds 0), Stage-1 only (Stage-2 disabled via expand_eos_confidence_threshold=0.0), full DAEDAL, SmartCrop, ProbeLen, oracle. The baseline script header records why -- the vanilla LLaDA decoder and the DAEDAL decoder produced 0/34 identical outputs at canvas 32 on the same problems, so any policy-vs-uniform delta measured across decoders mixes allocation with decoding. math_train is the 3262-problem fit set, kept disjoint from the math500 eval set. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The output-length work needs a decoder that can vary the canvas, which the companion repo does not have. It also needs its own labels: the two repos build different prompts and run different decoders, and at canvas 32 on the same 34 problems at temperature 0 they produced 0/34 identical outputs. A different prompt gives a different answer, and a different answer has a different length -- so an out_len measured there is not the target for a run here. No new Python was needed. metrics/math500.py already emits per_problem_tokens.csv with effective_tokens, and make_alloc_from_actual.py already converts it to the allocation schema; the labelling run is eval_LLaDA_Baseline.sh with LENGTHS=2048. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Listing them with diffstats rather than claiming the fork is additive-only -- the baseline script in particular is a real behaviour change (it now runs the same decoder as every other arm), and a reader comparing against upstream needs to know that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.