Accelerating Discrete Diffusion Models with Parallel-In-Time Sampling #17

Model/Pipeline/Scheduler description

Proposes adding a new scheduler to diffusers: a parallel-in-time τ-leaping sampler for absorbing discrete diffusion, based on Accelerating Discrete Diffusion Models with Parallel-In-Time Sampling (arXiv:2607.00773).

The paper contributes a sampling algorithm (not a model architecture or training method) for discrete diffusion: it parallelizes the τ-leaping sampler for absorbing discrete diffusion formulated as a Continuous-Time Markov Chain (CTMC). Using the continuous-time stochastic-integral form of τ-leaping combined with Picard iteration, it achieves parallel-in-time acceleration, reducing per-sample NFE complexity from O(d·log S) to O(log(d·log S)·log d). It reports 7–9× runtime speedup on synthetic distributions and 1.45–1.86× (with 50% fewer NFE) on image/text tasks on a single GPU.

What blocks a clean implementation

• No τ-leaping / CTMC infrastructure exists. A repo-wide search returns zero hits for tau-leap/τ-leap, CTMC, absorbing, or D3PM. The paper's contribution is accelerating τ-leaping — but the base τ-leaping sampler it speeds up is not implemented anywhere in the repo.
• The existing discrete schedulers use a different paradigm. scheduling_amused.py, scheduling_vq_diffusion.py, and scheduling_block_refinement.py (LLaDA2) sample via masked / multinomial / Gumbel schemes. None expose a CTMC rate matrix or a τ-leaping step that the parallel-in-time method could hook into.
• No natural call site, and no scoped slice was identified. No selection rationale named a host module. A new scheduling_tau_leaping*.py would be freestanding — none of the AMuSED / VQ-Diffusion / LLaDA2 pipelines would call it.
• No benchmark/eval path to consume as a lighter slice. benchmarks/ targets continuous models (Flux, SDXL, LTX, Wan) for latency/memory and reports no discrete-generation NFE/quality metrics. There is no released discrete-diffusion artifact the existing eval path can ingest.

Suggested scope spike

Prototype τ-leaping + Picard iteration on a synthetic absorbing-state CTMC (or against a discrete-diffusion checkpoint that already has a diffusers loader), and benchmark NFE/runtime against diffusers' existing masked discrete schedulers (AMuSED, LLaDA2 BlockRefinement). Confirm a real call site exists before committing to a full scheduler integration.

Why this is an Issue rather than a PR

The paper's contribution is a parallel-in-time τ-leaping sampler for absorbing discrete diffusion (CTMC), but diffusers has no τ-leaping/CTMC/absorbing infrastructure to parallelize — its discrete schedulers (AMuSED, VQ-Diffusion, LLaDA2) all use masked/multinomial/Gumbel sampling instead. No selection rationale named a call site, so there is no scoped slice that drops into existing code; a new τ-leaping scheduler would be freestanding and called by nothing. This needs a team decision before building: does the team want absorbing discrete diffusion / τ-leaping support at all (current discrete support is limited to masked-generation text via LLaDA2 and image via AMuSED)? Would we add a standalone τ-leaping scheduler, or retrofit it into an existing discrete pipeline that would need rewiring? Which absorbing-discrete-diffusion checkpoint would exercise it, and is there a loader path in diffusers today?

Open source status

• The model implementation is available.
• The model weights are available (Only relevant if addition is not a scheduler).

Provide useful links for the implementation

• Paper: Accelerating Discrete Diffusion Models with Parallel-In-Time Sampling (arXiv:2607.00773)
• No reference code repository surfaced in the paper, recommendation envelope, or arxiv abstract page — worth confirming whether the authors have an open release before investing in adoption.
• Authors: no GitHub usernames surfaced.

Drafted by Outrider — paper: arXiv:2607.00773.