Add CRegCCD: class-based regularised CCD with full support

[alexeid]

Adds CRegCCD, a full-support CCD that replaces KRegCCD's escape mechanism with additive smoothing over every bipartition of a clade. Paper section is in a PR on the manuscript repo (yangsoph/CCD_regularisation#10).

The model

The pseudocount depends only on how many novel clades a bipartition introduces:

class	bipartition	pseudocount
$A_0$	both children observed — the CCD0 split set	alpha per split
$A_1$	exactly one child observed	alpha1 total
$A_2$	neither child observed	alpha2 total

|A_2| follows by subtraction from 2^(m-1)-1, so the exponentially large class is never enumerated. There is no #P-hard counting, no reserve equation and no truncation — the cost is one O(#clades) scan per clade, comparable to the CCD0 expansion.

Per-class totals rather than per-split constants are essential: with a constant pseudocount the two-novel class swamps the data, leaving observed splits 6.3e-9 of the probability at a 40-taxon root clade (SplitClassSizeAnalysis).

Properties hold by construction rather than by correction:

• exactly normalised by the chain rule — no partition function, and the Theta(mu^2) maximality deficit KRegCCD carries cannot arise, because there are no regions
• full support
• regCCD nested exactly at alpha1 = alpha2 = 0
• an observed split always outranks an expanded one at the same clade
• the mass held back shrinks as f(C) grows

What is implemented

Scoring, sampling, MAP and entropy.

Sampling draws a class then a member — classes 0 and 1 by enumeration, class 2 by rejection from uniform bipartitions with an enumeration fallback when acceptance would be poor. Each draw is scored with the same routine the scorer uses, so the sampled and scored distributions coincide exactly; there is no sampling-fidelity choice to make.

MAP runs a DP over the observed-clade DAG, with an optional wider search admitting one-novel-clade splits, plus a bound that can certify global optimality.

Entropy is unbiased by Monte Carlo, since sampler and scorer agree. A deterministic recursion is also provided; it approximates novel subclades as structureless and is therefore optimistic, measured at +2.0% to +10.4% on real data, so Monte Carlo is the estimator of record.

Verification

In CRegCCDTest and CRegCCDMapEntropyTest:

• total mass 1.000000000000 by enumeration on 4–7 taxa, for several training sets and arbitrary pseudocounts
• agreement with RegCCD to 1.8e-15 at alpha1 = alpha2 = 0
• class sizes partitioning all 2^(m-1)-1 bipartitions on 135 clades
• sampled frequencies matching scored probabilities across all 105 five-taxon topologies (worst |z| = 2.58 at 1e6 draws), plus the entropy identity at six taxa
• MAP search matching brute force in twelve configurations, certificate firing in each

Also included

Measurement tests used to characterise the models: KRegNormalisationTest (KRegCCD is exactly normalised on four taxa, sub-normalised beyond, deficit scaling as mu^2 and never exceeding one), SplitClassSizeAnalysis, ClassUsageAnalysis, and RealDataHeadToHeadTest, which compares CCD1, regCCD, KRegCCD, MRegCCD and CRegCCD on a real posterior with hyperparameters selected on a disjoint validation split.

Known limitations

• Within class 2 the model is uniform, so a bipartition into two novel clades is chosen without regard to the observed structure it destroys. On a real 129-taxon posterior these account for 0.01% of internal nodes and 0.4% of total log probability, but that is where the model is weakest.
• The MAP certificate fires at 5–8 taxa but is inconclusive at 129 taxa; there the result is the MAP over the graph, the same guarantee CCD0 and regCCD give.
• alpha1 and alpha2 are class totals, not per-split pseudocounts, so the alpha naming is arguably misleading — mu1/mu2 is under consideration and not yet applied.

🤖 Generated with Claude Code