Skip to content

Measure this type against a bare double, and chart it per release - #85

Merged
matt-edmondson merged 1 commit into
mainfrom
claude/magical-knuth-idzj5r
Sep 16, 2026
Merged

matt-edmondson merged 1 commit into
mainfrom
claude/magical-knuth-idzj5r

Conversation

@matt-edmondson

Copy link
Copy Markdown
Contributor

Adds AbstractionCostBenchmarks and a third chart section carrying what it measures. The same class, with the same loops and the same methodology, goes into ktsu.SignificantNumber and ktsu.Semantics, so the three libraries answer one question the same way and their answers are comparable with each other as well as with double.

the chart as it stands

What it says

Every other benchmark here answers "how long does this take", which is only readable beside something. This supplies the something — the primitive a caller would otherwise have used.

release Add Multiply
1.8.0 99.7× 293.9×
1.9.0 99.7× 288.2×
2.0.0 82.8× 248.8×
2.0.1 83.2× 248.3×
2.0.2 83.8× 246.0×
2.0.3 83.7× 252.9×
2.0.4 83.1× 246.2×
2.0.5 83.6× 249.8×

Becoming a value type in 2.0 took about 15% off the price of arbitrary precision, and six releases have held it there. Neither half of that is readable from a column of nanoseconds, which is why the ratio is drawn rather than left to be divided by eye.

The ratio is not expected to be 1 and is not a defect for being large — a double cannot do this at all. What the section is for is noticing the day it moves.

How the number should be read

Three things decide that, and all three are in the class's remarks:

  • These are loops, deliberately. A single operation over operands that do not change is loop-invariant and the JIT hoists it out — which would leave the double side indistinguishable from an empty method, and a ratio against an empty method means nothing. Each iteration feeds the next, so there is nothing to hoist and both sides are measurable.
  • The loop's own cost biases toward 1, being paid identically by both sides, so a ratio is a floor on the real cost rather than the whole of it.
  • Both loops accumulate rather than compound. This type carries as many digits as the arithmetic produces, so a compounding chain would measure that growth instead of the operation. Operands are also chosen to be values a double can hold, so the two sides do the same arithmetic on the same numbers. How the cost grows with digits is a different question, answered by ArithmeticBenchmarks across its Digits axis.

The chart section

It derives its ratio from two benchmarks already stored rather than recording a new field. So there is no schema change, no history to rewrite, and an entry gathered before the pair existed simply draws as "not measured" — which is what it did when I first rendered it against the existing history.

The backfill now measures packages, not tag worktrees

Both were defensible until a benchmark was added. A tag predating one cannot run it, so this section could only ever have started from the next release, and the 1.x points above would not exist. Measuring the published packages with today's sources gives the new benchmark the same history as everything else, times every version by identical code, and matches how ktsu.SignificantNumber and ktsu.Semantics already backfill — so the three charts are now built the same way. The release path is unchanged and still measures the tag's own worktree.

Two consequences, both handled:

  • ConstructionBenchmarks measures through CreateFromComponents, which is internal and so invisible to a package. It is left out of those builds and its panel draws from the working copy onward.
  • ingest gained the guard the other two repositories already had, refusing a run where every row reads NA rather than recording a release with nothing under it. This copy predated that fix, and measuring packages is exactly the situation where a build-and-throw NA table arises.

Pre-2.0 releases also needed = default! on the fields: this type was a class then, where an unassigned field is a null reference the compiler rejects.

Verification

  • dotnet build -c Release — clean, 0 warnings.
  • dotnet test -c Release — 266 passed, 0 failed.
  • The table above is a real local seeding run: all 8 releases measured in one pass against a single reference reading, so the ratios are comparable as they stand.
  • Both package-build paths were exercised: BenchmarkAgainstVersion=2.0.4 (struct) and =1.8.0 (class).

🤖 Generated with Claude Code

https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8


Generated by Claude Code

…tch]

Adds AbstractionCostBenchmarks and a third chart section carrying what it
measures. The same class, with the same loops and the same methodology, goes
into ktsu.SignificantNumber and ktsu.Semantics, so the three libraries answer
one question the same way and their answers are comparable with each other as
well as with double.

Every other benchmark here answers "how long does this take", which is only
readable beside something. This supplies the something: the primitive a caller
would otherwise have used. Eight releases of it:

            add      multiply
    1.8.0   99.7x    293.9x
    1.9.0   99.7x    288.2x
    2.0.0   82.8x    248.8x
    2.0.1   83.2x    248.3x
    2.0.2   83.8x    246.0x
    2.0.3   83.7x    252.9x
    2.0.4   83.1x    246.2x
    2.0.5   83.6x    249.8x

Becoming a value type in 2.0 took about 15% off the price of arbitrary
precision, and six releases have held it there. Neither half of that is
readable from a column of nanoseconds, which is why the ratio is drawn rather
than left to be divided by eye. The number is not expected to be 1 and is not a
defect for being large -- a double cannot do this at all. What the section is
for is noticing the day it moves.

The section derives its ratio from two benchmarks already stored rather than
recording a new field, so there is no schema change, no history to rewrite, and
an entry gathered before the pair existed simply draws as not measured.

Three things decide how the number should be read, and all three are in the
remarks. The loops exist because a single operation over operands that do not
change is loop-invariant and the JIT hoists it out, leaving a double method
indistinguishable from an empty one and a ratio against it meaningless. The
loop's own cost is paid by both sides and pulls the ratio toward 1, so a ratio
is a floor rather than the whole cost. And both loops accumulate rather than
compound, because this type carries as many digits as the arithmetic produces
and a compounding chain would measure that growth instead of the operation.

The backfill now measures published packages rather than each tag's own
worktree. Both were defensible until a benchmark was added: a tag predating one
cannot run it, so this section could only ever have started from the next
release, and the 1.x points above would not exist. Packages give it the same
history as everything else, time every version by identical code, and match how
the other two repositories backfill. Two consequences follow.
ConstructionBenchmarks measures through an internal factory, which a package
does not expose, so it is left out of those builds and its panel draws from the
working copy onward. And ingest gained the guard the other two already had,
refusing a run where every row reads NA rather than recording a release with
nothing under it -- measuring packages is exactly where that arises.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017jrnV7N94UGL8fDRRE8Xt8
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants