Skip to content

issues: the system clock takes the TSC on one reading, and that reading misses its rate - #1082

Merged
Japabu merged 6 commits into
mainfrom
tsc-clock-issues
Oct 28, 2026
Merged

Japabu merged 6 commits into
mainfrom
tsc-clock-issues

Conversation

@Japabu

@Japabu Japabu commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Draft. Issues only, no code. The ToyOS clock records found while measuring the TSC for #1079, filed apart from it at the coordinator's request.

  • issues/the-system-clock-takes-the-tsc-on-one-reading-and-never-checks-it-again.md: the system clock never checks three things:

    • whether the TSC may keep time: the invariant bit, which only deep idle reads (idle.rs::facts);
    • whether the CPUs agree: an 11 ms bracket at AP start;
    • whether the rate still holds after its one calibration.

    The AMD's 1.7 ms early-boot step in 5 of 30 Linux boots is the case it would miss. Its exit asks for a loud fallback and a runtime bound that cannot repeat Linux's own miscalibration.

  • issues/the-kernels-50-ms-tsc-calibration-misses-the-tscs-rate.md: that calibration reads 41 to 66 ppm low on the AMD, against 1796.6248 MHz measured over minutes against the HPET and the ACPI PM timer, so the clock gains about 5 s a day. The HPET bracketing is labelled as the likely cause. On the T14 it is −285 and +59 ppm off a CPUID-stated rate that measures right within 0.1 ppm.

  • issues/ap-tsc-trail-is-assumed-and-never-checked.md (existing) gains the AMD's warp measurement: every pair's worst step positive, bounding each CPU's offset from CPU 0 by 342 cycles (about 190 ns). It also gains the 11 ms bracket smp now logs, and a citation to the clock record.

Evidence: #1079's evidence branch pr-1079, tsc/ in r3 and r4.

Checks: markdown only, so none ran. The files follow issues/README.md's frontmatter, and each has an owner and an exit.

🤖 Generated with Claude Code

…ng misses by tens of ppm

Found while measuring the AMD bench's TSC for PR #1079 (2026-10-28,
tsc-drift under its Linux: 1796.6248 MHz over 10 minutes against the HPET
and the ACPI PM timer, every phase alike, invariant bit set).

- The system clock reads CPUID 80000007H:EDX[8] nowhere (deep idle reads
  it, idle.rs::facts). It checks the CPUs' TSCs against an 11 ms bracket
  at start only, and checks the rate against no reference after its one
  50 ms calibration.
- That calibration reads 40 to 66 ppm low on the AMD bench in every boot
  log of the session, a bias and not noise. The T14's readings sit in two
  clusters, 285 ppm under and 59 ppm over its CPUID-stated rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Oct 28, 2026
… a labelled experiment

The planning architect's guidance (2026-10-28) is that a trustworthy
default Linux is the baseline, and a hardware-policy setting is a
controlled experiment beside it, never its replacement. The README now
says so. The frequency request ToyOS makes (EPP 0x80 on the T14,
performance on the AMD) is the experiment. The AMD's Linux on the HPET is
its baseline. A Linux forced onto the TSC has its stability checks
switched off, so it is no baseline. Four runs a side is where a row can
first resolve, not a precision guarantee.

The AMD clocksource record keeps the bench's default. Ten minutes of a
steady TSC rate (1796.6248 MHz against the HPET and the ACPI PM timer)
does not explain the boot-time watchdog's 1.7 ms over 502 ms, so that
cause is the record's exit.

The invariant-TSC finding leaves this PR for #1082, filed accurately:
deep idle already reads the bit, and the gap is the system clock's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ords

T14: the TSC counts 2419.2002 MHz against the HPET over 5 minutes, CPUID
15H's rate within 0.1 ppm, so the kernel's -285/+59 ppm readings are its
own. AMD: in 5 of 30 boots' logs, a 1.7 ms step early in boot that no
runtime probe reproduces, where ToyOS's one calibration also runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Japabu added a commit that referenced this pull request Oct 28, 2026
The 2026-10-28 validation run at 2eb28ee, one ToyOS boot and two
baseline Linux runs per bench, passed its row on both. It showed:

- The AMD's baseline Linux boosts at 4,022 to 4,126 MHz, dithering by some
  2% within any 50 ms, so the 1% plateau never formed and 84 rows read
  unsettled, at 0.5 s twice each. The governor steps the settle exists to
  catch are a quarter of the clock and more, so it now holds within 5%.
- The format-boot refusal moves to the DATA follow-up. Neither bench's
  p4 is designated (the AMD's stamp was cleared before any ToyOS boot
  read it), so nothing reaches it until ToyOS designates DATA.
- Why the AMD's Linux keeps time on the HPET. Over 30 boots' kernel logs
  its watchdog fired in 18. In 13, Linux's refined TSC calibration read
  1799.54-1799.61 MHz, 1,630 ppm high. In 5, with a right calibration, a
  1.7 ms step early in boot was seen, cause unknown. At runtime
  (tsc-probe, 15 min, 16 CPUs, idle gaps to 1 s) no CPU read behind
  another and no step passed 1.2 us. The README records it, and
  issues/the-amd-benchs-linux-reads-its-clock-through-a-syscall.md
  closes: the bench keeps its default, and the unexplained step is
  ToyOS's clock's concern (#1082).
- The T14's default Linux requests 0x80002a04 (IA32_HWP_REQUEST, every
  CPU): EPP 0x80 and highest 0x2a, as ToyOS asks, and lowest 4 against
  ToyOS's 1. The README says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Review: tsc-clock-issues @ 44813af08 (issues only)

Assessment. Two records with a sensible split:

  • how accurate the one calibration is;
  • the checks the system clock never makes.

Both are kind: defect and status: open, with opened, an Exit and an Owner, as issues/README.md requires. The changes are Markdown only, so this is a consistency review and needs no build. No sentence claims a cause the data lacks: the AMD step is "unknown", the T14 clusters' revisions are labelled an inference, and the 3% figure is conditional.

Checked against r3/r4 tsc/ (= scratch tsc/runs, tsc/runs2), toyos-calibrations.txt, the metal3 kernel logs and main's code.

  • AMD:
    • 1796.6248 MHz, against the HPET over 601 s and the ACPI PM timer over 300.5 s;
    • the phases at 1796.6247–.6249;
    • ToyOS's 273 calibration readings: 1796.506–1796.551 but one at 1796.597, so 41–66 ppm low and 15 ppm low;
    • one step of the stored period is 3,227 Hz, 1.8 ppm;
    • the SMP bracket is 19,762,668 cycles, 11.0 ms;
    • CPUID 8000_0007 EDX 0x6799, so the invariant bit is set, and it is read only at idle.rs:33.
  • T14: 2419.2002 MHz against 2419.2000 (0.08 ppm), worst residual 0.2 µs. The clusters are at 2418508360 Hz (−285 ppm) and 2419345083 Hz (+59 ppm).
  • The AMD steps: 5 boots with a skew of +1.698 to +1.722 ms, each with a refined calibration of 1796.616–1796.624 MHz.
  • The runtime probes: the warp and jump runs show what the issues say.

Required

  1. the-kernels-50-ms-tsc-calibration-…md, paragraph 1: the consequence has the wrong sign.

    • The issue says: "At 58 ppm a ToyOS clock on that machine loses about 5 s a day. Every interval it times reads that much short."
    • hpet.rs:66-68 sets the period to HPET femtoseconds over TSC cycles. A rate read 58 ppm low is a period 58 ppm long.
    • clock::ticks_to_nanos multiplies ticks by that period, so every interval reads 58 ppm long, and the clock gains about 5 s a day.
    • On the T14, the −285 ppm cluster would gain about 25 s a day, and the +59 cluster lose about 5.
  2. Slug and title: "misses the rate by tens of ppm" is false for the T14.

    • issues/README.md treats a slug as a claim.
    • The T14's largest cluster is 285 ppm off, and other readings are 404–440 and 759–766 ppm off.
    • Rename it, for example the-kernels-50-ms-tsc-calibration-misses-the-tscs-rate, and move the citation in the other issue in the same change.
  3. Figures in the same issue:

    • "in 344 and 250 log files": the evidence counts 345 at 2418508360 Hz.
    • "Over 10 minutes … against the same HPET and against the ACPI PM timer": the PM run was 300.5 s.
    • "read 1796.506 to 1796.551 MHz" contradicts "the highest, once, is 1796.597". Say "all but one".
  4. the-system-clock-takes-the-tsc-…md: "2 to 3 s into boot" for all five steps is not in the data.

    • Only boot −2's dmesg (r2 tsc/amd-dmesg-clocksource.txt, at 2.62 s) has a time since boot.
    • boots.txt has wall-clock times only, to the second.
    • Say "2.6 s into boot in the one boot whose dmesg was kept", or derive the other four.

Suggestions

  • "Where it comes from … is unread": a labelled inference is available from hpet.rs:32-64.
    • The HPET is read before the first TSC read and after the last, so the HPET interval brackets the TSC interval by up to two uncached MMIO reads.
    • About 1.5 µs over 50 ms is about 30 ppm low: the observed sign and order of size.
    • The period's truncation biases the other way, by under 1.8 ppm.
    • This cannot explain the T14's −285 ppm (14 µs) or +59 ppm, which need another cause. One labelled sentence would help whoever fixes it.
  • The clock issue's Exit allows "falling back to a counter that holds". Require the fallback to be loud.
    • Derive the runtime bound so that ToyOS's check cannot repeat what this investigation saw of Linux's watchdog: 12 of its 17 fires came from its own miscalibration.
    • Where the vendor documents a cheaper detector, prefer it, for example IA32_TSC_ADJUST for firmware writes to the TSC on Intel (SDM Vol. 3).

SEND BACK

Japabu and others added 2 commits October 28, 2026 11:13
…es are the evidence's

- A rate read low is a period kept long, so the AMD's clock gains about
  5 s a day, and the T14's -285 ppm cluster about 25.
- the-kernels-50-ms-tsc-calibration-misses-the-tscs-rate: the T14's
  readings are hundreds of ppm off, not tens. Its citation moves with it.
- 345 log files; the ACPI PM run is 5 minutes; all but one calibration in
  the range given; 41 to 66 ppm; "2.6 s into boot in the one boot whose
  dmesg was kept".
- Suggestions taken: the HPET-bracket inference, labelled, and an exit
  whose fallback is loud and whose runtime bound cannot repeat Linux's own
  miscalibration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ecord cites it

issues/ap-tsc-trail-is-assumed-and-never-checked.md owns cross-CPU
agreement. It gains the AMD's tsc-probe warp result (no CPU behind
another's last reading, within about 200 ns) and the 11 ms bracket smp
now logs. The system-clock record cites it rather than restating it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

r1 response, head de2abfd78

  1. Fixed. A rate read low is a period kept long, so the clock gains about 5 s a day at 58 ppm. On the T14 the −285 ppm cluster gains about 25 s a day and the +59 cluster loses about 5.
  2. Fixed. The issue is renamed to the-kernels-50-ms-tsc-calibration-misses-the-tscs-rate.md, and its citation in the clock record moved with it. No reference to the old slug remains in the tree.
  3. Fixed. 345 log files; the ACPI PM run is 5 minutes; "all but one … 41 to 66 ppm low", with the other at 15 ppm. The T14's outliers at 404–440 and 759–766 ppm are named.
  4. Fixed. "2.6 s into boot in the one boot whose dmesg was kept".

Suggestions taken:

  • the HPET-bracket inference, labelled, noting that it explains neither T14 cluster;
  • an exit with a loud fallback, a runtime bound derived so it cannot repeat Linux's own miscalibration (12 of its 17 fires), and a vendor detector preferred where documented (IA32_TSC_ADJUST).

Also: issues/ap-tsc-trail-is-assumed-and-never-checked.md already owned cross-CPU agreement. It now holds the AMD's warp result and the bracket smp logs, and the clock record cites it rather than restating it.

@Japabu

Japabu commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Review r2: tsc-clock-issues @ de2abfd78

Most of r1 is resolved; one new figure is wrong (required 1).

  • R1, resolved. The sign is fixed: "a period kept long … gains about 5 s a day". The T14's clusters are given as gaining about 25 s a day and losing about 5.
  • R2, resolved. The file is renamed to the-kernels-50-ms-tsc-calibration-misses-the-tscs-rate.md, and its citation in the clock record moved. git grep misses-the-rate-by-tens de2abfd78 finds nothing. The PR title still says "tens of ppm"; the title is the orchestrator's to change.
  • R3, resolved. 345 log files, the PM run over 5 minutes, "all but one … 41 to 66 ppm" with 15 ppm for the other, and the 404–440 and 759–766 ppm readings named.
  • R4, resolved. The text now says "2.6 s into boot in the one boot whose dmesg was kept".
  • Suggestions, taken. The HPET-bracket inference is labelled, and it says that it explains neither T14 cluster. The Exit now has a loud fallback, a derived bound, and IA32_TSC_ADJUST where the vendor documents it.

Folding the cross-core data into issues/ap-tsc-trail-is-assumed-and-never-checked.md is the right home: the clock record now cites it rather than restating it.

Required

  1. ap-tsc-trail-is-assumed-and-never-checked.md, new paragraph: the bound's derivation is wrong.
    • The paragraph says: "the smallest step was 54 cycles, so that CPU's TSCs agree to within one turn, some 200 ns".
    • In r4 tsc/amd/tsc-warp.txt a turn is 180 to 972 cycles, up to about 540 ns. So "within one turn" is not 200 ns, and the smallest step, 54 cycles, bounds only CPU 0 against CPU 1.
    • A non-negative step in each direction bounds a pair's offset by the larger of its two worst steps. Across the 15 pairs those reach 342 cycles (cpu0↔cpu10), about 190 ns.
    • Say, for example: "every pair's worst step was positive, 54 to 342 cycles, which bounds each CPU's offset from CPU 0 by 342 cycles, some 190 ns."

Suggestion

  • The clock record's line "in the one boot whose dmesg was kept. Every time it was the same size, and it came with a calibration that" is left unwrapped.

SEND BACK

A non-negative step each way bounds a pair's offset by the larger of its
two worst steps; across the 15 pairs that is 342 cycles (cpu0<->cpu10),
about 190 ns. One line rewrapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu Japabu changed the title issues: the system clock takes the TSC on one reading, and that reading misses by tens of ppm issues: the system clock takes the TSC on one reading, and that reading misses its rate Oct 28, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Japabu

Japabu commented Oct 28, 2026

Copy link
Copy Markdown
Collaborator Author

Orchestrator: the follow-up's one required fix is verified at da056ab. The cross-core bound is now the worst step across all pairs, 342 cycles (about 190 ns), and the title and wrap are corrected. Landing.

@Japabu
Japabu marked this pull request as ready for review October 28, 2026 10:36
@Japabu
Japabu added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 5870531 Oct 28, 2026
10 checks passed
@Japabu
Japabu deleted the tsc-clock-issues branch October 28, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant