Skip to content

perf: diagnose the four open benchmark findings (lu_solve, the new matmath() sites, gaussian kernel) #169

Description

@kalwalt

Summary

The #86 benchmark suite produced several findings that survived investigation of the others. Four remain open on merged dev, measured over four full-suite runs on an idle machine (see the Current status table in bench/README.md).

This issue tracks diagnosing them. Unlike #86 — which is measurement infrastructure and explicitly rules out micro-optimising — work here may touch src/, with the parity suite green at every step.

The open findings

case 4 runs notes
linalg.lu_solve (6x6) 1.43 / 1.49 / 1.46 / 1.47 The tightest signal in the suite — ±0.03, tighter than YAPE ever was. Never profiled.
homography2d.check_subset 1.35 / 1.27 / 1.28 / 1.63 One of three new matmath() sites
affine2d.run — 3 points 1.34 / 1.31 / 1.30 / 1.41 Same cause; worst when the real work is smallest
math.get_gaussian_kernel — size 7 1.23 / 1.21 / 1.16 / 1.15 4/4 jsfeat, at or just above the floor

lu_solve — the one to start with

#159 predicted in advance that its fix could not explain this one, because lu_solve constructs no internal matrix_t. That prediction held: every other linalg case dropped below the floor after #159, and lu_solve did not move.

One misattribution has already been ruled out: lu_solve reads JSFEAT_CONSTANTS.EPSILON inside a loop, which looks like the bug fixed in #165 — but original jsfeat reads jsfeat.EPSILON, also a property load. Both sides pay it, so it explains nothing here.

The three new matmath() sites

motion_model.ts lines 212 (affine2d.run), 352 (homography2d.run) and 541 (homography2d.check_subset) each construct a matmath per call; original jsfeat calls module-scope functions instead. #159 made each construction much cheaper — affine2d.run at 3 points fell from ~3.5x to ~1.35x — but did not remove them.

The shape of the remaining gap still fits a fixed per-call cost: affine2d.run is worst at 3 points, where the real arithmetic is smallest, and much better at 40.

get_gaussian_kernel size 7

Newly consistent rather than newly appeared: earlier series had it flipping sign, the merged-dev series has it 4/4 at 1.15–1.23. Lowest priority of the four, and possibly still floor noise — confirm it is real before investigating.

Also worth recording: an unexplained improvement

yape06 measured 1.14–1.47 on #166's branch, where that PR correctly reported the alias had not helped it. On merged dev it measures ~1.04 across four runs. The improvement is real but not attributable to any single one of #159/#165/#166. Not a defect, but if someone later needs to explain why yape06 is fast now, this is the open question.

Approach that worked before

#159 and #165 both came from the same loop: profile the workload the bench exercises, aggregate self-time by call frame, then isolate the suspected cost with a throwaway probe that changes exactly one thing. #165's probe compared five variants of one function; #166's replicated a whole function body under four structural shapes, asserting identical outputs before timing.

Two lessons from those, worth repeating:

  • Verify the hypothesis before fixing. perf(yape): alias imported helpers to module-scope consts #166's documented theory (cross-module imports blocking inlining) was disproven by the profile.
  • A control case is worth more than a big number. In motion_model, error() — the only method that allocated nothing — staying at noise while everything else moved was the strongest evidence in that investigation.

Acceptance criteria

  • Each of the four either has a documented cause, or a recorded "profiled, cause not found".
  • Any src/ change keeps npm test green and is measured before/after on an idle machine.
  • bench/README.md's Current status table updated — it is the single place that tracks this.
  • Disproven hypotheses recorded as disproven, not silently dropped.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions