Skip to content

benchmark: add --analyze option to benchmark/scatter.js - #65594

Open
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/improve-benchmark-scatter
Open

benchmark: add --analyze option to benchmark/scatter.js#65594
jasnell wants to merge 1 commit into
nodejs:mainfrom
jasnell:jasnell/improve-benchmark-scatter

Conversation

@jasnell

@jasnell jasnell commented Aug 27, 2026

Copy link
Copy Markdown
Member

Similar to compare.js, provide an R-free --analyze to make scatter.js more self-contained.

Allows scatter.js to produce data without requiring the R script to be used.

$ node benchmark/scatter.js --analyze --xaxis chunkLen --category encoding benchmark/string_decoder/string-decoder.js
chunkLen  encoding   samples          rate  confidence.interval        median    median.interval
      16  'ascii'          6   2,248,111.0    61,570.3 (±2.74%)   2,241,996.7   [-3.75%, +4.47%]
      16  'utf16le'        6   2,004,145.9    33,363.7 (±1.66%)   1,990,015.9   [-1.03%, +3.44%]
      16  'utf8'           6   1,156,990.9    84,288.2 (±7.29%)   1,186,284.7  [-16.23%, +1.15%]  (!)
     256  'ascii'          6  11,271,912.4   398,786.9 (±3.54%)  10,991,001.5   [-0.47%, +7.61%]
     256  'utf16le'        6  10,243,336.8   792,495.2 (±7.74%)   9,908,838.3  [-4.73%, +17.25%]  (!)
     256  'utf8'           6   8,247,482.7   316,510.5 (±3.84%)   8,282,009.5   [-5.80%, +3.19%]
    1024  'ascii'          6  11,130,650.9   438,019.6 (±3.94%)  10,838,937.5   [-1.34%, +7.10%]
    1024  'utf16le'        6   9,786,274.4   186,207.1 (±1.90%)   9,690,675.1   [-0.77%, +4.20%]
    1024  'utf8'           6   8,479,690.1   405,762.8 (±4.79%)   8,525,516.7   [-9.01%, +4.06%]  (!)

Rate in operations/second; longer is faster. │ marks the mean and the
shaded band (░) is its 95% confidence interval, so bars whose bands
overlap are not clearly different.

                                  0                           11,670,699.3
                                  +--------------------------------------+
chunkLen=16 encoding='ascii'      ███████│                                   2,248,111.0
chunkLen=16 encoding='utf16le'    ██████│                                    2,004,145.9
chunkLen=16 encoding='utf8'       ███│                                       1,156,990.9

chunkLen=256 encoding='ascii'     █████████████████████████████████████░│░  11,271,912.4
chunkLen=256 encoding='utf16le'   ████████████████████████████████░░░│░░    10,243,336.8
chunkLen=256 encoding='utf8'      ███████████████████████████░│              8,247,482.7

chunkLen=1024 encoding='ascii'    █████████████████████████████████████░│░  11,130,650.9
chunkLen=1024 encoding='utf16le'  █████████████████████████████████│         9,786,274.4
chunkLen=1024 encoding='utf8'     ████████████████████████████░│             8,479,690.1

Change between consecutive chunkLen values (Mann-Whitney U, Cliff's delta):

  encoding='ascii'
    16 -> 256    +401.39%  16.0x  exponent=+0.58  p=0.0039  delta=+1.000 (large)
    256 -> 1024    -1.25%   4.0x  exponent=-0.01  p=0.3367  delta=-0.333 (medium)

  encoding='utf16le'
    16 -> 256    +411.11%  16.0x  exponent=+0.59  p=0.0039  delta=+1.000 (large)
    256 -> 1024    -4.46%   4.0x  exponent=-0.03  p=0.1495  delta=-0.500 (large)

  encoding='utf8'
    16 -> 256    +612.84%  16.0x  exponent=+0.71  p=0.0039  delta=+1.000 (large)
    256 -> 1024    +2.82%   4.0x  exponent=+0.02  p=0.2002  delta=+0.444 (medium)

Similar to compare.js, provide an R-free --analyze to
make scatter.js more self-contained.

Signed-off-by: James M Snell <jasnell@gmail.com>
Assisted-by: Opencode
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/performance
  • @nodejs/tsc

@jasnell
jasnell requested review from RafaelGSS and mcollina August 27, 2026 16:21
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

@mcollina mcollina left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RSLGTM

@jasnell jasnell added benchmark Issues and PRs related to the benchmark subsystem. tools Issues and PRs related to the tools directory. author ready PRs that have at least one approval, no outstanding review comments, and a CI started. performance Issues and PRs related to the performance of Node.js. and removed tools Issues and PRs related to the tools directory. labels Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no outstanding review comments, and a CI started. benchmark Issues and PRs related to the benchmark subsystem. performance Issues and PRs related to the performance of Node.js.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants