Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,11 +68,11 @@
- Before a link session starts, the game's save is written and a copy of it
is kept as `<rom>.sav.link1`; the two copies before it move to `.link2`
and `.link3`. If the save cannot be written or copied, the session does
not start and the Link menu says why.
- Link > Restore pre-link save puts one of those copies back, chosen by its
date and time. The current save is kept as `<rom>.sav.bak`, and the game
restarts on the restored save. The item is not available during a
session.
not start and the Link Cable menu says why.
- Link Cable > Restore pre-link save puts one of those copies back, chosen
by its date and time. The current save is kept as `<rom>.sav.bak`, and
the game restarts on the restored save. The item is not available during
a session.
- The runtime log records how every `.sav` was read (its size, the save
chip chosen and whether they disagree), how long every save write took,
whether it worked and how many tries it needed, how long the save copy
Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,10 +84,10 @@ The optional desktop launcher has its own
`psp-build`, `package` and `artifact-smoke` use the release profile, the EBOOT
players get, unless given `PROFILE=diagnostic`. The diagnostic profile adds
the features that `source/src/build_profile.h` switches on (today the input
trace, Diagnostics > Pin a test toast, the `diag-autorun.txt` file that
`source/src/autorun_config.h` describes, the simulated network that its
`net_profile=` key puts on the ad-hoc link, and the rendering-off and
audio-off arms of Diagnostics > Unbound benchmark (docs/PERFORMANCE.md);
trace, Diagnostics > Pin a test toast, the Unbound benchmark rows with
rendering off and with audio off (docs/PERFORMANCE.md), the
`diag-autorun.txt` file that `source/src/autorun_config.h` describes, and the
simulated network that its `net_profile=` key puts on the ad-hoc link;
`DIAGNOSTIC_CORE_PROFILE=1` adds
the New core's profile and `DIAGNOSTIC_HW_PROBE=1` the memory system probe),
is titled
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -252,9 +252,9 @@ in `source/`: the corresponding source ZIP,

### Performance comparison

On PSP, use **Diagnostics > Start measurement**, play a repeatable section,
stop measurement and save the report. Collect three reports per build, then
run:
On PSP, use **Diagnostics > Start recording**, play a repeatable section,
then **Stop recording** and **Save report**. Collect three reports per
build, then run:

```sh
python3 scripts/compare-benchmarks.py \
Expand Down
7 changes: 4 additions & 3 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,10 @@ benchmarks/<model>/<game-code>-r<revision>/<version>/<run>.txt
- `<game-code>-r<revision>` is the report's `game-code` and `rom-revision`,
for example `BPRE-r0`.
- `<version>` is the report's `version`, for example `1.4.3`.
- `<run>.txt` is a report saved from Diagnostics > Start measurement,
unchanged. Every run in one directory must share the settings
`scripts/compare-benchmarks.py` checks (`MATCH_FIELDS`).
- `<run>.txt` is a report saved with Diagnostics > Start recording, Stop
recording and Save report, unchanged. Every run in one directory must
share the settings `scripts/compare-benchmarks.py` checks
(`MATCH_FIELDS`).

Rules:

Expand Down
20 changes: 12 additions & 8 deletions docs/PERFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@ How GBA Rush PSP speed is measured, and the baselines measured so far.

## Comparing two builds

On the PSP, use Diagnostics > Start measurement, play a repeatable section,
stop and save the report. Capture at least three reports per build with the
same ROM, save or save state, renderer, clock and settings, then compare:
On the PSP, use Diagnostics > Start recording, play a repeatable section,
then Stop recording and Save report. Capture at least three reports per
build with the same ROM, save or save state, renderer, clock and settings,
then compare:

```sh
python3 scripts/compare-benchmarks.py \
Expand Down Expand Up @@ -62,8 +63,9 @@ for the time per frame:

- `phase-main-update-gba-us`: update_gba's own time (scheduler, input,
presenting the frame, DMA), without the two phases below and without the
waits the work time leaves out (the VBlank waits and the GE wait). During
a lockstep link it also holds the link waits.
waits the work time leaves out (the VBlank waits, the sound sync's wait
for the audio thread and the GE wait). During a lockstep link it also
holds the link waits.
- `phase-main-scanline-us`: update_scanline.
- `phase-main-gbc-sound-us`: the PSG channels (update_gbc_sound). Direct
sound, hundreds of calls a frame, is not timed.
Expand Down Expand Up @@ -117,9 +119,11 @@ a capture is not counted.
Diagnostics > Unbound benchmark loads the current save state slot as the
slot menu does (a numbered slot backs up the progress it replaces to the
auto slot first, and the load asks before replacing the game), then runs
3600 frames as fast as the PSP can: every frame is drawn, nothing waits
for the VBlank, VSync is off, and the sound is mixed but dropped, so the
emulation never waits for audio. The game gets no input. The capture then
3600 frames as fast as the PSP can: every frame is drawn (with Exact on
the Media Engine, a frame the Media Engine has not finished in time shows
the previous one again and counts as late), nothing waits for the VBlank,
VSync is off, and the sound is mixed but dropped, so the emulation never
waits for audio. The game gets no input. The capture then
stops by itself and its report is saved to `benchmarks/`; the Diagnostics
status row shows its name, and the game returns to its normal speed. A
link refuses the benchmark. Opening the menu, a load or save state hotkey,
Expand Down
2 changes: 1 addition & 1 deletion docs/hardware-tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -509,7 +509,7 @@ Each test has four fields:
- Pass: `scripts/compare-benchmarks.py` reports no failure for any of the
5 games on either model.
- Steps: for each of 5 games, take 3 measurements with Diagnostics > Start
measurement on the previous release and 3 on the candidate, alternating
recording on the previous release and 3 on the candidate, alternating
the two builds, with the same save or state, renderer, clocks and
settings. Compare each game's two sets of reports with
`scripts/compare-benchmarks.py`.
Expand Down
16 changes: 8 additions & 8 deletions docs/wip/perf-method.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@
- **Branch:** `features/state-of-the-art-development/wip/perf-method`, staccato da `features/state-of-the-art-development/main`.
- **Ticket:** PRF-1, PRF-2, PRF-3, PRF-8.6, PRF-10.1 (19 storie, circa 30 gg).
- **File principali:** `scripts/compare-benchmarks.py`, `source/src/perf_*.c`, `tools/`.
- **Dipende da:** `wip/release-and-ci` (ENG-1.6, ENG-1.2, ENG-1.3, ENG-2.2), in main da 8ace097; restano `wip/platform` (PRF-4.1) e `wip/open-bios` (CMP-1.4) per PRF-1.5, `wip/replay-oracle` (ENG-3.2) per PRF-1.7.
- **Dipende da:** `wip/release-and-ci` (ENG-1.6, ENG-1.2, ENG-1.3, ENG-2.2), in main da 8ace097; `wip/platform` (PRF-4.1) in main da cf5ef4f; resta `wip/open-bios` (CMP-1.4) per PRF-1.5; per PRF-1.7 ENG-3.2 (`wip/replay-oracle`) è in main da 2ff83ab ma aspetta la prova su PSP.
- **Nota:** PRF-8.6 e PRF-10.1 sono misure che servono a entrambi i core.
- **Regole:** quelle di «Branch di lavoro dell'epica» in `docs/BACKLOG.md`. Il testo completo di ogni ticket (cosa, file, test, accettazione, licenza) è in `docs/BACKLOG.md` sotto il suo ID. Si spunta un ticket quando la sua accettazione è verificata; questo file si cancella nella PR che chiude il branch.
- **Legenda:** «subito» = nessuna dipendenza, si può iniziare oggi; «aspetta» = dipendenze in altri branch, si parte quando sono in main.

## 1.5

- [x] **PRF-2.1** — Tempo di lavoro per frame, frame in ritardo e oltre budget (report v7) · S, 2 gg · dipende da: ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_metrics e test_perf_report in make check e make test-sanitize, test_compare_benchmarks.py e check-performance.sh in make check, `psp_video.c` compilato nei due profili PSP_
- [x] **PRF-2.1** — Tempo di lavoro per frame, frame in ritardo e oltre budget (report v7) · S, 2 gg · dipende da: ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_metrics e test_perf_report in make check e make test-sanitize, test_compare_benchmarks.py e check-performance.sh in make check, `psp_video.c` compilato nei due profili PSP; revisione del 2026-11-16 (25cd5a1): il sonno del sound sync in attesa del thread audio (frameskip manuale o spento) non conta più come lavoro, con una lettura di `sceKernelGetSystemTimeWide` in più in `sound.c`, solo a cattura attiva e nei frame che aspettano il thread audio: il proprietario conferma che la «Cosa» («usa solo tempi già misurati») lo ammette; la build PSP di `sound.c` cambiato si vede nella CI della PR di revisione_
- [ ] **PRF-8.6** — Suite SMC e di riferimento misurata sulla 1.4.3 · M, 3 gg · dipende da: — · PSP: sì · **subito**

## 1.6
Expand All @@ -19,19 +19,19 @@
- [x] **PRF-1.2** — Gate sugli estremi: p99, massimo robusto, work p95, frame oltre 1,5× budget al minuto · S, 1 gg · dipende da: PRF-1.1, PRF-2.1, ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_compare_benchmarks.py in make check_
- [x] **PRF-1.3** — Ordine alternato verificato, ora di cattura e affermazione dichiarata prima · S, 1 gg · dipende da: PRF-1.1, PRF-2.1, ENG-1.6 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_report in make check e make test-sanitize, test_compare_benchmarks.py in make check, `main.c` compilato nei due profili PSP_
- [x] **PRF-1.4** — Archivio delle baseline e tabella per modello generata · M, 2 gg · dipende da: PRF-1.1 · PSP: no
- [ ] **PRF-1.5** — Sessione hardware: baseline 1.6 per PSP-1000 e uno slim · M, 2,5 gg · dipende da: PRF-1.3, PRF-1.4, PRF-2.2, PRF-2.3, PRF-3.1, PRF-4.1, CMP-1.4, ENG-1.6 · PSP: sì · aspetta: PRF-4.1 (`wip/platform`), CMP-1.4 (`wip/open-bios`) · _anche PRF-2.3, ferma per la decisione (m)_
- [ ] **PRF-1.5** — Sessione hardware: baseline 1.6 per PSP-1000 e uno slim · M, 2,5 gg · dipende da: PRF-1.3, PRF-1.4, PRF-2.2, PRF-2.3, PRF-3.1, PRF-4.1, CMP-1.4, ENG-1.6 · PSP: sì · aspetta: CMP-1.4 (`wip/open-bios`); PRF-4.1 è in main (cf5ef4f) · _anche PRF-2.3, ferma per la decisione (m)_
- [ ] **PRF-1.6** — Controllo delle prestazioni nel gate di release · S, 1 gg · dipende da: PRF-1.4, F-139, F-118 · PSP: no · _in parte: CI verde su 8d7bd38 (run 43664592279); i report si legano all'rc col `build-id` del manifest, non con lo sha256 di EBOOT.PBP (i report non lo scrivono), e il controllo vale solo per le release che richiedono `perf-baseline-5`: il proprietario decide se basta_
- [ ] **PRF-2.2** — Timer di fase su frame campionati: update_gba, update_scanline, PSG audio · M, 2,5 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: no · _in parte: CI verde su 8d7bd38 (run 43664592279); a cattura spenta, oltre al test di flag per scanline, update_gba ne fa due per chiamata (ingresso e uscita) e il PSG uno per aggiornamento: il proprietario decide se l'accettazione («un solo test di flag per scanline») lo ammette_
- [ ] **PRF-2.3** — Contatori di dispatch per regione e di SMC · S, 1 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: no · _ferma per la decisione (m): i contatori stanno nel core Classic (`cpu.c`); il proprietario sceglie se riportarla al core New (`arm_mips`) o congelarla fino a SOA-10.1_
- [x] **PRF-2.4** — Prelazioni del thread di emulazione, divise tra lavoro e attesa · S, 1 gg · dipende da: PRF-2.1 · PSP: no · _CI verde su 8d7bd38 (run 43664592279): test_perf_metrics e test_perf_report in make check e make test-sanitize, `main.c` compilato nei due profili PSP_
- [ ] **PRF-2.5** — Sonda del costo del lookup (solo build diagnostica) · S, 1 gg · dipende da: PRF-2.3, F-137 · PSP: sì · _ferma per la decisione (m): cronometra il lookup del core Classic per il gate di PRF-7, congelato; il proprietario sceglie se congelarla fino a SOA-10.1 o riportarla al dispatch del core New_
- [ ] **PRF-2.6** — Validazione su hardware della strumentazione · S, 1,5 gg · dipende da: PRF-2.2, PRF-2.4, PRF-3.2 · PSP: sì
- [ ] **PRF-3.1** — Benchmark senza limite da uno slot di stato · M, 2 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: sì · _codice pronto (e108fdd), manca la prova su PSP_
- [ ] **PRF-3.2** — Bracci render-off e audio-off (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _codice pronto (1a411a1), manca la prova su PSP_
- [ ] **PRF-3.3** — Replay di traccia nel bench (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _codice pronto (c4098c0), manca la prova su PSP; lo stesso stato si verifica con l'hash di F-120 (ENG-3, `wip/replay-oracle`), non ancora in main_
- [ ] **PRF-3.1** — Benchmark senza limite da uno slot di stato · M, 2 gg · dipende da: PRF-2.1, ENG-1.6 · PSP: sì · _codice pronto (e108fdd), manca la prova su PSP; nella prova anche un bench con Exact sul Media Engine, su un titolo leggero e su uno pesante: il bench non aspetta mai, e se il Media Engine resta indietro 300 frame di fila Exact passa al processore principale fino al riavvio dell'EBOOT, mentre il report scrive solo `renderer`; si vede avviando l'EBOOT, aprendo il gioco e lanciando subito il bench (che carica lo slot da sé), poi uscendo dall'EBOOT subito dopo il bench, nel runtime log (`python3 tools/decode_runtime_log.py`): `exact-renderer=cpu-too-slow` invece di `media-engine`, e i frame in ritardo nel secondo valore di `me-frames/frames-late/me-busy-us-per-frame` (gruppi 11 e 9 di LINK_COUNTERS, scritti all'uscita e contati dall'avvio dell'EBOOT: così il gruppo 9 tiene quasi solo i frame del bench)_
- [ ] **PRF-3.2** — Bracci render-off e audio-off (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _in parte: i bracci sono pronti (1a411a1), ma in main il bench non calcola nessun hash dello stato (`input_diagnostic_trace_begin_bench` in `input.c` scarta quelli del replay), quindi nessuna build controlla ancora lo stato a fine bench; il codice degli hash di F-120 (ENG-3.3, 91bc1ef) è in main da 2ff83ab, ma ENG-3.3 aspetta la prova su PSP; l'hash a fine bench (decisione del 2026-11-16: uno all'ultimo frame del bench con traccia, senza la regione audio, nel perf report come `bench-state-hash`) è scritto in `wip/replay-oracle` (bcf3657) e arriva in main con la sua PR di revisione; poi i bracci si confrontano con la stessa traccia e lo stesso slot, audio-off compreso perché l'hash lascia fuori i campioni, e si fa la prova su PSP_
- [ ] **PRF-3.3** — Replay di traccia nel bench (build diagnostica) · S, 1 gg · dipende da: PRF-3.1, F-137 · PSP: sì · _in parte: il replay della traccia è pronto (c4098c0), ma il bench non calcola nessun hash dello stato, quindi «lo stesso stato (hash F-120)» non si controlla ancora in main; il codice degli hash di F-120 (ENG-3.3, 91bc1ef) è in main da 2ff83ab, ma ENG-3.3 aspetta la prova su PSP; l'hash a fine bench (decisione del 2026-11-16: uno all'ultimo frame del bench con traccia, nel perf report come `bench-state-hash`) è scritto in `wip/replay-oracle` (bcf3657) e arriva in main con la sua PR di revisione; poi la prova su PSP_
- [ ] **PRF-3.4** — compare-benchmarks per i bench e rumore misurato su hardware · S, 1 gg · dipende da: PRF-3.1, PRF-1.1 · PSP: sì · _codice pronto (8ebfcb6), manca la prova su PSP: i 3+3 run sul PSP-1000 e il rumore in `docs/PERFORMANCE.md`_
- [ ] **PRF-10.1** — Esperimento di padding (il gate) · S, 1,5 gg · dipende da: PRF-3.1, PRF-1.1, F-137 · PSP: sì · _codice pronto (3018116), manca la prova su PSP: 12 run per modello e la decisione in `docs/PERFORMANCE.md`; la build PSP con `PADDING_PROBE_BYTES` non è mai stata fatta, nemmeno in CI_
- [ ] **PRF-10.1** — Esperimento di padding (il gate) · S, 1,5 gg · dipende da: PRF-3.1, PRF-1.1, F-137 · PSP: sì · _codice pronto (3018116), manca la prova su PSP: 12 run per modello e la decisione in `docs/PERFORMANCE.md`; la build PSP con `PADDING_PROBE_BYTES` non è mai stata fatta, nemmeno in CI; il confronto per cartelle salta i `-alloc.txt` delle build diagnostiche da e421190_

## 2.0

- [ ] **PRF-1.7** — Suite pubblica di almeno 40 scene per modello · M, 3 gg · dipende da: PRF-1.4, PRF-1.5, PRF-3.1, ENG-3.2 · PSP: sì · aspetta: ENG-3.2 (`wip/replay-oracle`)
- [ ] **PRF-1.7** — Suite pubblica di almeno 40 scene per modello · M, 3 gg · dipende da: PRF-1.4, PRF-1.5, PRF-3.1, ENG-3.2 · PSP: sì · aspetta: ENG-3.2 (`wip/replay-oracle`), in main da 2ff83ab ma non ancora provata su PSP
10 changes: 7 additions & 3 deletions scripts/compare-benchmarks.py
Original file line number Diff line number Diff line change
Expand Up @@ -656,6 +656,11 @@ def format_result(result):
# The report fields that make a group (title x model) in directory mode.
GROUP_FIELDS = ("psp-model", "game-code", "rom-revision")

# The files beside the reports that are no report: notes, the -core.txt
# profiles of a DIAGNOSTIC_CORE_PROFILE build and the -alloc.txt traces
# every diagnostic build writes.
NOT_REPORTS = (".md", "-core.txt", "-alloc.txt")


def group_name(key):
"""A group's name as the output prints it, such as 0/BPRE-r0."""
Expand All @@ -665,8 +670,7 @@ def group_name(key):
def collect_groups(*directories):
"""The reports anywhere under the directories, keyed by GROUP_FIELDS.

Every file but the .md notes and the -core.txt profiles of a
DIAGNOSTIC_CORE_PROFILE build must be a stopped .txt report, every
Every file but NOT_REPORTS must be a stopped .txt report, every
directory must hold at least one, and the runs of one group must all
come from the same version.
"""
Expand All @@ -675,7 +679,7 @@ def collect_groups(*directories):
for directory in map(pathlib.Path, directories):
count = 0
for path in sorted(directory.rglob("*")):
if not path.is_file() or path.name.endswith((".md", "-core.txt")):
if not path.is_file() or path.name.endswith(NOT_REPORTS):
continue
name = path.as_posix()
if path.suffix != ".txt":
Expand Down
9 changes: 5 additions & 4 deletions scripts/perf-table.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,9 +124,10 @@ def summarize(reports):
def collect(root):
"""The summarized groups of the archive, keyed (model, game, version).

Every file but the .md notes must be a .txt report three directories
down, and a group's reports must match the model, game and version
directories they are in.
Every file but the .md notes and the diagnostic files that
compare-benchmarks skips too (NOT_REPORTS) must be a .txt report
three directories down, and a group's reports must match the model,
game and version directories they are in.
"""
root = pathlib.Path(root)
if not root.is_dir():
Expand All @@ -135,7 +136,7 @@ def collect(root):
for path in root.rglob("*") if path.is_file())
groups = {}
for name, path in files:
if name.endswith(".md"):
if name.endswith(compare_benchmarks.NOT_REPORTS):
continue
parts = name.split("/")
if len(parts) != 4 or not name.endswith(".txt"):
Expand Down
7 changes: 6 additions & 1 deletion source/src/perf_metrics.c
Original file line number Diff line number Diff line change
Expand Up @@ -396,7 +396,8 @@ void perf_metrics_observe_memory_available(PerfMetrics *metrics,
}

/* Adds one VBlank wait to the count and total of its site and to the waits
of the current frame; an unknown site is ignored. */
of the current frame; the audio drain, no VBlank wait, goes only to the
frame's waits, and an unknown site is ignored. */
void perf_metrics_record_wait(PerfMetrics *metrics,
PerfMetricsWaitSite site, uint32_t wait_us)
{
Expand All @@ -420,6 +421,10 @@ void perf_metrics_record_wait(PerfMetrics *metrics,
count = &metrics->vblank_audio_count;
total = &metrics->vblank_audio_total_us;
break;
case PERF_METRICS_WAIT_AUDIO_DRAIN:
metrics->frame_wait_us = perf_metrics_saturating_add(
metrics->frame_wait_us, wait_us);
return;
default:
return;
}
Expand Down
7 changes: 5 additions & 2 deletions source/src/perf_metrics.h
Original file line number Diff line number Diff line change
Expand Up @@ -74,12 +74,15 @@ typedef struct
} PerfMetricsHistogram;

/* Where the emulator thread waited for the PSP VBlank: frame pacing in
synchronize(), before presenting a frame, or in the sound sync. */
synchronize(), before presenting a frame, or in the sound sync. The
last site is no VBlank wait: the sound sync sleeping until the audio
thread takes a block, which only leaves the frame's work time. */
typedef enum
{
PERF_METRICS_WAIT_BACKPRESSURE,
PERF_METRICS_WAIT_PRESENT,
PERF_METRICS_WAIT_AUDIO
PERF_METRICS_WAIT_AUDIO,
PERF_METRICS_WAIT_AUDIO_DRAIN
} PerfMetricsWaitSite;

/* What a phase timer measured. UPDATE_GBA is the whole call; the scanline
Expand Down
Loading