Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
74bc583
test(browser): add og-browser, a Rust headless-Chrome driver for the …
GeiserX Nov 17, 2026
971f8fb
test(live): move the replay and asset-scan checks to og-browser
GeiserX Nov 17, 2026
9f8478c
refactor(lookup): share the summary names rule and the query reader w…
GeiserX Nov 17, 2026
0b70e82
perf(search): build the results search index in og with the rule the …
GeiserX Nov 17, 2026
88fb283
docs(search): name og report index as the rule search.js mirrors
GeiserX Nov 17, 2026
bb03567
feat(y): write Yleaf's table for a pileup with no read in og
GeiserX Nov 17, 2026
8e509be
refactor(y): run Yleaf through its own CLI instead of a Python wrappe…
GeiserX Nov 17, 2026
9432051
ci(guard): quote the empty CI value in the og-browser control (SC1007)
GeiserX Nov 17, 2026
03e3b85
fix(y): stop with a clear message on a missing prediction or a compre…
GeiserX Nov 17, 2026
9cebacd
fix(search): veil each index item by its own mentions, as results_ind…
GeiserX Nov 17, 2026
909efcf
docs(browser): say the [SKIP] line shows only with --nocapture
GeiserX Nov 17, 2026
c3e3b49
Merge remote-tracking branch 'origin/main' into pgp/port-yleaf-wrap
GeiserX Nov 18, 2026
3a92582
Merge remote-tracking branch 'origin/main' into pgp/port-report-index
GeiserX Nov 18, 2026
623acb0
fix(browser): bound every DevTools wait, confine the test server and …
GeiserX Nov 18, 2026
d5be94d
fix(browser): read script attributes one by one and never answer a co…
GeiserX Nov 18, 2026
78cdec1
test(report-index): drop the Python index test now that the Rust inde…
GeiserX Nov 18, 2026
aba59e5
Merge #471 port-report-index into the batch c tip
GeiserX Nov 18, 2026
f988b85
Merge #472 port-yleaf-wrap into the batch c tip
GeiserX Nov 18, 2026
59e7119
Merge origin/main into the batch c tip
GeiserX Nov 18, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .github/workflows/guard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,38 @@ jobs:
- name: cargo test
run: cargo test --workspace --locked --no-fail-fast

# The browser checks of og-browser (tests/browser/README.md) ran in
# cargo test above, in the runner's Chrome. Two controls: a page error
# planted in the clean page must fail its test, and with no Chrome the
# tests must fail in CI and only skip outside it.
- name: The browser checks fail on a planted page error, and without Chrome in CI
if: ${{ !cancelled() }}
run: |
log="${RUNNER_TEMP}/og-browser-red.log"
if cargo test --locked --no-fail-fast -p og-browser --features planted-page-error --test cdp > "$log" 2>&1; then
cat "$log"
echo "ERROR: the clean page's test passed with a planted page error"
exit 1
fi
cat "$log"
grep -E 'test a_clean_page_loads_with_no_page_error \.\.\. FAILED' "$log"
grep -E "TypeError: .*\bplanted\b" "$log"
echo "Control: a planted page error failed the clean page's test."
if CHROME=/nonexistent cargo test --locked --no-fail-fast -p og-browser --test cdp > "$log" 2>&1; then
cat "$log"
echo "ERROR: the browser checks passed in CI without Chrome"
exit 1
fi
cat "$log"
grep -F "[FAIL] no Chrome or Chromium to load the test page" "$log"
Comment thread
geiresx marked this conversation as resolved.
Resolved
grep -E 'test a_clean_page_loads_with_no_page_error \.\.\. FAILED' "$log"
grep -E 'test a_planted_exception_and_console_error_are_reported \.\.\. FAILED' "$log"
echo "Control: without Chrome the browser checks fail in CI."
CHROME=/nonexistent CI='' cargo test --locked -p og-browser --test cdp -- --nocapture > "$log" 2>&1 || { cat "$log"; exit 1; }
cat "$log"
test "$(grep -c -F '[SKIP] no Chrome or Chromium to load the test page' "$log")" -eq 2
echo "Outside CI, without Chrome the browser checks skip."

# The planned-verb test (og/tests/port_scaffold.rs) must fail when a
# later merge drops a verb's arm.
- name: The planned-verb test fails when a verb is dropped
Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n

## Unreleased

- `og report index build --summary summary.json [--sample-dir DIR] -o <sample>_search_index.json` writes the run's search index in Rust. `og report index query --index FILE QUERY` answers one query with the rule of the live site's search box (`assets/live/plugins/search.js`); a cargo test runs that script under node against the Rust rule. The file is the one `bin/results_index.py` writes: the same terms, sections, anchors, counts and mentions, and the same answer to each query. No step calls the verb yet: step 24 and the Nextflow report still write the index with `bin/results_index.py`. One field is not written yet: `veil`, which tags an item that names a result a child's page keeps closed. It needs the live site's findings, which are not ported yet.
- Step 37 and the Nextflow `Y_HAPLOGROUP` process run Yleaf through its own command line. `bin/yleaf_run.py`, our Python launcher that patched Yleaf at run time (its data folder, its reference, its samtools calls and its process pools), is gone. Yleaf 4.1.7 has the options the launcher stood in for: `-rp` reads the pileup the step makes beside the chrY BAM, `--ref-fasta` names the reference so it downloads none, and `--no-update-check` keeps it offline. Yleaf now predicts with the marker table and tree its image ships; the image test checks they are the files of the archive `setup.sh --yleaf-data` installs, whose positions the pileup is made at, and that Yleaf still has those options, that pileup file and the samtools flags the pileup uses. When no read falls on any marker, Yleaf stops before it predicts, so `og y yleaf-no-reads` writes the row it gives for too few markers (`Hg` NA, the BAM's mapped reads), as the launcher did. On the fixture's chrY slice the prediction table and every file Yleaf writes are the same, byte for byte, as with the launcher, and so is the empty-pileup row. No option, output file or column changes. Step 37 now stops with a clear message when `REF_FASTA` does not end in `.fa`, `.fasta` or `.fna` (a compressed FASTA, say): Yleaf's `--ref-fasta` takes no other name, and the Nextflow `--reference` already asks for a plain FASTA. `setup.sh` installs `.fasta`.
- A new page, [User journeys](docs/user-journeys.md), records where a new user gets stuck on each surface: the docs, the demo, the shell and the local site. It gives the outcome of each journey in the first test-as-user run (2026-11-17, commit `65095e97`), the fix for each finding, and what the run could not reach. A fix that lands has a journey to walk again.
- VCF intake now refuses a record with GT after another FORMAT key, such as `DP:GT:GQ`, before a reader can mistake depth for the genotype. The check exits 2 and names the sample, file, first offending position and FORMAT field. It tells users to reorder the keys and sample values together or get a corrected VCF. Records without GT remain allowed by this ordering check. Valid input and output formats are unchanged. Part of #411 (clin-37).
- The `archaic-same-person` job of `public-evidence.yml` puts step 43's Neanderthal totals beside Chen et al.'s released calls with `og archaic compare` and `og archaic genomes`, in Rust. They replace `bin/archaic_compare.py`, which is gone with its test, `tests/test_archaic_compare.py`; those checks are now cargo tests in og-ancestry. The tables are the same, byte for byte, on 20 parity cases and on the inputs of the last green run of the job (NA12878, chromosomes 19 to 22). No step of the pipeline runs them, and no option or output changes. What changes on purpose: a missing file, an unreadable number or an empty spans file stops with an `ERROR:` line and exit 1 instead of a Python traceback; an interval with a negative start or an end before its start stops the command, where the Python counted a negative length; and a `--mine` value without `=` is a usage error (exit 2).
Expand Down
7 changes: 4 additions & 3 deletions assets/live/plugins/search.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@
// filters that index here in the browser: no request per keystroke, so the query is in no
// request and no server log. A hit lists each section that names the term, with a link to it in
// the report and, for a mention with a locus, to the reads in the genome viewer. The query rule
// is bin/results_index.py's query(): an rsID, a gene symbol (a prefix also lists the terms that
// start with it), or a position, which the index cannot answer (scripts/owngenome-lookup.sh can).
// is og report index's query() (rust/crates/og-render/src/index.rs) and bin/results_index.py's:
// an rsID, a gene symbol (a prefix also lists the terms that start with it, in sorted order), or
// a position, which the index cannot answer (scripts/owngenome-lookup.sh can).
// The box sits in the page header; on a person's page a hit opens its card (#s/<section>).
// tests/test_results_index.py runs match() under node against the Python rule.
// rust/crates/og-render/tests/index_search.rs runs match() under node against the Rust rule.
(function (root) {
'use strict';
var SUGGESTIONS = 20;
Expand Down
139 changes: 0 additions & 139 deletions bin/yleaf_run.py

This file was deleted.

2 changes: 1 addition & 1 deletion docs/24-html-report.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ After them, "What this report leaves out" lists the steps not run and what no st
- **Links to the step's own files**, relative to the report, when the summary says they exist: the PharmCAT HTML report, the CPIC recommendations per gene, the CPSR HTML report (or the `cpsr/` folder that holds it) and the Exomiser ranking table. The report sits in the sample folder beside each step's folder, in the bash layout and in a Nextflow `--outdir`. MultiQC and the couple report are not linked: the summary does not record them, and the couple report sits in the family folder.
- **Nothing is fetched.** A Content-Security-Policy (`default-src 'none'; img-src data:; style-src 'unsafe-inline'`) lets the browser load nothing but the file itself and its inline pictures, and there is no JavaScript. The only addresses off this computer are the haplogroup links (FamilyTreeDNA, YFull, the docs) and the project link in the footer, each marked "opens a website"; the FamilyTreeDNA and YFull ones add that the address carries the haplogroup. The genome viewer links go to `127.0.0.1`, this computer.
- **A Viewer column on every findings table.** The ClinVar hits, the secondary-findings genes, the mitochondrial variants, the known deep intronic sites, the cDNA scan, the AnnotSV class 4 and 5 SVs and the calls moved to class 3, slivar's pairs in trans, the mosaic alterations and Exomiser's ranked variants each end with a Viewer column: one link per row, to that row's position (a range for an SV, a region or a pair). The link opens the reads there in the [genome viewer](viewer.md) once `scripts/owngenome-view.sh <sample>` runs; a note under the table says so. The column is new, so every other cell keeps its text; it replaces the single line of links that sat under three of these tables. A row with no position has an empty Viewer cell. On the [live site](live-site.md) the served report points the same links at the site's own viewer of the run.
- **Genes and rsIDs in these results.** The closing list is built by `bin/results_index.py`. It holds every gene symbol and rsID that a section in state complete or stale names in `summary.json`, by the same rule `scripts/owngenome-lookup.sh` uses for "in the reports" (`names_of` in `bin/genome_lookup.py`, which `og lookup` ports in `rust/crates/og-lookup`; `tests/test_results_index.py` checks that the two agree). A section's locus name counts as a term too, when it holds an upper-case letter: a repeat expansion (HTT, FMR1, CFTR_TG), a Locityper locus (HLA-DRB1, FCGR) and a copy-number locus are indexed by their `locus` field, so a term is not always a gene symbol. An expansion record with no REPID or VARID takes its position as its locus (`chrX:147912050`), and that counts as a term only when it holds an upper-case letter (a chrX, chrY or chrM position). It adds the genes of the clinical filter (step 23) and variant prioritization (step 31), whose cards only count them: from their summary tables on the bash path, from the VCFs on the Nextflow path. The two give the same genes when a variant's most severe consequence is on the transcript the report ranks first (MANE, else canonical); the step 31 table keeps the gene of the most severe consequence, so where an overlapping gene's transcript carries the worse one, the two forms name different genes. A ClinVar hit's rsIDs come from step 13's VEP output (`Existing_variation`), because the hits file's ID column holds ClinVar's own id; without step 13 a hit has none. The Nextflow `HTML_REPORT` process does not get step 13's VEP output, so its index has no ClinVar rsIDs; step 24, which `run-all.sh` runs over the output folder afterwards, adds them. A failed, missing or unreadable section adds nothing. The browser's find (Ctrl+F) searches the list; there is still no script. Apart from those locus names a position is not in it, and the search does not answer a position: ask `scripts/owngenome-lookup.sh` for a genotype ([genotype lookup](lookup.md)).
- **Genes and rsIDs in these results.** The closing list is built by `bin/results_index.py`. It holds every gene symbol and rsID that a section in state complete or stale names in `summary.json`, by the same rule `scripts/owngenome-lookup.sh` uses for "in the reports" (`names_of` in `bin/genome_lookup.py`, which `og lookup` ports in `rust/crates/og-lookup`; `og report index` ports the list itself, and `rust/crates/og-render/tests/index_search.rs` checks the page search against it). A section's locus name counts as a term too, when it holds an upper-case letter: a repeat expansion (HTT, FMR1, CFTR_TG), a Locityper locus (HLA-DRB1, FCGR) and a copy-number locus are indexed by their `locus` field, so a term is not always a gene symbol. An expansion record with no REPID or VARID takes its position as its locus (`chrX:147912050`), and that counts as a term only when it holds an upper-case letter (a chrX, chrY or chrM position). It adds the genes of the clinical filter (step 23) and variant prioritization (step 31), whose cards only count them: from their summary tables on the bash path, from the VCFs on the Nextflow path. The two give the same genes when a variant's most severe consequence is on the transcript the report ranks first (MANE, else canonical); the step 31 table keeps the gene of the most severe consequence, so where an overlapping gene's transcript carries the worse one, the two forms name different genes. A ClinVar hit's rsIDs come from step 13's VEP output (`Existing_variation`), because the hits file's ID column holds ClinVar's own id; without step 13 a hit has none. The Nextflow `HTML_REPORT` process does not get step 13's VEP output, so its index has no ClinVar rsIDs; step 24, which `run-all.sh` runs over the output folder afterwards, adds them. A failed, missing or unreadable section adds nothing. The browser's find (Ctrl+F) searches the list; there is still no script. Apart from those locus names a position is not in it, and the search does not answer a position: ask `scripts/owngenome-lookup.sh` for a genotype ([genotype lookup](lookup.md)).
- **Dark on screen, light on paper.**
- **Print one section.** Open a section from the contents (or its "link to this section"), then print: only that section prints, with the header (sample, generation date, pipeline commit, ClinVar file date, ClinVar in CPSR's bundle, HLA database), the list of what is not assessed and the disclaimer. A chapter prints the same way from its title in the contents. Without an anchor in the address the whole report prints, as it does in a browser without CSS `:has()`.

Expand Down
Loading
Loading