You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 2207350
Browse filesBrowse the repository at this point in the historyBrowse files
One merge for a batch of independent pull requests. Each was reviewed in its own lane. The tip merges every head below, and one end-to-end run plus the fast checks passed on that exact tree.
- #489
- #474 refactor(docs): check the docs site with og dev instead of a Python script
- #468 refactor(steps): find vendor data paths with sh instead of inline Python
E2E evidence: https://github.com/geiresx/OwnGenome/actions/runs/44289882413
Copy file name to clipboardExpand all lines: CHANGELOG.md
+6Lines changed: 6 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -9,6 +9,12 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n
9
9
- The `archaic-same-person` job of `public-evidence.yml` puts step 43's Neanderthal totals beside Chen et al.'s released calls with `og archaic compare` and `og archaic genomes`, in Rust. They replace `bin/archaic_compare.py`, which is gone with its test, `tests/test_archaic_compare.py`; those checks are now cargo tests in og-ancestry. The tables are the same, byte for byte, on 20 parity cases and on the inputs of the last green run of the job (NA12878, chromosomes 19 to 22). No step of the pipeline runs them, and no option or output changes. What changes on purpose: a missing file, an unreadable number or an empty spans file stops with an `ERROR:` line and exit 1 instead of a Python traceback; an interval with a negative start or an end before its start stops the command, where the Python counted a negative length; and a `--mine` value without `=` is a usage error (exit 2).
10
10
- The live site's access log no longer names the finding a page opens. A finding's id is built from its gene and position, and the finding route wrote that id into the log's section column, though `docs/live-site.md` says the log never holds a result. The row now holds `-` there, as the other result routes do. The log keeps its columns and one row per page open. An og-live cargo test opens a finding page on a made-up run and reads the log; its control puts the old line back and must fail.
11
11
- The self-test of `scripts/ci/check-images.sh` copies the whole tree again once the list of tracked paths passes 128 KiB. It piped that list through `xargs` to `tar`, and GNU `xargs` split a list that long over two `tar` runs; the reading `tar` stopped after the first archive, so Guard's `images` job failed on a broken pipe (exit 123). One `tar` now reads the list itself (`tar --null -T -`). Nothing the check reports changes.
12
+
-`og report render` now writes the sections of the opt-in steps and of two small default ones in Rust: clonal haematopoiesis with its detection limit (step 49), viral DNA (step 47), mobile element insertions (step 50), MoChA's mosaic alterations (step 51), VNtyper (step 52), Locityper (step 53), the Exomiser ranking (step 48), telomere content and runs of homozygosity. Both reports match `bin/render_report.py` byte for byte on the parity corpus, except for these changes:
13
+
- When the BAM has no unplaced unmapped reads, step 47 looked at no viral genome. The viral section now says the genomes were not assessed. The Python said "No read on any of the 24 viral genomes", which reads as a negative result.
14
+
- A stat value from the summary is escaped in the HTML card. The Python put it into the page as it was.
15
+
- The genome chart of the ROH card is not drawn yet: it comes with the port of the charts.
16
+
- The lines of the SV callers and of the cDNA scan are ready as parts for the structural-variant and ClinVar sections, which come with the clinical sections.
17
+
- The checks of `tests/test_{chip,viral,mei,mocha,vntyper,locityper,exomiser}_report.py` moved to cargo tests in og-chip, og-reports, og-summary and og-render, and run-all.sh's plan and the DONE guards of step 53 to the fake-docker case `port-report-optin-locityper`. The seven Python files are gone. Part of #411.
12
18
- The phased family VCF is indexed by the Rust `og` binary. WhatsHap's step (step 38 and the Nextflow WHATSHAP_PED process) wrote `family.phased.vcf.gz.tbi` with a Python one-liner, `pysam.tabix_index`. It now runs `og vcf index -f`. The `.tbi` keeps its name and format, and its bytes, once inflated, are the ones htslib 1.24 writes (the htslib of the pysam and bcftools the pipeline pins): the same bins in the same order, the same chunks and the same linear index. Only the deflate stream can differ, as two deflate libraries differ. On a 5-million-record three-sample VCF it took 0.75 to 0.91 s and 31 MB at peak, where htslib's own `tabix` took 0.81 to 0.93 s and 19 MB.
13
19
- As in htslib 1.24, a record ends at the largest of INFO/END, the REF length, the SVLEN of a `<DEL>`, `<DUP>`, `<INV>` or `<CNV>` ALT, and a sample's LEN on a gVCF block. A line with no position to read, unsorted records, a sequence that starts again, a record past 2^29 and an empty file stop the index with a message, and nothing is written. Without `-f` it never writes over an index, even one that appears while it runs.
14
20
- Steps 02b and the variant benchmark resolve paths with `realpath`, then `readlink -f`, instead of a `python3` one-liner.
Copy file name to clipboardExpand all lines: assets/chip_genes.tsv
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -8,7 +8,7 @@
8
8
# residue (R882: any change of that residue), one exact change (V617F), a codon range (102-292)
9
9
# or exons:A-B.
10
10
# A change outside these classes is listed as other_change, not counted.
11
-
# og chip check-table checks the schema; tests/test_chip_report.py runs it.
11
+
# og chip check-table checks the schema; rust/crates/og-chip/tests/optin_chip.rs runs it.
12
12
gene truncating missense note pmids
13
13
DNMT3A all R882 The most common CHIP gene. R882 is the hotspot of DNMT3A in acute myeloid leukaemia; other missense changes are listed, not counted. 25426837;21067377
14
14
TET2 all none Second most common CHIP gene. Truncating changes only; missense changes are listed, not counted. 25426837;19474426
Copy file name to clipboardExpand all lines: docs/50-mei.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -84,7 +84,7 @@ On HG002's chr20 at 30x (13 million mapped reads) on a 4-CPU GitHub runner, MEGA
84
84
85
85
## Validation
86
86
87
-
The e2e case `tests/e2e/mei-1.sh` runs the step on HG002's chr20: the reads of chr20 in GIAB's 60x novoalign BAM, half the pairs (30x), realigned with minimap2 to chr20 alone, the unmapped reads dropped (so the case fits a runner's time), and the k-mer set of that chromosome. Its Alu calls are scored against the GIAB v5.0q insertions of HG002 on chr20 (RepeatMasker SINE/Alu, interspersed, 250 to 400 bp, inside the benchmark regions): recall must be at least 0.70 and precision at least 0.80, within 100 bp. On 2026-10-27 it found 12 of the 15 insertions (0.80), and 13 of its 15 Alu calls inside the benchmark regions matched a GIAB insertion (0.87); with the unmapped reads kept, the same 12 of 15 and 13 of 15. With every read pair within 1 kb of a GIAB Alu insertion removed, it found 0 of 15 and the recall check failed. The same case then runs AnnotSV on the calls with made-up AnnotSV data (the bundle of `tests/fixtures/annotsv/class45.sh` plus one made-up gene over all of chr20) and checks that every call comes back inside that gene, with its location there and class `NA`; the same VCF read without AnnotSV's table fails that check. It also runs the Nextflow pipeline with `--tools mei` and `--annotsv_annotations` on the same BAM and checks that its calls table is the bash step's, byte for byte. `tests/test_mei_report.py` plants an SVA in the FKTN window of a synthetic MEGAnE VCF and checks it is seen, and not seen without it, and joins a table in AnnotSV's columns back to the calls.
87
+
The e2e case `tests/e2e/mei-1.sh` runs the step on HG002's chr20: the reads of chr20 in GIAB's 60x novoalign BAM, half the pairs (30x), realigned with minimap2 to chr20 alone, the unmapped reads dropped (so the case fits a runner's time), and the k-mer set of that chromosome. Its Alu calls are scored against the GIAB v5.0q insertions of HG002 on chr20 (RepeatMasker SINE/Alu, interspersed, 250 to 400 bp, inside the benchmark regions): recall must be at least 0.70 and precision at least 0.80, within 100 bp. On 2026-10-27 it found 12 of the 15 insertions (0.80), and 13 of its 15 Alu calls inside the benchmark regions matched a GIAB insertion (0.87); with the unmapped reads kept, the same 12 of 15 and 13 of 15. With every read pair within 1 kb of a GIAB Alu insertion removed, it found 0 of 15 and the recall check failed. The same case then runs AnnotSV on the calls with made-up AnnotSV data (the bundle of `tests/fixtures/annotsv/class45.sh` plus one made-up gene over all of chr20) and checks that every call comes back inside that gene, with its location there and class `NA`; the same VCF read without AnnotSV's table fails that check. It also runs the Nextflow pipeline with `--tools mei` and `--annotsv_annotations` on the same BAM and checks that its calls table is the bash step's, byte for byte. `rust/crates/og-reports/tests/optin_mei.rs` plants an SVA in the FKTN window of a synthetic MEGAnE VCF and checks it is seen, and not seen without it, and joins a table in AnnotSV's columns back to the calls.
0 commit comments