Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ Each release's notes. From 1.0.0 on, a major or minor release has hand-written n

## Unreleased

- `og report render` writes most of the ancestry chapter and the polygenic score card in Rust: fine-grained ancestry with the painting and its legend, the mitochondrial haplogroup, the Y chromosome (Yleaf, the placement on the current public tree, the Y-STR markers in FamilyTreeDNA's order), archaic and ancient ancestry, and the PRS table with its ancestry and sex notes. Runs of homozygosity are not written yet, so both Rust reports leave them out even when the summary has them. Both reports read as `bin/render_report.py` writes them, except:
- A PRS row whose score, matched or total the summary has as null (a short row of pgsc_calc's table) reads "not available" in both reports, and in the HTML result cell when there is no score either. The Python wrote "None" in the text report and stopped the whole HTML report on that row.
- The painting and the ancient PCA plot are escaped in their `src` attribute. A real picture is base64 and does not change; a value that is not text renders instead of stopping the report.
- A haplogroup's explore address that is not a web address with a host (`http://` or `https://`), such as a `javascript:` one, shows as text in the HTML report. The Python made it a link, which would run it when clicked. Only the two docs pages the collector links get the plain "opens a website" note; any other address gets the note that it carries the haplogroup.
- A painting legend colour that is not `#rrggbb` shows as a fixed grey, so a summary cannot add CSS to the legend. The collector always writes `#rrggbb`.
- The charts are not part of these cards yet; `og report charts` will write them.
The tests of these sections moved from Python to cargo (`rust/crates/og-render/tests/ancestry_*.rs`): `tests/test_ancient_report.py`, `test_archaic_report.py`, `test_haplogroup_explore.py` and `test_ystr_report.py` are gone, and `test_ancestry_fine_report.py` no longer renders the reports. The IBDmix runs of the archaic test need docker and amd64; they run in CI.
- A new page, [User journeys](docs/user-journeys.md), records where a new user gets stuck on each surface: the docs, the demo, the shell and the local site. It gives the outcome of each journey in the first test-as-user run (2026-11-17, commit `65095e97`), the fix for each finding, and what the run could not reach. A fix that lands has a journey to walk again.
- VCF intake now refuses a record with GT after another FORMAT key, such as `DP:GT:GQ`, before a reader can mistake depth for the genotype. The check exits 2 and names the sample, file, first offending position and FORMAT field. It tells users to reorder the keys and sample values together or get a corrected VCF. Records without GT remain allowed by this ordering check. Valid input and output formats are unchanged. Part of #411 (clin-37).
- The `archaic-same-person` job of `public-evidence.yml` puts step 43's Neanderthal totals beside Chen et al.'s released calls with `og archaic compare` and `og archaic genomes`, in Rust. They replace `bin/archaic_compare.py`, which is gone with its test, `tests/test_archaic_compare.py`; those checks are now cargo tests in og-ancestry. The tables are the same, byte for byte, on 20 parity cases and on the inputs of the last green run of the job (NA12878, chromosomes 19 to 22). No step of the pipeline runs them, and no option or output changes. What changes on purpose: a missing file, an unreadable number or an empty spans file stops with an `ERROR:` line and exit 1 instead of a Python traceback; an interval with a negative start or an end before its start stops the command, where the Python counted a negative length; and a `--mine` value without `=` is a usage error (exit 2).
Expand Down
3 changes: 3 additions & 0 deletions rust/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions rust/crates/og-render/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,12 @@ og-io.workspace = true
og-summary.workspace = true
indexmap.workspace = true

# The ancestry chapter's tests (tests/ancestry_*.rs) run each step's own
# verb, then collect and render its result, as the run does.
[dev-dependencies]
og-ancestry.workspace = true
og-reports.workspace = true
flate2.workspace = true

[lints]
workspace = true
15 changes: 9 additions & 6 deletions rust/crates/og-render/src/chapters.rs
Original file line number Diff line number Diff line change
Expand Up @@ -207,21 +207,24 @@ pub fn writer(sid: &str) -> Option<(TextFn, HtmlFn)> {

"locityper" => None,

"ancestry-fine" => None,
"ancestry-fine" => Some((sections::ancestry_fine::text, sections::ancestry_fine::html)),

"mt-haplogroup" => None,
"mt-haplogroup" => Some((sections::haplogroups::text, sections::haplogroups::html)),

"y-chromosome" => None,
"y-chromosome" => Some((
sections::haplogroups::y_chromosome_text,
sections::haplogroups::y_chromosome_html,
)),

"archaic" => None,
"archaic" => Some((sections::archaic::text, sections::archaic::html)),

"ancient" => None,
"ancient" => Some((sections::ancient::text, sections::ancient::html)),

"roh" => None,

"traits-panel" => None,

"prs" => None,
"prs" => Some((sections::prs::text, sections::prs::html)),

"bloodtype" => None,

Expand Down
1 change: 1 addition & 0 deletions rust/crates/og-render/src/html.rs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ fn line(out: &mut Vec<String>, l: &Line) {
out.extend(items.iter().map(|t| format!(" <li>{}</li>", e(t))));
out.push(" </ul>".to_owned());
}
Line::Raw(t) => out.push(t.clone()),
}
}

Expand Down
6 changes: 3 additions & 3 deletions rust/crates/og-render/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@
//! [`css`] the page's style.
//!
//! Ported so far: the frame (chapters, contents, section frames, the run
//! facts, what the report leaves out) and the quality-control and variant
//! sections. A section not ported yet is left out of both reports; the index
//! and charts verbs say they are not ported and exit 2.
//! facts, what the report leaves out) and the sections
//! [`chapters::writer`] names. A section not ported yet is left out of both
//! reports; the index and charts verbs say they are not ported and exit 2.

pub mod chapters;
pub mod charts;
Expand Down
10 changes: 8 additions & 2 deletions rust/crates/og-render/src/model.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
//! `==`), and the document the section modules build from it, which the
//! text and HTML writers lay out.
//!
//! A section module never writes markup. It returns cards of typed lines;
//! every text in them is escaped once, by the HTML writer.
//! A section module returns cards of typed lines; every text in them is
//! escaped once, by the HTML writer. The one exception is [`Line::Raw`]:
//! a table or picture the section lays out itself, escaping each text it
//! puts in.

use std::fmt;

Expand Down Expand Up @@ -325,6 +327,10 @@ pub enum Line {
Para(String),
/// A plain bulleted list.
List(Vec<String>),
/// Markup a section built itself, each text in it escaped with
/// [`crate::escape::e`]: a table, a picture or a link whose layout the
/// report fixes byte for byte.
Raw(String),
}

/// One card: a title, the caveats shown under it, the notes saying its
Expand Down
204 changes: 202 additions & 2 deletions rust/crates/og-render/src/sections/ancestry.rs
Original file line number Diff line number Diff line change
@@ -1,2 +1,202 @@
//! The report section `ancestry`. Empty until its unit of the 1.0.0 plan
//! lands.
//! What the ancestry chapter's sections share: reading a row of the
//! summary the way the Python report did (`r["key"]`, `f"{v:<35}"`, a list
//! walked with `for`), the percentages of the fine-grained ancestry, and the
//! links where a haplogroup can be explored further (steps 12 and 37).
//!
//! Used by the fine-grained ancestry, haplogroup, Y chromosome, archaic,
//! ancient and polygenic score sections.

use pycompat::fmt::{fixed, ljust, rjust};
use pycompat::json::Value;

use crate::escape::e;
use crate::html::offsite_link;
use crate::model::{Line, Malformed, get, need, num, py, truthy};
use crate::notes::EXPLORE_NOTE;

/// Every explore address but the docs one carries the haplogroup in it.
pub const EXPLORE_DOCS: &str = "https://github.com/GeiserX/OwnGenome/blob/main/docs/";

/// The pages under [`EXPLORE_DOCS`] the collector links, one per haplogroup.
const EXPLORE_DOC_PAGES: [&str; 2] = [
"37-y-haplogroup.md#explore-your-branch-further",
"12-mito-haplogroup.md#explore-your-branch-further",
];

/// The mark of an address that leaves this computer with the haplogroup in it.
pub const OFFSITE_HAPLOGROUP: &str = "opens a website; the address carries the haplogroup";

/// `str(v)` of an optional value: `None` when it is absent.
pub fn py_or(v: Option<&Value>, default: &str) -> Result<String, Malformed> {
v.map_or(Ok(default.to_owned()), py)
}

/// `d.get(key, default)` as text: the default only when the key is absent.
pub fn get_text(d: &Value, key: &str, default: &str, path: &str) -> Result<String, Malformed> {
py_or(get(d, key, path)?, default)
}

/// `str(d[key])`.
pub fn need_text(d: &Value, key: &str, path: &str) -> Result<String, Malformed> {
py(need(d, key, path)?)
}

/// The items `for x in v` walks: a list's items, a string's characters, a
/// dict's keys. `None` and numbers cannot be walked.
pub fn items(v: &Value, path: &str) -> Result<Vec<Value>, Malformed> {
match v {
Value::Array(a) => Ok(a.clone()),
Value::Str(s) => Ok(s.chars().map(|c| Value::Str(c.to_string())).collect()),
Value::Object(o) => Ok(o.keys().map(|k| Value::Str(k.clone())).collect()),
_ => Err(Malformed(format!("{path} is not a list"))),
}
}

/// `d.get(key, [])` walked with `for`: nothing when the key is absent.
pub fn list(d: &Value, key: &str, path: &str) -> Result<Vec<Value>, Malformed> {
match get(d, key, path)? {
None => Ok(Vec::new()),
Some(v) => items(v, &format!("{path}.{key}")),
}
}

/// `len(v)`.
pub fn py_len(v: &Value, path: &str) -> Result<usize, Malformed> {
match v {
Value::Array(a) => Ok(a.len()),
Value::Str(s) => Ok(s.chars().count()),
Value::Object(o) => Ok(o.len()),
_ => Err(Malformed(format!("{path} has no length"))),
}
}

/// Which side `f"{v:<N}"` (left) or `f"{v:>N}"` (right) pads.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Align {
Left,
Right,
}

/// `f"{v:<N}"` or `f"{v:>N}"`: a string or a number padded to `width`
/// characters. A bool formats as the number it is, as in Python; `None`, a
/// list or a dict cannot take a width.
pub fn align(v: &Value, width: usize, side: Align, path: &str) -> Result<String, Malformed> {
let text = match v {
Value::Str(s) => s.clone(),
Value::Int(i) => i.to_string(),
Value::Float(_) => py(v)?,
Value::Bool(b) => u8::from(*b).to_string(),
_ => return Err(Malformed(format!("{path} cannot be padded"))),
};
Ok(match side {
Align::Left => ljust(&text, width),
Align::Right => rjust(&text, width),
})
}

/// `f"{d[key]:<N}"` and `f"{d[key]:>N}"`.
pub fn need_align(
d: &Value,
key: &str,
width: usize,
side: Align,
path: &str,
) -> Result<String, Malformed> {
align(need(d, key, path)?, width, side, &format!("{path}.{key}"))
}

/// A number as Python compares it with `>=`; anything else is an error.
pub fn number(v: &Value, path: &str) -> Result<f64, Malformed> {
num(Some(v)).ok_or_else(|| Malformed(format!("{path} is not a number")))
}

/// A fraction as a whole percentage (`95%`), `.` for none.
pub fn share(v: Option<&Value>, path: &str) -> Result<String, Malformed> {
match v {
None | Some(Value::Null) => Ok(".".to_owned()),
Some(x) => Ok(format!("{}%", fixed(100.0 * number(x, path)?, 0))),
}
}

/// A stat line whose value is markup built here.
pub fn stat_raw(label: &str, value: &str) -> Line {
Line::Raw(format!(
" <div class=\"stat\"><span class=\"label\">{}</span><span class=\"value\">{value}</span></div>",
e(label)
))
}

/// One explore entry of a haplogroup section: its name and address.
fn explore(d: &Value, path: &str) -> Result<Vec<(String, Value)>, Malformed> {
let at = format!("{path}.explore[]");
list(d, "explore", path)?
.iter()
.map(|x| Ok((need_text(x, "name", &at)?, need(x, "url", &at)?.clone())))
.collect()
}

/// The text report's explore lines: `name: url` for each link, then the
/// note that nothing is uploaded; nothing when there is no link.
pub fn explore_lines(d: &Value, path: &str) -> Result<Vec<String>, Malformed> {
let mut out = Vec::new();
for (name, url) in explore(d, path)? {
out.push(format!("{name}: {}", py(&url)?));
}
if !out.is_empty() {
out.push(EXPLORE_NOTE.to_owned());
}
Ok(out)
}

/// The HTML report's explore lines: one stat per link, each marked as
/// leaving this computer, and whether the card shows the explore caveat.
/// The address is escaped, so a branch name cannot open a tag or end the
/// attribute.
pub fn explore_html(d: &Value, path: &str) -> Result<(Vec<Line>, bool), Malformed> {
let mut out = Vec::new();
for (name, url) in explore(d, path)? {
let Value::Str(url) = url else {
return Err(Malformed(format!("{path}.explore[].url is not a text")));
};
// Only the two fixed pages the collector links get the plain
// warning: any other address, even under the docs folder, may carry
// the haplogroup in its query.
let why = if url
.strip_prefix(EXPLORE_DOCS)
.is_some_and(|page| EXPLORE_DOC_PAGES.contains(&page))
{
crate::html::OFFSITE
} else {
OFFSITE_HAPLOGROUP
};
// Only a web address with a host becomes a link: a `javascript:` or
// `data:` address in a summary would run when clicked, and a bare
// `https://` leads nowhere, so both show as text.
let web = ["http://", "https://"].iter().any(|s| {
url.get(..s.len())
.is_some_and(|head| head.eq_ignore_ascii_case(s))
&& url[s.len()..]
.split(['/', '?', '#'])
.next()
.is_some_and(|host| !host.is_empty())
});
Comment thread
geiresx marked this conversation as resolved.
Resolved
let value = if web {
offsite_link(&url, &url, why)
} else {
e(&url).into_owned()
};
out.push(stat_raw(&name, &value));
}
let note = !out.is_empty();
Ok((out, note))
}

/// `d.get(key, ".") not in (".", "")`: a value worth a line.
pub fn shown<'a>(d: &'a Value, key: &str, path: &str) -> Result<Option<&'a Value>, Malformed> {
Ok(get(d, key, path)?.filter(|v| !matches!(v, Value::Str(s) if s.is_empty() || s == ".")))
}

/// `bool(d.get(key))`.
pub fn has(d: &Value, key: &str, path: &str) -> Result<bool, Malformed> {
Ok(get(d, key, path)?.is_some_and(truthy))
}
Loading
Loading