Skip to content

Pair tumor and normal BAMs regardless of per-BAM kinetics - #219

Open
robert-a-forsyth wants to merge 3 commits into
devfrom
fix/pairing-keys
Open

robert-a-forsyth wants to merge 3 commits into
devfrom
fix/pairing-keys

Conversation

@robert-a-forsyth

Copy link
Copy Markdown
Collaborator

Tumor/normal pairing joined on a meta that carried METAEXTRACT's per-BAM kinetics flag, so a pair where only one BAM kept PacBio kinetics was silently dropped from all variant calling; its QC still reached the report join, which then crashed with "Invalid method invocation call".

  • Drop kinetics from meta after alignment; only the m6A branch reads it.
  • Stop early when a sample's BAMs resolve to different Clair models (validateSampleModels), since those also key the pairing joins.
  • Make the tumor/normal join failOnMismatch so meta drift can no longer drop a pair silently.

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

Tumor/normal pairing joined on a meta that carried METAEXTRACT's per-BAM
`kinetics` flag, so a pair where only one BAM kept PacBio kinetics was
silently dropped from all variant calling; its QC still reached the report
join, which then crashed with "Invalid method invocation call".

- Drop `kinetics` from meta after alignment; only the m6A branch reads it.
- Stop early when a sample's BAMs resolve to different Clair models
  (validateSampleModels), since those also key the pairing joins.
- Make the tumor/normal join failOnMismatch so meta drift can no longer
  drop a pair silently.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Comment thread workflows/lrsomatic.nf
}
.join(paired_normal_bams)
// failOnMismatch: a tumor/normal meta drift must stop the run, not drop the pair
.join(paired_normal_bams, failOnMismatch: true)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No regression test for the bug this PR fixes. The only new tests are the two validateSampleModels function tests. They don't run the pairing join itself.

Nothing tagged small covers a paired PacBio sample where one BAM has kinetics and the other doesn't, so CI never runs that case. The core fix (dropping kinetics from the subMaps) and the new failOnMismatch: true are therefore only checked indirectly.

Suggestions:

  • Add a small stub or fixture test with a tumor/normal pair whose METAEXTRACT kinetics differs. It should show that somatic_smallvar_input emits the pair and that the run reaches LRSOMATICREPORT.
  • If that isn't practical, run the Slurm test array with such a samplesheet and note the result in the PR.

Comment thread workflows/lrsomatic.nf
ch_samplesheet
.map { meta, _bam -> meta }
.collect()
.map { metas -> validateSampleModels(metas) }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing CHANGELOG.md entry. The PR changes behaviour that users will see:

  • Paired samples whose BAMs differ in kinetics are no longer dropped silently.
  • A run with mismatched per-BAM caller models now stops with an error.
  • The tumor/normal join now uses failOnMismatch.

REVIEW.md asks for an entry under v1.2.0dev that cites #219. This should probably go under Fixed, and possibly Changed for the new launch-time error.

@claude

claude Bot commented Oct 6, 2026

Copy link
Copy Markdown

Automated review: 2 findings, posted inline.

  1. No regression test for the actual bug. No small test runs a tumor/normal pair whose BAMs differ in kinetics. The two new tests cover only the validateSampleModels function.
  2. Missing CHANGELOG.md entry for Pair tumor and normal BAMs regardless of per-BAM kinetics #219.

The fix itself looks right. I removed kinetics everywhere it was used as a join key, and the m6A branch still reads it before alignment.

A human reviewer needs to run these, because I couldn't run anything:

  • Default small tests: check that all paired samples still pair with failOnMismatch: true. A mismatch on any other meta field (e.g. fiber or sex differing between the tumor and normal rows) now stops the run where it used to drop the pair silently.
  • A PacBio pair with mixed kinetics, via nf-test or the Slurm test array: check that it reaches small-variant calling and LRSOMATICREPORT.
  • validateSampleModels in the real pipeline: check that the call aborts the run when models mismatch. Its output channel is never consumed.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 1391c61

+| ✅ 232 tests passed       |+
#| ❔  22 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗  45 tests had warnings |!
Details

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 1.1.0
  • pipeline_todos - TODO string in nextflow.config: Specify your pipeline's command line flags
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in README.md: Include a figure that guides the user through the major workflow steps. Many nf-core
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in lint_log.txt: Named file extensions MUST be emitted for ALL output channels
  • pipeline_todos - TODO string in lint_log.txt: List additional required output channels/values here
  • pipeline_todos - TODO string in meta.yml: #Add a description of the module and list keywords
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in CONTRIBUTING.md: Add any pipeline specific contribution guidelines here, such as coding styles, procedures, checklists etc.
  • schema_description - Ungrouped param in schema: skip_modkit
  • schema_description - No description provided in schema for parameter: generate_gvcf
  • schema_description - No description provided in schema for parameter: autocorrelation
  • schema_description - No description provided in schema for parameter: vep_custom
  • schema_description - No description provided in schema for parameter: vep_custom_tbi
  • schema_description - No description provided in schema for parameter: severus_minsupport
  • schema_description - No description provided in schema for parameter: wakhan_chroms
  • local_component_structure - prepare_vep_plugins.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - ecdna.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - fiberseq.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - deepsomatic.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - small_variant_consensus.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_aa_data_repo.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_reference_files.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_annotation.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_signatures.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - reconplot_figures.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - phasing_haplotyping.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - padfoot_annotation.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • files_exist - File is ignored: CODE_OF_CONDUCT.md
  • files_exist - File is ignored: assets/nf-core-lrsomatic_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-lrsomatic_logo_light.png
  • files_exist - File is ignored: docs/images/nf-core-lrsomatic_logo_dark.png
  • files_exist - File is ignored: .github/ISSUE_TEMPLATE/config.yml
  • files_exist - File is ignored: .github/workflows/awstest.yml
  • files_exist - File is ignored: .github/workflows/awsfulltest.yml
  • files_exist - File is ignored: .github/CONTRIBUTING.md
  • nextflow_config - Config variable ignored: manifest.name
  • nextflow_config - Config variable ignored: manifest.homePage
  • nf_test_content - 'tests/sv_annotation_compose.nf.test' checking ignored
  • files_unchanged - File ignored due to lint config: CODE_OF_CONDUCT.md
  • files_unchanged - File ignored due to lint config: .github/ISSUE_TEMPLATE/bug_report.yml
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: .github/workflows/branch.yml
  • files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
  • files_unchanged - File ignored due to lint config: assets/email_template.txt
  • files_unchanged - File ignored due to lint config: assets/nf-core-lrsomatic_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-lrsomatic_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-lrsomatic_logo_dark.png
  • files_unchanged - File ignored due to lint config: docs/README.md
  • actions_awstest - 'awstest.yml' workflow not found: /home/runner/work/lrsomatic/lrsomatic/.github/workflows/awstest.yml

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 4.1.0
  • Run at 2026-10-13 09:19:51

robert-a-forsyth and others added 2 commits October 13, 2026 11:04
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
#223 combined the Severus, Wakhan and SAVANA inputs into one shared
sv_meta_keys list built from PHASING_HAPLOTYPING's haplotagged BAMs, and it
carried 'kinetics'. Take dev's structure and drop 'kinetics' from that list
too, so it matches the kinetics-free phased germline VCF meta it joins with.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ljwharbers

Copy link
Copy Markdown
Collaborator

Merged current dev (cab487f: #223 Severus fix, #224 template 4.2.1, #227 lint cleanup) into this branch.

Conflict in workflows/lrsomatic.nf. #223 replaced the separate Severus input (built from branched_minimap.tumor_only) and the savana_meta_keys block with one shared sv_meta_keys list. Severus, Wakhan and SAVANA all build their input from PHASING_HAPLOTYPING.out.tumor_normal_hapbams_ch, and that list included 'kinetics'. I kept dev's structure and dropped 'kinetics' from sv_meta_keys, in line with this PR:

def sv_meta_keys = ['id', 'paired_data', 'platform', 'sex', 'fiber',
    'clair3_model', 'clairS_model', 'clairSTO_model']

Checks that nothing needs it there:

  • Across modules/, subworkflows/, workflows/, conf/ and bin/, only METAEXTRACT (which sets it) and the fiberseq m6A branch (which runs on the unaligned BAMs before alignment) read kinetics. Severus, Wakhan, SAVANA, Padfoot, ReConPlot, ecDNA and ASCAT never do.
  • Severus fix #223 added kinetics only so the keys matched on both sides of the .join(PHASING_HAPLOTYPING.out.phased_germline_vcf). After this PR, the phased germline VCF meta comes from the same 8 keys, so both sides still match. The hapbams tumour/normal join, the Wakhan join with SEVERUS.out.all_vcf and the ReConPlot joins with the ASCAT outputs all use the same 8-key meta.
  • In the merged workflows/lrsomatic.nf, kinetics now appears only where the meta is built from METAEXTRACT, plus the comments describing it.
  • The PR's edits to phasing_haplotyping and small_variant_consensus carried over to their new subworkflows/local/<name>/main.nf paths, and no flat copies came back.

Lint: prek run --all-files (including nextflow lint) passes. nf-core pipelines lint (nextflow_config, files_unchanged, modules_json, template_strings) reports 0 failures; its warnings are the existing missing meta.yml ones on dev.

nf-test (apptainer, Slurm array 11767781, merge commit bf6bf58). Every pipeline snapshot matched with no re-recording:

task job result
default 11767782 PASSED
clair_only 11767783 PASSED
consensus 11767784 PASSED
deep_only 11767785 PASSED
union (2 tests) 11767786 PASSED
small (tag, 142 tests, includes both new validateSampleModels tests) 11767787 PASSED
chm13 11767781 PASSED

🤖 Generated with Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants