Skip to content

Add Kraken2 2.17.2 and prebuilt Kraken2 databases - #194

Merged
kennedydane merged 4 commits into
masterfrom
update/kraken
Oct 7, 2026
Merged

kennedydane merged 4 commits into
masterfrom
update/kraken

Conversation

@kennedydane

@kennedydane kennedydane commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds Kraken2 2.17.2 (bio, compiled) alongside the existing 2.1.3, and adds prebuilt Kraken2 databases at /software/data/bio/kraken2.

  • 2.17.2: built from the GitHub tag archive with install_kraken2.sh as before. The new k2 wrapper uses only the Python 3 standard library, so the system python3 is enough.
  • Databases (-t kraken2,kraken2-db): Standard, Viral and MinusB (2026-06-26), EuPathDB48 (2023-04-07), and the four 16S sets (Greengenes 13.5, RDP 11.5, Silva 132, Silva 138). Each one is extracted to <name>_<date>/, gets an INSTALLED.ansible marker, and is skipped on later runs. The archive is deleted after extraction.
  • Module: kraken2.lua now sets KRAKEN2_DB_PATH, so kraken2 --db standard_20260626 finds the database by name.

Checklist

  • Linters pass locally: cd ansible && uv run yamllint . && uv run ansible-lint (Profile 'production' ... passed); ansible-playbook site.yaml --syntax-check also passes.

  • CI is green on this PR.

  • Double tag wired up: kraken2 on the include; kraken2-2.17.2 / kraken2-db on the leaf blocks (both with never). The hyphenated version tag matches the existing kraken2-2.1.3.

  • Checksum set on every download. 2.17.2 uses sha512. Standard, Viral and MinusB use upstream's md5, taken from their published <collection>.md5 files. EuPathDB48 and the 16S sets publish no checksums, so their sha256 was computed from complete downloads, each size-checked against S3's Content-Length.

  • Module template: updated kraken2.lua.

  • Symbolic file modes.

  • Uses the path variables (bio_data_dir, bio_dir, bio_modules).

  • README software tables: Kraken2 isn't listed there today, so they're unchanged.

Notes for the maintainer

  • Run as:
    • ansible-playbook site.yaml -t kraken2,kraken2-2.17.2 for the software.
    • ansible-playbook site.yaml -t kraken2,kraken2-db for the databases.
  • Disk space: each step runs over every database before the next step starts, so all the archives (~98 GB) are still on disk while everything is extracted (~128 GB). Allow about 230 GB free on /software at the peak. Downloads are staged in /software/data/bio/kraken2 (tmp_dest), not ~/.ansible/tmp, so the home directory isn't used. Databases are listed smallest first, so the small ones finish before Standard starts. If a download is interrupted, a partial temp file may be left in that directory to delete by hand.
  • Pre-placed archives: if an archive is already at /software/data/bio/kraken2/<basename> and its checksum matches, get_url skips the download. EuPathDB48 was pre-placed this way. Run as the user that owns these files, because the tasks set modes, extract into the directory and delete the archive.
  • Extraction: tar is used instead of ansible.builtin.unarchive, which would decompress the ~80 GB archive twice. There's a # NOTE: at the task.
  • Runs per version: the database block sits inside the per-version include, so it runs once for each Kraken2 version. Repeat passes only check the markers.
  • Not changed: kraken2_dir's existing g=rx mode, and undated standard → standard_<date> symlinks.
  • Upstream:

🤖 Generated with Claude Code

https://claude.ai/code/session_01GttZpVB7Vniruv7g6F6jjU

kennedydane and others added 2 commits October 7, 2026 13:13
Upstream dropped the "v" prefix from the release tag, so the archive URL
differs from earlier versions. install_kraken2.sh is unchanged, so the
existing task builds it as-is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GttZpVB7Vniruv7g6F6jjU
Downloads Standard, Viral, MinusB, EuPathDB48 and the four 16S indexes to
/software/data/bio/kraken2/<name>, skipping any with an INSTALLED.ansible
marker. Upstream md5s are used where published; EuPathDB and 16S sha256s
were computed from complete downloads. The module now sets
KRAKEN2_DB_PATH so databases resolve by name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GttZpVB7Vniruv7g6F6jjU
Copilot AI balanced review requested due to automatic review settings October 7, 2026 11:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kennedydane
kennedydane requested a review from MikeCTZA October 7, 2026 11:21
kennedydane and others added 2 commits October 7, 2026 13:31
get_url defaults its temporary file to remote_tmp (~/.ansible/tmp), which
would put the ~80 GB Standard archive in a quota-limited home directory and
then copy it across filesystems. Staging in the database root makes the
final move a rename.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GttZpVB7Vniruv7g6F6jjU
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GttZpVB7Vniruv7g6F6jjU
@kennedydane
kennedydane merged commit 416a8d9 into master Oct 7, 2026
1 check passed
@kennedydane
kennedydane deleted the update/kraken branch October 7, 2026 11:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants