Single Cell Toolkit

Collection of scripts and tools for processing single cell (scATAC-seq, scRNA-seq and multiome) sequencing data: barcode extraction and correction for various single cell technologies, FASTQ manipulation, BAM processing, fragment file generation and quality control.

Most scripts print usage information when run without arguments.

Requirements

Depending on the script, the following tools are needed:

• Bash scripts:
  • bash
  • zstd
  • mawk
  • igzip
  • pigz
  • bgzip (HTSlib)
  • samtools
• Codon scripts (*.codon): Codon with the seq plugin. The bash wrapper scripts run them with codon run -plugin seq -release (override with the CODON_RUN_CMD environment variable).
• Python scripts: Python 3 with (depending on the script) numpy, pandas, polars, pyarrow, pyBigWig, numba, scipy, matplotlib, uncertainties and xopen.
• Rust tools (rust/): Rust toolchain; build with cargo build --release in the rust/ directory.

Overview

• Barcode extraction and correction
• FASTQ manipulation
• BAM processing
• Fragments files
• Quality control and statistics
• Barcode multiplet detection (barcard)
• Rust tools
• Miscellaneous

Barcode extraction and correction

Directory: barcode_extraction_and_correction/

Most technology specific scripts consist of a bash wrapper (.sh) and a Codon script (.codon) which does the actual work. Barcode correction scripts write, for each read, the read name with the raw barcode (CR:Z:), raw barcode quality (CY:Z:) and, if correctable, the corrected barcode (CB:Z:) as SAM tags (zstd compressed), together with a barcode correction statistics file.

Barcode correction per technology

Script	Technology	Description
correct_barcode_from_fastq.{sh,codon}	Generic (e.g. 10x)	Correct raw barcodes from an index 2 FASTQ file against a barcode whitelist.
extract_and_correct_hydrop_atac_barcode_from_fastq.{sh,codon}	HyDrop ATAC	Extract the 3 x 10 bp HyDrop ATAC barcode parts from index 2 reads and correct them against the whitelist.
extract_and_correct_hydrop_rna_barcode_from_fastq.{sh,codon}	HyDrop RNA	Extract the 3 x 10 bp HyDrop RNA barcode parts from read 1 and correct them against the whitelist.
extract_hydrop_atac_barcode_from_R2_fastq.sh	HyDrop ATAC	Extract HyDrop ATAC barcodes from R2 reads and write a FASTQ file with only the barcode.
extract_and_correct_shareseq_barcode_from_fastq.{sh,codon}	SHARE-seq (ATAC and RNA)	Extract the 3 x 8 bp SHARE-seq barcode parts from index 1 reads, check spacers, correct barcodes and report the barcode name as corrected barcode.
extract_and_correct_scalebio_atac_barcode_from_fastq.{sh,codon}	ScaleBio ATAC	Correct the tagmentation barcode and 10x ATAC barcode from index 2 reads.
extract_and_correct_and_split_scalebio_atac_barcode_from_fastq.sh	ScaleBio ATAC	Correct the tagmentation barcode and split read 1 and read 2 FASTQ files per tagmentation barcode.
extract_and_correct_biorad_barcode_from_fastq.{sh,codon}	Bio-Rad ATAC (v2.1)	Extract barcodes and the ATAC part of the reads, correct barcodes and write new FASTQ files with barcode info in the comment field.
create_issaac_seq_barcode_fastq_for_star_solo.sh + extract_and_correct_issaac_seq_barcode_from_fastq.codon	ISSAAC-seq	Create a corrected barcode FASTQ file (30 bp barcode + 10 bp UMI) for STARsolo.
create_scifi_rna_barcode_fastq_for_star_solo.sh + extract_and_correct_scifi_rna_barcode_from_fastq.codon	scifi-RNA	Create a corrected barcode FASTQ file (13 bp RT barcode + 16 bp 10x ATAC barcode + 8 bp UMI) for STARsolo.

Adding barcodes to reads

Script	Description
barcode_10x_scatac_fastqs.sh	Add the cell barcode from R2 to each read in R1 and R3 of 10x scATAC FASTQ files, as a comment or in front of the read name.
add_corrected_barcode_to_read_name.sh	Add raw barcode, raw barcode quality and corrected barcode (as SAM tags) to the read names (comment field) of scATAC R1 and R2 FASTQ files.
add_corrected_barcode_to_sam.sh	Append corrected barcode SAM tags (CR:Z:, CY:Z:, CB:Z:) from a corrected barcode file to each record of a SAM file.

Shared Codon library

File	Description
barcode_extraction_and_correction/barcode_correction.codon	Reading barcode whitelists (as set, list or barcode to name mapping) and barcode correction with up to 3 mismatches (quality-ordered, N-aware).
barcode_extraction_and_correction/utils.codon	Helpers for reading (gzipped) files and extracting barcode sequence/quality from FASTQ records (with automatic reverse complementing of index 2 reads depending on the sequencer).

FASTQ manipulation

Directory: fastq_manipulation/

Script	Description
sra_file_to_fastqs.sh	Convert an SRA file to FASTQ files with original Illumina read names, human readable sample names and CellRanger compatible filenames.
extract_cb_umi_as_fastq_from_sra_file.sh	Extract CB and UMI from the LINKAGE_GROUP field of SRA files (old 10x scRNA datasets) and write them as FASTQ.
fix_sra_fastq.sh	Fix read names of FASTQ files generated with SRA Toolkit (use original Illumina read name if available).
fix_R2_read_10x_scATAC_and_10x_multiome_ATAC_fastq.sh	Fix R2 reads for mixed 10x scATAC / 10x multiome ATAC demultiplexing runs (take the correct 16 bp barcode).
fix_scalebio_atac_fastqs.sh	Fix ScaleBio ATAC R2 and R3 reads after standard demultiplexing so R2 contains the barcodes and R3 the second ATAC read.
split_scalebio_atac_fastqs.sh	Split ScaleBio ATAC R1/R2 FASTQ files and corrected barcode file per tagmentation barcode.
limit_aviti_fastq_to_max_q40.sh	Cap AVITI FASTQ quality scores at Q40 (needed for CellRanger/CellRanger ATAC/ARC).
fastq_subset.sh	Get the first X reads from a FASTQ file.
filter_fastq_by_read_names.sh	Filter a FASTQ file by a list of read names.

BAM processing

Script	Description
mark_or_remove_duplicates_per_barcode.sh	Mark or remove duplicate reads per cell barcode.
subset_bam_per_cb.sh	Subset a BAM file per cell barcode.

See also the Rust tools for updating cell barcodes, splitting BAM files per cluster and creating per sample ParseBio BAM files.

Fragments files

Script	Description
fragments_to_cut_sites.sh	Convert a fragments file to a cut sites file.
fragments_to_bw.py	Calculate genome coverage of fragments and write it to a bigWig file.

See also create_fragments_file in the Rust tools for creating fragments files from BAM files.

Quality control and statistics

Script	Description
mapping_stats.sh	Get mapping statistics from a BAM file (samtools stats, uniquely/confidently mapped reads, fragment reads, ...).
calculate_saturation_from_fragments.py	Infer scATAC sequencing saturation by subsampling a fragments file.
calculate_saturation_10x_samples.py	Infer sequencing saturation of 10x scATAC/scRNA samples.
filter_star_solo_raw_count_matrix.py	Filter STARsolo raw count matrix files and only keep cell barcodes which appear at least once.

Barcode multiplet detection (barcard)

Directory: barcard/

Script	Description
calculate_jaccard_index_cbs.py	Calculate the Jaccard index for cell barcode pairs based on their common fragments.
calculate_jaccard_index_cbs_10x_compare.py	Calculate the Jaccard index between PUMATAC and CellRanger ARC fragments files for selected cell barcodes.
merge_cbs_over_jaccard_index_threshold.py	Merge cell barcodes with a Jaccard index above a precalculated threshold.

Rust tools

Directory: rust/ (build with cargo build --release, binaries end up in rust/target/release/).

Binary	Description
bam_update_cell_barcode	Update cell barcodes in a BAM file.
create_fragments_file	Create a fragments file from a BAM file.
create_fragments_file_htslib	Create a fragments file from a BAM file (rust-htslib based version).
create_parsebio_per_sample_bams	Create per sample BAM files from ParseBio sublibrary BAM files.
split_bams_per_cluster	Split BAM files in per cluster BAM files based on cell barcodes.
split_bams_per_cluster_htslib	Split (multiple) BAM files in per cluster BAM files based on a list of cell barcodes per cluster (rust-htslib based, multithreaded, with optional per strand output).
fastq_count	Count the number of reads in FASTQ files (in parallel).

Miscellaneous

Script	Description
run_codon_seq_script.sh	Run a Codon script that requires the seq plugin (codon run -plugin seq -release).