Module Requirements

Overview

Below is a table detailing minimum resource requirements for each module in the duet pipeline and D(h)MR workflow, relative to each profile. Your cloud tenancy or HPC cluster should be able to allocate a set of VMs/nodes with these resources.

The module specific settings and profiles can be found in the biomodal-duet-1.4.x/modules and biomodal-duet-1.4.x/conf directories. We do not recommend editing these files directly as any local change will not be reflected when you upgrade the pipeline.

You can edit resource requirements by referencing specific modules the nextflow.config file in the biomodal script folder to accommodate available hardware resources. Please see examples in section Set queue, CPU, RAM and DISK per module for more information on how to increase or decrease the resource requirements. However, the duet pipeline is less likely to run successfully if the minimum resources are too restrictive.

Important

The module requirement is per sample per node on a HPC cluster node or cloud VM. You need to multiply the number of samples by the resource requirement if you intend to only use a single node or VM.

If you are running the pipeline or D(h)MR workflow on a larger local node you can set a maximum ceiling for CPU and RAM per module. Please see section Limit local resources in the documentation for information of how to use parameters to limit the resource requirements for the local and local_deep_seq profiles.

Module requirements

Module / Requirement type	CPU (cores)	RAM (GB)	Disk (GB)	CPU (cores)	RAM (GB)	Disk (GB)	CPU (cores)	RAM (GB)	Disk (GB)
Profile name:	default	default	default	deep_seq	deep_seq	deep_seq	super_seq	super_seq	super_seq
ASM	16	32	-	-	-	-	32	128	1600
BAMLET	2	2	-	-	-	-	-	-	-
BAM_TO_CRAM	6	6	-	-	-	-	-	-	-
BAMLET_GENOME	2	2	-	8	8	800	8	8	3200
BAMLET_GENOME_PRIMARY_ASSEMBLY	2	2	-	8	8	800	8	8	3200
BAMLET_LONG_CTRL	2	2	-	4	4	800	4	4	1600
BAMLET_SHORT_CTRL	2	2	-	4	4	800	4	4	1600
BEDGRAPHS	2	8	-	-	-	-	-	-	-
BIGWIGS	2	8	-	-	-	-	-	-	-
BWAKIT	16	16	-	-	-	-	-	-	-
BWA_MEM2 (Human genome)	16	64	-	32	64	800	96	128	3200
BWA_MEM2 (Mouse genome)	32	128	-	32	128	800	96	128	3200
COMBINE_GENOME_AND_CTRLS	1	2	-	-	-	-	-	-	-
CONCATENATE_SAMPLE_STORES (chg_chh_contexts)	16	32	-	-	-	-	-	-	-
CONCATENATE_SAMPLE_STORES	8	16	-	-	-	-	-	-	-
COUPLET	32	32	-	-	-	-	96	320	1600
CURATE_QUANT	4	8	-	-	-	-	-	-	-
CURATE_QUANT (chg_chh_contexts)	8	16	-	-	-	-	-	-	-
CUTADAPT	16	16	-	-	-	-	32	32	1600
DEDUP_GENOME	1	4	-	4	16	800	4	16	1600
DEDUP_MERGED_BAM	1	4	-	1	16	800	4	16	1600
DEDUP_GENOME_INTERVAL	1	4	-	-	-	-	4	16	1600
DEDUP_LONG_CTRL	1	4	-	2	8	800	4	16	800
DEEPTOOLS_PLOTFINGERPRINT	4	16	-	-	-	-	8	32	1600
DOWNSAMPLE_BAM	2	2	-	-	-	-	-	-	-
DOWNSAMPLE_LONG_CTRL	2	2	-	-	-	-	-	-	-
DQSPY	8	32	-	-	-	-	-	-	-
DQSREPORT	4	16	-	-	-	-	-	-	-
EPIQUANT_BAM2ZARR	8	8	-	16	16	-	32	32	-
EPIQUANT_MBIAS or GENOME_MBIAS	4	4	-	16	16	-	-	-	-
EPIQUANT_QUANT	8	8	-	16	32	800	32	32	1600
EPIQUANT_SUMMARY	1	1	-	-	-	-	-	-	-
FASTQC	2	4	-	-	-	-	-	-	-
FASTQC_RAW	2	4	-	-	-	-	-	-	-
FASTQC_RESOLVED	2	4	-	-	-	-	-	-	-
FASTQC_TRIMMED	2	4	-	-	-	-	-	-	-
FILTER_BAM	8	16	-	-	-	-	-	-	-
FILTER_GENOME	8	16	-	-	-	-	8	16	1600
FLEXBAR_PRE_RESOLUTION	2	2	-	-	-	-	-	-	-
FLEXBAR_POST_RESOLUTION	2	-	-	-	-	-	-	-	-
FRAGMENT_LENGTH	2	8	-	-	-	-	8	32	1600
GENETIC_ACCURACY	8	16	-	-	-	-	-	-	-
HAPLOTYPE_CALLER	16	64	-	32	64	-	32	128	1600
INLINE_UMIS	16	32	500	-	-	-	-	-	1600
JOIN_ZARR_STORES	16	32	-	-	-	-	-	-	-
JOIN_ZARR_STORES (GC)	8	16	-	-	-	-	-	-	-
LONG_CTRL_GENETIC_ACCURACY	1	1	-	-	-	-	-	-	-
MAKE_BED_FILES	1	1	-	-	-	-	-	-	-
MAKE_FASTA_INDEX	1	1	-	-	-	-	-	-	-
MAKE_GENOME_PRIMARY_ASSEMBLY	1	1	-	-	-	-	-	-	-
MAKE_GENOME_SIZES_NAMES	1	1	-	-	-	-	-	-	-
MAKE_INTERVAL_FILE	1	1	-	-	-	-	-	-	-
MAKE_MOD_REF_FILES	1	1	-	-	-	-	-	-	-
MAKE_NON_POSITIONAL_QTABLE	1	1	-	-	-	-	-	-	-
MAKE_STORE (chg_chh_contexts)	16	32	-	-	-	-	-	-	-
MAKE_STORE	8	16	-	-	-	-	-	-	-
MAKE_TARGET_INTERVALS	1	1	-	-	-	-	-	-	-
MERGE_DEDUP_STATS	2	2	-	-	-	-	-	-	-
MERGE_UMI_STATS	2	2	-	-	-	-	-	-	-
MODALITY_DMR	(dynamic allocation) 8	(dynamic allocation) 16	-	-	-	-	-	-	-
MODALITY_EXPORT	(dynamic allocation) 4	(dynamic allocation) 16	(dynamic allocation) 6	(dynamic allocation) 128	-	-	-	-	-
MULTIQC	2	8	-	-	-	-	-	-	-
MUTECT2	16	64	-	32	64	-	64	256	1600
ONE_STEP_BAMLET	8	8	-	16	16	800	16	16	1600
PICARD_GCBIAS	2	8	-	-	-	-	8	32	1600
PICARD_MARKDUPLICATES	1	4	-	-	-	-	-	-	-
PICARD_TARGETED_METRICS	2	8	-	8	32	-	8	32	-
PICARD_TARGETED_METRICS (twist methylome)	8	32	-	8	32	-	8	32	-
PIPELINE_REPORT	1	1	-	-	-	-	-	-	-
PRELUDE	32	32	-	64	64	-	96	96	1600
PRESEQ	4	8	-	-	-	-	-	-	-
QUALIMAP_BAMQC	4	16	-	-	-	-	8	32	1600
REDUCE_FASTA	1	1	-	-	-	-	-	-	-
REHEADER_VCF	1	1	-	-	-	-	-	-	-
SAMTOOLS_MERGE	8	2	-	-	-	-	-	-	-
SAMTOOLS_MERGE_INTERVALS	16	16	200	-	-	-	8	8	3200
SAMTOOLS_MERGE_LANES (Human genome)	8	8	-	8	8	800	8	8	3200
SAMTOOLS_MERGE_LANES (Mouse genome)	8	8	-	16	64	800	16	64	3200
SAMTOOLS_STATS	2	2	-	-	-	800	-	-	1600
SEQTK_SAMPLE	8	8	-	32	64	-	32	64	1600
SPLIT_FASTQS	1	1	-	-	-	800	-	-	1600
SPLIT_INTERVALS	1	4	-	-	-	-	-	-	-
SUMMARY_REPORT	2	8	-	-	-	-	-	-	-
TSS_BIAS	2	8	-	-	-	-	-	-	1600
UMI_DEDUP_GENOME	1	4	-	4	16	800	4	16	1600
UMI_DEDUP_LONG_CTRL	1	4	-	-	-	-	4	16	800

Reference pipeline module requirements

Module / Requirement type	CPU (cores)	RAM (GB)
COMBINE_GENOME_AND_CTRLS	1	2
MAKE_BWA_MEM2_INDEX	2	128
MAKE_BWA_MEM2_INDEX (>=4.6Gbp)	2	320
MAKE_FASTA_INDEX_GENOME	1	1
MAKE_FASTA_INDEX_SHORT_CTRLS	1	1
MAKE_FASTA_INDEX_LONG_CTRLS	1	1
MAKE_BED_FILES_GENOME	1	1
MAKE_GENOME_PRIMARY_ASSEMBLY	1	1
MAKE_BED_FILES_SHORT_CTRLS	1	1
MAKE_BED_FILES_LONG_CTRLS	1	1
MAKE_SHORT_CTRLS_SIZES_NAMES	1	1
MAKE_LONG_CTRLS_SIZES_NAMES	1	1
MAKE_GENOME_SIZES_NAMES	1	1
MAKE_MOD_REF_FILES	1	1
MAKE_DICT_GENOME	2	4
MAKE_DICT_COMBINED	2	4