Module Requirements#
Overview#
Below is a table detailing minimum resource requirements for each module in the duet and make_reference pipeline, relative to each profile. Your cloud tenancy or HPC cluster should be able to allocate a set of VMs/nodes with these resources.
The module specific settings and profiles can be found in the ./pipelines/duet/1.5.x/modules and ./pipelines/duet/1.5.x/conf directories under your current instance directory.
We do not recommend editing these files directly as any local change will not be reflected when you upgrade the pipeline.
You can edit resource requirements by referencing specific modules in the nextflow_override.config file in the biomodal script folder to accommodate available hardware resources.
Please see examples in section Set queue, CPU, RAM and DISK per module for more information on how to increase or decrease the resource requirements.
However, the duet pipeline is less likely to run successfully if the minimum resources are too restrictive.
Important
The module requirement is per sample per node on a HPC cluster node or cloud VM. You need to multiply the number of samples by the resource requirement if you intend to only use a single node or VM.
If you are running the duet or make_reference pipeline on a larger local node you can set a maximum ceiling for CPU and RAM per module.
Please see section Limit local resources in the documentation for information of how to use parameters to limit the resource requirements for the local and local_deep_seq profiles.
Module requirements#
Module / Requirement type |
CPU (cores) |
RAM (GB) |
Disk (GB) |
CPU (cores) |
RAM (GB) |
Disk (GB) |
CPU (cores) |
RAM (GB) |
Disk (GB) |
|---|---|---|---|---|---|---|---|---|---|
Profile name: |
default |
default |
default |
deep_seq |
deep_seq |
deep_seq |
super_seq |
super_seq |
super_seq |
ASM |
16 |
32 |
- |
- |
- |
- |
32 |
128 |
1600 |
BAMLET |
2 |
2 |
- |
16 |
16 |
800 |
16 |
16 |
1600 |
BAM_TO_CRAM |
6 |
6 |
- |
- |
- |
- |
- |
- |
- |
BWA_MEM2 (Human genome) |
16 |
64 |
- |
32 |
64 |
800 |
96 |
128 |
3200 |
BWA_MEM2 (Mouse genome) |
32 |
128 |
- |
32 |
128 |
800 |
64 |
256 |
3200 |
DEEPTOOLS_PLOTFINGERPRINT |
4 |
16 |
- |
- |
- |
- |
8 |
32 |
1600 |
DOWNSAMPLE_BAM |
2 |
2 |
- |
- |
- |
- |
- |
- |
- |
DQSPY |
8 |
32 |
- |
- |
- |
- |
- |
- |
- |
EPIQUANT_BAM2ZARR |
8 |
8 |
- |
16 |
16 |
- |
32 |
32 |
- |
EPIQUANT_MBIAS or GENOME_MBIAS |
4 |
4 |
- |
16 |
16 |
- |
32 |
32 |
- |
FASTQC |
2 |
4 |
- |
- |
- |
- |
- |
- |
- |
FILTER_BAM |
8 |
16 |
- |
- |
- |
- |
- |
- |
- |
FLEXBAR_PRE_RESOLUTION |
2 |
2 |
- |
- |
- |
- |
- |
- |
- |
GENETIC_ACCURACY |
8 |
16 |
- |
- |
- |
- |
- |
- |
- |
HAPLOTYPE_CALLER |
16 |
64 |
- |
32 |
64 |
- |
32 |
128 |
1600 |
INLINE_UMIS |
16 |
32 |
500 |
- |
- |
800 |
- |
- |
1600 |
JOIN_ZARR_STORES |
16 |
32 |
- |
- |
- |
- |
- |
- |
- |
JOIN_ZARR_STORES (GC) |
8 |
16 |
- |
- |
- |
- |
- |
- |
- |
LONG_CTRL_GENETIC_ACCURACY |
1 |
1 |
- |
- |
- |
- |
- |
- |
- |
MAKE_NON_POSITIONAL_QTABLE |
1 |
1 |
- |
- |
- |
- |
- |
- |
- |
MERGE_UMI_STATS |
2 |
2 |
- |
- |
- |
- |
- |
- |
- |
MODALITY_EXPORT |
(dynamic allocation) 4 |
(dynamic allocation) 16 |
(dynamic allocation) 6 |
(dynamic allocation) 128 |
- |
- |
- |
- |
- |
MULTIQC |
2 |
8 |
- |
- |
- |
- |
- |
- |
- |
MUTECT2 |
16 |
64 |
- |
32 |
64 |
- |
64 |
256 |
1600 |
PICARD_GCBIAS |
2 |
8 |
- |
- |
- |
- |
8 |
32 |
1600 |
PICARD_TARGETED_METRICS |
2 |
8 |
- |
8 |
32 |
- |
8 |
32 |
- |
PICARD_TARGETED_METRICS (twist methylome) |
8 |
32 |
- |
8 |
32 |
- |
8 |
32 |
- |
PIPELINE_REPORT |
1 |
1 |
- |
- |
- |
- |
- |
- |
- |
PRELUDE |
32 |
32 |
- |
64 |
64 |
- |
96 |
96 |
1600 |
PRESEQ |
4 |
8 |
- |
- |
- |
800 |
- |
- |
1600 |
PUBLISH_CRAMS |
- |
- |
- |
- |
- |
800 |
- |
- |
1600 |
QUALIMAP_BAMQC |
4 |
16 |
- |
- |
- |
- |
8 |
32 |
1600 |
SAMTOOLS_MERGE_INTERVALS |
16 |
16 |
200 |
- |
- |
- |
16 |
16 |
3200 |
SAMTOOLS_MERGE_LANES (Human genome) |
8 |
8 |
- |
8 |
16 |
800 |
8 |
16 |
3200 |
SAMTOOLS_MERGE_LANES (Mouse genome) |
8 |
8 |
- |
16 |
64 |
800 |
32 |
96 |
3200 |
SEQTK_SAMPLE |
8 |
8 |
- |
32 |
64 |
- |
32 |
64 |
1600 |
SPLIT_INTERVALS |
1 |
4 |
- |
- |
- |
- |
- |
- |
- |
TSS_BIAS |
2 |
8 |
- |
- |
- |
- |
- |
- |
1600 |
Reference pipeline module requirements#
Module / Requirement type |
CPU (cores) |
RAM (GB) |
|---|---|---|
COMBINE_GENOME_AND_CTRLS |
1 |
2 |
MAKE_BWAMEM2_INDEX |
2 |
128 |
MAKE_BWAMEM2_INDEX (>=4.6Gbp) |
2 |
320 |
MAKE_FASTA_INDEX_GENOME |
1 |
1 |
MAKE_FASTA_INDEX_SHORT_CTRLS |
1 |
1 |
MAKE_FASTA_INDEX_LONG_CTRLS |
1 |
1 |
MAKE_BED_FILES_GENOME |
1 |
1 |
MAKE_GENOME_PRIMARY_ASSEMBLY |
1 |
1 |
MAKE_BED_FILES_SHORT_CTRLS |
1 |
1 |
MAKE_BED_FILES_LONG_CTRLS |
1 |
1 |
MAKE_SHORT_CTRLS_SIZES_NAMES |
1 |
1 |
MAKE_LONG_CTRLS_SIZES_NAMES |
1 |
1 |
MAKE_GENOME_SIZES_NAMES |
1 |
1 |
MAKE_MOD_REF_FILES |
1 |
1 |
MAKE_DICT_GENOME |
2 |
4 |
MAKE_DICT_COMBINED |
2 |
4 |