DRAGEN
Illumina Connected Software
  • Overview
    • Illumina® DRAGEN™ Secondary Analysis
    • DRAGEN Applications
    • Deployment Options
  • Product Guides
    • DRAGEN v4.4
      • Getting Started
      • DRAGEN Host Software
        • DRAGEN Secondary Analysis
      • Clinical Research Workflows
        • DRAGEN Heme WGS Tumor Only Pipeline
          • Quick Start
          • Sample Sheets
            • Introduction
            • Requirements
            • Templates
          • Run Planning
            • Sample Sheet Creation in BaseSpace
            • Custom Config Support
          • DRAGEN Server App
            • Getting Started
            • Launching Analysis
            • Command Line Options
            • Output
            • Advanced Topics
              • Custom Workflow
              • Custom Config Support
              • Illumina Connected Insights
          • ICA Cloud App
            • Getting Started
            • Launching Analysis
            • Advanced Topics
              • Custom Workflow
              • Custom Config Support
              • Post Processing
              • Illumina Connected Insights
          • Analysis Output
          • Analysis Methods
          • Troubleshooting
        • DRAGEN Solid WGS Tumor Normal Pipeline
          • Quick Start
          • Sample Sheets
            • Introduction
            • Requirements
            • Templates
          • Run Planning
            • Sample Sheet Creation in BaseSpace
            • Custom Config Support
          • DRAGEN Server App
            • Quick Start
            • Getting Started
            • Launching Analysis
            • Command Line Options
            • Output
            • Advanced Topics
            • Custom Workflow
              • Custom Config Support
            • Illumina Connected Insights
          • ICA Cloud App
            • Getting Started
            • Launching Analysis
            • Output
            • Advanced Topics
              • Custom Workflow
              • Custom Config Support
              • Post Processing
              • Illumina Connected Insights
          • Analysis Output
          • Analysis Methods
          • Troubleshooting
      • DRAGEN Recipes
        • DNA Germline Panel UMI
        • DNA Germline Panel
        • DNA Germline WES UMI
        • DNA Germline WES
        • DNA Germline WGS UMI
        • DNA Germline WGS
        • DNA Somatic Tumor-Normal Solid Panel UMI
        • DNA Somatic Tumor-Normal Solid Panel
        • DNA Somatic Tumor-Normal Solid WES UMI
        • DNA Somatic Tumor-Normal Solid WES
        • DNA Somatic Tumor-Normal Solid WGS UMI
        • DNA Somatic Tumor-Normal Solid WGS
        • DNA Somatic Tumor-Only Heme WGS
        • DNA Somatic Tumor-Only Solid Panel UMI
        • DNA Somatic Tumor-Only Solid Panel
        • DNA Somatic Tumor-Only Solid WES UMI
        • DNA Somatic Tumor-Only Solid WES
        • DNA Somatic Tumor-Only Solid WGS UMI
        • DNA Somatic Tumor-Only Solid WGS
        • DNA Somatic Tumor-Only ctDNA Panel UMI
        • Illumina scRNA
        • Other scRNA prep
        • RNA Panel
        • RNA WTS
      • DRAGEN Reference Support
        • Prepare a Reference Genome
      • DRAGEN DNA Pipeline
        • DNA Mapping
        • Read Trimming
        • DRAGEN FASTQC
        • Sorting and Duplicate Marking
        • Small Variant Calling
          • ROH Caller
          • B-Allele Frequency Output
          • Somatic Mode
          • Pedigree Analysis
          • De Novo Small Variant Filtering
          • Autogenerated MD5SUM for VCF Files
          • Force Genotyping
          • Machine Learning for Variant Calling
          • Evidence BAM
          • Mosaic Detection
          • VCF Imputation
          • Multi-Region Joint Detection
        • Copy Number Variant Calling
          • Available pipelines
            • Germline CNV Calling (WGS/WES)
            • Germline CNV Calling ASCN (WGS)
            • Multisample Germline CNV Calling
            • Somatic CNV Calling ASCN (WGS)
            • Somatic CNV Calling WES
            • Somatic CNV Calling ASCN (WES)
          • Additional documentation
            • CNV Input
            • CNV Preprocessing
            • CNV Segmentation
            • CNV Output
            • CNV ASCN module
            • CNV with SV Support
            • Cytogenetics Modality
        • Repeat Expansion Detection
          • De Novo Repeat Expansion Detection
        • Targeted Caller
          • CYPDB6 Caller
          • CYP2D6 Caller
          • CYP21A2 Caller
          • GBA Caller
          • HBA Caller
          • LPA Caller
          • Rh Caller
          • SMN Caller
        • Structural Variant Calling
          • Structural Variant De Novo Quality Scoring
          • Structural Variant IGV Tutorial
        • VNTR Calling
        • Population Genotyping
        • Filter Duplicate Variants
        • Ploidy Calling
          • Ploidy Estimator
          • Ploidy Caller
        • Multi Caller
        • QC Metrics Reporting
        • JSON Metrics Reporting
        • HLA Typing
        • Biomarkers
          • Tumor Mutational Burden
          • Microsatellite Instability
          • Homologous Recombination Deficiency
          • BRCA Large Genomic Rearrangment
          • DRAGEN Fragmentomics
        • Downsampling
          • Fractional (Raw Reads) Downsampling
        • Unique Molecular Identifiers
        • Indel Re-aligner (Beta)
        • Star Allele Caller
        • High Coverage Analysis
        • CheckFingerprint
        • Population Haplotyping (Beta)
        • DUX4 Rearrangement Caller
      • DRAGEN RNA Pipeline
        • RNA Alignment
        • Gene Fusion Detection
        • Gene Expression Quantification
        • RNA Variant Calling
        • Splice Variant Caller
      • DRAGEN Single Cell Pipeline
        • Illumina PIPseq scRNA
        • Other scRNA Prep
        • scATAC
        • Single-Cell Multiomics
      • DRAGEN Methylation Pipeline
      • DRAGEN MRD Pipeline
      • DRAGEN Amplicon Pipeline
      • Explify Analysis Pipeline
        • Kmer Classifier
        • Kmer Classifier Database Builder
      • BCL conversion
      • Illumina Connected Annotations
      • ORA Compression
      • Command Line Options
        • Docker Requirements
      • DRAGEN Reports
      • Tools and Utilities
    • DRAGEN v4.3
      • Getting Started
      • DRAGEN Host Software
        • DRAGEN Secondary Analysis
      • DRAGEN Reference Support
        • Prepare a Reference Genome
      • DRAGEN DNA Pipeline
        • DNA Mapping
        • Read Trimming
        • DRAGEN FASTQC
        • Sorting and Duplicate Marking
        • Small Variant Calling
          • ROH Caller
          • B-Allele Frequency Output
          • Somatic Mode
          • Joint Analysis
          • De Novo Small Variant Filtering
          • Autogenerated MD5SUM for VCF Files
          • Force Genotyping
          • Machine Learning for Variant Calling
          • Evidence BAM
          • Mosaic Detection
          • VCF Imputation
          • Multi-Region Joint Detection
        • Copy Number Variant Calling
          • CNV Output
          • CNV with SV Support
          • Multisample CNV Calling
          • Somatic CNV Calling WGS
          • Somatic CNV Calling WES
          • Allele Specific CNV for Somatic WES CNV
        • Repeat Expansion Detection
          • De Novo Repeat Expansion Detection
        • Targeted Caller
          • CYPDB6 Caller
          • CYP2D6 Caller
          • CYP21A2 Caller
          • GBA Caller
          • HBA Caller
          • LPA Caller
          • Rh Caller
          • SMN Caller
        • Structural Variant Calling
          • Structural Variant De Novo Quality Scoring
        • VNTR Calling
        • Filter Duplicate Variants
        • Ploidy Calling
          • Ploidy Estimator
          • Ploidy Caller
        • Multi Caller
        • QC Metrics Reporting
        • HLA Typing
        • Biomarkers
          • Tumor Mutational Burden
          • Microsatellite Instability
          • Homologous Recombination Deficiency
          • BRCA Large Genomic Rearrangment
          • DRAGEN Fragmentomics
        • Downsampling
          • Fractional (Raw Reads) Downsampling
          • Effective Coverage Downsampling
        • Unique Molecular Identifiers
        • Indel Re-aligner (Beta)
        • Star Allele Caller
        • High Coverage Analysis
        • CheckFingerprint
        • Population Haplotyping (Beta)
        • DUX4 Rearrangement Caller
      • DRAGEN RNA Pipeline
        • RNA Alignment
        • Gene Fusion Detection
        • Gene Expression Quantification
        • RNA Variant Calling
        • Splice Variant Caller
      • DRAGEN Single-Cell Pipeline
        • scRNA
        • scATAC
        • Single-Cell Multiomics
      • DRAGEN Methylation Pipeline
      • DRAGEN Amplicon Pipeline
      • Explify Analysis Pipeline
        • Kmer Classifier
        • Kmer Classifier Database Builder
      • DRAGEN Recipes
        • DNA Germline Panel UMI
        • DNA Germline Panel
        • DNA Germline WES UMI
        • DNA Germline WES
        • DNA Germline WGS UMI
        • DNA Germline WGS
        • DNA Somatic Tumor-Normal Solid Panel UMI
        • DNA Somatic Tumor-Normal Solid Panel
        • DNA Somatic Tumor-Normal Solid WES UMI
        • DNA Somatic Tumor-Normal Solid WES
        • DNA Somatic Tumor-Normal Solid WGS UMI
        • DNA Somatic Tumor-Normal Solid WGS
        • DNA Somatic Tumor-Only Heme WGS
        • DNA Somatic Tumor-Only Solid Panel UMI
        • DNA Somatic Tumor-Only Solid Panel
        • DNA Somatic Tumor-Only Solid WES UMI
        • DNA Somatic Tumor-Only Solid WES
        • DNA Somatic Tumor-Only Solid WGS UMI
        • DNA Somatic Tumor-Only Solid WGS
        • DNA Somatic Tumor-Only ctDNA Panel UMI
        • RNA Panel
        • RNA WTS
      • BCL conversion
      • Illumina Connected Annotations
      • ORA Compression
      • Command Line Options
      • DRAGEN Reports
      • Tools and Utilities
  • Reference
    • DRAGEN Server
    • DRAGEN Multi-Cloud
      • DRAGEN on AWS
      • DRAGEN on AWS Batch
      • DRAGEN on Microsoft Azure
        • Run DRAGEN VM on Azure
      • DRAGEN on Microsoft Azure Batch
        • Azure Batch Run Modes
    • DRAGEN Licensing
      • DRAGEN Server Licensing
      • DRAGEN Cloud Licensing
    • DRAGEN Application Manager
    • Support
    • Resource Files
      • Noise Baselines
    • Supplementary Information
    • Troubleshooting
    • Citing DRAGEN software
    • Release Notes
    • Revision History
Powered by GitBook
On this page
  • Requirements
  • Supported Regions
  • Deployment Steps
  • Simple DRAGEN Run Example with AWS Batch

Was this helpful?

Export as PDF
  1. Reference
  2. DRAGEN Multi-Cloud

DRAGEN on AWS Batch

PreviousDRAGEN on AWSNextDRAGEN on Microsoft Azure

Last updated 2 days ago

Was this helpful?

DRAGEN on AWS Batch deployment guide was created by Illumina in collaboration with Amazon Web Services (AWS). This helps people to deploy popular technologies on AWS according to AWS best practices. If you’re unfamiliar with AWS Batch stacks, refer to the .

This deployment guide provides instructions for deploying the Illumina DRAGEN in the AWS Cloud using AWS Batch. The AWS Batch is a fully managed service that simplifies the process of running and scaling batch computing workloads in AWS cloud. For those who prefer using an AWS EC2 virtual machine for DRAGEN analyses, please use the DRAGEN Complete Suite.

Important: This template is provided as a starting point. Users are expected to tailor the cloudformation configuration, input data, and parameters to meet their specific workflow and requirements.

Requirements

To use DRAGEN on AWS Batch, the following are required:

  • Subscription to the DRAGEM AMI (marketplace or private)

  • s3 bucket

  • Quota for EC2 f1 or f2 instances

Supported Regions

DRAGEN on AWS Batch is available with Marketplace AMIs in the following regions for f1 and f2 instances:

  • us-east-1

  • us-west-2

  • eu-central-1

  • eu-west-1

  • ap-southeast-2

For BYOL users using a private AMI, the appropriate AMI ID must be specified in the template for their region. Additionally, the SupportedRegionRule may need to be removed.

Deployment Steps

  1. Modify the CloudFormation template to fit your needs (e.g., specify your private AMI).

  2. Upload the modified templates to your S3 bucket, e.g., s3://your-bucket-name/DRAGEN-on-AWS-Batch/.

  3. In the AWS Console, go to CloudFormation > Stacks > Create Stack, choose "With existing template"

  4. provide the S3 URL

    • Create a new VPC and deploy DRAGEN on AWS Batch

      • https://your-bucket-name.s3.<aws_region>.amazonaws.com/DRAGEN-on-AWS-Batch/templates/dragen-main.template.yaml

    • Deploy DRAGEN on AWS Batch in an existing VPC

      • https://your-bucket-name.s3.<aws_region>.amazonaws.com/DRAGEN-on-AWS-Batch/templates/dragen.template.yaml

  5. Configure stack settings:

    • Stack name: Provide a name for your stack

    • Availability Zones: select two AZs

    • Key pair: Use the key pair you want to use for SSH access to instances

    • Instance type: f2.6xlarge

    • Max vCPU: 24

    • Genomics Data Bucket: s3://your-bucket-genomic-data/

    • Quick Start S3 region: <aws_region>

    • Quick Start S3 Key Prefix: DRAGEN-on-AWS-Batch/

  6. Click Submit to launch the stack.

Simple DRAGEN Run Example with AWS Batch

After successful deployment, you can initiate a sample run:

cat > e2e-test.json << EOF
{
    "jobName": "e2e-job",
    "jobQueue": "dragen-queue",
    "jobDefinition": "dragen",
    "containerOverrides": {
        "vcpus": 24,
        "memory": 240000,
        "command": [
            "-f", "-r", "s3://your-bucket-genomic-data/ref/hg38-alt_masked.cnv.graph.hla.methyl_cg.rna-11-r5.0-2.tar.gz",
            "-1", "s3://your-bucket-genomic-data/input/NA24385-AJ-Son-R1-NS_S33_L001_R1_001.fastq.gz",
            "-2", "s3://your-bucket-genomic-data/input/NA24385-AJ-Son-R1-NS_S33_L001_R2_001.fastq.gz",
            "--RGID", "1",
            "--RGSM", "HG002",
            "--enable-bam-indexing", "true",
            "--enable-map-align-output", "true",
            "--enable-sort", "true",
            "--output-file-prefix", "RGMS",
            "--enable-map-align", "true",
            "--output-format", "BAM",
            "--output-directory", "s3://your-bucket-genomic-data/output/",
            "--enable-variant-caller", "true",
            "--lic-server", https://<ID>:<PASSWD>@license.dragen.illumina.com # requires for BYOL users
        ]
    },
    "retryStrategy": {
        "attempts": 1
    }
}
EOF
 
aws batch submit-job --cli-input-json file://e2e-test.json --region <aws_region>

Note:

  • Input files from the S3 bucket are automatically copied to the instance.

  • The job runs using local copies of the S3 input files along with the specified parameters.

  • Output files are first saved locally, then transferred to the designated S3 output folder.

  • Job status can be monitored via the AWS Batch console.

  • Logs are available in both CloudWatch and the S3 output folder.

Clone the repository: (include submodules as needed).

AWS Batch user guide
DRAGEN on AWS Batch deployment guide