Pipeline Outputs & Directory Structure¶
This document describes the pipeline output files and the tools used to generate them. The results are organized into logical subdirectories within the specified output directory (default: results/).
Example execution: nextflow run JD2112/milou --outdir results
1. Quality Control & Trimmed Data¶
1.1 Read Processing (FastQC & Trim Galore!)¶
This directory contains the initial quality checks and the adapter-trimmed sequencing reads.
| Path | Description |
|---|---|
read_processing/fastqc/<SAMPLE_ID>_R1_fastqc.html |
FastQC report containing an overview of sequencing quality. |
read_processing/fastqc/<SAMPLE_ID>_R1_fastqc.zip |
Raw data used for FastQC report generation. |
read_processing/trim_galore/<SAMPLE_ID>_R1_val_1.fq.gz |
High-quality, adapter-trimmed reads used for alignment by Trim Galore!. |
read_processing/trim_galore/<SAMPLE_ID>_R1.fastq.gz_trimming_report.txt |
Trimming report containing summary of operations performed by Trim Galore. |
2. Reference Genome Indexing¶
2.1 Bismark & BWA-meth Indices¶
Contains the indices required for bisulfite alignment. The content adapts based on whether the CPU (Bismark) or GPU (BWA-meth) track is used.
| Path | Description |
|---|---|
prepare_genome/bismark_genome_preparation/bismark_index/Bisulfite_Genome/ |
Directory containing the C->T and G->A converted indices created during CPU Indexing with Bismark. |
prepare_genome/bismark_genome_preparation/bismark_index/hg38.fa |
The reference genome FASTA sequence used for the index. |
parabricks_analysis/bwameth_index/ |
Directory containing the BWA-meth specific index files built for GPU execution. |
3. Read Alignment & BAM Processing¶
milou supports two alignment tracks. Outputs are stored in either bismark_analysis or parabricks_analysis.
3.1 CPU Track (Bismark Suite)¶
The standard path for bisulfite and EM-seq mapping and deduplication.
| Path | Description |
|---|---|
bismark_analysis/bismark_align/<SAMPLE_ID>_PE_report.txt |
Mapping efficiency statistics generated by Bismark. |
bismark_analysis/bismark_align/<SAMPLE_ID>_pe.bam |
Aligned reads produced by Bismark mapping. |
bismark_analysis/bismark_deduplicate/<SAMPLE_ID>_pe.deduplicated.bam |
Deduplicated aligned reads with PCR duplicates removed. |
bismark_analysis/bismark_deduplicate/<SAMPLE_ID>_pe.deduplication_report.txt |
Details on the duplicate removal process. |
bismark_analysis/samtools_sort/<SAMPLE_ID>.sorted.bam |
Sorted BAM file of the deduplicated aligned reads. |
bismark_analysis/samtools_index/<SAMPLE_ID>.sorted.bam.bai |
BAM index file for fast access to the sorted deduplicated reads. |
3.2 GPU Track (NVIDIA Clara Parabricks)¶
Ultra-fast mapping and deduplication using NVIDIA GPUs.
| Path | Description |
|---|---|
parabricks_analysis/parabricks_fq2bammeth/<SAMPLE_ID>.bam |
GPU-accelerated sorted and deduplicated BAM file produced by NVIDIA Parabricks. |
parabricks_analysis/samtools_index/<SAMPLE_ID>.bam.bai |
BAM index file for the Parabricks aligned BAM. |
4. Methylation Calling & Extraction¶
4.1 Bismark Extractor (CPU Track)¶
Standard extraction of C->T conversion counts.
| Path | Description |
|---|---|
bismark_analysis/bismark_methylation_extractor/<SAMPLE_ID>_pe.deduplicated.bismark.cov.gz |
Compressed coverage file detailing CpG methylation counts (Methylated/Unmethylated). |
bismark_analysis/bismark_methylation_extractor/<SAMPLE_ID>_pe.deduplicated.bedGraph.gz |
Genome browser compatible coverage levels and methylation percentages. |
bismark_analysis/bismark_methylation_extractor/<SAMPLE_ID>_pe.deduplicated_splitting_report.txt |
Extractor splitting report with basic statistics. |
bismark_analysis/bismark_report/<SAMPLE_ID>_bismark_report.html |
Bismark summary HTML report visualizing mapping and extraction metrics. |
4.2 MethylDackel Extractor (GPU Track)¶
High-throughput extraction directly from GPU-aligned BAMs.
| Path | Description |
|---|---|
parabricks_analysis/methyldackel_extract/<SAMPLE_ID>_CpG.bedGraph |
BedGraph file containing high-throughput CpG methylation metrics extracted by MethylDackel. |
5. Post-Alignment Quality Mapping¶
5.1 Qualimap Coverage & Insert Size Metrics¶
Comprehensive metrics on mapping coverage and insert sizes using Qualimap.
| Path | Description |
|---|---|
qualimap/<SAMPLE_ID>/qualimapReport.html |
Comprehensive HTML report assessing mapping quality and genome coverage by Qualimap. |
qualimap/<SAMPLE_ID>/genome_results.txt |
Text file containing genome-wide quality metrics. |
qualimap/<SAMPLE_ID>/images_qualimapReport/genome_coverage_histogram.png |
Histogram plot showing depth of coverage distribution. |
qualimap/<SAMPLE_ID>/images_qualimapReport/genome_insert_size_histogram.png |
Histogram plot showing the distribution of read insert sizes. |
6. Differential Methylation Analysis¶
6.1 Statistical Differential Engines (DSS, edgeR, methylKit)¶
Statistical results identifying significantly different CpG sites between groups using DSS, EdgeR, and MethylKit.
| Path | Description |
|---|---|
differential_methylation/edger_analysis/EdgeR_group_<COMPARISON>_coverage3.csv |
Table containing EdgeR Log2 Fold Change, P-values, and FDR for differentially methylated regions. |
differential_methylation/edger_analysis/edger_log.txt |
Execution log for the EdgeR analysis step. |
differential_methylation/methylkit_analysis/MethylKit_<COMPARISON>.csv |
Differential methylation analysis results generated by MethylKit. |
differential_methylation/methylkit_analysis/methylkit_log.txt |
Execution log for the MethylKit analysis step. |
differential_methylation/dss_analysis/DSS_group_<COMPARISON>_coverage3.csv |
Table containing DSS Difference, P-values, and FDR for differentially methylated regions. |
differential_methylation/dss_analysis/dss_log.txt |
Execution log for the DSS analysis step. |
7. Functional Annotation & Visualizations¶
7.1 Gene Annotation & Locus Mapping¶
| Path | Description |
|---|---|
result_analysis/annotate_results/EdgeR_group_<COMPARISON>_annotated.csv |
DMRs fully annotated with gene names, overlaps, and distances to TSS. |
7.2 Volcano & MA Plots¶
| Path | Description |
|---|---|
result_analysis/post_processing_edger/<COMPARISON>/edger_volcano_plot.png |
Volcano plot showing EdgeR biological (Fold Change) vs. statistical (P-value) significance. |
result_analysis/post_processing_edger/<COMPARISON>/edger_summary_stats.csv |
Summary statistics of EdgeR hyper- and hypo-methylated regions. |
result_analysis/post_processing_edger/<COMPARISON>/edger_ma_or_scatter_plot.png |
Visualizes global consistency of methylation levels between groups for EdgeR. |
result_analysis/post_processing_methylkit/<COMPARISON>/methylkit_volcano_plot.png |
Volcano plot for MethylKit differential methylation results. |
result_analysis/post_processing_methylkit/<COMPARISON>/methylkit_summary_stats.csv |
Summary statistics of MethylKit hyper- and hypo-methylated regions. |
result_analysis/post_processing_dss/<COMPARISON>/dss_volcano_plot.png |
Volcano plot showing DSS biological (Methylation Difference) vs. statistical (FDR) significance. |
result_analysis/post_processing_dss/<COMPARISON>/dss_summary_stats.csv |
Summary statistics of DSS hyper- and hypo-methylated regions. |
result_analysis/post_processing_dss/<COMPARISON>/dss_ma_or_scatter_plot.png |
Visualizes global consistency of methylation levels between groups for DSS. |
7.3 Gene Ontology (GO) & Pathway Enrichment¶
| Path | Description |
|---|---|
result_analysis/go_analysis_edger/<COMPARISON>_go_enrichment_results.csv |
Table containing Gene Ontology enrichment analysis results from EdgeR DMRs using clusterProfiler. |
result_analysis/go_analysis_edger/<COMPARISON>_gochord_plot.png |
Chord diagram depicting relationships between genes and EdgeR GO terms. |
result_analysis/go_analysis_methylkit/<COMPARISON>_go_enrichment_results.csv |
Table containing Gene Ontology enrichment analysis results from MethylKit DMRs. |
result_analysis/go_analysis_dss/<COMPARISON>_go_enrichment_results.csv |
Table containing Gene Ontology enrichment analysis results from DSS DMRs. |
8. Unified Results Layer & Consensus Scoring¶
The consolidated analytical layer synthesizing orthogonal differential callers via the \(\pi\)-value framework.
8.1 Multi-Method Consensus Rankings (π-Value)¶
| Path | Description |
|---|---|
unified_layer/unified_consensus_ranking.csv |
Master candidate gene table prioritized by the cross-method \(\pi\)-value score ($\pi = \overline{ |
unified_layer/consensus_pi_value_ranking.tsv |
Tab-delimited candidate ranking with individual engine metrics (DSS, edgeR, methylKit). |
unified_layer/consensus_genes.csv |
List of high-confidence genes supported by cross-caller agreement (filtered by majority voting in clinical mode). |
unified_layer/dml_consensus_summary.csv |
Summary table of CpG-level and DMR-level concordance metrics. |
9. Clinical Reporting Layer¶
Diagnostic-ready reports and compliance provenance generated by the Quarto reporting engine.
9.1 Interactive Quarto Reports & Audit Artifacts¶
| Path | Description |
|---|---|
clinical_reporting/milou_clinical_report.html |
Interactive, standalone clinical summary report with integrated MultiQC metrics, consensus rankings, and interactive volcano plots. |
clinical_reporting/milou_clinical_report.pdf |
Print-ready, archival publication/clinical PDF summary report. |
clinical_reporting/sha256_checksums.txt |
Cryptographic SHA256 audit log verifying input FASTQ integrity. |
clinical_reporting/qc_gate_audit.json |
Automated target coverage and conversion quality pass/fail evaluation log. |
10. MultiQC & Metadata¶
Aggregates all logs into a single interactive HTML report.
10.1 Aggregated MultiQC Dashboard & Software Provenance¶
| Path | Description |
|---|---|
multiqc/multiqc_report.html |
Aggregated report of all QC and processing steps into a single, interactive dashboard using MultiQC. |
multiqc/multiqc_data/multiqc_bismark_alignment.txt |
Raw data used for the MultiQC Bismark alignment module. |
multiqc/multiqc_data/multiqc_general_stats.txt |
General MultiQC stats overview in a tabular text format. |
multiqc/multiqc_data/multiqc_data.json |
JSON format containing all metrics aggregated by MultiQC. |
multiqc/versions.yml |
Software versions of tools used during the pipeline run. |
multiqc/software_versions.csv |
A CSV summary table of all tools and their respective versions retrieved during the run. |
11. Pipeline Runtime Information¶
Technical reports on resource usage and execution flow.
11.1 Execution Reports, Traces & DAGs¶
| Path | Description |
|---|---|
pipeline_info/execution_report.html |
HTML report containing Nextflow performance metrics and task executions. |
pipeline_info/execution_timeline.html |
Task timeline report detailing execution overlaps and delays. |
pipeline_info/execution_trace.txt |
Trace file specifying resource usage (memory, CPU, time) per process. |
pipeline_info/pipeline_dag.html |
Interactive DAG visualization of the workflow. |