[med-svn] [Git][med-team/flexbar][upstream] New upstream version 3.5.0
Andreas Tille
gitlab at salsa.debian.org
Sat Jul 27 20:32:02 BST 2019
Andreas Tille pushed to branch upstream at Debian Med / flexbar
Commits:
4c5fb050 by Andreas Tille at 2019-07-27T19:27:45Z
New upstream version 3.5.0
- - - - -
14 changed files:
- CMakeLists.txt
- README.md
- + galaxy/flexbar.pl
- + galaxy/flexbar.xml
- src/Flexbar.cpp
- src/FlexbarIO.h
- src/FlexbarTypes.h
- src/Options.h
- src/PairedAlign.h
- src/PairedOutput.h
- src/QualTrimming.h
- src/SeqAlign.h
- src/SeqAlignPair.h
- src/SeqInput.h
Changes:
=====================================
CMakeLists.txt
=====================================
@@ -2,7 +2,7 @@ cmake_minimum_required( VERSION 2.8.2 )
project( FLEXBAR )
-set( SEQAN_APP_VERSION "3.4.0" )
+set( SEQAN_APP_VERSION "3.5.0" )
include_directories( ${FLEXBAR_SOURCE_DIR}/include )
# link_directories( ${FLEXBAR_SOURCE_DIR}/lib )
=====================================
README.md
=====================================
@@ -1,6 +1,6 @@
## Flexbar – flexible barcode and adapter removal
-The program Flexbar preprocesses high-throughput sequencing data efficiently. It demultiplexes barcoded runs and removes adapter sequences. Moreover, trimming and filtering features are provided. Flexbar increases read mapping rates and improves genome as well as transcriptome assemblies. Unique molecular identifiers can be extracted in a flexible way. The program supports sequencing data in fasta and fastq format, e.g. from the Illumina platform.
+The program Flexbar preprocesses high-throughput sequencing data efficiently. It demultiplexes barcoded runs and removes adapter sequences. Several adapter removal presets for Illumina libraries are included. Flexbar computes exact overlap alignments using SIMD and multicore parallelism. Moreover, trimming and filtering features are provided, e.g. trimming of homopolymers at read ends. Flexbar increases read mapping rates and improves genome as well as transcriptome assemblies. Unique molecular identifiers can be extracted in a flexible way. The software supports data in fasta and fastq format from multiple sequencing platforms.
Refer to the [manual](https://github.com/seqan/flexbar/wiki) or contact [Johannes Roehr](https://github.com/jtroehr) for support with this application.
@@ -22,7 +22,15 @@ See article on [PubMed](https://www.ncbi.nlm.nih.gov/pubmed/24832523)
### Download
-Flexbar source code as well as binaries for Linux and Mac OS can be downloaded on the [release](https://github.com/seqan/flexbar/releases) page. Please follow instructions for building or setup of binaries below. Additionally, Flexbar is available via package manager on Debian systems and in Conda. Versions before 2.4 can be found on the [old](https://sourceforge.net/projects/flexbar) page.
+Flexbar source code as well as binaries for Linux and Mac OS can be downloaded on the [release](https://github.com/seqan/flexbar/releases) page. Please follow instructions for building or setup of binaries below. Additionally, Flexbar is available via package manager on Debian systems, in Homebrew, and in Bioconda. Versions before 2.4 can be found on the [old](https://sourceforge.net/projects/flexbar) page.
+
+Installation with package managers:
+
+* Debian: `sudo apt install flexbar`
+* Homebrew: `brew install brewsci/science/flexbar`
+* Bioconda: `conda install -c bioconda flexbar`
+
+To get the latest version and best performance consider to build Flexbar from source.
### Building from source
@@ -30,20 +38,20 @@ Flexbar source code as well as binaries for Linux and Mac OS can be downloaded o
Make sure that `cmake` is available, as well as development and runtime files of the TBB library 4.0 or later (Intel Threading Building Blocks). For example on Debian systems, install the packages `libtbb-dev` and `libtbb2`. Furthermore, the SeqAn library and a compiler that supports C++14 is required:
* Get SeqAn library version 2.4.0 [here](https://github.com/seqan/seqan/releases/download/seqan-v2.4.0/seqan-library-2.4.0.tar.xz)
-* Download Flexbar 3.4 source code [release](https://github.com/seqan/flexbar/releases)
+* Download Flexbar 3.5.0 source code [release](https://github.com/seqan/flexbar/releases)
Decompress both files:
- tar xzf flexbar-3.4.0.tar.gz
+ tar xzf flexbar-3.5.0.tar.gz
tar xJf seqan-library-2.4.0.tar.xz
Move SeqAn include folder to Flexbar:
- mv seqan-library-2.4.0/include flexbar-3.4.0
+ mv seqan-library-2.4.0/include flexbar-3.5.0
Use these commands for building:
- cd flexbar-3.4.0
+ cd flexbar-3.5.0
cmake .
make
@@ -57,12 +65,12 @@ For execution of provided Flexbar binaries, the corresponding TBB library has to
#### Linux
Adjust lib search path to include the absolute path of the Flexbar directory containing the lib file libtbb.so.2 for the current terminal session, or permanently in shell startup scripts:
- export LD_LIBRARY_PATH=/path/FlexbarDir:$LD_LIBRARY_PATH
+ export LD_LIBRARY_PATH=/YourPath/flexbar-3.5.0-linux:$LD_LIBRARY_PATH
#### Mac OS
It applies the same as for Linux. Make the file libtbb.dylib available by setting the lib search path:
- export DYLD_LIBRARY_PATH=/path/FlexbarDir:$DYLD_LIBRARY_PATH
+ export DYLD_LIBRARY_PATH=/YourPath/flexbar-3.5.0-macos:$DYLD_LIBRARY_PATH
### Program usage
@@ -95,7 +103,7 @@ To remove adapter sequences from single-end reads, specify a file with adapters
For paired-end libraries, specify both files with paired reads and a fasta file with adapters for removal. Given adapters are trimmed in right mode per default. It is recommended to activate the pair overlap detection in case of standard paired reads. This increases the sensitivity by removing very short parts of adapters if an overlap is detected for a pair.
- flexbar -r r1.fq -p r2.fq -a adapters.fa -ap ON
+ flexbar -r r1.fq -p r2.fq -a a1.fa -a2 a2.fa -ap ON
#### Adapter removal presets
=====================================
galaxy/flexbar.pl
=====================================
@@ -0,0 +1,86 @@
+#!/usr/bin/env perl
+
+# Flexbar wrapper for Galaxy tool definition, version 3.5.0
+# Author: Johannes Roehr
+
+use warnings;
+use strict;
+
+my $format;
+my @inFiles;
+my @outFiles;
+
+my $compression = "";
+
+foreach(0..$#ARGV){
+ my $arg = $ARGV[$_];
+
+ if($arg =~ /\.(fastq\w+)$/ || $arg =~ /\.(fastq\w+\.gz)$/ || $arg =~ /\.(fastq\w+\.bz2)$/){
+
+ if(defined $format && $format ne $1){
+ warn "Read files should have the same format.\n";
+ exit 1;
+ }
+ $format = $1;
+
+ my $file = $arg;
+
+ $arg =~ s/\.fastq\w+$/\.fastq/;
+ $arg =~ s/\.fastq\w+\.gz$/\.fastq\.gz/;
+ $arg =~ s/\.fastq\w+\.bz2$/\.fastq\.bz2/;
+
+ $compression = "GZ" if $arg =~ /\.fastq\.gz$/;
+ $compression = "BZ2" if $arg =~ /\.fastq\.bz2$/;
+
+ $ARGV[$_] = $arg;
+
+ if($arg =~ /\.dat_input\w\.fastq$/ || $arg =~ /\.dat_input\w\.fastq\.gz$/ || $arg =~ /\.dat_input\w\.fastq\.bz2$/){
+ push @inFiles, $arg;
+ rename $file, $arg;
+ }
+
+ push @outFiles, $arg if $arg =~ /\.dat\.fastq$/ || $arg =~ /\.dat\.fastq\.gz$/ || $arg =~ /\.dat\.fastq\.bz2$/;
+ }
+}
+
+my $barcoded = 0;
+
+$barcoded = 1 if $ARGV[$#ARGV] =~ /barcoded$/;
+
+my $call = join " ", @ARGV[0..($#ARGV - $barcoded)];
+
+# $call = $call ." --zip-output ". $compression if $barcoded && $compression ne "";
+
+system $call and exit 1;
+
+
+unlink $_ or warn "Could not unlink $_: $!" foreach(@inFiles);
+
+if($barcoded){
+ $format =~ s/\.gz//;
+ $format =~ s/\.bz2//;
+
+ foreach(<$ARGV[$#ARGV]/flexbarOut*.fastq*>){
+
+ my $file = $_;
+
+ s/fastq$/$format/;
+ # s/fastq\.gz$/$format/;
+ # s/fastq\.bz2$/$format/;
+
+ rename $file, $_;
+ }
+}
+else{
+ foreach(@outFiles){
+
+ my $file = $_;
+
+ s/\.fastq$//;
+ s/\.fastq\.gz$//;
+ s/\.fastq\.bz2$//;
+
+ rename $file, $_;
+ }
+}
+
=====================================
galaxy/flexbar.xml
=====================================
@@ -0,0 +1,739 @@
+
+<!-- Flexbar tool definition for Galaxy, version 3.5.0 -->
+<!-- Author: Johannes Roehr -->
+
+
+<tool id="flexbar" name="Flexbar" version="3.5.0">
+
+ <description>- flexible barcode and adapter removal</description>
+
+ <requirements>
+ <requirement type="package" version="3.5.0">flexbar</requirement>
+ </requirements>
+
+ <version_command>flexbar --versions</version_command>
+
+ <command detect_errors="aggressive">
+
+ <![CDATA[
+
+ #if $reads.mode == "se":
+ ln -s '$reads.rs' '$outlog'_input1.'$reads.rs.extension' &&
+ #elif $reads.mode == "pe":
+ ln -s '$reads.r1' '$outlog'_input1.'$reads.r1.extension' &&
+ ln -s '$reads.r2' '$outlog'_input2.'$reads.r2.extension' &&
+ #else:
+ ln -s '$reads.rc.forward' '$outlog'_input1.'$reads.rc.forward.extension' &&
+ ln -s '$reads.rc.reverse' '$outlog'_input2.'$reads.rc.reverse.extension' &&
+ #end if
+
+ #if $cBarcodes.select == "on":
+
+ mkdir flexbar_barcoded &&
+
+ #if $cBarcodes.bAdvanced.cbReads.select == "yes":
+ ln -s '$cBarcodes.bAdvanced.cbReads.bReads' '$outlog'_input3.'$cBarcodes.bAdvanced.cbReads.bReads.extension' &&
+ #end if
+ #end if
+
+
+ '$__tool_directory__/flexbar.pl' flexbar
+
+ --threads \${GALAXY_SLOTS:-1}
+
+ #if $reads.mode == "se":
+ --reads '$outlog'_input1.'$reads.rs.extension'
+ #elif $reads.mode == "pe":
+ --reads '$outlog'_input1.'$reads.r1.extension'
+ --reads2 '$outlog'_input2.'$reads.r2.extension'
+ #else:
+ --reads '$outlog'_input1.'$reads.rc.forward.extension'
+ --reads2 '$outlog'_input2.'$reads.rc.reverse.extension'
+ #end if
+
+ --max-uncalled $maxUncalled
+ --min-read-length $minReadLen
+
+ #if $trimEnds.select == "on":
+ --pre-trim-left $trimEnds.trimLeft
+ --pre-trim-right $trimEnds.trimRight
+ #end if
+
+
+ #if $cTrimQual.select != "off":
+ #if $reads.mode == "se":
+ #if $reads.rs.ext == "fastqsanger":
+ --qtrim-format sanger
+ #end if
+ #if $reads.rs.ext == "fastqsolexa":
+ --qtrim-format solexa
+ #end if
+ #if $reads.rs.ext == "fastqillumina":
+ --qtrim-format i1.3
+ #end if
+ #elif $reads.mode == "pe":
+ #if $reads.r1.ext == "fastqsanger":
+ --qtrim-format sanger
+ #end if
+ #if $reads.r1.ext == "fastqsolexa":
+ --qtrim-format solexa
+ #end if
+ #if $reads.r1.ext == "fastqillumina":
+ --qtrim-format i1.3
+ #end if
+ #else:
+ #if $reads.rc.forward.ext == "fastqsanger":
+ --qtrim-format sanger
+ #end if
+ #if $reads.rc.forward.ext == "fastqsolexa":
+ --qtrim-format solexa
+ #end if
+ #if $reads.rc.forward.ext == "fastqillumina":
+ --qtrim-format i1.3
+ #end if
+ #end if
+ #end if
+
+ #if $cTrimQual.select == "tail":
+ --qtrim TAIL
+ --qtrim-threshold $cTrimQual.trimQualTail
+ $cTrimQual.trimQualPostTail
+ #end if
+ #if $cTrimQual.select == "win":
+ --qtrim WIN
+ --qtrim-threshold $cTrimQual.trimQualWin
+ --qtrim-win-size $cTrimQual.trimQualWinSize
+ $cTrimQual.trimQualPostWin
+ #end if
+ #if $cTrimQual.select == "bwa":
+ --qtrim BWA
+ --qtrim-threshold $cTrimQual.trimQualBWA
+ $cTrimQual.trimQualPostBWA
+ #end if
+
+
+ #if $cBarcodes.select == "on":
+
+ --target flexbar_barcoded/flexbarOut
+
+ #if $cBarcodes.ccBarcodes.select == "file":
+ --barcodes $cBarcodes.ccBarcodes.barcodeFile
+ #end if
+ #if $cBarcodes.ccBarcodes.select == "files":
+ --barcodes $cBarcodes.ccBarcodes.barcodeFile1
+ --barcodes2 $cBarcodes.ccBarcodes.barcodeFile2
+ #end if
+
+ --barcode-error-rate $cBarcodes.bThresh
+
+ --barcode-trim-end $cBarcodes.bTrimEnd
+
+ #if $cBarcodes.bAdvanced.cbReads.select == "yes":
+ --barcode-reads '$outlog'_input3.'$cBarcodes.bAdvanced.cbReads.bReads.extension'
+ #end if
+
+ #if $cBarcodes.bAdvanced.cbMinOverlap.select == "yes":
+ --barcode-min-overlap $cBarcodes.bAdvanced.cbMinOverlap.bMinOverlap
+ #end if
+
+ $cBarcodes.bAdvanced.bKeep
+ $cBarcodes.bAdvanced.bUnassigned
+
+ #if $cBarcodes.bAdvanced.cbTailLen.select == "yes":
+ --barcode-tail-length $cBarcodes.bAdvanced.cbTailLen.bTailLen
+ #end if
+
+ #if $cBarcodes.bAdvanced.cbAlignScores.select == "yes":
+ --barcode-match $cBarcodes.bAdvanced.cbAlignScores.bMatch
+ --barcode-mismatch $cBarcodes.bAdvanced.cbAlignScores.bMismatch
+ --barcode-gap $cBarcodes.bAdvanced.cbAlignScores.bGap
+ #end if
+ #end if
+
+
+ #if $cAdapters.select == "on":
+
+ #if $cAdapters.ccAdapters.select == "file":
+ --adapters '$cAdapters.ccAdapters.adapterFile'
+ #end if
+ #if $cAdapters.ccAdapters.select == "files":
+ --adapters '$cAdapters.ccAdapters.adapterFile1'
+ --adapters2 '$cAdapters.ccAdapters.adapterFile2'
+ #end if
+ #if $cAdapters.ccAdapters.select == "preset":
+ --adapter-preset '$cAdapters.ccAdapters.adapterPreset'
+ #end if
+ #if $cAdapters.ccAdapters.select == "seq":
+ --adapter-seq $cAdapters.ccAdapters.adapterSeq
+ #end if
+
+ --adapter-trim-end $cAdapters.aTrimEnd
+ --adapter-min-overlap $cAdapters.aMinOverlap
+ --adapter-error-rate $cAdapters.aThresh
+
+ #if $cAdapters.cPairOverlap.select == "on":
+ --adapter-pair-overlap $cAdapters.cPairOverlap.aPairOverlap
+ --adapter-min-poverlap $cAdapters.cPairOverlap.aMinPoverlap
+ #end if
+
+ #if $cAdapters.aAdvanced.caTailLen.select == "yes":
+ --adapter-tail-length $cAdapters.aAdvanced.caTailLen.aTailLen
+ #end if
+
+ #if $cAdapters.aAdvanced.cRevComp.select == "on":
+ --adapter-revcomp ON
+ #if $cAdapters.aAdvanced.cRevComp.aRevCompEnd != "off":
+ --adapter-revcomp-end $cAdapters.aAdvanced.cRevComp.aRevCompEnd
+ #end if
+ #end if
+ #if $cAdapters.aAdvanced.cRevComp.select == "only":
+ --adapter-revcomp ONLY
+ #end if
+
+ $cAdapters.aAdvanced.aReadSet
+
+ --adapter-cycles $cAdapters.aAdvanced.aCycles
+
+ #if $cAdapters.aAdvanced.caAlignScores.select == "yes":
+ --adapter-match $cAdapters.aAdvanced.caAlignScores.aMatch
+ --adapter-mismatch $cAdapters.aAdvanced.caAlignScores.aMismatch
+ --adapter-gap $cAdapters.aAdvanced.caAlignScores.aGap
+ #end if
+ #end if
+
+
+ #if $hTrim.select == "on":
+ #if $hTrim.hTrimLeft != "":
+ --htrim-left $hTrim.hTrimLeft
+ #end if
+ #if $hTrim.hTrimRight != "":
+ --htrim-right $hTrim.hTrimRight
+ #end if
+
+ --htrim-min-length $hTrim.hMinLength
+ --htrim-error-rate $hTrim.hErrorRate
+
+ #if $hTrim.hAdvanced.hMinLength2 != "":
+ --htrim-min-length2 $hTrim.hAdvanced.hMinLength2
+ #end if
+ #if $hTrim.hAdvanced.hMaxLength != "":
+ --htrim-max-length $hTrim.hAdvanced.hMaxLength
+ #end if
+
+ $hTrim.hAdvanced.hMaxFirst
+ $hTrim.hAdvanced.hTrimAdapter
+ #end if
+
+
+ #if $cTrimLen.select == "on":
+ --post-trim-length $cTrimLen.trimLen
+ #end if
+
+ #if $cAddOptions.select == "show":
+ $cAddOptions.logLevel
+ $cAddOptions.numTags
+ $cAddOptions.remTags
+ $cAddOptions.rndTags
+ $cAddOptions.singleReadsP
+
+ #if $cAddOptions.cReadBundle.select == "show":
+ --bundle $cAddOptions.cReadBundle.bundle
+
+ #if $cAddOptions.cReadBundle.bundles != "":
+ --bundles $cAddOptions.cReadBundle.bundles
+ #end if
+ #end if
+ #end if
+
+ #if $cBarcodes.select == "off":
+ #if $reads.mode == "se":
+ --output-reads '$rsout'.'$reads.rs.extension'
+ #elif $reads.mode == "pe":
+ --output-reads '$r1out'.'$reads.r1.extension'
+ --output-reads2 '$r2out'.'$reads.r2.extension'
+ #else:
+ --output-reads '$rcout.forward'.'$reads.rc.forward.extension'
+ --output-reads2 '$rcout.reverse'.'$reads.rc.reverse.extension'
+ #end if
+ #end if
+
+ --output-log '$outlog'
+
+ #if $cBarcodes.select == "on":
+ flexbar_barcoded
+ #end if
+
+ ]]>
+ </command>
+
+
+ <inputs>
+
+ <conditional name="reads">
+ <param name="mode" type="select" label="Sequencing reads">
+ <option value="se">Single read file</option>
+ <option value="pe" selected="true">Paired read files</option>
+ <option value="pc">Paired read collection</option>
+ </param>
+ <when value="se">
+ <param name="rs" type="data" format="fastqsanger,fastqsolexa,fastqillumina,fastqsanger.gz,fastqsolexa.gz,fastqillumina.gz,fastqsanger.bz2,fastqsolexa.bz2,fastqillumina.bz2" label="Fastq read file"/>
+ </when>
+ <when value="pe">
+ <param name="r1" type="data" format="fastqsanger,fastqsolexa,fastqillumina,fastqsanger.gz,fastqsolexa.gz,fastqillumina.gz,fastqsanger.bz2,fastqsolexa.bz2,fastqillumina.bz2" label="Fastq read file 1"/>
+ <param name="r2" type="data" format="fastqsanger,fastqsolexa,fastqillumina,fastqsanger.gz,fastqsolexa.gz,fastqillumina.gz,fastqsanger.bz2,fastqsolexa.bz2,fastqillumina.bz2" label="Fastq read file 2"/>
+ </when>
+ <when value="pc">
+ <param name="rc" format="fastqsanger,fastqsolexa,fastqillumina,fastqsanger.gz,fastqsolexa.gz,fastqillumina.gz,fastqsanger.bz2,fastqsolexa.bz2,fastqillumina.bz2" type="data_collection" collection_type="paired" label="Fastq pair collection"/>
+ </when>
+ </conditional>
+
+ <param name="maxUncalled" size="4" type="integer" value="0" label="1) Maximum uncalled bases"/>
+
+ <conditional name="trimEnds">
+ <param name="select" type="select" label="2) Trimming of read ends">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <param name="trimLeft" size="4" type="integer" value="0" label="Left"/>
+ <param name="trimRight" size="4" type="integer" value="0" label="Right" help="trims specified number of bases at ends"/>
+ </when>
+ </conditional>
+
+ <conditional name="cTrimQual">
+ <param name="select" type="select" label="3) Quality-based trimming">
+ <option value="off" selected="true">Off</option>
+ <option value="tail">Tail</option>
+ <option value="win">Window</option>
+ <option value="bwa">BWA</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="tail">
+ <param name="trimQualTail" size="4" type="integer" value="20" label="Quality threshold" help="trim right end until specified quality or higher"/>
+
+ <param name="trimQualPostTail" type="select" label="After adapter removal">
+ <option value="" selected="true">No</option>
+ <option value="--qtrim-post-removal">Yes</option>
+ </param>
+ </when>
+ <when value="win">
+ <param name="trimQualWin" size="4" type="integer" value="20" label="Quality threshold"/>
+ <param name="trimQualWinSize" size="4" type="integer" value="5" label="Window length" help="sliding window average quality approach from left end"/>
+
+ <param name="trimQualPostWin" type="select" label="After adapter removal">
+ <option value="" selected="true">No</option>
+ <option value="--qtrim-post-removal">Yes</option>
+ </param>
+ </when>
+ <when value="bwa">
+ <param name="trimQualBWA" size="4" type="integer" value="20" label="Quality threshold" help="trimming approach used by BWA"/>
+
+ <param name="trimQualPostBWA" type="select" label="After adapter removal">
+ <option value="" selected="true">No</option>
+ <option value="--qtrim-post-removal">Yes</option>
+ </param>
+ </when>
+ </conditional>
+
+
+ <conditional name="cBarcodes">
+ <param name="select" type="select" label="4) Barcode detection">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <conditional name="ccBarcodes">
+ <param name="select" type="select" label="Barcodes">
+ <option value="file" selected="true">Barcode file</option>
+ <option value="files">Two barcode files</option>
+ </param>
+ <when value="file">
+ <param format="fasta" name="barcodeFile" type="data" label="Fasta file with barcodes for reads 1"/>
+ </when>
+ <when value="files">
+ <param format="fasta" name="barcodeFile1" type="data" label="Fasta file with barcodes for reads 1"/>
+ <param format="fasta" name="barcodeFile2" type="data" label="Fasta file with barcodes for reads 2"/>
+ </when>
+ </conditional>
+
+ <param name="bThresh" size="4" type="float" value="0.0" label="Error rate" help="allowed fraction of mismatches and indels"/>
+
+ <param name="bTrimEnd" type="select" label="Trim-end mode">
+ <option value="LTAIL" selected="true">Left tail</option>
+ <option value="LEFT">Left</option>
+ <option value="RTAIL">Right tail</option>
+ <option value="RIGHT">Right</option>
+ <option value="ANY">Any</option>
+ </param>
+
+ <section name="bAdvanced" title="Advanced options" expanded="false">
+
+ <conditional name="cbReads">
+ <param name="select" type="select" label="Separate barcode reads">
+ <option value="no" selected="true">No</option>
+ <option value="yes">Yes</option>
+ </param>
+ <when value="no">
+ </when>
+ <when value="yes">
+ <param name="bReads" type="data" label="Fastq file with separate barcode reads" format="fastqsanger,fastqsolexa,fastqillumina,fastqsanger.gz,fastqsolexa.gz,fastqillumina.gz,fastqsanger.bz2,fastqsolexa.bz2,fastqillumina.bz2"/>
+ </when>
+ </conditional>
+
+ <conditional name="cbMinOverlap">
+ <param name="select" type="select" label="Change minimum overlap" help="default: barcode length">
+ <option value="no" selected="true">No</option>
+ <option value="yes">Yes</option>
+ </param>
+ <when value="no">
+ </when>
+ <when value="yes">
+ <param name="bMinOverlap" size="4" type="integer" value="" label="Minimum overlap"/>
+ </when>
+ </conditional>
+
+ <param name="bKeep" type="select" label="Remove barcodes within reads">
+ <option value="" selected="true">Yes</option>
+ <option value="--barcode-keep">No</option>
+ </param>
+
+ <param name="bUnassigned" type="select" label="Include unassigned reads">
+ <option value="" selected="true">No</option>
+ <option value="--barcode-unassigned">Yes</option>
+ </param>
+
+ <conditional name="cbTailLen">
+ <param name="select" type="select" label="Change tail length">
+ <option value="no" selected="true">No</option>
+ <option value="yes">Yes</option>
+ </param>
+ <when value="no">
+ </when>
+ <when value="yes">
+ <param name="bTailLen" size="4" type="integer" value="" label="Tail length"/>
+ </when>
+ </conditional>
+
+ <conditional name="cbAlignScores">
+ <param name="select" type="select" label="Modify alignment scores">
+ <option value="no" selected="true">No</option>
+ <option value="yes">Yes</option>
+ </param>
+ <when value="no">
+ </when>
+ <when value="yes">
+ <param name="bMatch" size="3" type="integer" value="1" label="Match"/>
+ <param name="bMismatch" size="3" type="integer" value="-1" label="Mismatch"/>
+ <param name="bGap" size="3" type="integer" value="-9" label="Gap"/>
+ </when>
+ </conditional>
+ </section>
+ </when>
+ </conditional>
+
+
+ <conditional name="cAdapters">
+ <param name="select" type="select" label="5) Adapter removal">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <conditional name="ccAdapters">
+ <param name="select" type="select" label="Adapter source">
+ <option value="file" selected="true">Adapter file</option>
+ <option value="files">Two adapter files</option>
+ <option value="preset">Illumina preset</option>
+ <option value="seq">Single sequence</option>
+ </param>
+ <when value="file">
+ <param format="fasta" name="adapterFile" type="data" label="Fasta adapter file"/>
+ </when>
+ <when value="files">
+ <param format="fasta" name="adapterFile1" type="data" label="Fasta adapter file for reads 1"/>
+ <param format="fasta" name="adapterFile2" type="data" label="Fasta adapter file for reads 2"/>
+ </when>
+ <when value="preset">
+ <param name="adapterPreset" type="select" label="Adapter preset">
+ <option value="TruSeq" selected="true">TruSeq LT and HT-based kits</option>
+ <option value="SmallRNA">TruSeq Small RNA</option>
+ <option value="Methyl">TruSeq DNA Methylation and ScriptSeq</option>
+ <option value="Ribo">TruSeq Ribo Profile</option>
+ <option value="Nextera">Nextera, AmpliSeq, and TruSight</option>
+ <option value="NexteraMP">Nextera Mate Pair</option>
+ </param>
+ </when>
+ <when value="seq">
+ <param name="adapterSeq" size="40" label="Adapter sequence" type="text" value=""/>
+ </when>
+ </conditional>
+
+ <param name="aMinOverlap" size="4" type="integer" value="3" label="Minimum overlap"/>
+ <param name="aThresh" size="4" type="float" value="0.1" label="Error rate" help="allowed fraction of mismatches and indels"/>
+
+ <param name="aTrimEnd" type="select" label="Trim-end mode">
+ <option value="RIGHT" selected="true">Right</option>
+ <option value="LEFT">Left</option>
+ <option value="RTAIL">Right tail</option>
+ <option value="LTAIL">Left tail</option>
+ <option value="ANY">Any</option>
+ </param>
+
+ <conditional name="cPairOverlap">
+ <param name="select" type="select" label="Pair overlap detection">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <param name="aPairOverlap" type="select" label="Detection mode">
+ <option value="ON" selected="true">Standard</option>
+ <option value="SHORT">Trim short</option>
+ </param>
+ <param name="aMinPoverlap" size="4" type="integer" value="40" label="Minimum pair overlap"/>
+ </when>
+ </conditional>
+
+ <section name="aAdvanced" title="Advanced options" expanded="false">
+
+ <conditional name="caTailLen">
+ <param name="select" type="select" label="Change tail length">
+ <option value="no" selected="true">No</option>
+ <option value="yes">Yes</option>
+ </param>
+ <when value="no">
+ </when>
+ <when value="yes">
+ <param name="aTailLen" size="4" type="integer" value="" label="Tail length"/>
+ </when>
+ </conditional>
+
+ <conditional name="cRevComp">
+ <param name="select" type="select" label="Reverse complements of adapters">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ <option value="only">Only</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <param name="aRevCompEnd" type="select" label="Different trim-end for reverse complements">
+ <option value="off" selected="true">Off</option>
+ <option value="RIGHT">Right</option>
+ <option value="LEFT">Left</option>
+ <option value="RTAIL">Right tail</option>
+ <option value="LTAIL">Left tail</option>
+ <option value="ANY">Any</option>
+ </param>
+ </when>
+ </conditional>
+
+
+ <param name="aReadSet" type="select" label="Removal for single read set">
+ <option value="" selected="true">Off</option>
+ <option value="--adapter-read-set 1">1st</option>
+ <option value="--adapter-read-set 2">2nd</option>
+ </param>
+
+ <param name="aCycles" size="4" type="integer" value="1" label="Number of removal cycles"/>
+
+ <conditional name="caAlignScores">
+ <param name="select" type="select" label="Modify alignment scores">
+ <option value="no" selected="true">No</option>
+ <option value="yes">Yes</option>
+ </param>
+ <when value="no">
+ </when>
+ <when value="yes">
+ <param name="aMatch" size="3" type="integer" value="1" label="Match"/>
+ <param name="aMismatch" size="3" type="integer" value="-1" label="Mismatch"/>
+ <param name="aGap" size="3" type="integer" value="-6" label="Gap"/>
+ </when>
+ </conditional>
+ </section>
+ </when>
+ </conditional>
+
+
+ <conditional name="hTrim">
+ <param name="select" type="select" label="6) Trimming of homopolymers">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <param name="hTrimLeft" size="4" label="Trim homopolymers on left end" type="text" value="" optional="true"/>
+ <param name="hTrimRight" size="4" label="Trim homopolymers on right end" type="text" value="" optional="true" help="for example AT"/>
+
+ <param name="hMinLength" size="4" type="integer" value="3" label="Minimum length"/>
+ <param name="hErrorRate" size="4" type="float" value="0.1" label="Error rate"/>
+
+ <section name="hAdvanced" title="Advanced options" expanded="false">
+
+ <param name="hMaxLength" size="4" type="integer" value="" label="Maximum length" optional="true"/>
+
+ <param name="hMaxFirst" type="select" label="Maximum length only for first one">
+ <option value="" selected="true">Off</option>
+ <option value="--htrim-max-first">On</option>
+ </param>
+
+ <param name="hMinLength2" size="4" type="integer" value="" label="Minimum length after first one" optional="true"/>
+
+ <param name="hTrimAdapter" type="select" label="Only in case of adapter on same side">
+ <option value="" selected="true">Off</option>
+ <option value="--htrim-adapter">On</option>
+ </param>
+ </section>
+ </when>
+ </conditional>
+
+
+ <conditional name="cTrimLen">
+ <param name="select" type="select" label="7) Trimming to length">
+ <option value="off" selected="true">Off</option>
+ <option value="on">On</option>
+ </param>
+ <when value="off">
+ </when>
+ <when value="on">
+ <param name="trimLen" size="4" type="integer" value="" label="Length" help="trim reads to certain length from right"/>
+ </when>
+ </conditional>
+
+ <param name="minReadLen" size="4" type="integer" value="20" label="8) Minimum read length"/>
+
+ <conditional name="cAddOptions">
+ <param name="select" type="select" label="Additional options">
+ <option value="hide" selected="true">Hide</option>
+ <option value="show">Show</option>
+ </param>
+ <when value="hide">
+ </when>
+ <when value="show">
+ <param name="logLevel" type="select" label="Alignment logging">
+ <option value="" selected="true">Off</option>
+ <option value="--align-log ALL">All</option>
+ <option value="--align-log MOD">Modified</option>
+ <option value="--align-log TAB">Tabular</option>
+ </param>
+
+ <param name="numTags" type="select" label="Number tags">
+ <option value="" selected="true">Off</option>
+ <option value="--number-tags">On</option>
+ </param>
+
+ <param name="remTags" type="select" label="Removal tags">
+ <option value="" selected="true">Off</option>
+ <option value="--removal-tags">On</option>
+ </param>
+
+ <param name="rndTags" type="select" label="UMI tags">
+ <option value="" selected="true">Off</option>
+ <option value="--umi-tags">On</option>
+ </param>
+
+ <param name="singleReadsP" type="select" label="Single reads as paired">
+ <option value="" selected="true">Off</option>
+ <option value="--single-reads-paired">On</option>
+ </param>
+
+ <conditional name="cReadBundle">
+ <param name="select" type="select" label="Read bundle options">
+ <option value="hide" selected="true">Hide</option>
+ <option value="show">Show</option>
+ </param>
+ <when value="hide">
+ </when>
+ <when value="show">
+ <param name="bundle" size="4" type="integer" value="256" label="Number of paired reads per bundle"/>
+ <param name="bundles" size="4" type="integer" value="" optional="true" label="Only certain number of bundles for testing"/>
+ </when>
+ </conditional>
+ </when>
+ </conditional>
+
+ </inputs>
+
+ <outputs>
+ <data name="outlog" format="txt" label="Flexbar log for ${on_string}"/>
+
+ <data name="rsout" label="Flexbar on ${reads.rs.name}" format_source="rs">
+ <filter>reads['mode'] == 'se'</filter>
+ <filter>cBarcodes['select'] == "off"</filter>
+ </data>
+
+ <data name="r1out" label="Flexbar on ${reads.r1.name}" format_source="r1">
+ <filter>reads['mode'] == "pe"</filter>
+ <filter>cBarcodes['select'] == "off"</filter>
+ </data>
+ <data name="r2out" label="Flexbar on ${reads.r2.name}" format_source="r2">
+ <filter>reads['mode'] == "pe"</filter>
+ <filter>cBarcodes['select'] == "off"</filter>
+ </data>
+
+ <collection name="rcout" type="paired" label="Flexbar on ${on_string}">
+ <filter>reads['mode'] == "pc"</filter>
+ <filter>cBarcodes['select'] == "off"</filter>
+ <data name="forward" label="Flexbar on ${reads.rc.forward.name}" format_source="rc['forward']"/>
+ <data name="reverse" label="Flexbar on ${reads.rc.reverse.name}" format_source="rc['reverse']"/>
+ </collection>
+
+ <collection type="list" label="Flexbar on ${on_string}" name="barcoded">
+ <filter>cBarcodes['select'] == "on"</filter>
+ <discover_datasets pattern="__designation_and_ext__" directory="flexbar_barcoded" visible="true"/>
+ </collection>
+
+ </outputs>
+
+ <help>
+
+**Description**
+
+The program Flexbar preprocesses high-throughput sequencing data efficiently. It demultiplexes barcoded runs and removes adapter sequences. Moreover, trimming and filtering features are provided. Flexbar increases read mapping rates and improves genome as well as transcriptome assemblies. Unique molecular identifiers can be extracted in a flexible way. The program supports sequencing data in fastq format, e.g. from the Illumina platform. Flexbar is available on the project_ page.
+
+.. _project: https://github.com/seqan/flexbar
+
+------
+
+**Trim-end modes**
+
+**Any:** longer side of read remains after overlap removal
+
+**Left:** right side remains after removal, align before or at read end
+
+**Right:** left part remains after removal, align after or at read start
+
+**Left tail:** consider first n bases of reads in alignment
+
+**Right tail:** use only last n bases, see tail-length options
+
+------
+
+**Documentation**
+
+Further documentation is available on the `manual`__ page and via the command line help screen.
+
+.. __: https://github.com/seqan/flexbar/wiki
+
+------
+
+**References**
+
+Johannes T. Roehr, Christoph Dieterich, Knut Reinert:
+Flexbar 3.0 – SIMD and multicore parallelization. Bioinformatics 2017.
+
+Matthias Dodt, Johannes T. Roehr, Rina Ahmed, Christoph Dieterich:
+Flexbar – flexible barcode and adapter processing for next-generation sequencing platforms. Biology 2012.
+
+ </help>
+
+</tool>
+
=====================================
src/Flexbar.cpp
=====================================
@@ -2,7 +2,7 @@
Flexbar - flexible barcode and adapter removal
- Version 3.4.0
+ Version 3.5.0
BSD 3-Clause License
@@ -29,8 +29,8 @@ int main(int argc, const char* argv[]){
using namespace std;
using namespace seqan;
- const string version = "3.4.0";
- const string date = "June 2018";
+ const string version = "3.5.0";
+ const string date = "May 2019";
ArgumentParser parser("flexbar");
=====================================
src/FlexbarIO.h
=====================================
@@ -295,8 +295,7 @@ void checkInputType(const std::string path, flexbar::FileFormat &format, const b
try{
if(! atEnd(seqFileIn)){
- FSeqStr seq;
- FString id, qual;
+ FString id, seq, qual;
readRecord(id, seq, qual, seqFileIn);
=====================================
src/FlexbarTypes.h
=====================================
@@ -11,14 +11,15 @@ class SeqRead {
TSeqStr seq;
TString id, qual, umi;
- bool rmAdapter, rmAdapterRC, pairOverlap;
+ bool rmAdapter, rmAdapterRC, pairOverlap, poRemoval;
SeqRead(TSeqStr& sequence, TString& seqID) :
seq(sequence),
id(seqID),
rmAdapter(false),
rmAdapterRC(false),
- pairOverlap(false){
+ pairOverlap(false),
+ poRemoval(false){
}
SeqRead(TSeqStr& sequence, TString& seqID, TString& quality) :
@@ -27,7 +28,8 @@ class SeqRead {
qual(quality),
rmAdapter(false),
rmAdapterRC(false),
- pairOverlap(false){
+ pairOverlap(false),
+ poRemoval(false){
}
};
@@ -233,6 +235,12 @@ namespace flexbar{
AOFF
};
+ enum AdapterTrimmed {
+ ATON,
+ ATOFF,
+ ATONLY
+ };
+
enum RunType {
SINGLE,
PAIRED,
=====================================
src/Options.h
=====================================
@@ -22,8 +22,8 @@ struct Options{
bool isPaired, useAdapterFile, useNumberTag, useRemovalTag, umiTags, logStdout;
bool switch2Fasta, writeUnassigned, writeSingleReads, writeSingleReadsP, writeLengthDist;
- bool useStdin, useStdout, relaxRegion, useRcTrimEnd, qtrimPostRm;
- bool interleavedInput, htrimAdapterRm, htrimMaxFirstOnly;
+ bool useStdin, useStdout, relaxRegion, useRcTrimEnd, qtrimPostRm, addBarcodeAdapter;
+ bool interleavedInput, iupacInput, htrimAdapterRm, htrimMaxFirstOnly;
int cutLen_begin, cutLen_end, cutLen_read, a_tail_len, b_tail_len, p_min_overlap;
int qtrimThresh, qtrimWinSize, a_overhang, htrimMinLength, htrimMinLength2, htrimMaxLength;
@@ -44,6 +44,7 @@ struct Options{
flexbar::RevCompMode rcMode;
flexbar::PairOverlap poMode;
flexbar::AdapterPreset aPreset;
+ flexbar::AdapterTrimmed aTrimmed;
tbb::concurrent_vector<flexbar::TBar> barcodes, adapters, barcodes2, adapters2;
@@ -80,10 +81,12 @@ struct Options{
logStdout = false;
umiTags = false;
interleavedInput = false;
+ iupacInput = false;
useStdin = false;
useStdout = false;
relaxRegion = false;
useRcTrimEnd = false;
+ addBarcodeAdapter = false;
qtrimPostRm = false;
htrimAdapterRm = false;
htrimMaxFirstOnly = false;
@@ -114,6 +117,7 @@ struct Options{
arc_end = RIGHT;
b_end = LTAIL;
aPreset = APOFF;
+ aTrimmed = ATON;
}
};
@@ -149,7 +153,7 @@ const std::string getFlexbarURL(){
const std::string getFlexbarDescription(){
- return "The program Flexbar preprocesses high-throughput sequencing data efficiently. It demultiplexes barcoded runs and removes adapter sequences. Moreover, trimming and filtering features are provided. Flexbar increases read mapping rates and improves genome as well as transcriptome assemblies. Unique molecular identifiers can be extracted in a flexible way. The program supports sequencing data in fasta and fastq format, e.g. from the Illumina platform. Refer to the manual on github.com/seqan/flexbar/wiki or contact Johannes Roehr on github.com/jtroehr for support with this application.";
+ return "The program Flexbar preprocesses high-throughput sequencing data efficiently. It demultiplexes barcoded runs and removes adapter sequences. Several adapter removal presets for Illumina libraries are included. Flexbar computes exact overlap alignments using SIMD and multicore parallelism. Moreover, trimming and filtering features are provided, e.g. trimming of homopolymers at read ends. Flexbar increases read mapping rates and improves genome as well as transcriptome assemblies. Unique molecular identifiers can be extracted in a flexible way. The software supports data in fasta and fastq format from multiple sequencing platforms. Refer to the manual on github.com/seqan/flexbar/wiki or contact Johannes Roehr on github.com/jtroehr for support with this application.";
}
@@ -184,6 +188,7 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
addOption(parser, ArgParseOption("r", "reads", "Fasta/q file or stdin (-) with reads that may contain barcodes.", ARG::INPUT_FILE));
addOption(parser, ArgParseOption("p", "reads2", "Second input file of paired reads, gz and bz2 files supported.", ARG::INPUT_FILE));
addOption(parser, ArgParseOption("i", "interleaved", "Interleaved format for first input set with paired reads."));
+ addOption(parser, ArgParseOption("I", "iupac", "Accept iupac symbols in reads and convert to N if not ATCG."));
addSection(parser, "Barcode detection");
addOption(parser, ArgParseOption("b", "barcodes", "Fasta file with barcodes for demultiplexing, may contain N.", ARG::INPUT_FILE));
@@ -214,7 +219,9 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
addOption(parser, ArgParseOption("av", "adapter-min-poverlap", "Minimum overlap of paired reads for detection.", ARG::INTEGER));
addOption(parser, ArgParseOption("ac", "adapter-revcomp", "Include reverse complements of adapters.", ARG::STRING));
addOption(parser, ArgParseOption("ad", "adapter-revcomp-end", "Use different trim-end for reverse complements of adapters.", ARG::STRING));
+ addOption(parser, ArgParseOption("ab", "adapter-add-barcode", "Add reverse complement of detected barcode to adapters."));
addOption(parser, ArgParseOption("ar", "adapter-read-set", "Consider only single read set for adapters.", ARG::STRING));
+ addOption(parser, ArgParseOption("ak", "adapter-trimmed-out", "Modify that trimmed reads are kept.", ARG::STRING));
addOption(parser, ArgParseOption("ay", "adapter-cycles", "Number of adapter removal cycles.", ARG::INTEGER));
addOption(parser, ArgParseOption("am", "adapter-match", "Alignment match score.", ARG::INTEGER));
addOption(parser, ArgParseOption("ai", "adapter-mismatch", "Alignment mismatch score.", ARG::INTEGER));
@@ -257,7 +264,7 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
addOption(parser, ArgParseOption("z", "zip-output", "Direct compression of output files.", ARG::STRING));
addOption(parser, ArgParseOption("1", "stdout-reads", "Write reads to stdout, tagged and interleaved if needed."));
addOption(parser, ArgParseOption("R", "output-reads", "Output file for reads instead of target prefix usage.", ARG::OUTPUT_FILE));
- addOption(parser, ArgParseOption("D", "output-reads2", "Output file for reads2 instead of target prefix usage.", ARG::OUTPUT_FILE));
+ addOption(parser, ArgParseOption("P", "output-reads2", "Output file for reads2 instead of target prefix usage.", ARG::OUTPUT_FILE));
addOption(parser, ArgParseOption("j", "length-dist", "Generate length distribution for read output files."));
addOption(parser, ArgParseOption("s", "single-reads", "Write single reads for too short counterparts in pairs."));
addOption(parser, ArgParseOption("S", "single-reads-paired", "Write paired single reads with N for short counterparts."));
@@ -287,6 +294,8 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
setAdvanced(parser, "adapter-min-poverlap");
setAdvanced(parser, "adapter-revcomp");
setAdvanced(parser, "adapter-revcomp-end");
+ setAdvanced(parser, "adapter-add-barcode");
+ setAdvanced(parser, "adapter-trimmed-out");
setAdvanced(parser, "adapter-read-set");
setAdvanced(parser, "adapter-cycles");
setAdvanced(parser, "adapter-match");
@@ -308,6 +317,7 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
setAdvanced(parser, "bundle");
setAdvanced(parser, "bundles");
setAdvanced(parser, "interleaved");
+ setAdvanced(parser, "iupac");
setAdvanced(parser, "length-dist");
setAdvanced(parser, "single-reads");
setAdvanced(parser, "single-reads-paired");
@@ -359,6 +369,7 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
setValidValues(parser, "adapter-read-set", "1 2");
setValidValues(parser, "adapter-revcomp", "ON ONLY");
+ setValidValues(parser, "adapter-trimmed-out", "OFF ONLY");
setValidValues(parser, "adapter-pair-overlap", "ON SHORT ONLY");
setValidValues(parser, "adapter-preset", "TruSeq SmallRNA Methyl Ribo Nextera NexteraMP");
@@ -403,7 +414,7 @@ void defineOptions(seqan::ArgumentParser &parser, const std::string version, con
addText(parser._toolDoc, "\\fBflexbar\\fP \\fB-r\\fP reads.fq \\fB-t\\fP target \\fB-q\\fP TAIL \\fB-qf\\fP i1.8", false);
addText(parser._toolDoc, "\\fBflexbar\\fP \\fB-r\\fP reads.fq \\fB-b\\fP barcodes.fa \\fB-bt\\fP LTAIL", false);
addText(parser._toolDoc, "\\fBflexbar\\fP \\fB-r\\fP reads.fq \\fB-a\\fP adapters.fa \\fB-ao\\fP 3 \\fB-ae\\fP 0.1", false);
- addText(parser._toolDoc, "\\fBflexbar\\fP \\fB-r\\fP r1.fq \\fB-p\\fP r2.fq \\fB-a\\fP adapters.fa \\fB-ap\\fP ON", false);
+ addText(parser._toolDoc, "\\fBflexbar\\fP \\fB-r\\fP r1.fq \\fB-p\\fP r2.fq \\fB-a\\fP a1.fa \\fB-a2\\fP a2.fa \\fB-ap\\fP ON", false);
addText(parser._toolDoc, "\\fBflexbar\\fP \\fB-r\\fP r1.fq \\fB-p\\fP r2.fq \\fB-aa\\fP TruSeq \\fB-ap\\fP ON");
}
@@ -613,6 +624,11 @@ void loadOptions(Options &o, seqan::ArgumentParser &parser){
}
}
+ if(isSet(parser, "iupac")){
+ *out << "Iupac reads: on" << endl;
+ o.iupacInput = true;
+ }
+
// barcode and adapter file options
@@ -871,12 +887,10 @@ void loadOptions(Options &o, seqan::ArgumentParser &parser){
if(isSet(parser, "output-reads") && (isSet(parser, "output-reads2") || o.runType == SINGLE)){
getOptionValue(o.outReadsFile, parser, "output-reads");
- *out << "output-reads: " << o.outReadsFile << endl;
}
if(isSet(parser, "output-reads2") && isSet(parser, "output-reads") && o.runType == PAIRED){
getOptionValue(o.outReadsFile2, parser, "output-reads2");
- *out << "output-reads2: " << o.outReadsFile2 << endl;
if(o.outReadsFile == o.outReadsFile2){
cerr << "\n" << "Output reads and reads2 file should not be the same.\n" << endl;
@@ -1043,6 +1057,22 @@ void loadOptions(Options &o, seqan::ArgumentParser &parser){
o.relaxRegion = true;
}
+ if(isSet(parser, "adapter-add-barcode") && o.isPaired && o.a_end == RIGHT && o.rcMode != RCON &&
+ o.barDetect != BARCODE_READ && o.barDetect != BOFF && o.b_end == LTAIL){
+
+ *out << "adapter-add-barcode: on" << endl;
+ o.addBarcodeAdapter = true;
+ }
+
+ if(isSet(parser, "adapter-trimmed-out")){
+ string a_trimmed_out;
+ getOptionValue(a_trimmed_out, parser, "adapter-trimmed-out");
+ *out << "adapter-trimmed-out: " << a_trimmed_out << endl;
+
+ if(a_trimmed_out == "OFF") o.aTrimmed = ATOFF;
+ else if(a_trimmed_out == "ONLY") o.aTrimmed = ATONLY;
+ }
+
if(isSet(parser, "adapter-read-set") && o.isPaired && o.adapRm != NORMAL2){
string a_read_set;
getOptionValue(a_read_set, parser, "adapter-read-set");
=====================================
src/PairedAlign.h
=====================================
@@ -14,7 +14,7 @@ class PairedAlign : public tbb::filter {
private:
const bool m_writeUnassigned, m_twoBarcodes, m_umiTags, m_useRcTrimEnd;
- const bool m_htrim, m_htrimAdapterRm, m_htrimMaxFirstOnly;
+ const bool m_htrim, m_htrimAdapterRm, m_htrimMaxFirstOnly, m_addBarcodeAdapter;
const std::string m_htrimLeft, m_htrimRight;
@@ -61,6 +61,7 @@ public:
m_umiTags(o.umiTags),
m_useRcTrimEnd(o.useRcTrimEnd),
m_writeUnassigned(o.writeUnassigned),
+ m_addBarcodeAdapter(o.addBarcodeAdapter),
m_htrimLeft(o.htrimLeft),
m_htrimRight(o.htrimRight),
m_htrimMinLength(o.htrimMinLength),
@@ -106,11 +107,11 @@ public:
using namespace flexbar;
switch(m_barType){
- case BARCODE_READ: pRead->barID = m_b1->alignSeqRead(pRead->b, false, alBundle[0], cycle[0], idxAl[0], alMode, m_bTrimEnd); break;
- case WITHIN_READ_REMOVAL2: pRead->barID2 = m_b2->alignSeqRead(pRead->r2, true, alBundle[2], cycle[2], idxAl[2], alMode, m_bTrimEnd);
- case WITHIN_READ_REMOVAL: pRead->barID = m_b1->alignSeqRead(pRead->r1, true, alBundle[1], cycle[1], idxAl[1], alMode, m_bTrimEnd); break;
- case WITHIN_READ2: pRead->barID2 = m_b2->alignSeqRead(pRead->r2, false, alBundle[2], cycle[2], idxAl[2], alMode, m_bTrimEnd);
- case WITHIN_READ: pRead->barID = m_b1->alignSeqRead(pRead->r1, false, alBundle[1], cycle[1], idxAl[1], alMode, m_bTrimEnd); break;
+ case BARCODE_READ: pRead->barID = m_b1->alignSeqRead(pRead->b, false, alBundle[0], cycle[0], idxAl[0], alMode, m_bTrimEnd, ""); break;
+ case WITHIN_READ_REMOVAL2: pRead->barID2 = m_b2->alignSeqRead(pRead->r2, true, alBundle[2], cycle[2], idxAl[2], alMode, m_bTrimEnd, "");
+ case WITHIN_READ_REMOVAL: pRead->barID = m_b1->alignSeqRead(pRead->r1, true, alBundle[1], cycle[1], idxAl[1], alMode, m_bTrimEnd, ""); break;
+ case WITHIN_READ2: pRead->barID2 = m_b2->alignSeqRead(pRead->r2, false, alBundle[2], cycle[2], idxAl[2], alMode, m_bTrimEnd, "");
+ case WITHIN_READ: pRead->barID = m_b1->alignSeqRead(pRead->r1, false, alBundle[1], cycle[1], idxAl[1], alMode, m_bTrimEnd, ""); break;
case BOFF: break;
}
@@ -125,12 +126,49 @@ public:
using namespace flexbar;
- if(m_adapRem != ATWO)
- m_a1->alignSeqRead(pRead->r1, true, alBundle[0], cycle[0], idxAl[0], alMode, trimEnd);
+ if(m_adapRem != ATWO){
+
+ TSeqStr addBarcode = "";
+
+ if(m_addBarcodeAdapter && pRead->r2 != NULL && pRead->barID2 > 0){
+ addBarcode = m_barcodes2->at(pRead->barID2 - 1).seq;
+
+ if(m_umiTags && pRead->r2->umi != ""){
+ unsigned int umiPos = 1;
+
+ for(unsigned int i = 0; i < length(addBarcode); ++i){
+ if(addBarcode[i] == 'N' && length(pRead->r2->umi) > umiPos){
+ addBarcode[i] = pRead->r2->umi[umiPos++];
+ }
+ }
+ }
+ seqan::reverseComplement(addBarcode);
+ }
+
+ m_a1->alignSeqRead(pRead->r1, true, alBundle[0], cycle[0], idxAl[0], alMode, trimEnd, addBarcode);
+ }
if(pRead->r2 != NULL && m_adapRem != AONE){
- if(m_adapRem != NORMAL2) m_a1->alignSeqRead(pRead->r2, true, alBundle[1], cycle[1], idxAl[1], alMode, trimEnd);
- else m_a2->alignSeqRead(pRead->r2, true, alBundle[1], cycle[1], idxAl[1], alMode, trimEnd);
+
+ TSeqStr addBarcode = "";
+
+ if(m_addBarcodeAdapter && pRead->barID > 0){
+ addBarcode = m_barcodes->at(pRead->barID - 1).seq;
+
+ if(m_umiTags && pRead->r1->umi != ""){
+ unsigned int umiPos = 1;
+
+ for(unsigned int i = 0; i < length(addBarcode); ++i){
+ if(addBarcode[i] == 'N' && length(pRead->r1->umi) > umiPos){
+ addBarcode[i] = pRead->r1->umi[umiPos++];
+ }
+ }
+ }
+ seqan::reverseComplement(addBarcode);
+ }
+
+ if(m_adapRem != NORMAL2) m_a1->alignSeqRead(pRead->r2, true, alBundle[1], cycle[1], idxAl[1], alMode, trimEnd, addBarcode);
+ else m_a2->alignSeqRead(pRead->r2, true, alBundle[1], cycle[1], idxAl[1], alMode, trimEnd, addBarcode);
}
}
=====================================
src/PairedOutput.h
=====================================
@@ -26,6 +26,7 @@ private:
const flexbar::RunType m_runType;
const flexbar::BarcodeDetect m_barDetect;
const flexbar::QualTrimType m_qtrim;
+ const flexbar::AdapterTrimmed m_aTrimmed;
typedef SeqOutput<TSeqStr, TString> TSeqOutput;
typedef SeqOutputFiles<TSeqStr, TString> TOutFiles;
@@ -50,6 +51,7 @@ public:
m_qtrimThresh(o.qtrimThresh),
m_qtrimWinSize(o.qtrimWinSize),
m_qtrimPostRm(o.qtrimPostRm),
+ m_aTrimmed(o.aTrimmed),
m_isPaired(o.isPaired),
m_writeUnassigned(o.writeUnassigned),
m_writeSingleReads(o.writeSingleReads),
@@ -239,7 +241,7 @@ public:
using namespace flexbar;
- bool l1ok = false, l2ok = false;
+ bool r1ok = false, r2ok = false;
switch(m_runType){
@@ -253,11 +255,13 @@ public:
if(qualTrim(pRead->r1, m_qtrim, m_qtrimThresh, m_qtrimWinSize)) ++m_nLowPhred;
}
- if(length(pRead->r1->seq) >= m_minLength){
-
- m_outMap[pRead->barID].f1->writeRead(pRead->r1);
- }
+ if(length(pRead->r1->seq) >= m_minLength) r1ok = true;
else m_outMap[pRead->barID].m_nShort_1++;
+
+ if (m_aTrimmed == ATOFF && (pRead->r1->rmAdapter || pRead->r1->rmAdapterRC)) r1ok = false;
+ else if(m_aTrimmed == ATONLY && ! pRead->r1->rmAdapter && ! pRead->r1->rmAdapterRC) r1ok = false;
+
+ if(r1ok) m_outMap[pRead->barID].f1->writeRead(pRead->r1);
}
}
break;
@@ -284,14 +288,23 @@ public:
if(qualTrim(pRead->r2, m_qtrim, m_qtrimThresh, m_qtrimWinSize)) ++m_nLowPhred;
}
- if(length(pRead->r1->seq) >= m_minLength) l1ok = true;
- if(length(pRead->r2->seq) >= m_minLength) l2ok = true;
+ if(length(pRead->r1->seq) >= m_minLength) r1ok = true;
+ if(length(pRead->r2->seq) >= m_minLength) r2ok = true;
- if(l1ok && l2ok){
+ if(! r1ok) m_outMap[outIdx].m_nShort_1++;
+ if(! r2ok) m_outMap[outIdx].m_nShort_2++;
+
+ if (m_aTrimmed == ATOFF && (pRead->r1->rmAdapter || pRead->r1->rmAdapterRC || pRead->r1->poRemoval)) r1ok = false;
+ else if(m_aTrimmed == ATONLY && ! pRead->r1->rmAdapter && ! pRead->r1->rmAdapterRC && ! pRead->r1->poRemoval) r1ok = false;
+
+ if (m_aTrimmed == ATOFF && (pRead->r2->rmAdapter || pRead->r2->rmAdapterRC || pRead->r2->poRemoval)) r2ok = false;
+ else if(m_aTrimmed == ATONLY && ! pRead->r2->rmAdapter && ! pRead->r2->rmAdapterRC && ! pRead->r2->poRemoval) r2ok = false;
+
+ if(r1ok && r2ok){
m_outMap[outIdx].f1->writeRead(pRead->r1);
m_outMap[outIdx].f2->writeRead(pRead->r2);
}
- else if(l1ok && ! l2ok){
+ else if(r1ok && ! r2ok){
m_nSingleReads++;
if(m_writeSingleReads){
@@ -308,7 +321,7 @@ public:
m_outMap[outIdx].f2->writeRead(pRead->r2);
}
}
- else if(! l1ok && l2ok){
+ else if(! r1ok && r2ok){
m_nSingleReads++;
if(m_writeSingleReads){
@@ -325,9 +338,6 @@ public:
m_outMap[outIdx].f2->writeRead(pRead->r2);
}
}
-
- if(! l1ok) m_outMap[outIdx].m_nShort_1++;
- if(! l2ok) m_outMap[outIdx].m_nShort_2++;
}
}
}
=====================================
src/QualTrimming.h
=====================================
@@ -43,10 +43,10 @@ unsigned qualTrimming(const TString& qual, unsigned const _cutoff, Window const
unsigned window = spec.size;
unsigned avg = 0, i = 0;
-
- // Work with absolute cutoff in window to avoid divisions
+
+ // Absolute cutoff in window to avoid divisions
unsigned cutoff = _cutoff * window;
-
+
// Calculate average quality of initial window
for (i = 0; i < window; ++i){
avg += getQuality(qual, i);
@@ -57,9 +57,15 @@ unsigned qualTrimming(const TString& qual, unsigned const _cutoff, Window const
// Take care only not to go over the end of the sequence. Shorten window near the end
avg -= getQuality(qual, i);
- avg += i + window < length(qual) ? getQuality(qual, i + window) : 0;
+
+ if(i + window < length(qual)){
+ avg += getQuality(qual, i + window);
+ }
+ else{
+ cutoff = _cutoff * ((length(qual) - 1) - i);
+ }
}
- return i; // i holds start of first window that turned bad
+ return i; // holds start of first window that turned bad
}
=====================================
src/SeqAlign.h
=====================================
@@ -15,7 +15,7 @@ private:
const flexbar::FileFormat m_format;
const flexbar::PairOverlap m_poMode;
- const bool m_isBarcoding, m_writeTag, m_umiTags, m_strictRegion;
+ const bool m_isBarcoding, m_writeTag, m_umiTags, m_strictRegion, m_addBarcodeAdapter;
const int m_minLength, m_minOverlap, m_tailLength;
const float m_errorRate;
const unsigned int m_bundleSize;
@@ -41,6 +41,7 @@ public:
m_log(o.logAlign),
m_format(o.format),
m_writeTag(o.useRemovalTag),
+ m_addBarcodeAdapter(o.addBarcodeAdapter),
m_strictRegion(! o.relaxRegion),
m_bundleSize(o.bundleSize),
m_out(o.out),
@@ -53,7 +54,7 @@ public:
};
- int alignSeqRead(flexbar::TSeqRead* sr, const bool performRemoval, flexbar::Alignments &alignments, flexbar::ComputeCycle &cycle, unsigned int &idxAl, const flexbar::AlignmentMode &alMode, const flexbar::TrimEnd trimEnd){
+ int alignSeqRead(flexbar::TSeqRead* sr, const bool performRemoval, flexbar::Alignments &alignments, flexbar::ComputeCycle &cycle, unsigned int &idxAl, const flexbar::AlignmentMode &alMode, const flexbar::TrimEnd trimEnd, const TSeqStr &addBarcode){
using namespace std;
using namespace flexbar;
@@ -81,12 +82,18 @@ public:
if (alMode == ALIGNRCOFF && m_queries->at(i).rcAdapter) continue;
else if(alMode == ALIGNRC && ! m_queries->at(i).rcAdapter) continue;
- TSeqStr &qseq = m_queries->at(i).seq;
+ TSeqStr *qseq = &m_queries->at(i).seq;
TSeqStr *rseq = &seqRead.seq;
- TSeqStr tmp;
+ TSeqStr tmp, tmpq;
+
+ if(! m_isBarcoding && m_addBarcodeAdapter && addBarcode != ""){
+ tmpq = addBarcode;
+ append(tmpq, m_queries->at(i).seq);
+ qseq = &tmpq;
+ }
if(trimEnd == LTAIL || trimEnd == RTAIL){
- int tailLength = (m_tailLength > 0) ? m_tailLength : length(qseq);
+ int tailLength = (m_tailLength > 0) ? m_tailLength : length(*qseq);
if(tailLength < readLength){
if(trimEnd == LTAIL) tmp = prefix(seqRead.seq, tailLength);
@@ -100,7 +107,7 @@ public:
resize(rows(alignments.aset[idxAl]), 2);
assignSource(row(alignments.aset[idxAl], 0), *rseq);
- assignSource(row(alignments.aset[idxAl], 1), qseq);
+ assignSource(row(alignments.aset[idxAl], 1), *qseq);
++idxAl;
}
@@ -124,6 +131,11 @@ public:
m_algo.alignGlobal(a, alignments, cycle, idxAl++, trimEnd);
a.queryLength = length(m_queries->at(i).seq);
+
+ if(! m_isBarcoding && m_addBarcodeAdapter && addBarcode != ""){
+ a.queryLength += length(addBarcode);
+ }
+
a.tailLength = (m_tailLength > 0) ? m_tailLength : a.queryLength;
a.overlapLength = a.endPos - a.startPos;
=====================================
src/SeqAlignPair.h
=====================================
@@ -114,6 +114,8 @@ public:
++m_modified;
+ seqRead2.poRemoval = true;
+
if(m_writeTag) append(seqRead2.id, "_Flexbar_removal_PO");
}
}
@@ -132,6 +134,8 @@ public:
++m_modified;
+ seqRead.poRemoval = true;
+
if(m_writeTag) append(seqRead.id, "_Flexbar_removal_PO");
}
}
=====================================
src/SeqInput.h
=====================================
@@ -16,7 +16,7 @@ private:
const flexbar::QualTrimType m_qtrim;
const flexbar::FileFormat m_format;
- const bool m_preProcess, m_useStdin, m_qtrimPostRm;
+ const bool m_preProcess, m_useStdin, m_qtrimPostRm, m_iupacInput;
const int m_maxUncalled, m_preTrimBegin, m_preTrimEnd, m_qtrimThresh, m_qtrimWinSize;
tbb::atomic<unsigned long> m_nrReads, m_nrChars, m_nLowPhred;
@@ -33,6 +33,7 @@ public:
m_qtrimThresh(o.qtrimThresh),
m_qtrimWinSize(o.qtrimWinSize),
m_qtrimPostRm(o.qtrimPostRm),
+ m_iupacInput(o.iupacInput),
m_format(o.format),
m_nrReads(0),
m_nrChars(0),
@@ -76,12 +77,30 @@ public:
reserve(seqs, nReads);
reserve(uncalled, nReads);
- if(m_format == FASTA){
- readRecords(ids, seqs, seqFileIn, nReads);
+ if(! m_iupacInput){
+
+ if(m_format == FASTA){
+ readRecords(ids, seqs, seqFileIn, nReads);
+ }
+ else{
+ reserve(quals, nReads);
+ readRecords(ids, seqs, quals, seqFileIn, nReads);
+ }
}
else{
- reserve(quals, nReads);
- readRecords(ids, seqs, quals, seqFileIn, nReads);
+ seqan::StringSet<seqan::IupacString> seqsIupac;
+
+ reserve(seqsIupac, nReads);
+
+ if(m_format == FASTA){
+ readRecords(ids, seqsIupac, seqFileIn, nReads);
+ }
+ else{
+ reserve(quals, nReads);
+ readRecords(ids, seqsIupac, quals, seqFileIn, nReads);
+ }
+
+ seqs = seqsIupac;
}
for(unsigned int i = 0; i < length(ids); ++i){
@@ -133,6 +152,7 @@ public:
}
}
}
+
m_nrReads += length(ids);
return length(ids);
View it on GitLab: https://salsa.debian.org/med-team/flexbar/commit/4c5fb0505cefa6300f8011db975946bd7b156b6e
--
View it on GitLab: https://salsa.debian.org/med-team/flexbar/commit/4c5fb0505cefa6300f8011db975946bd7b156b6e
You're receiving this email because of your account on salsa.debian.org.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://alioth-lists.debian.net/pipermail/debian-med-commit/attachments/20190727/9d20bf28/attachment-0001.html>
More information about the debian-med-commit
mailing list