[med-svn] [bowtie2] 02/06: New upstream version 2.3.3
Alex Mestiashvili
malex-guest at moszumanska.debian.org
Mon Sep 11 11:23:09 UTC 2017
This is an automated email from the git hooks/post-receive script.
malex-guest pushed a commit to branch master
in repository bowtie2.
commit c97d572fa8a3956f12a4fbdd816df254a84172d0
Author: Alexandre Mestiashvili <alex at biotec.tu-dresden.de>
Date: Mon Sep 11 10:22:09 2017 +0200
New upstream version 2.3.3
---
.gitignore | 26 -
.travis.yml | 21 -
AUTHORS | 6 +
MANUAL | 2573 +++++++++++++++++++++--------------------
MANUAL.markdown | 831 ++++++--------
Makefile | 3 +-
NEWS | 35 +-
VERSION | 2 +-
aligner_cache.cpp | 23 +-
aligner_cache.h | 141 ++-
aligner_result.h | 57 +-
aligner_seed.cpp | 18 +-
aligner_seed.h | 3 +-
aligner_seed_policy.cpp | 11 +-
aligner_sw_common.h | 4 +-
aligner_sw_driver.cpp | 20 +-
aligner_swsse.h | 3 +-
aligner_swsse_ee_i16.cpp | 2 +-
aligner_swsse_loc_i16.cpp | 2 +-
aligner_swsse_loc_u8.cpp | 2 +-
aln_sink.cpp | 40 +-
aln_sink.h | 12 +-
blockwise_sa.h | 205 ++--
bowtie2 | 7 +-
bowtie_main.cpp | 2 +-
bt2_build.cpp | 25 +-
bt2_search.cpp | 2620 +++++++++++++++++++++++-------------------
diff_sample.h | 324 +++---
doc/manual.html | 139 ++-
doc/strip_markdown.pl | 45 -
doc/website/manual.ssi | 141 ++-
doc/website/recent_news.ssi | 38 +-
doc/website/rhsidebar.ssi | 4 +-
ds.cpp | 4 +-
ds.h | 10 +-
endian_swap.h | 4 +-
filebuf.h | 40 +-
group_walk.h | 27 +-
multikey_qsort.h | 6 +-
opts.h | 4 +
outq.cpp | 72 +-
outq.h | 38 +-
pat.cpp | 133 ++-
pat.h | 53 +-
presets.cpp | 21 +-
read_qseq.cpp | 12 +-
scripts/sim/Sim.pm | 2 +-
scripts/test/simple_tests.pl | 85 +-
search_globals.h | 2 -
threading.h | 38 +-
50 files changed, 4219 insertions(+), 3717 deletions(-)
diff --git a/.gitignore b/.gitignore
deleted file mode 100644
index f5e3327..0000000
--- a/.gitignore
+++ /dev/null
@@ -1,26 +0,0 @@
-.project
-bowtie2-align
-bowtie2-build
-bowtie2-inspect
-*.zip
-*.bt2
-*.bt2l
-*.dSYM
-*.pyc
-*.swp
-.run.pl.child.*
-.simple_tests*
-.idea
-bowtie2-build-s
-bowtie2-build-l
-bowtie2-align-s
-bowtie2-align-l
-bowtie2-inspect-s
-bowtie2-inspect-l
-bowtie2-build-s-debug
-bowtie2-build-l-debug
-bowtie2-align-s-debug
-bowtie2-align-l-debug
-bowtie2-inspect-s-debug
-bowtie2-inspect-l-debug
-Xcode
diff --git a/.travis.yml b/.travis.yml
deleted file mode 100644
index 1381f64..0000000
--- a/.travis.yml
+++ /dev/null
@@ -1,21 +0,0 @@
-language: c++
-compiler:
-- clang
-- gcc
-cache: apt
-addons:
- apt:
- packages:
- - libtbb-dev
-branches:
- only:
- - master
- - cleaner_parsing
- - batch_parsing
- - no-io
- - rel2.3
- - v2.3.2
-script: make allall && make simple-test
-notifications:
- slack:
- secure: tfzT8N1fNV+oSV7tide9WrAj3ifs+LONJ3fCH1tUzexqrx23te4lE0oAj9C1cEMJ4evyRYwHNG8HZoLCOy8EfapqbWm6vgLIlkIBpeZ9E6f2jG6v0YuVDWWpqQC3qdGXCqWtHPjgs3i5OLsLwwQ/LItLoTqpBk2aYv+vGNs2F9g=
diff --git a/AUTHORS b/AUTHORS
index d22b8b2..14b921e 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -26,4 +26,10 @@ check http://tinythreadpp.bitsnbites.eu/
Various users have kindly supplied patches, bug reports and feature requests
over the years. Many, many thanks go to them.
+Contributors to this project include:
+ * Christopher Wilks -- cwilks3 at jhu.edu,
+ * Aidan Reilly -- areilly8 at jhu.edu
+ * Valentin Antonescu -- valentin at jhu.edu
+ * Rone Charles -- charro at cs.jhu.edu
+
September 2011
diff --git a/MANUAL b/MANUAL
index b82d8bf..5328c74 100644
--- a/MANUAL
+++ b/MANUAL
@@ -1,212 +1,187 @@
-Introduction
-============
+
+INTRODUCTION
+
What is Bowtie 2?
------------------
-
-[Bowtie 2] is an ultrafast and memory-efficient tool for aligning sequencing
-reads to long reference sequences. It is particularly good at aligning reads of
-about 50 up to 100s or 1,000s of characters to relatively long (e.g. mammalian)
-genomes. Bowtie 2 indexes the genome with an [FM Index] (based on the
-[Burrows-Wheeler Transform] or [BWT]) to keep its memory footprint small: for
-the human genome, its memory footprint is typically around 3.2 gigabytes of RAM.
- Bowtie 2 supports gapped, local, and paired-end alignment modes. Multiple
-processors can be used simultaneously to achieve greater alignment speed.
-Bowtie 2 outputs alignments in [SAM] format, enabling interoperation with a
-large number of other tools (e.g. [SAMtools], [GATK]) that use SAM. Bowtie 2 is
-distributed under the [GPLv3 license], and it runs on the command line under
-Windows, Mac OS X and Linux.
-
-[Bowtie 2] is often the first step in pipelines for comparative genomics,
-including for variation calling, ChIP-seq, RNA-seq, BS-seq. [Bowtie 2] and
-[Bowtie] (also called "[Bowtie 1]" here) are also tightly integrated into some
-tools, including [TopHat]: a fast splice junction mapper for RNA-seq reads,
-[Cufflinks]: a tool for transcriptome assembly and isoform quantitiation from
-RNA-seq reads, [Crossbow]: a cloud-enabled software tool for analyzing
-resequencing data, and [Myrna]: a cloud-enabled software tool for aligning
-RNA-seq reads and measuring differential gene expression.
-
-If you use [Bowtie 2] for your published research, please cite the [Bowtie
-paper]. Thank you!
-
-[Bowtie 2]: http://bowtie-bio.sf.net/bowtie2
-[Bowtie]: http://bowtie-bio.sf.net
-[Bowtie 1]: http://bowtie-bio.sf.net
-[Burrows-Wheeler Transform]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
-[BWT]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
-[FM Index]: http://en.wikipedia.org/wiki/FM-index
-[SAM]: http://samtools.sourceforge.net/SAM1.pdf
-[SAMtools]: http://samtools.sourceforge.net
-[GATK]: http://www.broadinstitute.org/gsa/wiki/index.php/The_Genome_Analysis_Toolkit
-[TopHat]: http://tophat.cbcb.umd.edu/
-[Cufflinks]: http://cufflinks.cbcb.umd.edu/
-[Crossbow]: http://bowtie-bio.sf.net/crossbow
-[Myrna]: http://bowtie-bio.sf.net/myrna
-[Bowtie paper]: http://genomebiology.com/2009/10/3/R25
-[GPLv3 license]: http://www.gnu.org/licenses/gpl-3.0.html
+
+Bowtie 2 is an ultrafast and memory-efficient tool for aligning
+sequencing reads to long reference sequences. It is particularly good at
+aligning reads of about 50 up to 100s or 1,000s of characters to
+relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome
+with an FM Index (based on the Burrows-Wheeler Transform or BWT) to keep
+its memory footprint small: for the human genome, its memory footprint
+is typically around 3.2 gigabytes of RAM. Bowtie 2 supports gapped,
+local, and paired-end alignment modes. Multiple processors can be used
+simultaneously to achieve greater alignment speed. Cufflinks: a tool for
+transcriptome assembly and isoform quantitiation from Bowtie 2 outputs
+alignments in SAM format, enabling interoperation with a large number of
+other tools (e.g. SAMtools, GATK) that use SAM. Bowtie 2 is distributed
+under the GPLv3 license, and it runs on the command line under Windows,
+Mac OS X and Linux.
+
+Bowtie 2 is often the first step in pipelines for comparative genomics,
+including for variation calling, ChIP-seq, RNA-seq, BS-seq. Bowtie 2 and
+Bowtie (also called "Bowtie 1" here) are also tightly integrated into
+some tools, including TopHat: a fast splice junction mapper for RNA-seq
+reads, RNA-seq reads, Crossbow: a cloud-enabled software tool for
+analyzing resequencing data, and Myrna: a cloud-enabled software tool
+for aligning RNA-seq reads and measuring differential gene expression.
+
+If you use Bowtie 2 for your published research, please cite the Bowtie
+paper. Thank you!
+
How is Bowtie 2 different from Bowtie 1?
-----------------------------------------
-Bowtie 1 was released in 2009 and was geared toward aligning the relatively
-short sequencing reads (up to 25-50 nucleotides) prevalent at the time. Since
-then, technology has improved both sequencing throughput (more nucleotides
-produced per sequencer per day) and read length (more nucleotides per read).
+Bowtie 1 was released in 2009 and was geared toward aligning the
+relatively short sequencing reads (up to 25-50 nucleotides) prevalent at
+the time. Since then, technology has improved both sequencing throughput
+(more nucleotides produced per sequencer per day) and read length (more
+nucleotides per read).
The chief differences between Bowtie 1 and Bowtie 2 are:
-1. For reads longer than about 50 bp Bowtie 2 is generally faster, more
-sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g.
-less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive.
-B
+1. For reads longer than about 50 bp Bowtie 2 is generally faster, more
+ sensitive, and uses less memory than Bowtie 1. For relatively short
+ reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or
+ more sensitive.
-2. Bowtie 2 supports gapped alignment with affine gap penalties. Number of gaps
-and gap lengths are not restricted, except by way of the configurable scoring
-scheme. Bowtie 1 finds just ungapped alignments.
+2. Bowtie 2 supports gapped alignment with affine gap penalties. Number
+ of gaps and gap lengths are not restricted, except by way of the
+ configurable scoring scheme. Bowtie 1 finds just ungapped
+ alignments.
-3. Bowtie 2 supports [local alignment], which doesn't require reads to align
-end-to-end. Local alignments might be "trimmed" ("soft clipped") at one or both
-extremes in a way that optimizes alignment score. Bowtie 2 also supports
-[end-to-end alignment] which, like Bowtie 1, requires that the read align
-entirely.
+3. Bowtie 2 supports local alignment, which doesn't require reads to
+ align end-to-end. Local alignments might be "trimmed" ("soft
+ clipped") at one or both extremes in a way that optimizes alignment
+ score. Bowtie 2 also supports end-to-end alignment which, like
+ Bowtie 1, requires that the read align entirely.
-4. There is no upper limit on read length in Bowtie 2. Bowtie 1 had an upper
-limit of around 1000 bp.
+4. There is no upper limit on read length in Bowtie 2. Bowtie 1 had an
+ upper limit of around 1000 bp.
-5. Bowtie 2 allows alignments to [overlap ambiguous characters] (e.g. `N`s) in
-the reference. Bowtie 1 does not.
+5. Bowtie 2 allows alignments to overlap ambiguous characters (e.g. Ns)
+ in the reference. Bowtie 1 does not.
-6. Bowtie 2 does away with Bowtie 1's notion of alignment "stratum", and its
-distinction between "Maq-like" and "end-to-end" modes. In Bowtie 2 all
-alignments lie along a continuous spectrum of alignment scores where the
-[scoring scheme], similar to [Needleman-Wunsch] and [Smith-Waterman].
+6. Bowtie 2 does away with Bowtie 1's notion of alignment "stratum",
+ and its distinction between "Maq-like" and "end-to-end" modes. In
+ Bowtie 2 all alignments lie along a continuous spectrum of alignment
+ scores where the scoring scheme, similar to Needleman-Wunsch and
+ Smith-Waterman.
-7. Bowtie 2's [paired-end alignment] is more flexible. E.g. for pairs that do
-not align in a paired fashion, Bowtie 2 attempts to find unpaired alignments for
-each mate.
+7. Bowtie 2's paired-end alignment is more flexible. E.g. for pairs
+ that do not align in a paired fashion, Bowtie 2 attempts to find
+ unpaired alignments for each mate.
-8. Bowtie 2 reports a spectrum of mapping qualities, in contrast for Bowtie 1
-which reports either 0 or high.
+8. Bowtie 2 reports a spectrum of mapping qualities, in contrast for
+ Bowtie 1 which reports either 0 or high.
-9. Bowtie 2 does not align colorspace reads.
+9. Bowtie 2 does not align colorspace reads.
-Bowtie 2 is not a "drop-in" replacement for Bowtie 1. Bowtie 2's command-line
-arguments and genome index format are both different from Bowtie 1's.
+Bowtie 2 is not a "drop-in" replacement for Bowtie 1. Bowtie 2's
+command-line arguments and genome index format are both different from
+Bowtie 1's.
-[Needleman-Wunsch]: http://en.wikipedia.org/wiki/Needleman-Wunsch_algorithm
-[Smith-Waterman]: http://en.wikipedia.org/wiki/Smith_waterman
What isn't Bowtie 2?
---------------------
-
-Bowtie 1 and Bowtie 2 are not general-purpose alignment tools like [MUMmer],
-[BLAST] or [Vmatch]. Bowtie 2 works best when aligning to large genomes, though
-it supports arbitrarily small reference sequences (e.g. amplicons). It handles
-very long reads (i.e. upwards of 10s or 100s of kilobases), but it is optimized
-for the read lengths and error modes yielded by recent sequencers, such as the
-Illumina HiSeq 2000, Roche 454, and Ion Torrent instruments.
-
-If your goal is to align two very large sequences (e.g. two genomes), consider
-using [MUMmer]. If your goal is very sensitive alignment to a relatively short
-reference sequence (e.g. a bacterial genome), this can be done with Bowtie 2 but
-you may want to consider using tools like [NUCmer], [BLAT], or [BLAST]. These
-tools can be extremely slow when the reference genome is long, but are often
-adequate when the reference is short.
+
+Bowtie 1 and Bowtie 2 are not general-purpose alignment tools like
+MUMmer, BLAST or Vmatch. Bowtie 2 works best when aligning to large
+genomes, though it supports arbitrarily small reference sequences (e.g.
+amplicons). It handles very long reads (i.e. upwards of 10s or 100s of
+kilobases), but it is optimized for the read lengths and error modes
+yielded by recent sequencers, such as the Illumina HiSeq 2000, Roche
+454, and Ion Torrent instruments.
+
+If your goal is to align two very large sequences (e.g. two genomes),
+consider using MUMmer. If your goal is very sensitive alignment to a
+relatively short reference sequence (e.g. a bacterial genome), this can
+be done with Bowtie 2 but you may want to consider using tools like
+NUCmer, BLAT, or BLAST. These tools can be extremely slow when the
+reference genome is long, but are often adequate when the reference is
+short.
Bowtie 2 does not support alignment of colorspace reads.
-[MUMmer]: http://mummer.sourceforge.net/
-[NUCmer]: http://mummer.sourceforge.net/manual/#nucmer
-[BLAST]: http://blast.ncbi.nlm.nih.gov/Blast.cgi
-[BLAT]: http://genome.ucsc.edu/cgi-bin/hgBlat?command=start
-[Vmatch]: http://www.vmatch.de/
What does it mean that some older Bowtie 2 versions are "beta"?
---------------------------------------------------------------
-We said those Bowtie 2 versions were in "beta" to convey that it was not as
-polished as a tool that had been around for a while, and was still in flux.
-Since version 2.0.1, we declared Bowtie 2 was no longer "beta".
+We said those Bowtie 2 versions were in "beta" to convey that it was not
+as polished as a tool that had been around for a while, and was still in
+flux. Since version 2.0.1, we declared Bowtie 2 was no longer "beta".
+
+
+
+OBTAINING BOWTIE 2
+
-Obtaining Bowtie 2
-==================
+Download Bowtie 2 sources and binaries from the Download section of the
+Sourceforge site. Binaries are available for the Intel x86_64
+architecture running Linux, Mac OS X, and Windows. If you plan to
+compile Bowtie 2 yourself, make sure to get the source package, i.e.,
+the filename that ends in "-source.zip".
-Download Bowtie 2 sources and binaries from the [Download] section of the
-Sourceforge site. Binaries are available for the Intel `x86_64` architecture
-running Linux, Mac OS X, and Windows. If you plan to compile Bowtie 2 yourself,
-make sure to get the source package, i.e., the filename that ends in
-"-source.zip".
Building from source
---------------------
-Building Bowtie 2 from source requires a GNU-like environment with GCC, GNU Make
-and other basics. It should be possible to build Bowtie 2 on most vanilla Linux
-installations or on a Mac installation with [Xcode] installed. Bowtie 2 can
-also be built on Windows using a 64-bit MinGW distribution and MSYS. In order
-to simplify the MinGW setup it might be worth investigating popular MinGW personal
-builds since these are coming already prepared with most of the toolchains needed.
+Building Bowtie 2 from source requires a GNU-like environment with GCC,
+GNU Make and other basics. It should be possible to build Bowtie 2 on
+most vanilla Linux installations or on a Mac installation with Xcode
+installed. Bowtie 2 can also be built on Windows using a 64-bit MinGW
+distribution and MSYS. In order to simplify the MinGW setup it might be
+worth investigating popular MinGW personal builds since these are coming
+already prepared with most of the toolchains needed.
-First, download the source package from the [sourceforge site]. Make sure
+First, download the source package from the sourceforge site. Make sure
you're getting the source package; the file downloaded should end in
-`-source.zip`. Unzip the file, change to the unzipped directory, and build the
-Bowtie 2 tools by running GNU `make` (usually with the command `make`, but
-sometimes with `gmake`) with no arguments. If building with MinGW, run `make`
-from the MSYS environment.
-
-+Bowtie 2 is using the multithreading software model in order to
-+speed up execution times on SMP architectures where this is possible.
-+The Threading Building Blocks library, TBB, is now the default
-+threading library in Bowtie 2. On POSIX platforms (like Linux, Mac
-+OS, etc.) if TBB is not available the pthread library will be used.
-+Although it is possible to use pthread library on Windows, a non-POSIX
-+platform, due to performance reasons Bowtie 2 will try to use Windows
-+native multithreading if possible. We recommend that you first
-+install the [Threading Building Blocks library], but if unable to
-+do so please specify `make NO_TBB=1`. TBB comes installed by default
-+on many popular Linux distros. Please note, packages built without
-+TBB will have _-legacy_ appended to the name.
-
-[MinGW]: http://www.mingw.org/
-[MSYS]: http://www.mingw.org/wiki/msys
-[pthreads]: http://sourceware.org/pthreads-win32/
-[GnuWin32]: http://gnuwin32.sf.net/packages/coreutils.htm
-[Threading Building Blocks library]: https://www.threadingbuildingblocks.org
-[Download]: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/
-[sourceforge site]: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/
-[Xcode]: http://developer.apple.com/xcode/
+-source.zip. Unzip the file, change to the unzipped directory, and build
+the Bowtie 2 tools by running GNU make (usually with the command make,
+but sometimes with gmake) with no arguments. If building with MinGW, run
+make from the MSYS environment.
+
+Bowtie 2 is using the multithreading software model in order to speed up
+execution times on SMP architectures where this is possible. The
+Threading Building Blocks library, TBB, is now the default threading
+library in Bowtie 2. On POSIX platforms (like Linux, Mac OS, etc.) if
+TBB is not available the pthread library will be used. Although it is
+possible to use pthread library on Windows, a non-POSIX platform, due to
+performance reasons Bowtie 2 will try to use Windows native
+multithreading if possible. We recommend that you first install the
+Threading Building Blocks library, but if unable to do so please specify
+make NO_TBB=1. TBB comes installed by default on many popular Linux
+distros. Please note, packages built without TBB will have _-legacy_
+appended to the name.
+
Adding to PATH
---------------
-By adding your new Bowtie 2 directory to your [PATH environment variable], you
-ensure that whenever you run `bowtie2`, `bowtie2-build` or `bowtie2-inspect`
-from the command line, you will get the version you just installed without
-having to specify the entire path. This is recommended for most users. To do
-this, follow your operating system's instructions for adding the directory to
-your [PATH].
+By adding your new Bowtie 2 directory to your PATH environment variable,
+you ensure that whenever you run bowtie2, bowtie2-build or
+bowtie2-inspect from the command line, you will get the version you just
+installed without having to specify the entire path. This is recommended
+for most users. To do this, follow your operating system's instructions
+for adding the directory to your PATH.
+
+If you would like to install Bowtie 2 by copying the Bowtie 2 executable
+files to an existing directory in your PATH, make sure that you copy all
+the executables, including bowtie2, bowtie2-align-s, bowtie2-align-l,
+bowtie2-build, bowtie2-build-s, bowtie2-build-l, bowtie2-inspect,
+bowtie2-inspect-s and bowtie2-inspect-l.
-If you would like to install Bowtie 2 by copying the Bowtie 2 executable files
-to an existing directory in your [PATH], make sure that you copy all the
-executables, including `bowtie2`, `bowtie2-align-s`, `bowtie2-align-l`,
-`bowtie2-build`, `bowtie2-build-s`, `bowtie2-build-l`, `bowtie2-inspect`,
-`bowtie2-inspect-s` and `bowtie2-inspect-l`.
-[PATH environment variable]: http://en.wikipedia.org/wiki/PATH_(variable)
-[PATH]: http://en.wikipedia.org/wiki/PATH_(variable)
-The `bowtie2` aligner
-=====================
+THE bowtie2 ALIGNER
-`bowtie2` takes a Bowtie 2 index and a set of sequencing read files and outputs
-a set of alignments in SAM format.
-"Alignment" is the process by which we discover how and where the read sequences
-are similar to the reference sequence. An "alignment" is a result from this
-process, specifically: an alignment is a way of "lining up" some or all of the
-characters in the read with some characters from the reference in a way that
-reveals how they're similar. For example:
+bowtie2 takes a Bowtie 2 index and a set of sequencing read files and
+outputs a set of alignments in SAM format.
+
+"Alignment" is the process by which we discover how and where the read
+sequences are similar to the reference sequence. An "alignment" is a
+result from this process, specifically: an alignment is a way of "lining
+up" some or all of the characters in the read with some characters from
+the reference in a way that reveals how they're similar. For example:
Read: GACTGGGCGATCTCGACTTCG
||||| |||||||||| |||
@@ -215,248 +190,256 @@ reveals how they're similar. For example:
Where dash symbols represent gaps and vertical bars show where aligned
characters match.
-We use alignment to make an educated guess as to where a read originated with
-respect to the reference genome. It's not always possible to determine this
-with certainty. For instance, if the reference genome contains several long
-stretches of As (`AAAAAAAAA` etc.) and the read sequence is a short stretch of As
-(`AAAAAAA`), we cannot know for certain exactly where in the sea of `A`s the
-read originated.
+We use alignment to make an educated guess as to where a read originated
+with respect to the reference genome. It's not always possible to
+determine this with certainty. For instance, if the reference genome
+contains several long stretches of As (AAAAAAAAA etc.) and the read
+sequence is a short stretch of As (AAAAAAA), we cannot know for certain
+exactly where in the sea of As the read originated.
+
End-to-end alignment versus local alignment
--------------------------------------------
-By default, Bowtie 2 performs end-to-end read alignment. That is, it searches
-for alignments involving all of the read characters. This is also called an
-"untrimmed" or "unclipped" alignment.
+By default, Bowtie 2 performs end-to-end read alignment. That is, it
+searches for alignments involving all of the read characters. This is
+also called an "untrimmed" or "unclipped" alignment.
-When the --local option is specified, Bowtie 2 performs local read alignment. In
-this mode, Bowtie 2 might "trim" or "clip" some read characters from one or both
-ends of the alignment if doing so maximizes the alignment score.
+When the --local option is specified, Bowtie 2 performs local read
+alignment. In this mode, Bowtie 2 might "trim" or "clip" some read
+characters from one or both ends of the alignment if doing so maximizes
+the alignment score.
-### End-to-end alignment example
+End-to-end alignment example
The following is an "end-to-end" alignment because it involves all the
-characters in the read. Such an alignment can be produced by Bowtie 2 in either
-end-to-end mode or in local mode.
+characters in the read. Such an alignment can be produced by Bowtie 2 in
+either end-to-end mode or in local mode.
Read: GACTGGGCGATCTCGACTTCG
Reference: GACTGCGATCTCGACATCG
-
+
Alignment:
Read: GACTGGGCGATCTCGACTTCG
||||| |||||||||| |||
Reference: GACTG--CGATCTCGACATCG
-### Local alignment example
+Local alignment example
-The following is a "local" alignment because some of the characters at the ends
-of the read do not participate. In this case, 4 characters are omitted (or
-"soft trimmed" or "soft clipped") from the beginning and 3 characters are
-omitted from the end. This sort of alignment can be produced by Bowtie 2 only
-in local mode.
+The following is a "local" alignment because some of the characters at
+the ends of the read do not participate. In this case, 4 characters are
+omitted (or "soft trimmed" or "soft clipped") from the beginning and 3
+characters are omitted from the end. This sort of alignment can be
+produced by Bowtie 2 only in local mode.
Read: ACGGTTGCGTTAATCCGCCACG
Reference: TAACTTGCGTTAAATCCGCCTGG
-
+
Alignment:
Read: ACGGTTGCGTTAA-TCCGCCACG
||||||||| ||||||
Reference: TAACTTGCGTTAAATCCGCCTGG
+
Scores: higher = more similar
------------------------------
-An alignment score quantifies how similar the read sequence is to the reference
-sequence aligned to. The higher the score, the more similar they are. A score
-is calculated by subtracting penalties for each difference (mismatch, gap, etc.)
-and, in local alignment mode, adding bonuses for each match.
+An alignment score quantifies how similar the read sequence is to the
+reference sequence aligned to. The higher the score, the more similar
+they are. A score is calculated by subtracting penalties for each
+difference (mismatch, gap, etc.) and, in local alignment mode, adding
+bonuses for each match.
-The scores can be configured with the `--ma` (match bonus), `--mp` (mismatch
-penalty), `--np` (penalty for having an N in either the read or the
-reference), `--rdg` (affine read gap penalty) and `--rfg` (affine reference
+The scores can be configured with the --ma (match bonus), --mp (mismatch
+penalty), --np (penalty for having an N in either the read or the
+reference), --rdg (affine read gap penalty) and --rfg (affine reference
gap penalty) options.
-### End-to-end alignment score example
+End-to-end alignment score example
-A mismatched base at a high-quality position in the read receives a penalty of
--6 by default. A length-2 read gap receives a penalty of -11 by default (-5 for
-the gap open, -3 for the first extension, -3 for the second extension). Thus,
-in end-to-end alignment mode, if the read is 50 bp long and it matches the
-reference exactly except for one mismatch at a high-quality position and one
-length-2 read gap, then the overall score is -(6 + 11) = -17.
+A mismatched base at a high-quality position in the read receives a
+penalty of -6 by default. A length-2 read gap receives a penalty of -11
+by default (-5 for the gap open, -3 for the first extension, -3 for the
+second extension). Thus, in end-to-end alignment mode, if the read is 50
+bp long and it matches the reference exactly except for one mismatch at
+a high-quality position and one length-2 read gap, then the overall
+score is -(6 + 11) = -17.
-The best possible alignment score in end-to-end mode is 0, which happens when
-there are no differences between the read and the reference.
+The best possible alignment score in end-to-end mode is 0, which happens
+when there are no differences between the read and the reference.
-### Local alignment score example
+Local alignment score example
-A mismatched base at a high-quality position in the read receives a penalty of
--6 by default. A length-2 read gap receives a penalty of -11 by default (-5 for
-the gap open, -3 for the first extension, -3 for the second extension). A base
-that matches receives a bonus of +2 be default. Thus, in local alignment mode,
-if the read is 50 bp long and it matches the reference exactly except for one
-mismatch at a high-quality position and one length-2 read gap, then the overall
-score equals the total bonus, 2 * 49, minus the total penalty, 6 + 11, = 81.
+A mismatched base at a high-quality position in the read receives a
+penalty of -6 by default. A length-2 read gap receives a penalty of -11
+by default (-5 for the gap open, -3 for the first extension, -3 for the
+second extension). A base that matches receives a bonus of +2 be
+default. Thus, in local alignment mode, if the read is 50 bp long and it
+matches the reference exactly except for one mismatch at a high-quality
+position and one length-2 read gap, then the overall score equals the
+total bonus, 2 * 49, minus the total penalty, 6 + 11, = 81.
-The best possible score in local mode equals the match bonus times the length of
-the read. This happens when there are no differences between the read and the
-reference.
+The best possible score in local mode equals the match bonus times the
+length of the read. This happens when there are no differences between
+the read and the reference.
-### Valid alignments meet or exceed the minimum score threshold
+Valid alignments meet or exceed the minimum score threshold
+
+For an alignment to be considered "valid" (i.e. "good enough") by Bowtie
+2, it must have an alignment score no less than the minimum score
+threshold. The threshold is configurable and is expressed as a function
+of the read length. In end-to-end alignment mode, the default minimum
+score threshold is -0.6 + -0.6 * L, where L is the read length. In local
+alignment mode, the default minimum score threshold is 20 + 8.0 * ln(L),
+where L is the read length. This can be configured with the --score-min
+option. For details on how to set options like --score-min that
+correspond to functions, see the section on setting function options.
-For an alignment to be considered "valid" (i.e. "good enough") by Bowtie 2, it
-must have an alignment score no less than the minimum score threshold. The
-threshold is configurable and is expressed as a function of the read length. In
-end-to-end alignment mode, the default minimum score threshold is `-0.6 + -0.6 *
-L`, where `L` is the read length. In local alignment mode, the default minimum
-score threshold is `20 + 8.0 * ln(L)`, where L is the read length. This can be
-configured with the `--score-min` option. For details on how to set options
-like `--score-min` that correspond to functions, see the section on [setting
-function options].
Mapping quality: higher = more unique
--------------------------------------
The aligner cannot always assign a read to its point of origin with high
-confidence. For instance, a read that originated inside a repeat element might
-align equally well to many occurrences of the element throughout the genome,
-leaving the aligner with no basis for preferring one over the others.
-
-Aligners characterize their degree of confidence in the point of origin by
-reporting a mapping quality: a non-negative integer Q = -10 log10 p, where p is
-an estimate of the probability that the alignment does not correspond to the
-read's true point of origin. Mapping quality is sometimes abbreviated MAPQ, and
-is recorded in the [SAM] `MAPQ` field.
-
-Mapping quality is related to "uniqueness." We say an alignment is unique if it
-has a much higher alignment score than all the other possible alignments. The
-bigger the gap between the best alignment's score and the second-best
-alignment's score, the more unique the best alignment, and the higher its mapping
-quality should be.
-
-Accurate mapping qualities are useful for downstream tools like variant callers.
-For instance, a variant caller might choose to ignore evidence from alignments
-with mapping quality less than, say, 10. A mapping quality of 10 or less
-indicates that there is at least a 1 in 10 chance that the read truly originated
-elsewhere.
-
-[SAM]: http://samtools.sourceforge.net/SAM1.pdf
-
-Aligning pairs
---------------
-
-A "paired-end" or "mate-pair" read consists of pair of mates, called mate 1 and
-mate 2. Pairs come with a prior expectation about (a) the relative orientation
-of the mates, and (b) the distance separating them on the original DNA molecule.
-Exactly what expectations hold for a given dataset depends on the lab procedures
-used to generate the data. For example, a common lab procedure for producing
-pairs is Illumina's Paired-end Sequencing Assay, which yields pairs with a
-relative orientation of FR ("forward, reverse") meaning that if mate 1 came from
-the Watson strand, mate 2 very likely came from the Crick strand and vice versa.
- Also, this protocol yields pairs where the expected genomic distance from end
-to end is about 200-500 base pairs.
-
-For simplicity, this manual uses the term "paired-end" to refer to any pair of
-reads with some expected relative orientation and distance. Depending on the
-protocol, these might actually be referred to as "paired-end" or "mate-paired."
-Also, we always refer to the individual sequences making up the pair as "mates."
-
-### Paired inputs
-
-Pairs are often stored in a pair of files, one file containing the mate 1s and
-the other containing the mates 2s. The first mate in the file for mate 1 forms
-a pair with the first mate in the file for mate 2, the second with the second,
-and so on. When aligning pairs with Bowtie 2, specify the file with the mate 1s
-mates using the `-1` argument and the file with the mate 2s using the `-2`
-argument. This causes Bowtie 2 to take the paired nature of the reads into
-account when aligning them.
-
-### Paired SAM output
-
-When Bowtie 2 prints a SAM alignment for a pair, it prints two records (i.e. two
-lines of output), one for each mate. The first record describes the alignment
-for mate 1 and the second record describes the alignment for mate 2. In both
-records, some of the fields of the SAM record describe various properties of the
-alignment; for instance, the 7th and 8th fields (`RNEXT` and `PNEXT`
-respectively) indicate the reference name and position where the other mate
-aligned, and the 9th field indicates the inferred length of the DNA fragment
-from which the two mates were sequenced. See the [SAM specification] for more
-details regarding these fields.
-
-### Concordant pairs match pair expectations, discordant pairs don't
-
-A pair that aligns with the expected relative mate orientation and with the
-expected range of distances between mates is said to align "concordantly". If
-both mates have unique alignments, but the alignments do not match paired-end
-expectations (i.e. the mates aren't in the expected relative orientation, or
-aren't within the expected distance range, or both), the pair is said to align
-"discordantly". Discordant alignments may be of particular interest, for
-instance, when seeking [structural variants].
-
-The expected relative orientation of the mates is set using the `--ff`,
-`--fr`, or `--rf` options. The expected range of inter-mates distances (as
-measured from the furthest extremes of the mates; also called "outer distance")
-is set with the `-I` and `-X` options. Note that setting `-I` and `-X`
-far apart makes Bowtie 2 slower. See documentation for `-I` and `-X`.
-
-To declare that a pair aligns discordantly, Bowtie 2 requires that both mates
-align uniquely. This is a conservative threshold, but this is often desirable
-when seeking structural variants.
-
-By default, Bowtie 2 searches for both concordant and discordant alignments,
-though searching for discordant alignments can be disabled with the
-`--no-discordant` option.
-
-[structural variants]: http://www.ncbi.nlm.nih.gov/dbvar/content/overview/
-
-### Mixed mode: paired where possible, unpaired otherwise
-
-If Bowtie 2 cannot find a paired-end alignment for a pair, by default it will go
-on to look for unpaired alignments for the constituent mates. This is called
-"mixed mode." To disable mixed mode, set the `--no-mixed` option.
-
-Bowtie 2 runs a little faster in `--no-mixed` mode, but will only consider
-alignment status of pairs per se, not individual mates.
-
-### Some SAM FLAGS describe paired-end properties
+confidence. For instance, a read that originated inside a repeat element
+might align equally well to many occurrences of the element throughout
+the genome, leaving the aligner with no basis for preferring one over
+the others.
+
+Aligners characterize their degree of confidence in the point of origin
+by reporting a mapping quality: a non-negative integer Q = -10 log10 p,
+where p is an estimate of the probability that the alignment does not
+correspond to the read's true point of origin. Mapping quality is
+sometimes abbreviated MAPQ, and is recorded in the SAM MAPQ field.
+
+Mapping quality is related to "uniqueness." We say an alignment is
+unique if it has a much higher alignment score than all the other
+possible alignments. The bigger the gap between the best alignment's
+score and the second-best alignment's score, the more unique the best
+alignment, and the higher its mapping quality should be.
+
+Accurate mapping qualities are useful for downstream tools like variant
+callers. For instance, a variant caller might choose to ignore evidence
+from alignments with mapping quality less than, say, 10. A mapping
+quality of 10 or less indicates that there is at least a 1 in 10 chance
+that the read truly originated elsewhere.
-The SAM `FLAGS` field, the second field in a SAM record, has multiple bits that
-describe the paired-end nature of the read and alignment. The first (least
-significant) bit (1 in decimal, 0x1 in hexadecimal) is set if the read is part
-of a pair. The second bit (2 in decimal, 0x2 in hexadecimal) is set if the read
-is part of a pair that aligned in a paired-end fashion. The fourth bit (8 in
-decimal, 0x8 in hexadecimal) is set if the read is part of a pair and the other
-mate in the pair had at least one valid alignment. The sixth bit (32 in
-decimal, 0x20 in hexadecimal) is set if the read is part of a pair and the other
-mate in the pair aligned to the Crick strand (or, equivalently, if the reverse
-complement of the other mate aligned to the Watson strand). The seventh bit (64
-in decimal, 0x40 in hexadecimal) is set if the read is mate 1 in a pair. The
-eighth bit (128 in decimal, 0x80 in hexadecimal) is set if the read is mate 2 in
-a pair. See the [SAM specification] for a more detailed description of the
-`FLAGS` field.
-### Some SAM optional fields describe more paired-end properties
-
-The last several fields of each SAM record usually contain SAM optional fields,
-which are simply tab-separated strings conveying additional information about
-the reads and alignments. A SAM optional field is formatted like this: "XP:i:1"
-where "XP" is the `TAG`, "i" is the `TYPE` ("integer" in this case), and "1" is
-the `VALUE`. See the [SAM specification] for details regarding SAM optional
-fields.
+Aligning pairs
-### Mates can overlap, contain, or dovetail each other
+A "paired-end" or "mate-pair" read consists of pair of mates, called
+mate 1 and mate 2. Pairs come with a prior expectation about (a) the
+relative orientation of the mates, and (b) the distance separating them
+on the original DNA molecule. Exactly what expectations hold for a given
+dataset depends on the lab procedures used to generate the data. For
+example, a common lab procedure for producing pairs is Illumina's
+Paired-end Sequencing Assay, which yields pairs with a relative
+orientation of FR ("forward, reverse") meaning that if mate 1 came from
+the Watson strand, mate 2 very likely came from the Crick strand and
+vice versa. Also, this protocol yields pairs where the expected genomic
+distance from end to end is about 200-500 base pairs.
+
+For simplicity, this manual uses the term "paired-end" to refer to any
+pair of reads with some expected relative orientation and distance.
+Depending on the protocol, these might actually be referred to as
+"paired-end" or "mate-paired." Also, we always refer to the individual
+sequences making up the pair as "mates."
+
+Paired inputs
+
+Pairs are often stored in a pair of files, one file containing the mate
+1s and the other containing the mates 2s. The first mate in the file for
+mate 1 forms a pair with the first mate in the file for mate 2, the
+second with the second, and so on. When aligning pairs with Bowtie 2,
+specify the file with the mate 1s mates using the -1 argument and the
+file with the mate 2s using the -2 argument. This causes Bowtie 2 to
+take the paired nature of the reads into account when aligning them.
+
+Paired SAM output
+
+When Bowtie 2 prints a SAM alignment for a pair, it prints two records
+(i.e. two lines of output), one for each mate. The first record
+describes the alignment for mate 1 and the second record describes the
+alignment for mate 2. In both records, some of the fields of the SAM
+record describe various properties of the alignment; for instance, the
+7th and 8th fields (RNEXT and PNEXT respectively) indicate the reference
+name and position where the other mate aligned, and the 9th field
+indicates the inferred length of the DNA fragment from which the two
+mates were sequenced. See the SAM specification for more details
+regarding these fields.
+
+Concordant pairs match pair expectations, discordant pairs don't
+
+A pair that aligns with the expected relative mate orientation and with
+the expected range of distances between mates is said to align
+"concordantly". If both mates have unique alignments, but the alignments
+do not match paired-end expectations (i.e. the mates aren't in the
+expected relative orientation, or aren't within the expected distance
+range, or both), the pair is said to align "discordantly". Discordant
+alignments may be of particular interest, for instance, when seeking
+structural variants.
+
+The expected relative orientation of the mates is set using the --ff,
+--fr, or --rf options. The expected range of inter-mates distances (as
+measured from the furthest extremes of the mates; also called "outer
+distance") is set with the -I and -X options. Note that setting -I and
+-X far apart makes Bowtie 2 slower. See documentation for -I and -X.
+
+To declare that a pair aligns discordantly, Bowtie 2 requires that both
+mates align uniquely. This is a conservative threshold, but this is
+often desirable when seeking structural variants.
+
+By default, Bowtie 2 searches for both concordant and discordant
+alignments, though searching for discordant alignments can be disabled
+with the --no-discordant option.
+
+Mixed mode: paired where possible, unpaired otherwise
+
+If Bowtie 2 cannot find a paired-end alignment for a pair, by default it
+will go on to look for unpaired alignments for the constituent mates.
+This is called "mixed mode." To disable mixed mode, set the --no-mixed
+option.
-The fragment and read lengths might be such that alignments for the two mates
-from a pair overlap each other. Consider this example:
+Bowtie 2 runs a little faster in --no-mixed mode, but will only consider
+alignment status of pairs per se, not individual mates.
-(For these examples, assume we expect mate 1 to align to the left of mate 2.)
+Some SAM FLAGS describe paired-end properties
+
+The SAM FLAGS field, the second field in a SAM record, has multiple bits
+that describe the paired-end nature of the read and alignment. The first
+(least significant) bit (1 in decimal, 0x1 in hexadecimal) is set if the
+read is part of a pair. The second bit (2 in decimal, 0x2 in
+hexadecimal) is set if the read is part of a pair that aligned in a
+paired-end fashion. The fourth bit (8 in decimal, 0x8 in hexadecimal) is
+set if the read is part of a pair and the other mate in the pair had at
+least one valid alignment. The sixth bit (32 in decimal, 0x20 in
+hexadecimal) is set if the read is part of a pair and the other mate in
+the pair aligned to the Crick strand (or, equivalently, if the reverse
+complement of the other mate aligned to the Watson strand). The seventh
+bit (64 in decimal, 0x40 in hexadecimal) is set if the read is mate 1 in
+a pair. The eighth bit (128 in decimal, 0x80 in hexadecimal) is set if
+the read is mate 2 in a pair. See the SAM specification for a more
+detailed description of the FLAGS field.
+
+Some SAM optional fields describe more paired-end properties
+
+The last several fields of each SAM record usually contain SAM optional
+fields, which are simply tab-separated strings conveying additional
+information about the reads and alignments. A SAM optional field is
+formatted like this: "XP:i:1" where "XP" is the TAG, "i" is the TYPE
+("integer" in this case), and "1" is the VALUE. See the SAM
+specification for details regarding SAM optional fields.
+
+Mates can overlap, contain, or dovetail each other
+
+The fragment and read lengths might be such that alignments for the two
+mates from a pair overlap each other. Consider this example:
+
+(For these examples, assume we expect mate 1 to align to the left of
+mate 2.)
Mate 1: GCAGATTATATGAGTCAGCTACGATATTGTT
Mate 2: TGTTTGGGGTGACACATTACGCGTCTTTGAC
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
-It's also possible, though unusual, for one mate alignment to contain the other,
-as in these examples:
+It's also possible, though unusual, for one mate alignment to contain
+the other, as in these examples:
Mate 1: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGC
Mate 2: TGTTTGGGGTGACACATTACGC
@@ -466,243 +449,245 @@ as in these examples:
Mate 2: CTACGATATTGTTTGGGGTGAC
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
-And it's also possible, though unusual, for the mates to "dovetail", with the
-mates seemingly extending "past" each other as in this example:
+And it's also possible, though unusual, for the mates to "dovetail",
+with the mates seemingly extending "past" each other as in this example:
Mate 1: GTCAGCTACGATATTGTTTGGGGTGACACATTACGC
Mate 2: TATGAGTCAGCTACGATATTGTTTGGGGTGACACAT
Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC
-In some situations, it's desirable for the aligner to consider all these cases
-as "concordant" as long as other paired-end constraints are not violated. Bowtie
-2's default behavior is to consider overlapping and containing as being
-consistent with concordant alignment. By default, dovetailing is considered
-inconsistent with concordant alignment.
+In some situations, it's desirable for the aligner to consider all these
+cases as "concordant" as long as other paired-end constraints are not
+violated. Bowtie 2's default behavior is to consider overlapping and
+containing as being consistent with concordant alignment. By default,
+dovetailing is considered inconsistent with concordant alignment.
+
+These defaults can be overridden. Setting --no-overlap causes Bowtie 2
+to consider overlapping mates as non-concordant. Setting --no-contain
+causes Bowtie 2 to consider cases where one mate alignment contains the
+other as non-concordant. Setting --dovetail causes Bowtie 2 to consider
+cases where the mate alignments dovetail as concordant.
-These defaults can be overridden. Setting `--no-overlap` causes Bowtie 2 to
-consider overlapping mates as non-concordant. Setting `--no-contain` causes
-Bowtie 2 to consider cases where one mate alignment contains the other as
-non-concordant. Setting `--dovetail` causes Bowtie 2 to consider cases where
-the mate alignments dovetail as concordant.
Reporting
----------
-
-The reporting mode governs how many alignments Bowtie 2 looks for, and how to
-report them. Bowtie 2 has three distinct reporting modes. The default
-reporting mode is similar to the default reporting mode of many other read
-alignment tools, including [BWA]. It is also similar to Bowtie 1's `-M`
-alignment mode.
-
-In general, when we say that a read has an alignment, we mean that it has a
-[valid alignment]. When we say that a read has multiple alignments, we mean
-that it has multiple alignments that are valid and distinct from one another.
-
-[BWA]: http://bio-bwa.sourceforge.net/
-
-### Distinct alignments map a read to different places
-
-Two alignments for the same individual read are "distinct" if they map the same
-read to different places. Specifically, we say that two alignments are distinct
-if there are no alignment positions where a particular read offset is aligned
-opposite a particular reference offset in both alignments with the same
-orientation. E.g. if the first alignment is in the forward orientation and
-aligns the read character at read offset 10 to the reference character at
-chromosome 3, offset 3,445,245, and the second alignment is also in the forward
-orientation and also aligns the read character at read offset 10 to the
-reference character at chromosome 3, offset 3,445,245, they are not distinct
+
+The reporting mode governs how many alignments Bowtie 2 looks for, and
+how to report them. Bowtie 2 has three distinct reporting modes. The
+default reporting mode is similar to the default reporting mode of many
+other read alignment tools, including BWA. It is also similar to Bowtie
+1's -M alignment mode.
+
+In general, when we say that a read has an alignment, we mean that it
+has a valid alignment. When we say that a read has multiple alignments,
+we mean that it has multiple alignments that are valid and distinct from
+one another.
+
+Distinct alignments map a read to different places
+
+Two alignments for the same individual read are "distinct" if they map
+the same read to different places. Specifically, we say that two
+alignments are distinct if there are no alignment positions where a
+particular read offset is aligned opposite a particular reference offset
+in both alignments with the same orientation. E.g. if the first
+alignment is in the forward orientation and aligns the read character at
+read offset 10 to the reference character at chromosome 3, offset
+3,445,245, and the second alignment is also in the forward orientation
+and also aligns the read character at read offset 10 to the reference
+character at chromosome 3, offset 3,445,245, they are not distinct
alignments.
-Two alignments for the same pair are distinct if either the mate 1s in the two
-paired-end alignments are distinct or the mate 2s in the two alignments are
-distinct or both.
-
-### Default mode: search for multiple alignments, report the best one
-
-By default, Bowtie 2 searches for distinct, valid alignments for each read. When
-it finds a valid alignment, it generally will continue to look for alignments
-that are nearly as good or better. It will eventually stop looking, either
-because it exceeded a limit placed on search effort (see `-D` and `-R`) or
-because it already knows all it needs to know to report an alignment.
-Information from the best alignments are used to estimate mapping quality (the
-`MAPQ` [SAM] field) and to set SAM optional fields, such as `AS:i` and
-`XS:i`. Bowtie 2 does not guarantee that the alignment reported is the best
-possible in terms of alignment score.
-
-See also: `-D`, which puts an upper limit on the number of dynamic programming
-problems (i.e. seed extensions) that can "fail" in a row before Bowtie 2 stops
-searching. Increasing `-D` makes Bowtie 2 slower, but increases the
-likelihood that it will report the correct alignment for a read that aligns many
-places.
-
-See also: `-R`, which sets the maximum number of times Bowtie 2 will "re-seed"
-when attempting to align a read with repetitive seeds. Increasing `-R` makes
-Bowtie 2 slower, but increases the likelihood that it will report the correct
-alignment for a read that aligns many places.
-
-### -k mode: search for one or more alignments, report each
-
-In `-k` mode, Bowtie 2 searches for up to N distinct, valid alignments for
-each read, where N equals the integer specified with the `-k` parameter. That
-is, if `-k 2` is specified, Bowtie 2 will search for at most 2 distinct
-alignments. It reports all alignments found, in descending order by alignment
-score. The alignment score for a paired-end alignment equals the sum of the
-alignment scores of the individual mates. Each reported read or pair alignment
-beyond the first has the SAM 'secondary' bit (which equals 256) set in its FLAGS
-field. See the [SAM specification] for details.
-
-Bowtie 2 does not "find" alignments in any specific order, so for reads that
-have more than N distinct, valid alignments, Bowtie 2 does not guarantee that
-the N alignments reported are the best possible in terms of alignment score.
-Still, this mode can be effective and fast in situations where the user cares
-more about whether a read aligns (or aligns a certain number of times) than
-where exactly it originated.
-
-[SAM specification]: http://samtools.sourceforge.net/SAM1.pdf
-
-### -a mode: search for and report all alignments
-
-`-a` mode is similar to `-k` mode except that there is no upper limit on the
-number of alignments Bowtie 2 should report. Alignments are reported in
-descending order by alignment score. The alignment score for a paired-end
-alignment equals the sum of the alignment scores of the individual mates. Each
-reported read or pair alignment beyond the first has the SAM 'secondary' bit
-(which equals 256) set in its FLAGS field. See the [SAM specification] for
-details.
-
-Some tools are designed with this reporting mode in mind. Bowtie 2 is not! For
-very large genomes, this mode is very slow.
-
-[SAM specification]: http://samtools.sourceforge.net/SAM1.pdf
-
-### Randomness in Bowtie 2
-
-Bowtie 2's search for alignments for a given read is "randomized." That is,
-when Bowtie 2 encounters a set of equally-good choices, it uses a pseudo-random
-number to choose. For example, if Bowtie 2 discovers a set of 3 equally-good
-alignments and wants to decide which to report, it picks a pseudo-random integer
-0, 1 or 2 and reports the corresponding alignment. Arbitrary choices can crop up
-at various points during alignment.
-
-The pseudo-random number generator is re-initialized for every read, and the
-seed used to initialize it is a function of the read name, nucleotide string,
-quality string, and the value specified with `--seed`. If you run the same
-version of Bowtie 2 on two reads with identical names, nucleotide strings, and
-quality strings, and if `--seed` is set the same for both runs, Bowtie 2 will
-produce the same output; i.e., it will align the read to the same place, even if
-there are multiple equally good alignments. This is intuitive and desirable in
-most cases. Most users expect Bowtie to produce the same output when run twice
-on the same input.
-
-However, when the user specifies the `--non-deterministic` option, Bowtie 2
-will use the current time to re-initialize the pseudo-random number generator.
-When this is specified, Bowtie 2 might report different alignments for identical
-reads. This is counter-intuitive for some users, but might be more appropriate
-in situations where the input consists of many identical reads.
+Two alignments for the same pair are distinct if either the mate 1s in
+the two paired-end alignments are distinct or the mate 2s in the two
+alignments are distinct or both.
+
+Default mode: search for multiple alignments, report the best one
+
+By default, Bowtie 2 searches for distinct, valid alignments for each
+read. When it finds a valid alignment, it generally will continue to
+look for alignments that are nearly as good or better. It will
+eventually stop looking, either because it exceeded a limit placed on
+search effort (see -D and -R) or because it already knows all it needs
+to know to report an alignment. Information from the best alignments are
+used to estimate mapping quality (the MAPQ SAM field) and to set SAM
+optional fields, such as AS:i and XS:i. Bowtie 2 does not guarantee that
+the alignment reported is the best possible in terms of alignment score.
+
+See also: -D, which puts an upper limit on the number of dynamic
+programming problems (i.e. seed extensions) that can "fail" in a row
+before Bowtie 2 stops searching. Increasing -D makes Bowtie 2 slower,
+but increases the likelihood that it will report the correct alignment
+for a read that aligns many places.
+
+See also: -R, which sets the maximum number of times Bowtie 2 will
+"re-seed" when attempting to align a read with repetitive seeds.
+Increasing -R makes Bowtie 2 slower, but increases the likelihood that
+it will report the correct alignment for a read that aligns many places.
+
+-k mode: search for one or more alignments, report each
+
+In -k mode, Bowtie 2 searches for up to N distinct, valid alignments for
+each read, where N equals the integer specified with the -k parameter.
+That is, if -k 2 is specified, Bowtie 2 will search for at most 2
+distinct alignments. It reports all alignments found, in descending
+order by alignment score. The alignment score for a paired-end alignment
+equals the sum of the alignment scores of the individual mates. Each
+reported read or pair alignment beyond the first has the SAM 'secondary'
+bit (which equals 256) set in its FLAGS field. See the SAM specification
+for details.
+
+Bowtie 2 does not "find" alignments in any specific order, so for reads
+that have more than N distinct, valid alignments, Bowtie 2 does not
+guarantee that the N alignments reported are the best possible in terms
+of alignment score. Still, this mode can be effective and fast in
+situations where the user cares more about whether a read aligns (or
+aligns a certain number of times) than where exactly it originated.
+
+-a mode: search for and report all alignments
+
+-a mode is similar to -k mode except that there is no upper limit on the
+number of alignments Bowtie 2 should report. Alignments are reported in
+descending order by alignment score. The alignment score for a
+paired-end alignment equals the sum of the alignment scores of the
+individual mates. Each reported read or pair alignment beyond the first
+has the SAM 'secondary' bit (which equals 256) set in its FLAGS field.
+See the SAM specification for details.
+
+Some tools are designed with this reporting mode in mind. Bowtie 2 is
+not! For very large genomes, this mode is very slow.
+
+Randomness in Bowtie 2
+
+Bowtie 2's search for alignments for a given read is "randomized." That
+is, when Bowtie 2 encounters a set of equally-good choices, it uses a
+pseudo-random number to choose. For example, if Bowtie 2 discovers a set
+of 3 equally-good alignments and wants to decide which to report, it
+picks a pseudo-random integer 0, 1 or 2 and reports the corresponding
+alignment. Arbitrary choices can crop up at various points during
+alignment.
+
+The pseudo-random number generator is re-initialized for every read, and
+the seed used to initialize it is a function of the read name,
+nucleotide string, quality string, and the value specified with --seed.
+If you run the same version of Bowtie 2 on two reads with identical
+names, nucleotide strings, and quality strings, and if --seed is set the
+same for both runs, Bowtie 2 will produce the same output; i.e., it will
+align the read to the same place, even if there are multiple equally
+good alignments. This is intuitive and desirable in most cases. Most
+users expect Bowtie to produce the same output when run twice on the
+same input.
+
+However, when the user specifies the --non-deterministic option, Bowtie
+2 will use the current time to re-initialize the pseudo-random number
+generator. When this is specified, Bowtie 2 might report different
+alignments for identical reads. This is counter-intuitive for some
+users, but might be more appropriate in situations where the input
+consists of many identical reads.
+
Multiseed heuristic
--------------------
-
-To rapidly narrow the number of possible alignments that must be considered,
-Bowtie 2 begins by extracting substrings ("seeds") from the read and its reverse
-complement and aligning them in an ungapped fashion with the help of the [FM
-Index]. This is "multiseed alignment" and it is similar to what [Bowtie 1
-does], except Bowtie 1 attempts to align the entire read this way.
-
-This initial step makes Bowtie 2 much faster than it would be without such a
-filter, but at the expense of missing some valid alignments. For instance, it
-is possible for a read to have a valid overall alignment but to have no valid
-seed alignments because each potential seed alignment is interrupted by too many
-mismatches or gaps.
-
-The trade-off between speed and sensitivity/accuracy can be adjusted by setting
-the seed length (`-L`), the interval between extracted seeds (`-i`), and the
-number of mismatches permitted per seed (`-N`). For more sensitive alignment,
-set these parameters to (a) make the seeds closer together, (b) make the seeds
-shorter, and/or (c) allow more mismatches. You can adjust these options
-one-by-one, though Bowtie 2 comes with some useful combinations of options
-prepackaged as "[preset options]."
-
-`-D` and `-R` are also options that adjust the trade-off between speed and
+
+To rapidly narrow the number of possible alignments that must be
+considered, Bowtie 2 begins by extracting substrings ("seeds") from the
+read and its reverse complement and aligning them in an ungapped fashion
+with the help of the FM Index. This is "multiseed alignment" and it is
+similar to what Bowtie 1 does, except Bowtie 1 attempts to align the
+entire read this way.
+
+This initial step makes Bowtie 2 much faster than it would be without
+such a filter, but at the expense of missing some valid alignments. For
+instance, it is possible for a read to have a valid overall alignment
+but to have no valid seed alignments because each potential seed
+alignment is interrupted by too many mismatches or gaps.
+
+The trade-off between speed and sensitivity/accuracy can be adjusted by
+setting the seed length (-L), the interval between extracted seeds (-i),
+and the number of mismatches permitted per seed (-N). For more sensitive
+alignment, set these parameters to (a) make the seeds closer together,
+(b) make the seeds shorter, and/or (c) allow more mismatches. You can
+adjust these options one-by-one, though Bowtie 2 comes with some useful
+combinations of options prepackaged as "preset options."
+
+-D and -R are also options that adjust the trade-off between speed and
sensitivity/accuracy.
-### FM Index memory footprint
+FM Index memory footprint
-Bowtie 2 uses the [FM Index] to find ungapped alignments for seeds. This step
-accounts for the bulk of Bowtie 2's memory footprint, as the [FM Index] itself
-is typically the largest data structure used. For instance, the memory
-footprint of the [FM Index] for the human genome is about 3.2 gigabytes of RAM.
+Bowtie 2 uses the FM Index to find ungapped alignments for seeds. This
+step accounts for the bulk of Bowtie 2's memory footprint, as the FM
+Index itself is typically the largest data structure used. For instance,
+the memory footprint of the FM Index for the human genome is about 3.2
+gigabytes of RAM.
-[Bowtie 1 does]: http://genomebiology.com/2009/10/3/R25
-[Bowtie 1 paper]: http://genomebiology.com/2009/10/3/R25
-[FM Index]: http://portal.acm.org/citation.cfm?id=796543
-[bi-directional BWT approach]: http://www.computer.org/portal/web/csdl/doi/10.1109/BIBM.2009.42
Ambiguous characters
---------------------
-Non-whitespace characters besides A, C, G or T are considered "ambiguous." N is
-a common ambiguous character that appears in reference sequences. Bowtie 2
-considers all ambiguous characters in the reference (including [IUPAC nucleotide
-codes]) to be Ns.
+Non-whitespace characters besides A, C, G or T are considered
+"ambiguous." N is a common ambiguous character that appears in reference
+sequences. Bowtie 2 considers all ambiguous characters in the reference
+(including IUPAC nucleotide codes) to be Ns.
-Bowtie 2 allows alignments to overlap ambiguous characters in the reference. An
-alignment position that contains an ambiguous character in the read, reference,
-or both, is penalized according to `--np`. `--n-ceil` sets an upper limit
-on the number of positions that may contain ambiguous reference characters in a
-valid alignment. The optional field `XN:i` reports the number of ambiguous
-reference characters overlapped by an alignment.
+Bowtie 2 allows alignments to overlap ambiguous characters in the
+reference. An alignment position that contains an ambiguous character in
+the read, reference, or both, is penalized according to --np. --n-ceil
+sets an upper limit on the number of positions that may contain
+ambiguous reference characters in a valid alignment. The optional field
+XN:i reports the number of ambiguous reference characters overlapped by
+an alignment.
-Note that the [multiseed heuristic] cannot find *seed* alignments that overlap
-ambiguous reference characters. For an alignment overlapping an ambiguous
-reference character to be found, it must have one or more seed alignments that
-do not overlap ambiguous reference characters.
+Note that the multiseed heuristic cannot find _seed_ alignments that
+overlap ambiguous reference characters. For an alignment overlapping an
+ambiguous reference character to be found, it must have one or more seed
+alignments that do not overlap ambiguous reference characters.
-[IUPAC nucleotide codes]: http://www.bioinformatics.org/sms/iupac.html
Presets: setting many settings at once
---------------------------------------
-Bowtie 2 comes with some useful combinations of parameters packaged into shorter
-"preset" parameters. For example, running Bowtie 2 with the `--very-sensitive`
-option is the same as running with options: `-D 20 -R 3 -N 0 -L 20 -i S,1,0.50`.
- The preset options that come with Bowtie 2 are designed to cover a wide area of
-the speed/sensitivity/accuracy trade-off space, with the presets ending in `fast`
-generally being faster but less sensitive and less accurate, and the presets
-ending in `sensitive` generally being slower but more sensitive and more
-accurate. See the [documentation for the preset options] for details.
+Bowtie 2 comes with some useful combinations of parameters packaged into
+shorter "preset" parameters. For example, running Bowtie 2 with the
+--very-sensitive option is the same as running with options:
+-D 20 -R 3 -N 0 -L 20 -i S,1,0.50. The preset options that come with
+Bowtie 2 are designed to cover a wide area of the
+speed/sensitivity/accuracy trade-off space, with the presets ending in
+fast generally being faster but less sensitive and less accurate, and
+the presets ending in sensitive generally being slower but more
+sensitive and more accurate. See the documentation for the preset
+options for details.
+
Filtering
----------
-
-Some reads are skipped or "filtered out" by Bowtie 2. For example, reads may be
-filtered out because they are extremely short or have a high proportion of
-ambiguous nucleotides. Bowtie 2 will still print a SAM record for such a read,
-but no alignment will be reported and the `YF:i` SAM optional field will be
-set to indicate the reason the read was filtered.
-
-* `YF:Z:LN`: the read was filtered because it had length less than or equal to
-the number of seed mismatches set with the `-N` option.
-* `YF:Z:NS`: the read was filtered because it contains a number of ambiguous
-characters (usually `N` or `.`) greater than the ceiling specified with
-`--n-ceil`.
-* `YF:Z:SC`: the read was filtered because the read length and the match bonus
-(set with `--ma`) are such that the read can't possibly earn an alignment
-score greater than or equal to the threshold set with `--score-min`
-* `YF:Z:QC`: the read was filtered because it was marked as failing quality
-control and the user specified the `--qc-filter` option. This only happens
-when the input is in Illumina's QSEQ format (i.e. when `--qseq` is specified)
-and the last (11th) field of the read's QSEQ record contains `1`.
-
-If a read could be filtered for more than one reason, the value `YF:Z` flag will
-reflect only one of those reasons.
+
+Some reads are skipped or "filtered out" by Bowtie 2. For example, reads
+may be filtered out because they are extremely short or have a high
+proportion of ambiguous nucleotides. Bowtie 2 will still print a SAM
+record for such a read, but no alignment will be reported and the YF:i
+SAM optional field will be set to indicate the reason the read was
+filtered.
+
+- YF:Z:LN: the read was filtered because it had length less than or
+ equal to the number of seed mismatches set with the -N option.
+- YF:Z:NS: the read was filtered because it contains a number of
+ ambiguous characters (usually N or .) greater than the ceiling
+ specified with --n-ceil.
+- YF:Z:SC: the read was filtered because the read length and the match
+ bonus (set with --ma) are such that the read can't possibly earn an
+ alignment score greater than or equal to the threshold set with
+ --score-min
+- YF:Z:QC: the read was filtered because it was marked as failing
+ quality control and the user specified the --qc-filter option. This
+ only happens when the input is in Illumina's QSEQ format (i.e. when
+ --qseq is specified) and the last (11th) field of the read's QSEQ
+ record contains 1.
+
+If a read could be filtered for more than one reason, the value YF:Z
+flag will reflect only one of those reasons.
+
Alignment summary
-------------------
-When Bowtie 2 finishes running, it prints messages summarizing what happened.
-These messages are printed to the "standard error" ("stderr") filehandle. For
-datasets consisting of unpaired reads, the summary might look like this:
+When Bowtie 2 finishes running, it prints messages summarizing what
+happened. These messages are printed to the "standard error" ("stderr")
+filehandle. For datasets consisting of unpaired reads, the summary might
+look like this:
20000 reads; of these:
20000 (100.00%) were unpaired; of these:
@@ -731,137 +716,142 @@ For datasets consisting of pairs, the summary might look like this:
The indentation indicates how subtotals relate to totals.
+
Wrapper scripts
----------------
-The `bowtie2`, `bowtie2-build` and `bowtie2-inspect` executables are actually
-wrapper scripts that call binary programs as appropriate. The wrappers shield
-users from having to distinguish between "small" and "large" index formats,
-discussed briefly in the following section. Also, the `bowtie2` wrapper
-provides some key functionality, like the ability to handle compressed inputs,
-and the functionality for `--un`, `--al` and related options.
+The bowtie2, bowtie2-build and bowtie2-inspect executables are actually
+wrapper scripts that call binary programs as appropriate. The wrappers
+shield users from having to distinguish between "small" and "large"
+index formats, discussed briefly in the following section. Also, the
+bowtie2 wrapper provides some key functionality, like the ability to
+handle compressed inputs, and the functionality for --un, --al and
+related options.
+
+It is recommended that you always run the bowtie2 wrappers and not run
+the binaries directly.
-It is recommended that you always run the bowtie2 wrappers and not run the
-binaries directly.
Small and large indexes
------------------------
-`bowtie2-build` can index reference genomes of any size. For genomes less than
-about 4 billion nucleotides in length, `bowtie2-build` builds a "small" index
-using 32-bit numbers in various parts of the index. When the genome is longer,
-`bowtie2-build` builds a "large" index using 64-bit numbers. Small indexes are
-stored in files with the `.bt2` extension, and large indexes are stored in
-files with the `.bt2l` extension. The user need not worry about whether a
-particular index is small or large; the wrapper scripts will automatically build
-and use the appropriate index.
+bowtie2-build can index reference genomes of any size. For genomes less
+than about 4 billion nucleotides in length, bowtie2-build builds a
+"small" index using 32-bit numbers in various parts of the index. When
+the genome is longer, bowtie2-build builds a "large" index using 64-bit
+numbers. Small indexes are stored in files with the .bt2 extension, and
+large indexes are stored in files with the .bt2l extension. The user
+need not worry about whether a particular index is small or large; the
+wrapper scripts will automatically build and use the appropriate index.
+
Performance tuning
-------------------
-1. If your computer has multiple processors/cores, use `-p`
+1. If your computer has multiple processors/cores, use -p
- The `-p` option causes Bowtie 2 to launch a specified number of parallel
- search threads. Each thread runs on a different processor/core and all
- threads find alignments in parallel, increasing alignment throughput by
- approximately a multiple of the number of threads (though in practice,
- speedup is somewhat worse than linear).
+ The -p option causes Bowtie 2 to launch a specified number of
+ parallel search threads. Each thread runs on a different
+ processor/core and all threads find alignments in parallel,
+ increasing alignment throughput by approximately a multiple of the
+ number of threads (though in practice, speedup is somewhat worse
+ than linear).
2. If reporting many alignments per read, try reducing
- `bowtie2-build --offrate`
+ bowtie2-build --offrate
- If you are using `-k` or `-a` options and Bowtie 2 is reporting many
- alignments per read, using an index with a denser SA sample can speed
- things up considerably. To do this, specify a smaller-than-default
- `-o`/`--offrate` value when running `bowtie2-build`.
+ If you are using -k or -a options and Bowtie 2 is reporting many
+ alignments per read, using an index with a denser SA sample can
+ speed things up considerably. To do this, specify a
+ smaller-than-default -o/--offrate value when running bowtie2-build.
A denser SA sample yields a larger index, but is also particularly
- effective at speeding up alignment when many alignments are reported per
- read.
+ effective at speeding up alignment when many alignments are reported
+ per read.
-3. If `bowtie2` "thrashes", try increasing `bowtie2-build --offrate`
+3. If bowtie2 "thrashes", try increasing bowtie2-build --offrate
- If `bowtie2` runs very slowly on a relatively low-memory computer, try
- setting `-o`/`--offrate` to a *larger* value when building the index.
+ If bowtie2 runs very slowly on a relatively low-memory computer, try
+ setting -o/--offrate to a _larger_ value when building the index.
This decreases the memory footprint of the index.
+
Command Line
-------------
-### Setting function options
+Setting function options
-Some Bowtie 2 options specify a function rather than an individual number or
-setting. In these cases the user specifies three parameters: (a) a function
-type `F`, (b) a constant term `B`, and (c) a coefficient `A`. The available
-function types are constant (`C`), linear (`L`), square-root (`S`), and natural
-log (`G`). The parameters are specified as `F,B,A` - that is, the function type,
-the constant term, and the coefficient are separated by commas with no
-whitespace. The constant term and coefficient may be negative and/or
-floating-point numbers.
+Some Bowtie 2 options specify a function rather than an individual
+number or setting. In these cases the user specifies three parameters:
+(a) a function type F, (b) a constant term B, and (c) a coefficient A.
+The available function types are constant (C), linear (L), square-root
+(S), and natural log (G). The parameters are specified as F,B,A - that
+is, the function type, the constant term, and the coefficient are
+separated by commas with no whitespace. The constant term and
+coefficient may be negative and/or floating-point numbers.
-For example, if the function specification is `L,-0.4,-0.6`, then the function
-defined is:
+For example, if the function specification is L,-0.4,-0.6, then the
+function defined is:
f(x) = -0.4 + -0.6 * x
-If the function specification is `G,1,5.4`, then the function defined is:
+If the function specification is G,1,5.4, then the function defined is:
f(x) = 1.0 + 5.4 * ln(x)
-See the documentation for the option in question to learn what the parameter `x`
-is for. For example, in the case if the `--score-min` option, the function
-`f(x)` sets the minimum alignment score necessary for an alignment to be
-considered valid, and `x` is the read length.
+See the documentation for the option in question to learn what the
+parameter x is for. For example, in the case if the --score-min option,
+the function f(x) sets the minimum alignment score necessary for an
+alignment to be considered valid, and x is the read length.
-### Usage
+Usage
- bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r>} -S [<hit>]
+ bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i>} -S [<sam>]
-### Main arguments
+Main arguments
-x <bt2-idx>
-The basename of the index for the reference genome. The basename is the name of
-any of the index files up to but not including the final `.1.bt2` / `.rev.1.bt2`
-/ etc. `bowtie2` looks for the specified index first in the current directory,
-then in the directory specified in the `BOWTIE2_INDEXES` environment variable.
+The basename of the index for the reference genome. The basename is the
+name of any of the index files up to but not including the final .1.bt2
+/ .rev.1.bt2 / etc. bowtie2 looks for the specified index first in the
+current directory, then in the directory specified in the
+BOWTIE2_INDEXES environment variable.
-1 <m1>
-Comma-separated list of files containing mate 1s (filename usually includes
-`_1`), e.g. `-1 flyA_1.fq,flyB_1.fq`. Sequences specified with this option must
-correspond file-for-file and read-for-read with those specified in `<m2>`. Reads
-may be a mix of different lengths. If `-` is specified, `bowtie2` will read the
-mate 1s from the "standard in" or "stdin" filehandle.
+Comma-separated list of files containing mate 1s (filename usually
+includes _1), e.g. -1 flyA_1.fq,flyB_1.fq. Sequences specified with this
+option must correspond file-for-file and read-for-read with those
+specified in <m2>. Reads may be a mix of different lengths. If - is
+specified, bowtie2 will read the mate 1s from the "standard in" or
+"stdin" filehandle.
-2 <m2>
-Comma-separated list of files containing mate 2s (filename usually includes
-`_2`), e.g. `-2 flyA_2.fq,flyB_2.fq`. Sequences specified with this option must
-correspond file-for-file and read-for-read with those specified in `<m1>`. Reads
-may be a mix of different lengths. If `-` is specified, `bowtie2` will read the
-mate 2s from the "standard in" or "stdin" filehandle.
+Comma-separated list of files containing mate 2s (filename usually
+includes _2), e.g. -2 flyA_2.fq,flyB_2.fq. Sequences specified with this
+option must correspond file-for-file and read-for-read with those
+specified in <m1>. Reads may be a mix of different lengths. If - is
+specified, bowtie2 will read the mate 2s from the "standard in" or
+"stdin" filehandle.
-U <r>
-Comma-separated list of files containing unpaired reads to be aligned, e.g.
-`lane1.fq,lane2.fq,lane3.fq,lane4.fq`. Reads may be a mix of different lengths.
-If `-` is specified, `bowtie2` gets the reads from the "standard in" or "stdin"
-filehandle.
+Comma-separated list of files containing unpaired reads to be aligned,
+e.g. lane1.fq,lane2.fq,lane3.fq,lane4.fq. Reads may be a mix of
+different lengths. If - is specified, bowtie2 gets the reads from the
+"standard in" or "stdin" filehandle.
- -S <hit>
+ -S <sam>
-File to write SAM alignments to. By default, alignments are written to the
-"standard out" or "stdout" filehandle (i.e. the console).
+File to write SAM alignments to. By default, alignments are written to
+the "standard out" or "stdout" filehandle (i.e. the console).
-### Options
+Options
-#### Input options
+Input options
-q
-Reads (specified with `<m1>`, `<m2>`, `<s>`) are FASTQ files. FASTQ files
-usually have extension `.fq` or `.fastq`. FASTQ is the default format. See
-also: `--solexa-quals` and `--int-quals`.
+Reads (specified with <m1>, <m2>, <s>) are FASTQ files. FASTQ files
+usually have extension .fq or .fastq. FASTQ is the default format. See
+also: --solexa-quals and --int-quals.
--interleaved
@@ -872,143 +862,143 @@ represent a mate pair.
Each read or pair is on a single line. An unpaired read line is
[name]\t[seq]\t[qual]\n. A paired-end read line is
-[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n. An input file can be a
-mix of unpaired and paired-end reads and Bowtie 2 recognizes each
-according to the number of fields, handling each as it should.
+[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n. An input file can be a mix
+of unpaired and paired-end reads and Bowtie 2 recognizes each according
+to the number of fields, handling each as it should.
--tab6
-Similar to `--tab5` except, for paired-end reads, the second end can have a
-different name from the first:
+Similar to --tab5 except, for paired-end reads, the second end can have
+a different name from the first:
[name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n
--qseq
-Reads (specified with `<m1>`, `<m2>`, `<s>`) are QSEQ files. QSEQ files usually
-end in `_qseq.txt`. See also: `--solexa-quals` and `--int-quals`.
+Reads (specified with <m1>, <m2>, <s>) are QSEQ files. QSEQ files
+usually end in _qseq.txt. See also: --solexa-quals and --int-quals.
-f
-Reads (specified with `<m1>`, `<m2>`, `<s>`) are FASTA files. FASTA files
-usually have extension `.fa`, `.fasta`, `.mfa`, `.fna` or similar. FASTA files
-do not have a way of specifying quality values, so when `-f` is set, the result
-is as if `--ignore-quals` is also set.
+Reads (specified with <m1>, <m2>, <s>) are FASTA files. FASTA files
+usually have extension .fa, .fasta, .mfa, .fna or similar. FASTA files
+do not have a way of specifying quality values, so when -f is set, the
+result is as if --ignore-quals is also set.
-r
-Reads (specified with `<m1>`, `<m2>`, `<s>`) are files with one input sequence
-per line, without any other information (no read names, no qualities). When
-`-r` is set, the result is as if `--ignore-quals` is also set.
+Reads (specified with <m1>, <m2>, <s>) are files with one input sequence
+per line, without any other information (no read names, no qualities).
+When -r is set, the result is as if --ignore-quals is also set.
-c
-The read sequences are given on command line. I.e. `<m1>`, `<m2>` and
-`<singles>` are comma-separated lists of reads rather than lists of read files.
-There is no way to specify read names or qualities, so `-c` also implies
-`--ignore-quals`.
+The read sequences are given on command line. I.e. <m1>, <m2> and
+<singles> are comma-separated lists of reads rather than lists of read
+files. There is no way to specify read names or qualities, so -c also
+implies --ignore-quals.
-s/--skip <int>
-Skip (i.e. do not align) the first `<int>` reads or pairs in the input.
+Skip (i.e. do not align) the first <int> reads or pairs in the input.
-u/--qupto <int>
-Align the first `<int>` reads or read pairs from the input (after the
-`-s`/`--skip` reads or pairs have been skipped), then stop. Default: no limit.
+Align the first <int> reads or read pairs from the input (after the
+-s/--skip reads or pairs have been skipped), then stop. Default: no
+limit.
-5/--trim5 <int>
-Trim `<int>` bases from 5' (left) end of each read before alignment (default: 0).
+Trim <int> bases from 5' (left) end of each read before alignment
+(default: 0).
-3/--trim3 <int>
-Trim `<int>` bases from 3' (right) end of each read before alignment (default:
-0).
+Trim <int> bases from 3' (right) end of each read before alignment
+(default: 0).
--phred33
-Input qualities are ASCII chars equal to the [Phred quality] plus 33. This is
-also called the "Phred+33" encoding, which is used by the very latest Illumina
-pipelines.
-
-[Phred quality]: http://en.wikipedia.org/wiki/Phred_quality_score
+Input qualities are ASCII chars equal to the Phred quality plus 33. This
+is also called the "Phred+33" encoding, which is used by the very latest
+Illumina pipelines.
--phred64
-[Phred quality]: http://en.wikipedia.org/wiki/Phred_quality_score
-
-Input qualities are ASCII chars equal to the [Phred quality] plus 64. This is
-also called the "Phred+64" encoding.
+Input qualities are ASCII chars equal to the Phred quality plus 64. This
+is also called the "Phred+64" encoding.
--solexa-quals
-Convert input qualities from [Solexa][Phred quality] (which can be negative) to
-[Phred][Phred quality] (which can't). This scheme was used in older Illumina GA
-Pipeline versions (prior to 1.3). Default: off.
+Convert input qualities from Solexa (which can be negative) to Phred
+(which can't). This scheme was used in older Illumina GA Pipeline
+versions (prior to 1.3). Default: off.
--int-quals
-Quality values are represented in the read input file as space-separated ASCII
-integers, e.g., `40 40 30 40`..., rather than ASCII characters, e.g., `II?I`....
- Integers are treated as being on the [Phred quality] scale unless
-`--solexa-quals` is also specified. Default: off.
+Quality values are represented in the read input file as space-separated
+ASCII integers, e.g., 40 40 30 40..., rather than ASCII characters,
+e.g., II?I.... Integers are treated as being on the Phred quality scale
+unless --solexa-quals is also specified. Default: off.
-#### Preset options in `--end-to-end` mode
+Preset options in --end-to-end mode
--very-fast
-Same as: `-D 5 -R 1 -N 0 -L 22 -i S,0,2.50`
+Same as: -D 5 -R 1 -N 0 -L 22 -i S,0,2.50
--fast
-Same as: `-D 10 -R 2 -N 0 -L 22 -i S,0,2.50`
+Same as: -D 10 -R 2 -N 0 -L 22 -i S,0,2.50
--sensitive
-Same as: `-D 15 -R 2 -L 22 -i S,1,1.15` (default in `--end-to-end` mode)
+Same as: -D 15 -R 2 -L 22 -i S,1,1.15 (default in --end-to-end mode)
--very-sensitive
-Same as: `-D 20 -R 3 -N 0 -L 20 -i S,1,0.50`
+Same as: -D 20 -R 3 -N 0 -L 20 -i S,1,0.50
-#### Preset options in `--local` mode
+Preset options in --local mode
--very-fast-local
-Same as: `-D 5 -R 1 -N 0 -L 25 -i S,1,2.00`
+Same as: -D 5 -R 1 -N 0 -L 25 -i S,1,2.00
--fast-local
-Same as: `-D 10 -R 2 -N 0 -L 22 -i S,1,1.75`
+Same as: -D 10 -R 2 -N 0 -L 22 -i S,1,1.75
--sensitive-local
-Same as: `-D 15 -R 2 -N 0 -L 20 -i S,1,0.75` (default in `--local` mode)
+Same as: -D 15 -R 2 -N 0 -L 20 -i S,1,0.75 (default in --local mode)
--very-sensitive-local
-Same as: `-D 20 -R 3 -N 0 -L 20 -i S,1,0.50`
+Same as: -D 20 -R 3 -N 0 -L 20 -i S,1,0.50
-#### Alignment options
+Alignment options
-N <int>
-Sets the number of mismatches to allowed in a seed alignment during [multiseed
-alignment]. Can be set to 0 or 1. Setting this higher makes alignment slower
-(often much slower) but increases sensitivity. Default: 0.
+Sets the number of mismatches to allowed in a seed alignment during
+multiseed alignment. Can be set to 0 or 1. Setting this higher makes
+alignment slower (often much slower) but increases sensitivity. Default:
+0.
-L <int>
-Sets the length of the seed substrings to align during [multiseed alignment].
-Smaller values make alignment slower but more sensitive. Default: the
-`--sensitive` preset is used by default, which sets `-L` to 20 both in
-`--end-to-end` mode and in `--local` mode.
+Sets the length of the seed substrings to align during multiseed
+alignment. Smaller values make alignment slower but more sensitive.
+Default: the --sensitive preset is used by default, which sets -L to 20
+both in --end-to-end mode and in --local mode.
-i <func>
-Sets a function governing the interval between seed substrings to use during
-[multiseed alignment]. For instance, if the read has 30 characters, and seed
-length is 10, and the seed interval is 6, the seeds extracted will be:
+Sets a function governing the interval between seed substrings to use
+during multiseed alignment. For instance, if the read has 30 characters,
+and seed length is 10, and the seed interval is 6, the seeds extracted
+will be:
Read: TAGCTACGCTCTACGCTATCATGCATAAAC
Seed 1 fw: TAGCTACGCT
@@ -1020,329 +1010,346 @@ length is 10, and the seed interval is 6, the seeds extracted will be:
Seed 4 fw: TCATGCATAA
Seed 4 rc: TTATGCATGA
-Since it's best to use longer intervals for longer reads, this parameter sets
-the interval as a function of the read length, rather than a single
-one-size-fits-all number. For instance, specifying `-i S,1,2.5` sets the
-interval function `f` to `f(x) = 1 + 2.5 * sqrt(x)`, where x is the read length.
-See also: [setting function options]. If the function returns a result less than
-1, it is rounded up to 1. Default: the `--sensitive` preset is used by
-default, which sets `-i` to `S,1,1.15` in `--end-to-end` mode to `-i S,1,0.75`
-in `--local` mode.
+Since it's best to use longer intervals for longer reads, this parameter
+sets the interval as a function of the read length, rather than a single
+one-size-fits-all number. For instance, specifying -i S,1,2.5 sets the
+interval function f to f(x) = 1 + 2.5 * sqrt(x), where x is the read
+length. See also: setting function options. If the function returns a
+result less than 1, it is rounded up to 1. Default: the --sensitive
+preset is used by default, which sets -i to S,1,1.15 in --end-to-end
+mode to -i S,1,0.75 in --local mode.
--n-ceil <func>
-Sets a function governing the maximum number of ambiguous characters (usually
-`N`s and/or `.`s) allowed in a read as a function of read length. For instance,
-specifying `-L,0,0.15` sets the N-ceiling function `f` to `f(x) = 0 + 0.15 * x`,
-where x is the read length. See also: [setting function options]. Reads
-exceeding this ceiling are [filtered out]. Default: `L,0,0.15`.
+Sets a function governing the maximum number of ambiguous characters
+(usually Ns and/or .s) allowed in a read as a function of read length.
+For instance, specifying -L,0,0.15 sets the N-ceiling function f to
+f(x) = 0 + 0.15 * x, where x is the read length. See also: setting
+function options. Reads exceeding this ceiling are filtered out.
+Default: L,0,0.15.
--dpad <int>
-"Pads" dynamic programming problems by `<int>` columns on either side to allow
-gaps. Default: 15.
+"Pads" dynamic programming problems by <int> columns on either side to
+allow gaps. Default: 15.
--gbar <int>
-Disallow gaps within `<int>` positions of the beginning or end of the read.
-Default: 4.
+Disallow gaps within <int> positions of the beginning or end of the
+read. Default: 4.
--ignore-quals
-When calculating a mismatch penalty, always consider the quality value at the
-mismatched position to be the highest possible, regardless of the actual value.
-I.e. input is treated as though all quality values are high. This is also the
-default behavior when the input doesn't specify quality values (e.g. in `-f`,
-`-r`, or `-c` modes).
+When calculating a mismatch penalty, always consider the quality value
+at the mismatched position to be the highest possible, regardless of the
+actual value. I.e. input is treated as though all quality values are
+high. This is also the default behavior when the input doesn't specify
+quality values (e.g. in -f, -r, or -c modes).
--nofw/--norc
-If `--nofw` is specified, `bowtie2` will not attempt to align unpaired reads to
-the forward (Watson) reference strand. If `--norc` is specified, `bowtie2` will
-not attempt to align unpaired reads against the reverse-complement (Crick)
-reference strand. In paired-end mode, `--nofw` and `--norc` pertain to the
-fragments; i.e. specifying `--nofw` causes `bowtie2` to explore only those
-paired-end configurations corresponding to fragments from the reverse-complement
-(Crick) strand. Default: both strands enabled.
+If --nofw is specified, bowtie2 will not attempt to align unpaired reads
+to the forward (Watson) reference strand. If --norc is specified,
+bowtie2 will not attempt to align unpaired reads against the
+reverse-complement (Crick) reference strand. In paired-end mode, --nofw
+and --norc pertain to the fragments; i.e. specifying --nofw causes
+bowtie2 to explore only those paired-end configurations corresponding to
+fragments from the reverse-complement (Crick) strand. Default: both
+strands enabled.
--no-1mm-upfront
-By default, Bowtie 2 will attempt to find either an exact or a 1-mismatch
-end-to-end alignment for the read *before* trying the [multiseed heuristic]. Such
-alignments can be found very quickly, and many short read alignments have exact or
-near-exact end-to-end alignments. However, this can lead to unexpected
-alignments when the user also sets options governing the [multiseed heuristic],
-like `-L` and `-N`. For instance, if the user specifies `-N 0` and `-L` equal
-to the length of the read, the user will be surprised to find 1-mismatch alignments
-reported. This option prevents Bowtie 2 from searching for 1-mismatch end-to-end
-alignments before using the [multiseed heuristic], which leads to the expected
-behavior when combined with options such as `-L` and `-N`. This comes at the
-expense of speed.
+By default, Bowtie 2 will attempt to find either an exact or a
+1-mismatch end-to-end alignment for the read _before_ trying the
+multiseed heuristic. Such alignments can be found very quickly, and many
+short read alignments have exact or near-exact end-to-end alignments.
+However, this can lead to unexpected alignments when the user also sets
+options governing the multiseed heuristic, like -L and -N. For instance,
+if the user specifies -N 0 and -L equal to the length of the read, the
+user will be surprised to find 1-mismatch alignments reported. This
+option prevents Bowtie 2 from searching for 1-mismatch end-to-end
+alignments before using the multiseed heuristic, which leads to the
+expected behavior when combined with options such as -L and -N. This
+comes at the expense of speed.
--end-to-end
-In this mode, Bowtie 2 requires that the entire read align from one end to the
-other, without any trimming (or "soft clipping") of characters from either end.
-The match bonus `--ma` always equals 0 in this mode, so all alignment scores
-are less than or equal to 0, and the greatest possible alignment score is 0.
-This is mutually exclusive with `--local`. `--end-to-end` is the default mode.
+In this mode, Bowtie 2 requires that the entire read align from one end
+to the other, without any trimming (or "soft clipping") of characters
+from either end. The match bonus --ma always equals 0 in this mode, so
+all alignment scores are less than or equal to 0, and the greatest
+possible alignment score is 0. This is mutually exclusive with --local.
+--end-to-end is the default mode.
--local
-In this mode, Bowtie 2 does not require that the entire read align from one end
-to the other. Rather, some characters may be omitted ("soft clipped") from the
-ends in order to achieve the greatest possible alignment score. The match bonus
-`--ma` is used in this mode, and the best possible alignment score is equal to
-the match bonus (`--ma`) times the length of the read. Specifying `--local`
-and one of the presets (e.g. `--local --very-fast`) is equivalent to specifying
-the local version of the preset (`--very-fast-local`). This is mutually
-exclusive with `--end-to-end`. `--end-to-end` is the default mode.
+In this mode, Bowtie 2 does not require that the entire read align from
+one end to the other. Rather, some characters may be omitted ("soft
+clipped") from the ends in order to achieve the greatest possible
+alignment score. The match bonus --ma is used in this mode, and the best
+possible alignment score is equal to the match bonus (--ma) times the
+length of the read. Specifying --local and one of the presets (e.g.
+--local --very-fast) is equivalent to specifying the local version of
+the preset (--very-fast-local). This is mutually exclusive with
+--end-to-end. --end-to-end is the default mode.
-#### Scoring options
+Scoring options
--ma <int>
-Sets the match bonus. In `--local` mode `<int>` is added to the alignment
-score for each position where a read character aligns to a reference character
-and the characters match. Not used in `--end-to-end` mode. Default: 2.
+Sets the match bonus. In --local mode <int> is added to the alignment
+score for each position where a read character aligns to a reference
+character and the characters match. Not used in --end-to-end mode.
+Default: 2.
--mp MX,MN
-Sets the maximum (`MX`) and minimum (`MN`) mismatch penalties, both integers. A
-number less than or equal to `MX` and greater than or equal to `MN` is
-subtracted from the alignment score for each position where a read character
-aligns to a reference character, the characters do not match, and neither is an
-`N`. If `--ignore-quals` is specified, the number subtracted quals `MX`.
-Otherwise, the number subtracted is `MN + floor( (MX-MN)(MIN(Q, 40.0)/40.0) )`
-where Q is the Phred quality value. Default: `MX` = 6, `MN` = 2.
+Sets the maximum (MX) and minimum (MN) mismatch penalties, both
+integers. A number less than or equal to MX and greater than or equal to
+MN is subtracted from the alignment score for each position where a read
+character aligns to a reference character, the characters do not match,
+and neither is an N. If --ignore-quals is specified, the number
+subtracted quals MX. Otherwise, the number subtracted is
+MN + floor( (MX-MN)(MIN(Q, 40.0)/40.0) ) where Q is the Phred quality
+value. Default: MX = 6, MN = 2.
--np <int>
-Sets penalty for positions where the read, reference, or both, contain an
-ambiguous character such as `N`. Default: 1.
+Sets penalty for positions where the read, reference, or both, contain
+an ambiguous character such as N. Default: 1.
--rdg <int1>,<int2>
-Sets the read gap open (`<int1>`) and extend (`<int2>`) penalties. A read gap of
-length N gets a penalty of `<int1>` + N * `<int2>`. Default: 5, 3.
+Sets the read gap open (<int1>) and extend (<int2>) penalties. A read
+gap of length N gets a penalty of <int1> + N * <int2>. Default: 5, 3.
--rfg <int1>,<int2>
-Sets the reference gap open (`<int1>`) and extend (`<int2>`) penalties. A
-reference gap of length N gets a penalty of `<int1>` + N * `<int2>`. Default:
-5, 3.
+Sets the reference gap open (<int1>) and extend (<int2>) penalties. A
+reference gap of length N gets a penalty of <int1> + N * <int2>.
+Default: 5, 3.
--score-min <func>
-Sets a function governing the minimum alignment score needed for an alignment to
-be considered "valid" (i.e. good enough to report). This is a function of read
-length. For instance, specifying `L,0,-0.6` sets the minimum-score function `f`
-to `f(x) = 0 + -0.6 * x`, where `x` is the read length. See also: [setting
-function options]. The default in `--end-to-end` mode is `L,-0.6,-0.6` and
-the default in `--local` mode is `G,20,8`.
+Sets a function governing the minimum alignment score needed for an
+alignment to be considered "valid" (i.e. good enough to report). This is
+a function of read length. For instance, specifying L,0,-0.6 sets the
+minimum-score function f to f(x) = 0 + -0.6 * x, where x is the read
+length. See also: setting function options. The default in --end-to-end
+mode is L,-0.6,-0.6 and the default in --local mode is G,20,8.
-#### Reporting options
+Reporting options
-k <int>
-By default, `bowtie2` searches for distinct, valid alignments for each read.
-When it finds a valid alignment, it continues looking for alignments that are
-nearly as good or better. The best alignment found is reported (randomly
-selected from among best if tied). Information about the best alignments is
-used to estimate mapping quality and to set SAM optional fields, such as
-`AS:i` and `XS:i`.
-
-When `-k` is specified, however, `bowtie2` behaves differently. Instead, it
-searches for at most `<int>` distinct, valid alignments for each read. The
-search terminates when it can't find more distinct valid alignments, or when it
-finds `<int>`, whichever happens first. All alignments found are reported in
-descending order by alignment score. The alignment score for a paired-end
-alignment equals the sum of the alignment scores of the individual mates. Each
-reported read or pair alignment beyond the first has the SAM 'secondary' bit
-(which equals 256) set in its FLAGS field. For reads that have more than
-`<int>` distinct, valid alignments, `bowtie2` does not guarantee that the
-`<int>` alignments reported are the best possible in terms of alignment score.
-`-k` is mutually exclusive with `-a`.
-
-Note: Bowtie 2 is not designed with large values for `-k` in mind, and when
-aligning reads to long, repetitive genomes large `-k` can be very, very slow.
+By default, bowtie2 searches for distinct, valid alignments for each
+read. When it finds a valid alignment, it continues looking for
+alignments that are nearly as good or better. The best alignment found
+is reported (randomly selected from among best if tied). Information
+about the best alignments is used to estimate mapping quality and to set
+SAM optional fields, such as AS:i and XS:i.
+
+When -k is specified, however, bowtie2 behaves differently. Instead, it
+searches for at most <int> distinct, valid alignments for each read. The
+search terminates when it can't find more distinct valid alignments, or
+when it finds <int>, whichever happens first. All alignments found are
+reported in descending order by alignment score. The alignment score for
+a paired-end alignment equals the sum of the alignment scores of the
+individual mates. Each reported read or pair alignment beyond the first
+has the SAM 'secondary' bit (which equals 256) set in its FLAGS field.
+For reads that have more than <int> distinct, valid alignments, bowtie2
+does not guarantee that the <int> alignments reported are the best
+possible in terms of alignment score. -k is mutually exclusive with -a.
+
+Note: Bowtie 2 is not designed with large values for -k in mind, and
+when aligning reads to long, repetitive genomes large -k can be very,
+very slow.
-a
-Like `-k` but with no upper limit on number of alignments to search for. `-a`
-is mutually exclusive with `-k`.
+Like -k but with no upper limit on number of alignments to search for.
+-a is mutually exclusive with -k.
-Note: Bowtie 2 is not designed with `-a` mode in mind, and when
-aligning reads to long, repetitive genomes this mode can be very, very slow.
+Note: Bowtie 2 is not designed with -a mode in mind, and when aligning
+reads to long, repetitive genomes this mode can be very, very slow.
-#### Effort options
+Effort options
-D <int>
-Up to `<int>` consecutive seed extension attempts can "fail" before Bowtie 2
-moves on, using the alignments found so far. A seed extension "fails" if it
-does not yield a new best or a new second-best alignment. This limit is
-automatically adjusted up when -k or -a are specified. Default: 15.
+Up to <int> consecutive seed extension attempts can "fail" before Bowtie
+2 moves on, using the alignments found so far. A seed extension "fails"
+if it does not yield a new best or a new second-best alignment. This
+limit is automatically adjusted up when -k or -a are specified. Default:
+15.
-R <int>
-`<int>` is the maximum number of times Bowtie 2 will "re-seed" reads with
-repetitive seeds. When "re-seeding," Bowtie 2 simply chooses a new set of reads
-(same length, same number of mismatches allowed) at different offsets and
-searches for more alignments. A read is considered to have repetitive seeds if
-the total number of seed hits divided by the number of seeds that aligned at
-least once is greater than 300. Default: 2.
+<int> is the maximum number of times Bowtie 2 will "re-seed" reads with
+repetitive seeds. When "re-seeding," Bowtie 2 simply chooses a new set
+of reads (same length, same number of mismatches allowed) at different
+offsets and searches for more alignments. A read is considered to have
+repetitive seeds if the total number of seed hits divided by the number
+of seeds that aligned at least once is greater than 300. Default: 2.
-#### Paired-end options
+Paired-end options
-I/--minins <int>
-The minimum fragment length for valid paired-end alignments. E.g. if `-I 60` is
-specified and a paired-end alignment consists of two 20-bp alignments in the
-appropriate orientation with a 20-bp gap between them, that alignment is
-considered valid (as long as `-X` is also satisfied). A 19-bp gap would not
-be valid in that case. If trimming options `-3` or `-5` are also used, the
-`-I` constraint is applied with respect to the untrimmed mates.
+The minimum fragment length for valid paired-end alignments. E.g. if
+-I 60 is specified and a paired-end alignment consists of two 20-bp
+alignments in the appropriate orientation with a 20-bp gap between them,
+that alignment is considered valid (as long as -X is also satisfied). A
+19-bp gap would not be valid in that case. If trimming options -3 or -5
+are also used, the -I constraint is applied with respect to the
+untrimmed mates.
-The larger the difference between `-I` and `-X`, the slower Bowtie 2 will
-run. This is because larger differences between `-I` and `-X` require that
-Bowtie 2 scan a larger window to determine if a concordant alignment exists.
-For typical fragment length ranges (200 to 400 nucleotides), Bowtie 2 is very
-efficient.
+The larger the difference between -I and -X, the slower Bowtie 2 will
+run. This is because larger differences between -I and -X require that
+Bowtie 2 scan a larger window to determine if a concordant alignment
+exists. For typical fragment length ranges (200 to 400 nucleotides),
+Bowtie 2 is very efficient.
-Default: 0 (essentially imposing no minimum)
+Default: 0 (essentially imposing no minimum)
-X/--maxins <int>
-The maximum fragment length for valid paired-end alignments. E.g. if `-X 100`
-is specified and a paired-end alignment consists of two 20-bp alignments in the
-proper orientation with a 60-bp gap between them, that alignment is considered
-valid (as long as `-I` is also satisfied). A 61-bp gap would not be valid in
-that case. If trimming options `-3` or `-5` are also used, the `-X`
-constraint is applied with respect to the untrimmed mates, not the trimmed
-mates.
+The maximum fragment length for valid paired-end alignments. E.g. if
+-X 100 is specified and a paired-end alignment consists of two 20-bp
+alignments in the proper orientation with a 60-bp gap between them, that
+alignment is considered valid (as long as -I is also satisfied). A 61-bp
+gap would not be valid in that case. If trimming options -3 or -5 are
+also used, the -X constraint is applied with respect to the untrimmed
+mates, not the trimmed mates.
-The larger the difference between `-I` and `-X`, the slower Bowtie 2 will
-run. This is because larger differences between `-I` and `-X` require that
-Bowtie 2 scan a larger window to determine if a concordant alignment exists.
-For typical fragment length ranges (200 to 400 nucleotides), Bowtie 2 is very
-efficient.
+The larger the difference between -I and -X, the slower Bowtie 2 will
+run. This is because larger differences between -I and -X require that
+Bowtie 2 scan a larger window to determine if a concordant alignment
+exists. For typical fragment length ranges (200 to 400 nucleotides),
+Bowtie 2 is very efficient.
Default: 500.
--fr/--rf/--ff
-The upstream/downstream mate orientations for a valid paired-end alignment
-against the forward reference strand. E.g., if `--fr` is specified and there is
-a candidate paired-end alignment where mate 1 appears upstream of the reverse
-complement of mate 2 and the fragment length constraints (`-I` and `-X`) are
-met, that alignment is valid. Also, if mate 2 appears upstream of the reverse
-complement of mate 1 and all other constraints are met, that too is valid.
-`--rf` likewise requires that an upstream mate1 be reverse-complemented and a
-downstream mate2 be forward-oriented. ` --ff` requires both an upstream mate 1
-and a downstream mate 2 to be forward-oriented. Default: `--fr` (appropriate
-for Illumina's Paired-end Sequencing Assay).
+The upstream/downstream mate orientations for a valid paired-end
+alignment against the forward reference strand. E.g., if --fr is
+specified and there is a candidate paired-end alignment where mate 1
+appears upstream of the reverse complement of mate 2 and the fragment
+length constraints (-I and -X) are met, that alignment is valid. Also,
+if mate 2 appears upstream of the reverse complement of mate 1 and all
+other constraints are met, that too is valid. --rf likewise requires
+that an upstream mate1 be reverse-complemented and a downstream mate2 be
+forward-oriented. --ff requires both an upstream mate 1 and a downstream
+mate 2 to be forward-oriented. Default: --fr (appropriate for Illumina's
+Paired-end Sequencing Assay).
--no-mixed
-By default, when `bowtie2` cannot find a concordant or discordant alignment for
-a pair, it then tries to find alignments for the individual mates. This option
-disables that behavior.
+By default, when bowtie2 cannot find a concordant or discordant
+alignment for a pair, it then tries to find alignments for the
+individual mates. This option disables that behavior.
--no-discordant
-By default, `bowtie2` looks for discordant alignments if it cannot find any
-concordant alignments. A discordant alignment is an alignment where both mates
-align uniquely, but that does not satisfy the paired-end constraints
-(`--fr`/`--rf`/`--ff`, `-I`, `-X`). This option disables that behavior.
+By default, bowtie2 looks for discordant alignments if it cannot find
+any concordant alignments. A discordant alignment is an alignment where
+both mates align uniquely, but that does not satisfy the paired-end
+constraints (--fr/--rf/--ff, -I, -X). This option disables that
+behavior.
--dovetail
If the mates "dovetail", that is if one mate alignment extends past the
-beginning of the other such that the wrong mate begins upstream, consider that
-to be concordant. See also: [Mates can overlap, contain or dovetail each
-other]. Default: mates cannot dovetail in a concordant alignment.
+beginning of the other such that the wrong mate begins upstream,
+consider that to be concordant. See also: Mates can overlap, contain or
+dovetail each other. Default: mates cannot dovetail in a concordant
+alignment.
--no-contain
-If one mate alignment contains the other, consider that to be non-concordant.
-See also: [Mates can overlap, contain or dovetail each other]. Default: a mate
-can contain the other in a concordant alignment.
+If one mate alignment contains the other, consider that to be
+non-concordant. See also: Mates can overlap, contain or dovetail each
+other. Default: a mate can contain the other in a concordant alignment.
--no-overlap
If one mate alignment overlaps the other at all, consider that to be
-non-concordant. See also: [Mates can overlap, contain or dovetail each other].
-Default: mates can overlap in a concordant alignment.
+non-concordant. See also: Mates can overlap, contain or dovetail each
+other. Default: mates can overlap in a concordant alignment.
-#### Output options
+Output options
-t/--time
-Print the wall-clock time required to load the index files and align the reads.
-This is printed to the "standard error" ("stderr") filehandle. Default: off.
+Print the wall-clock time required to load the index files and align the
+reads. This is printed to the "standard error" ("stderr") filehandle.
+Default: off.
--un <path>
--un-gz <path>
--un-bz2 <path>
--un-lz4 <path>
-Write unpaired reads that fail to align to file at `<path>`. These reads
-correspond to the SAM records with the FLAGS `0x4` bit set and neither the
-`0x40` nor `0x80` bits set. If `--un-gz` is specified, output will be gzip
-compressed. If `--un-bz2` or `--un-lz4` is specified, output will be bzip2 or
-lz4 compressed. Reads written in this way will appear exactly as they did in
-the input file, without any modification (same sequence, same name, same quality
-string, same quality encoding). Reads will not necessarily appear in the same
-order as they did in the input.
+Write unpaired reads that fail to align to file at <path>. These reads
+correspond to the SAM records with the FLAGS 0x4 bit set and neither the
+0x40 nor 0x80 bits set. If --un-gz is specified, output will be gzip
+compressed. If --un-bz2 or --un-lz4 is specified, output will be bzip2
+or lz4 compressed. Reads written in this way will appear exactly as they
+did in the input file, without any modification (same sequence, same
+name, same quality string, same quality encoding). Reads will not
+necessarily appear in the same order as they did in the input.
--al <path>
--al-gz <path>
--al-bz2 <path>
--al-lz4 <path>
-Write unpaired reads that align at least once to file at `<path>`. These reads
-correspond to the SAM records with the FLAGS `0x4`, `0x40`, and `0x80` bits
-unset. If `--al-gz` is specified, output will be gzip compressed. If `--al-bz2`
-is specified, output will be bzip2 compressed. Similarly if `--al-lz4` is specified,
-output will be lz4 compressed. Reads written in this way will
-appear exactly as they did in the input file, without any modification (same
-sequence, same name, same quality string, same quality encoding). Reads will
-not necessarily appear in the same order as they did in the input.
+Write unpaired reads that align at least once to file at <path>. These
+reads correspond to the SAM records with the FLAGS 0x4, 0x40, and 0x80
+bits unset. If --al-gz is specified, output will be gzip compressed. If
+--al-bz2 is specified, output will be bzip2 compressed. Similarly if
+--al-lz4 is specified, output will be lz4 compressed. Reads written in
+this way will appear exactly as they did in the input file, without any
+modification (same sequence, same name, same quality string, same
+quality encoding). Reads will not necessarily appear in the same order
+as they did in the input.
--un-conc <path>
--un-conc-gz <path>
--un-conc-bz2 <path>
--un-conc-lz4 <path>
-Write paired-end reads that fail to align concordantly to file(s) at `<path>`.
-These reads correspond to the SAM records with the FLAGS `0x4` bit set and
-either the `0x40` or `0x80` bit set (depending on whether it's mate #1 or #2).
-`.1` and `.2` strings are added to the filename to distinguish which file
-contains mate #1 and mate #2. If a percent symbol, `%`, is used in `<path>`,
-the percent symbol is replaced with `1` or `2` to make the per-mate filenames.
-Otherwise, `.1` or `.2` are added before the final dot in `<path>` to make the
-per-mate filenames. Reads written in this way will appear exactly as they did
-in the input files, without any modification (same sequence, same name, same
-quality string, same quality encoding). Reads will not necessarily appear in
-the same order as they did in the inputs.
+Write paired-end reads that fail to align concordantly to file(s) at
+<path>. These reads correspond to the SAM records with the FLAGS 0x4 bit
+set and either the 0x40 or 0x80 bit set (depending on whether it's mate
+#1 or #2). .1 and .2 strings are added to the filename to distinguish
+which file contains mate #1 and mate #2. If a percent symbol, %, is used
+in <path>, the percent symbol is replaced with 1 or 2 to make the
+per-mate filenames. Otherwise, .1 or .2 are added before the final dot
+in <path> to make the per-mate filenames. Reads written in this way will
+appear exactly as they did in the input files, without any modification
+(same sequence, same name, same quality string, same quality encoding).
+Reads will not necessarily appear in the same order as they did in the
+inputs.
--al-conc <path>
--al-conc-gz <path>
--al-conc-bz2 <path>
--al-conc-lz4 <path>
-Write paired-end reads that align concordantly at least once to file(s) at
-`<path>`. These reads correspond to the SAM records with the FLAGS `0x4` bit
-unset and either the `0x40` or `0x80` bit set (depending on whether it's mate #1
-or #2). `.1` and `.2` strings are added to the filename to distinguish which
-file contains mate #1 and mate #2. If a percent symbol, `%`, is used in
-`<path>`, the percent symbol is replaced with `1` or `2` to make the per-mate
-filenames. Otherwise, `.1` or `.2` are added before the final dot in `<path>` to
-make the per-mate filenames. Reads written in this way will appear exactly as
-they did in the input files, without any modification (same sequence, same name,
-same quality string, same quality encoding). Reads will not necessarily appear
-in the same order as they did in the inputs.
+Write paired-end reads that align concordantly at least once to file(s)
+at <path>. These reads correspond to the SAM records with the FLAGS 0x4
+bit unset and either the 0x40 or 0x80 bit set (depending on whether it's
+mate #1 or #2). .1 and .2 strings are added to the filename to
+distinguish which file contains mate #1 and mate #2. If a percent
+symbol, %, is used in <path>, the percent symbol is replaced with 1 or 2
+to make the per-mate filenames. Otherwise, .1 or .2 are added before the
+final dot in <path> to make the per-mate filenames. Reads written in
+this way will appear exactly as they did in the input files, without any
+modification (same sequence, same name, same quality string, same
+quality encoding). Reads will not necessarily appear in the same order
+as they did in the inputs.
--quiet
@@ -1350,23 +1357,23 @@ Print nothing besides alignments and serious errors.
--met-file <path>
-Write `bowtie2` metrics to file `<path>`. Having alignment metric can be useful
-for debugging certain problems, especially performance issues. See also:
-`--met`. Default: metrics disabled.
+Write bowtie2 metrics to file <path>. Having alignment metric can be
+useful for debugging certain problems, especially performance issues.
+See also: --met. Default: metrics disabled.
--met-stderr <path>
-Write `bowtie2` metrics to the "standard error" ("stderr") filehandle. This is
-not mutually exclusive with `--met-file`. Having alignment metric can be
-useful for debugging certain problems, especially performance issues. See also:
-`--met`. Default: metrics disabled.
+Write bowtie2 metrics to the "standard error" ("stderr") filehandle.
+This is not mutually exclusive with --met-file. Having alignment metric
+can be useful for debugging certain problems, especially performance
+issues. See also: --met. Default: metrics disabled.
--met <int>
-Write a new `bowtie2` metrics record every `<int>` seconds. Only matters if
-either `--met-stderr` or `--met-file` are specified. Default: 1.
+Write a new bowtie2 metrics record every <int> seconds. Only matters if
+either --met-stderr or --met-file are specified. Default: 1.
-#### SAM options
+SAM options
--no-unal
@@ -1374,95 +1381,109 @@ Suppress SAM records for reads that failed to align.
--no-hd
-Suppress SAM header lines (starting with `@`).
+Suppress SAM header lines (starting with @).
--no-sq
-Suppress `@SQ` SAM header lines.
+Suppress @SQ SAM header lines.
--rg-id <text>
-Set the read group ID to `<text>`. This causes the SAM `@RG` header line to be
-printed, with `<text>` as the value associated with the `ID:` tag. It also
-causes the `RG:Z:` extra field to be attached to each SAM output record, with
-value set to `<text>`.
+Set the read group ID to <text>. This causes the SAM @RG header line to
+be printed, with <text> as the value associated with the ID: tag. It
+also causes the RG:Z: extra field to be attached to each SAM output
+record, with value set to <text>.
--rg <text>
-Add `<text>` (usually of the form `TAG:VAL`, e.g. `SM:Pool1`) as a field on the
-`@RG` header line. Note: in order for the `@RG` line to appear, `--rg-id`
-must also be specified. This is because the `ID` tag is required by the [SAM
-Spec][SAM]. Specify `--rg` multiple times to set multiple fields. See the
-[SAM Spec][SAM] for details about what fields are legal.
+Add <text> (usually of the form TAG:VAL, e.g. SM:Pool1) as a field on
+the @RG header line. Note: in order for the @RG line to appear, --rg-id
+must also be specified. This is because the ID tag is required by the
+SAM Spec. Specify --rg multiple times to set multiple fields. See the
+SAM Spec for details about what fields are legal.
--omit-sec-seq
-When printing secondary alignments, Bowtie 2 by default will write out the `SEQ`
-and `QUAL` strings. Specifying this option causes Bowtie 2 to print an asterisk
-in those fields instead.
+When printing secondary alignments, Bowtie 2 by default will write out
+the SEQ and QUAL strings. Specifying this option causes Bowtie 2 to
+print an asterisk in those fields instead.
+
+ --soft-clipped-unmapped-tlen
+
+Consider soft-clipped bases unmapped when calculating TLEN.
+
+ --sam-no-qname-trunc
+
+Suppress standard behavior of truncating readname at first whitespace at
+the expense of generating non-standard SAM
-#### Performance options
+ --xeq
+
+Use '='/'X', instead of 'M', to specify matches/mismatches in SAM record
+
+Performance options
-o/--offrate <int>
-Override the offrate of the index with `<int>`. If `<int>` is greater
-than the offrate used to build the index, then some row markings are
-discarded when the index is read into memory. This reduces the memory
+Override the offrate of the index with <int>. If <int> is greater than
+the offrate used to build the index, then some row markings are
+discarded when the index is read into memory. This reduces the memory
footprint of the aligner but requires more time to calculate text
-offsets. `<int>` must be greater than the value used to build the
-index.
+offsets. <int> must be greater than the value used to build the index.
-p/--threads NTHREADS
-Launch `NTHREADS` parallel search threads (default: 1). Threads will run on
-separate processors/cores and synchronize when parsing reads and outputting
-alignments. Searching for alignments is highly parallel, and speedup is close
-to linear. Increasing `-p` increases Bowtie 2's memory footprint. E.g. when
-aligning to a human genome index, increasing `-p` from 1 to 8 increases the
-memory footprint by a few hundred megabytes. This option is only available if
-`bowtie` is linked with the `pthreads` library (i.e. if `BOWTIE_PTHREADS=0` is
-not specified at build time).
+Launch NTHREADS parallel search threads (default: 1). Threads will run
+on separate processors/cores and synchronize when parsing reads and
+outputting alignments. Searching for alignments is highly parallel, and
+speedup is close to linear. Increasing -p increases Bowtie 2's memory
+footprint. E.g. when aligning to a human genome index, increasing -p
+from 1 to 8 increases the memory footprint by a few hundred megabytes.
+This option is only available if bowtie is linked with the pthreads
+library (i.e. if BOWTIE_PTHREADS=0 is not specified at build time).
--reorder
-Guarantees that output SAM records are printed in an order corresponding to the
-order of the reads in the original input file, even when `-p` is set greater
-than 1. Specifying `--reorder` and setting `-p` greater than 1 causes Bowtie
-2 to run somewhat slower and use somewhat more memory then if `--reorder` were
-not specified. Has no effect if `-p` is set to 1, since output order will
-naturally correspond to input order in that case.
+Guarantees that output SAM records are printed in an order corresponding
+to the order of the reads in the original input file, even when -p is
+set greater than 1. Specifying --reorder and setting -p greater than 1
+causes Bowtie 2 to run somewhat slower and use somewhat more memory then
+if --reorder were not specified. Has no effect if -p is set to 1, since
+output order will naturally correspond to input order in that case.
--mm
Use memory-mapped I/O to load the index, rather than typical file I/O.
-Memory-mapping allows many concurrent `bowtie` processes on the same computer to
-share the same memory image of the index (i.e. you pay the memory overhead just
-once). This facilitates memory-efficient parallelization of `bowtie` in
-situations where using `-p` is not possible or not preferable.
+Memory-mapping allows many concurrent bowtie processes on the same
+computer to share the same memory image of the index (i.e. you pay the
+memory overhead just once). This facilitates memory-efficient
+parallelization of bowtie in situations where using -p is not possible
+or not preferable.
-#### Other options
+Other options
--qc-filter
-Filter out reads for which the QSEQ filter field is non-zero. Only has an
-effect when read format is `--qseq`. Default: off.
+Filter out reads for which the QSEQ filter field is non-zero. Only has
+an effect when read format is --qseq. Default: off.
--seed <int>
-Use `<int>` as the seed for pseudo-random number generator. Default: 0.
+Use <int> as the seed for pseudo-random number generator. Default: 0.
--non-deterministic
-Normally, Bowtie 2 re-initializes its pseudo-random generator for each read. It
-seeds the generator with a number derived from (a) the read name, (b) the
-nucleotide sequence, (c) the quality sequence, (d) the value of the `--seed`
-option. This means that if two reads are identical (same name, same
-nucleotides, same qualities) Bowtie 2 will find and report the same alignment(s)
-for both, even if there was ambiguity. When `--non-deterministic` is specified,
-Bowtie 2 re-initializes its pseudo-random generator for each read using the
-current time. This means that Bowtie 2 will not necessarily report the same
-alignment for two identical reads. This is counter-intuitive for some users,
-but might be more appropriate in situations where the input consists of many
+Normally, Bowtie 2 re-initializes its pseudo-random generator for each
+read. It seeds the generator with a number derived from (a) the read
+name, (b) the nucleotide sequence, (c) the quality sequence, (d) the
+value of the --seed option. This means that if two reads are identical
+(same name, same nucleotides, same qualities) Bowtie 2 will find and
+report the same alignment(s) for both, even if there was ambiguity. When
+--non-deterministic is specified, Bowtie 2 re-initializes its
+pseudo-random generator for each read using the current time. This means
+that Bowtie 2 will not necessarily report the same alignment for two
+identical reads. This is counter-intuitive for some users, but might be
+more appropriate in situations where the input consists of many
identical reads.
--version
@@ -1473,31 +1494,31 @@ Print version information and quit.
Print usage information and quit.
+
SAM output
-----------
-Following is a brief description of the [SAM] format as output by `bowtie2`.
-For more details, see the [SAM format specification][SAM].
+Following is a brief description of the SAM format as output by bowtie2.
+For more details, see the SAM format specification.
-By default, `bowtie2` prints a SAM header with `@HD`, `@SQ` and `@PG` lines.
-When one or more `--rg` arguments are specified, `bowtie2` will also print
-an `@RG` line that includes all user-specified `--rg` tokens separated by
+By default, bowtie2 prints a SAM header with @HD, @SQ and @PG lines.
+When one or more --rg arguments are specified, bowtie2 will also print
+an @RG line that includes all user-specified --rg tokens separated by
tabs.
-Each subsequent line describes an alignment or, if the read failed to align, a
-read. Each line is a collection of at least 12 fields separated by tabs; from
-left to right, the fields are:
+Each subsequent line describes an alignment or, if the read failed to
+align, a read. Each line is a collection of at least 12 fields separated
+by tabs; from left to right, the fields are:
1. Name of read that aligned.
- Note that the [SAM specification] disallows whitespace in the read name.
- If the read name contains any whitespace characters, Bowtie 2 will truncate
- the name at the first whitespace character. This is similar to the
- behavior of other tools. The standard behavior of truncating at the first
- whitespace can be suppressed with `--sam-noqname-trunc` at the expense of
- generating non-standard SAM.
+ Note that the SAM specification disallows whitespace in the read
+ name. If the read name contains any whitespace characters, Bowtie 2
+ will truncate the name at the first whitespace character. This is
+ similar to the behavior of other tools. The standard behavior of
+ truncating at the first whitespace can be suppressed with
+ --sam-noqname-trunc at the expense of generating non-standard SAM.
-2. Sum of all applicable flags. Flags relevant to Bowtie are:
+2. Sum of all applicable flags. Flags relevant to Bowtie are:
1
@@ -1521,8 +1542,8 @@ left to right, the fields are:
32
- The other mate in the paired-end alignment is aligned to the
- reverse reference strand
+ The other mate in the paired-end alignment is aligned to the reverse
+ reference strand
64
@@ -1533,7 +1554,7 @@ left to right, the fields are:
The read is mate 2 in a pair
Thus, an unpaired read that aligns to the reverse reference strand
- will have flag 16. A paired-end read that aligns and is the first
+ will have flag 16. A paired-end read that aligns and is the first
mate in the pair will have flag 83 (= 64 + 16 + 2 + 1).
3. Name of reference sequence where alignment occurs
@@ -1545,271 +1566,277 @@ left to right, the fields are:
6. CIGAR string representation of alignment
-7. Name of reference sequence where mate's alignment occurs. Set to `=` if the
-mate's reference sequence is the same as this alignment's, or `*` if there is no
-mate.
+7. Name of reference sequence where mate's alignment occurs. Set to =
+ if the mate's reference sequence is the same as this alignment's, or
+ * if there is no mate.
+
+8. 1-based offset into the forward reference strand where leftmost
+ character of the mate's alignment occurs. Offset is 0 if there is no
+ mate.
-8. 1-based offset into the forward reference strand where leftmost character of
-the mate's alignment occurs. Offset is 0 if there is no mate.
+9. Inferred fragment length. Size is negative if the mate's alignment
+ occurs upstream of this alignment. Size is 0 if the mates did not
+ align concordantly. However, size is non-0 if the mates aligned
+ discordantly to the same chromosome.
-9. Inferred fragment length. Size is negative if the mate's alignment occurs
-upstream of this alignment. Size is 0 if the mates did not align concordantly.
-However, size is non-0 if the mates aligned discordantly to the same
-chromosome.
+10. Read sequence (reverse-complemented if aligned to the reverse
+ strand)
-10. Read sequence (reverse-complemented if aligned to the reverse strand)
+11. ASCII-encoded read qualities (reverse-complemented if the read
+ aligned to the reverse strand). The encoded quality values are on
+ the Phred quality scale and the encoding is ASCII-offset by 33
+ (ASCII char !), similarly to a FASTQ file.
-11. ASCII-encoded read qualities (reverse-complemented if the read aligned to
-the reverse strand). The encoded quality values are on the [Phred quality]
-scale and the encoding is ASCII-offset by 33 (ASCII char `!`), similarly to a
-[FASTQ] file.
+12. Optional fields. Fields are tab-separated. bowtie2 outputs zero or
+ more of these optional fields for each alignment, depending on the
+ type of the alignment:
-12. Optional fields. Fields are tab-separated. `bowtie2` outputs zero or more
-of these optional fields for each alignment, depending on the type of the
-alignment:
+ AS:i:<N>
- AS:i:<N>
+Alignment score. Can be negative. Can be greater than 0 in --local mode
+(but not in --end-to-end mode). Only present if SAM record is for an
+aligned read.
- Alignment score. Can be negative. Can be greater than 0 in `--local`
- mode (but not in `--end-to-end` mode). Only present if SAM record is for
- an aligned read.
+ XS:i:<N>
- XS:i:<N>
+Alignment score for the best-scoring alignment found other than the
+alignment reported. Can be negative. Can be greater than 0 in --local
+mode (but not in --end-to-end mode). Only present if the SAM record is
+for an aligned read and more than one alignment was found for the read.
+Note that, when the read is part of a concordantly-aligned pair, this
+score could be greater than AS:i.
- Alignment score for the best-scoring alignment found other than the
- alignment reported. Can be negative. Can be greater than 0 in `--local`
- mode (but not in `--end-to-end` mode). Only present if the SAM record is
- for an aligned read and more than one alignment was found for the read.
- Note that, when the read is part of a concordantly-aligned pair, this score
- could be greater than `AS:i`.
+ YS:i:<N>
- YS:i:<N>
+Alignment score for opposite mate in the paired-end alignment. Only
+present if the SAM record is for a read that aligned as part of a
+paired-end alignment.
+
+ XN:i:<N>
- Alignment score for opposite mate in the paired-end alignment. Only present
- if the SAM record is for a read that aligned as part of a paired-end
- alignment.
+The number of ambiguous bases in the reference covering this alignment.
+Only present if SAM record is for an aligned read.
- XN:i:<N>
+ XM:i:<N>
- The number of ambiguous bases in the reference covering this alignment.
- Only present if SAM record is for an aligned read.
+The number of mismatches in the alignment. Only present if SAM record is
+for an aligned read.
- XM:i:<N>
+ XO:i:<N>
- The number of mismatches in the alignment. Only present if SAM record is
- for an aligned read.
+The number of gap opens, for both read and reference gaps, in the
+alignment. Only present if SAM record is for an aligned read.
- XO:i:<N>
+ XG:i:<N>
- The number of gap opens, for both read and reference gaps, in the alignment.
- Only present if SAM record is for an aligned read.
+The number of gap extensions, for both read and reference gaps, in the
+alignment. Only present if SAM record is for an aligned read.
- XG:i:<N>
+ NM:i:<N>
- The number of gap extensions, for both read and reference gaps, in the
- alignment. Only present if SAM record is for an aligned read.
+The edit distance; that is, the minimal number of one-nucleotide edits
+(substitutions, insertions and deletions) needed to transform the read
+string into the reference string. Only present if SAM record is for an
+aligned read.
- NM:i:<N>
+ YF:Z:<S>
- The edit distance; that is, the minimal number of one-nucleotide edits
- (substitutions, insertions and deletions) needed to transform the read
- string into the reference string. Only present if SAM record is for an
- aligned read.
+String indicating reason why the read was filtered out. See also:
+Filtering. Only appears for reads that were filtered out.
- YF:Z:<S>
+ YT:Z:<S>
- String indicating reason why the read was filtered out. See also:
- [Filtering]. Only appears for reads that were filtered out.
+Value of UU indicates the read was not part of a pair. Value of CP
+indicates the read was part of a pair and the pair aligned concordantly.
+Value of DP indicates the read was part of a pair and the pair aligned
+discordantly. Value of UP indicates the read was part of a pair but the
+pair failed to aligned either concordantly or discordantly.
- YT:Z:<S>
+ MD:Z:<S>
- Value of `UU` indicates the read was not part of a pair. Value of `CP`
- indicates the read was part of a pair and the pair aligned concordantly.
- Value of `DP` indicates the read was part of a pair and the pair aligned
- discordantly. Value of `UP` indicates the read was part of a pair but the
- pair failed to aligned either concordantly or discordantly.
+A string representation of the mismatched reference bases in the
+alignment. See SAM Tags format specification for details. Only present
+if SAM record is for an aligned read.
- MD:Z:<S>
- A string representation of the mismatched reference bases in the alignment.
- See [SAM] format specification for details. Only present if SAM record is
- for an aligned read.
-[SAM format specification]: http://samtools.sf.net/SAM1.pdf
-[FASTQ]: http://en.wikipedia.org/wiki/FASTQ_format
+THE bowtie2-build INDEXER
-The `bowtie2-build` indexer
-===========================
-`bowtie2-build` builds a Bowtie index from a set of DNA sequences.
-`bowtie2-build` outputs a set of 6 files with suffixes `.1.bt2`, `.2.bt2`,
-`.3.bt2`, `.4.bt2`, `.rev.1.bt2`, and `.rev.2.bt2`. In the case of a large
-index these suffixes will have a `bt2l` termination. These files together
+bowtie2-build builds a Bowtie index from a set of DNA sequences.
+bowtie2-build outputs a set of 6 files with suffixes .1.bt2, .2.bt2,
+.3.bt2, .4.bt2, .rev.1.bt2, and .rev.2.bt2. In the case of a large index
+these suffixes will have a bt2l termination. These files together
constitute the index: they are all that is needed to align reads to that
-reference. The original sequence FASTA files are no longer used by Bowtie 2
-once the index is built.
-
-Bowtie 2's `.bt2` index format is different from Bowtie 1's `.ebwt` format, and
-they are not compatible with each other.
-
-Use of Karkkainen's [blockwise algorithm] allows `bowtie2-build` to trade off
-between running time and memory usage. `bowtie2-build` has three options
-governing how it makes this trade: `-p`/`--packed`, `--bmax`/`--bmaxdivn`,
-and `--dcv`. By default, `bowtie2-build` will automatically search for the
-settings that yield the best running time without exhausting memory. This
-behavior can be disabled using the `-a`/`--noauto` option.
-
-The indexer provides options pertaining to the "shape" of the index, e.g.
-`--offrate` governs the fraction of [Burrows-Wheeler]
-rows that are "marked" (i.e., the density of the suffix-array sample; see the
-original [FM Index] paper for details). All of these options are potentially
-profitable trade-offs depending on the application. They have been set to
-defaults that are reasonable for most cases according to our experiments. See
-[Performance tuning] for details.
-
-`bowtie2-build` can generate either [small or large indexes]. The wrapper
-will decide which based on the length of the input genome. If the reference
-does not exceed 4 billion characters but a large index is preferred, the user
-can specify `--large-index` to force `bowtie2-build` to build a large index
-instead.
-
-The Bowtie 2 index is based on the [FM Index] of Ferragina and Manzini, which in
-turn is based on the [Burrows-Wheeler] transform. The algorithm used to build
-the index is based on the [blockwise algorithm] of Karkkainen.
-
-[Blockwise algorithm]: http://portal.acm.org/citation.cfm?id=1314852
-[blockwise algorithm]: http://portal.acm.org/citation.cfm?id=1314852
-[FM Index]: http://portal.acm.org/citation.cfm?id=796543
-[Burrows-Wheeler]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
+reference. The original sequence FASTA files are no longer used by
+Bowtie 2 once the index is built.
+
+Bowtie 2's .bt2 index format is different from Bowtie 1's .ebwt format,
+and they are not compatible with each other.
+
+Use of Karkkainen's blockwise algorithm allows bowtie2-build to trade
+off between running time and memory usage. bowtie2-build has three
+options governing how it makes this trade: -p/--packed,
+--bmax/--bmaxdivn, and --dcv. By default, bowtie2-build will
+automatically search for the settings that yield the best running time
+without exhausting memory. This behavior can be disabled using the
+-a/--noauto option.
+
+The indexer provides options pertaining to the "shape" of the index,
+e.g. --offrate governs the fraction of Burrows-Wheeler rows that are
+"marked" (i.e., the density of the suffix-array sample; see the original
+FM Index paper for details). All of these options are potentially
+profitable trade-offs depending on the application. They have been set
+to defaults that are reasonable for most cases according to our
+experiments. See Performance tuning for details.
+
+bowtie2-build can generate either small or large indexes. The wrapper
+will decide which based on the length of the input genome. If the
+reference does not exceed 4 billion characters but a large index is
+preferred, the user can specify --large-index to force bowtie2-build to
+build a large index instead.
+
+The Bowtie 2 index is based on the FM Index of Ferragina and Manzini,
+which in turn is based on the Burrows-Wheeler transform. The algorithm
+used to build the index is based on the blockwise algorithm of
+Karkkainen.
+
Command Line
-------------
Usage:
bowtie2-build [options]* <reference_in> <bt2_base>
-### Main arguments
+Main arguments
+
+ <reference_in>
-A comma-separated list of FASTA files containing the reference sequences to be
-aligned to, or, if `-c` is specified, the sequences
-themselves. E.g., `<reference_in>` might be `chr1.fa,chr2.fa,chrX.fa,chrY.fa`,
-or, if `-c` is specified, this might be
-`GGTCATCCT,ACGGGTCGT,CCGTTCTATGCGGCTTA`.
+A comma-separated list of FASTA files containing the reference sequences
+to be aligned to, or, if -c is specified, the sequences themselves.
+E.g., <reference_in> might be chr1.fa,chr2.fa,chrX.fa,chrY.fa, or, if -c
+is specified, this might be GGTCATCCT,ACGGGTCGT,CCGTTCTATGCGGCTTA.
-The basename of the index files to write. By default, `bowtie2-build` writes
-files named `NAME.1.bt2`, `NAME.2.bt2`, `NAME.3.bt2`, `NAME.4.bt2`,
-`NAME.rev.1.bt2`, and `NAME.rev.2.bt2`, where `NAME` is `<bt2_base>`.
+ <bt2_base>
-### Options
+The basename of the index files to write. By default, bowtie2-build
+writes files named NAME.1.bt2, NAME.2.bt2, NAME.3.bt2, NAME.4.bt2,
+NAME.rev.1.bt2, and NAME.rev.2.bt2, where NAME is <bt2_base>.
+
+Options
-f
-The reference input files (specified as `<reference_in>`) are FASTA files
-(usually having extension `.fa`, `.mfa`, `.fna` or similar).
+The reference input files (specified as <reference_in>) are FASTA files
+(usually having extension .fa, .mfa, .fna or similar).
-c
-The reference sequences are given on the command line. I.e. `<reference_in>` is
-a comma-separated list of sequences rather than a list of FASTA files.
+The reference sequences are given on the command line. I.e.
+<reference_in> is a comma-separated list of sequences rather than a list
+of FASTA files.
--large-index
-Force `bowtie2-build` to build a [large index], even if the reference is less
-than ~ 4 billion nucleotides inlong.
+Force bowtie2-build to build a large index, even if the reference is
+less than ~ 4 billion nucleotides inlong.
-a/--noauto
-Disable the default behavior whereby `bowtie2-build` automatically selects
-values for the `--bmax`, `--dcv` and `--packed` parameters according to
-available memory. Instead, user may specify values for those parameters. If
-memory is exhausted during indexing, an error message will be printed; it is up
-to the user to try new parameters.
+Disable the default behavior whereby bowtie2-build automatically selects
+values for the --bmax, --dcv and --packed parameters according to
+available memory. Instead, user may specify values for those parameters.
+If memory is exhausted during indexing, an error message will be
+printed; it is up to the user to try new parameters.
-p/--packed
-Use a packed (2-bits-per-nucleotide) representation for DNA strings. This saves
-memory but makes indexing 2-3 times slower. Default: off. This is configured
-automatically by default; use `-a`/`--noauto` to configure manually.
+Use a packed (2-bits-per-nucleotide) representation for DNA strings.
+This saves memory but makes indexing 2-3 times slower. Default: off.
+This is configured automatically by default; use -a/--noauto to
+configure manually.
--bmax <int>
-The maximum number of suffixes allowed in a block. Allowing more suffixes per
-block makes indexing faster, but increases peak memory usage. Setting this
-option overrides any previous setting for `--bmax`, or `--bmaxdivn`.
-Default (in terms of the `--bmaxdivn` parameter) is `--bmaxdivn` 4 * number of threads. This is
-configured automatically by default; use `-a`/`--noauto` to configure manually.
+The maximum number of suffixes allowed in a block. Allowing more
+suffixes per block makes indexing faster, but increases peak memory
+usage. Setting this option overrides any previous setting for --bmax, or
+--bmaxdivn. Default (in terms of the --bmaxdivn parameter) is --bmaxdivn
+4 * number of threads. This is configured automatically by default; use
+-a/--noauto to configure manually.
--bmaxdivn <int>
-The maximum number of suffixes allowed in a block, expressed as a fraction of
-the length of the reference. Setting this option overrides any previous setting
-for `--bmax`, or `--bmaxdivn`. Default: `--bmaxdivn` 4 * number of threads. This is
-configured automatically by default; use `-a`/`--noauto` to configure manually.
+The maximum number of suffixes allowed in a block, expressed as a
+fraction of the length of the reference. Setting this option overrides
+any previous setting for --bmax, or --bmaxdivn. Default: --bmaxdivn 4 *
+number of threads. This is configured automatically by default; use
+-a/--noauto to configure manually.
--dcv <int>
-Use `<int>` as the period for the difference-cover sample. A larger period
-yields less memory overhead, but may make suffix sorting slower, especially if
-repeats are present. Must be a power of 2 no greater than 4096. Default: 1024.
- This is configured automatically by default; use `-a`/`--noauto` to configure
-manually.
+Use <int> as the period for the difference-cover sample. A larger period
+yields less memory overhead, but may make suffix sorting slower,
+especially if repeats are present. Must be a power of 2 no greater than
+4096. Default: 1024. This is configured automatically by default; use
+-a/--noauto to configure manually.
--nodc
-Disable use of the difference-cover sample. Suffix sorting becomes
+Disable use of the difference-cover sample. Suffix sorting becomes
quadratic-time in the worst case (where the worst case is an extremely
-repetitive reference). Default: off.
+repetitive reference). Default: off.
-r/--noref
-Do not build the `NAME.3.bt2` and `NAME.4.bt2` portions of the index, which
+Do not build the NAME.3.bt2 and NAME.4.bt2 portions of the index, which
contain a bitpacked version of the reference sequences and are used for
paired-end alignment.
-3/--justref
-Build only the `NAME.3.bt2` and `NAME.4.bt2` portions of the index, which
+Build only the NAME.3.bt2 and NAME.4.bt2 portions of the index, which
contain a bitpacked version of the reference sequences and are used for
paired-end alignment.
-o/--offrate <int>
-To map alignments back to positions on the reference sequences, it's necessary
-to annotate ("mark") some or all of the [Burrows-Wheeler] rows with their
-corresponding location on the genome.
-`-o`/`--offrate` governs how many rows get marked:
-the indexer will mark every 2^`<int>` rows. Marking more rows makes
-reference-position lookups faster, but requires more memory to hold the
-annotations at runtime. The default is 5 (every 32nd row is marked; for human
-genome, annotations occupy about 340 megabytes).
+To map alignments back to positions on the reference sequences, it's
+necessary to annotate ("mark") some or all of the Burrows-Wheeler rows
+with their corresponding location on the genome. -o/--offrate governs
+how many rows get marked: the indexer will mark every 2^<int> rows.
+Marking more rows makes reference-position lookups faster, but requires
+more memory to hold the annotations at runtime. The default is 5 (every
+32nd row is marked; for human genome, annotations occupy about 340
+megabytes).
-t/--ftabchars <int>
-The ftab is the lookup table used to calculate an initial [Burrows-Wheeler]
-range with respect to the first `<int>` characters of the query. A larger
-`<int>` yields a larger lookup table but faster query times. The ftab has size
-4^(`<int>`+1) bytes. The default setting is 10 (ftab is 4MB).
+The ftab is the lookup table used to calculate an initial
+Burrows-Wheeler range with respect to the first <int> characters of the
+query. A larger <int> yields a larger lookup table but faster query
+times. The ftab has size 4^(<int>+1) bytes. The default setting is 10
+(ftab is 4MB).
--seed <int>
-Use `<int>` as the seed for pseudo-random number generator.
+Use <int> as the seed for pseudo-random number generator.
--cutoff <int>
-Index only the first `<int>` bases of the reference sequences (cumulative across
-sequences) and ignore the rest.
+Index only the first <int> bases of the reference sequences (cumulative
+across sequences) and ignore the rest.
-q/--quiet
-`bowtie2-build` is verbose by default. With this option `bowtie2-build` will
+bowtie2-build is verbose by default. With this option bowtie2-build will
print only error messages.
--threads <int>
-By default `bowtie2-build` is using only one thread. Increasing the number
+By default bowtie2-build is using only one thread. Increasing the number
of threads will speed up the index building considerably in most cases.
-
+
-h/--help
Print usage information and quit.
@@ -1818,36 +1845,41 @@ Print usage information and quit.
Print version information and quit.
-The `bowtie2-inspect` index inspector
-=====================================
-`bowtie2-inspect` extracts information from a Bowtie index about what kind of
-index it is and what reference sequences were used to build it. When run without
-any options, the tool will output a FASTA file containing the sequences of the
-original references (with all non-`A`/`C`/`G`/`T` characters converted to `N`s).
- It can also be used to extract just the reference sequence names using the
-`-n`/`--names` option or a more verbose summary using the `-s`/`--summary`
-option.
+
+THE bowtie2-inspect INDEX INSPECTOR
+
+
+bowtie2-inspect extracts information from a Bowtie index about what kind
+of index it is and what reference sequences were used to build it. When
+run without any options, the tool will output a FASTA file containing
+the sequences of the original references (with all non-A/C/G/T
+characters converted to Ns). It can also be used to extract just the
+reference sequence names using the -n/--names option or a more verbose
+summary using the -s/--summary option.
+
Command Line
-------------
Usage:
bowtie2-inspect [options]* <bt2_base>
-### Main arguments
+Main arguments
+
+ <bt2_base>
-The basename of the index to be inspected. The basename is name of any of the
-index files but with the `.X.bt2` or `.rev.X.bt2` suffix omitted.
-`bowtie2-inspect` first looks in the current directory for the index files, then
-in the directory specified in the `BOWTIE2_INDEXES` environment variable.
+The basename of the index to be inspected. The basename is name of any
+of the index files but with the .X.bt2 or .rev.X.bt2 suffix omitted.
+bowtie2-inspect first looks in the current directory for the index
+files, then in the directory specified in the BOWTIE2_INDEXES
+environment variable.
-### Options
+Options
-a/--across <int>
-When printing FASTA output, output a newline character every `<int>` bases
+When printing FASTA output, output a newline character every <int> bases
(default: 60).
-n/--names
@@ -1856,18 +1888,20 @@ Print reference sequence names, one per line, and quit.
-s/--summary
-Print a summary that includes information about index settings, as well as the
-names and lengths of the input sequences. The summary has this format:
+Print a summary that includes information about index settings, as well
+as the names and lengths of the input sequences. The summary has this
+format:
- Colorspace <0 or 1>
- SA-Sample 1 in <sample>
- FTab-Chars <chars>
- Sequence-1 <name> <len>
- Sequence-2 <name> <len>
+ Colorspace <0 or 1>
+ SA-Sample 1 in <sample>
+ FTab-Chars <chars>
+ Sequence-1 <name> <len>
+ Sequence-2 <name> <len>
...
- Sequence-N <name> <len>
+ Sequence-N <name> <len>
-Fields are separated by tabs. Colorspace is always set to 0 for Bowtie 2.
+Fields are separated by tabs. Colorspace is always set to 0 for Bowtie
+2.
-v/--verbose
@@ -1881,63 +1915,60 @@ Print version information and quit.
Print usage information and quit.
-Getting started with Bowtie 2: Lambda phage example
-===================================================
-Bowtie 2 comes with some example files to get you started. The example files
-are not scientifically significant; we use the [Lambda phage] reference genome
-simply because it's short, and the reads were generated by a computer program,
-not a sequencer. However, these files will let you start running Bowtie 2 and
-downstream tools right away.
-First follow the manual instructions to [obtain Bowtie 2]. Set the `BT2_HOME`
-environment variable to point to the new Bowtie 2 directory containing the
-`bowtie2`, `bowtie2-build` and `bowtie2-inspect` binaries. This is important,
-as the `BT2_HOME` variable is used in the commands below to refer to that
-directory.
+GETTING STARTED WITH BOWTIE 2: LAMBDA PHAGE EXAMPLE
+
+
+Bowtie 2 comes with some example files to get you started. The example
+files are not scientifically significant; we use the Lambda phage
+reference genome simply because it's short, and the reads were generated
+by a computer program, not a sequencer. However, these files will let
+you start running Bowtie 2 and downstream tools right away.
+
+First follow the manual instructions to obtain Bowtie 2. Set the
+BT2_HOME environment variable to point to the new Bowtie 2 directory
+containing the bowtie2, bowtie2-build and bowtie2-inspect binaries. This
+is important, as the BT2_HOME variable is used in the commands below to
+refer to that directory.
-[Lambda phage]: http://en.wikipedia.org/wiki/Lambda_phage
Indexing a reference genome
----------------------------
-To create an index for the [Lambda phage] reference genome included with Bowtie
-2, create a new temporary directory (it doesn't matter where), change into that
-directory, and run:
+To create an index for the Lambda phage reference genome included with
+Bowtie 2, create a new temporary directory (it doesn't matter where),
+change into that directory, and run:
$BT2_HOME/bowtie2-build $BT2_HOME/example/reference/lambda_virus.fa lambda_virus
-The command should print many lines of output then quit. When the command
-completes, the current directory will contain four new files that all start with
-`lambda_virus` and end with `.1.bt2`, `.2.bt2`, `.3.bt2`, `.4.bt2`,
-`.rev.1.bt2`, and `.rev.2.bt2`. These files constitute the index - you're done!
+The command should print many lines of output then quit. When the
+command completes, the current directory will contain four new files
+that all start with lambda_virus and end with .1.bt2, .2.bt2, .3.bt2,
+.4.bt2, .rev.1.bt2, and .rev.2.bt2. These files constitute the index -
+you're done!
-You can use `bowtie2-build` to create an index for a set of FASTA files obtained
-from any source, including sites such as [UCSC], [NCBI], and [Ensembl]. When
-indexing multiple FASTA files, specify all the files using commas to separate
-file names. For more details on how to create an index with `bowtie2-build`,
-see the [manual section on index building]. You may also want to bypass this
-process by obtaining a pre-built index. See [using a pre-built index] below
-for an example.
+You can use bowtie2-build to create an index for a set of FASTA files
+obtained from any source, including sites such as UCSC, NCBI, and
+Ensembl. When indexing multiple FASTA files, specify all the files using
+commas to separate file names. For more details on how to create an
+index with bowtie2-build, see the manual section on index building. You
+may also want to bypass this process by obtaining a pre-built index. See
+using a pre-built index below for an example.
-[UCSC]: http://genome.ucsc.edu/cgi-bin/hgGateway
-[NCBI]: http://www.ncbi.nlm.nih.gov/sites/genome
-[Ensembl]: http://www.ensembl.org/
Aligning example reads
-----------------------
-Stay in the directory created in the previous step, which now contains the
-`lambda_virus` index files. Next, run:
+Stay in the directory created in the previous step, which now contains
+the lambda_virus index files. Next, run:
$BT2_HOME/bowtie2 -x lambda_virus -U $BT2_HOME/example/reads/reads_1.fq -S eg1.sam
-This runs the Bowtie 2 aligner, which aligns a set of unpaired reads to the
-[Lambda phage] reference genome using the index generated in the previous step.
-The alignment results in SAM format are written to the file `eg1.sam`, and a
-short alignment summary is written to the console. (Actually, the summary is
-written to the "standard error" or "stderr" filehandle, which is typically
-printed to the console.)
+This runs the Bowtie 2 aligner, which aligns a set of unpaired reads to
+the Lambda phage reference genome using the index generated in the
+previous step. The alignment results in SAM format are written to the
+file eg1.sam, and a short alignment summary is written to the console.
+(Actually, the summary is written to the "standard error" or "stderr"
+filehandle, which is typically printed to the console.)
To see the first few lines of the SAM output, run:
@@ -1945,72 +1976,73 @@ To see the first few lines of the SAM output, run:
You will see something like this:
- @HD VN:1.0 SO:unsorted
- @SQ SN:gi|9626243|ref|NC_001416.1| LN:48502
- @PG ID:bowtie2 PN:bowtie2 VN:2.0.1
- r1 0 gi|9626243|ref|NC_001416.1| 18401 42 122M * 0 0 TGAATGCGAACTCCGGGACGCTCAGTAATGTGACGATAGCTGAAAACTGTACGATAAACNGTACGCTGAGGGCAGAAAAAATCGTCGGGGACATTNTAAAGGCGGCGAGCGCGGCTTTTCCG +"@6<:27(F&5)9)"B:%B+A-%5A?2$HCB0B+0=D<7E/<.03#!.F77 at 6B==?C"7>;))%;,3-$.A06+<-1/@@?,26">=?*@'0;$:;??G+:#+(A?9+10!8!?()?7C> AS:i:-5 XN:i:0 XM:i:3 XO:i:0 XG:i:0 NM:i:3 MD:Z:59G13G21G26 YT:Z:UU
- r2 0 gi|9626243|ref|NC_001416.1| 8886 42 275M * 0 0 NTTNTGATGCGGGCTTGTGGAGTTCAGCCGATCTGACTTATGTCATTACCTATGAAATGTGAGGACGCTATGCCTGTACCAAATCCTACAATGCCGGTGAAAGGTGCCGGGATCACCCTGTGGGTTTATAAGGGGATCGGTGACCCCTACGCGAATCCGCTTTCAGACGTTGACTGGTCGCGTCTGGCAAAAGTTAAAGACCTGACGCCCGGCGAACTGACCGCTGAGNCCTATGACGACAGCTATCTCGATGATGAAGATGCAGACTGGACTGC '%)%!+!(&++)''"#"#&#"!'!("%'""("+&%$%*%%#$%#%#!)*'(($&$'&%+&#%*)*#*%*')(%+!%%*"$%"#+)$&&+)&)*+!"*)!*!("&&"*#+"&"'(%)*("'!$*!!%$&&&$!!&&"(*"$&"#&!$%'%"#)$#+%*+)! [...]
- r3 16 gi|9626243|ref|NC_001416.1| 11599 42 338M * 0 0 GGGCGCGTTACTGGGATGATCGTGAAAAGGCCCGTCTTGCGCTTGAAGCCGCCCGAAAGAAGGCTGAGCAGCAGACTCAAGAGGAGAAAAATGCGCAGCAGCGGAGCGATACCGAAGCGTCACGGCTGAAATATACCGAAGAGGCGCAGAAGGCTNACGAACGGCTGCAGACGCCGCTGCAGAAATATACCGCCCGTCAGGAAGAACTGANCAAGGCACNGAAAGACGGGAAAATCCTGCAGGCGGATTACAACACGCTGATGGCGGCGGCGAAAAAGGATTATGAAGCGACGCTGTAAAAGCCGAAACAGTCCAGCGTGAAGGTGTCTGCGGGCGAT 7F$%6=$:9B@/F'>=?!D?@0(:A*)7/>9C>6#1<6:C(.CC;#.;>;2'$4D:?&B!>689?(0(G7+0=@37F)GG=>?958.D2E04C<E [...]
- r4 0 gi|9626243|ref|NC_001416.1| 40075 42 184M * 0 0 GGGCCAATGCGCTTACTGATGCGGAATTACGCCGTAAGGCCGCAGATGAGCTTGTCCATATGACTGCGAGAATTAACNGTGGTGAGGCGATCCCTGAACCAGTAAAACAACTTCCTGTCATGGGCGGTAGACCTCTAAATCGTGCACAGGCTCTGGCGAAGATCGCAGAAATCAAAGCTAAGT(=8B)GD04*G%&4F,1'A>.C&7=F$,+#6!))43C,5/5+)?-/0>/D3=-,2/+.1?@->;)00!'3!7BH$G)HG+ADC'#-9F)7<7"$?&.>0)@5;4,!0-#C!15CF8&HB+B==H>7,/)C5)5*+(F5A%D,EA<(>G9E0>7&/E?4%;#'92)<5+ at 7:A.(BG at BG86@.G AS:i:-1 XN:i:0 XM:i:1 XO:i:0 XG:i:0 NM:i:1 MD:Z:77C106 YT:Z:UU
- r5 0 gi|9626243|ref|NC_001416.1| 48010 42 138M * 0 0 GTCAGGAAAGTGGTAAAACTGCAACTCAATTACTGCAATGCCCTCGTAATTAAGTGAATTTACAATATCGTCCTGTTCGGAGGGAAGAACGCGGGATGTTCATTCTTCATCACTTTTAATTGATGTATATGCTCTCTT 9''%<D)A03E1-*7=),:F/0!6,D9:H,<9D%:0B(%'E,(8EFG$E89B$27G8F*2+4,-!,0D5()&=(FGG:5;3*@/.0F-G#5#3->('FDFEG?)5.!)"AGADB3?6(@H(:B<>6!>;>6>G,."?% AS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:138 YT:Z:UU
- r6 16 gi|9626243|ref|NC_001416.1| 41607 42 72M2D119M * 0 0 TCGATTTGCAAATACCGGAACATCTCGGTAACTGCATATTCTGCATTAAAAAATCAACGCAAAAAATCGGACGCCTGCAAAGATGAGGAGGGATTGCAGCGTGTTTTTAATGAGGTCATCACGGGATNCCATGTGCGTGACGGNCATCGGGAAACGCCAAAGGAGATTATGTACCGAGGAAGAATGTCGCT 1H#G;H"$E*E#&"*)2%66?=9/9'=;4)4/>@%+5#@#$4A*!<D=="8#1*A9BA=:(1+#C&.#(3#H=9E)AC*5,AC#E'536*2?)H14?>9'B=7(3H/B:+A:8%1-+#(E%&$$&14"76D?>7(&20H5%*&CF8!G5B+A4F$7(:"'?0$?G+$)B-?2<0<F=D!38BH,%=8&5 at + AS:i:-13 XN:i:0 XM:i:2 XO:i:1 XG:i:2 NM:i:4 M [...]
- r7 16 gi|9626243|ref|NC_001416.1| 4692 42 143M * 0 0 TCAGCCGGACGCGGGCGCTGCAGCCGTACTCGGGGATGACCGGTTACAACGGCATTATCGCCCGTCTGCAACAGGCTGCCAGCGATCCGATGGTGGACAGCATTCTGCTCGATATGGACANGCCCGGCGGGATGGTGGCGGGG -"/@*7A0)>2,AAH@&"%B)*5*23B/,)90.B@%=FE,E063C9?,:26$-0:,.,1849'4.;F>FA;76+5&$<C":$!A*,<B,<)@<'85D%C*:)30 at 85;?.B$05=@95DCDH<53!8G:F:B7/A.E':434> AS:i:-6 XN:i:0 XM:i:2 XO:i:0 XG:i:0 NM:i:2 MD:Z:98G21C22 YT:Z:UU
-
-The first few lines (beginning with `@`) are SAM header lines, and the rest of
-the lines are SAM alignments, one line per read or mate. See the [Bowtie 2
-manual section on SAM output] and the [SAM specification] for details about how
-to interpret the SAM file format.
+ @HD VN:1.0 SO:unsorted
+ @SQ SN:gi|9626243|ref|NC_001416.1| LN:48502
+ @PG ID:bowtie2 PN:bowtie2 VN:2.0.1
+ r1 0 gi|9626243|ref|NC_001416.1| 18401 42 122M * 0 0 TGAATGCGAACTCCGGGACGCTCAGTAATGTGACGATAGCTGAAAACTGTACGATAAACNGTACGCTGAGGGCAGAAAAAATCGTCGGGGACATTNTAAAGGCGGCGAGCGCGGCTTTTCCG +"@6<:27(F&5)9)"B:%B+A-%5A?2$HCB0B+0=D<7E/<.03#!.F77 at 6B==?C"7>;))%;,3-$.A06+<-1/@@?,26">=?*@'0;$:;??G+:#+(A?9+10!8!?()?7C> AS:i:-5 XN:i:0 XM:i:3 XO:i:0 XG:i:0 NM:i:3 MD:Z:59G13G21G26 YT:Z:UU
+ r2 0 gi|9626243|ref|NC_001416.1| 8886 42 275M * 0 0 NTTNTGATGCGGGCTTGTGGAGTTCAGCCGATCTGACTTATGTCATTACCTATGAAATGTGAGGACGCTATGCCTGTACCAAATCCTACAATGCCGGTGAAAGGTGCCGGGATCACCCTGTGGGTTTATAAGGGGATCGGTGACCCCTACGCGAATCCGCTTTCAGACGTTGACTGGTCGCGTCTGGCAAAAGTTAAAGACCTGACGCCCGGCGAACTGACCGCTGAGNCCTATGACGACAGCTATCTCGATGATGAAGATGCAGACTGGACTGC (#!!'+!$""%+(+)'%)%!+!(&++)''"#"#&#"!'!("%'""("+&%$%*%%#$%#%#!)*'(#")(($&$'&%+&#%*)*#*%*')(%+!%%*"$%"#+)$&&+)&)*+!"*)!*!("&&"*#+"&"'(%)*("'!$*!! [...]
+ r3 16 gi|9626243|ref|NC_001416.1| 11599 42 338M * 0 0 GGGCGCGTTACTGGGATGATCGTGAAAAGGCCCGTCTTGCGCTTGAAGCCGCCCGAAAGAAGGCTGAGCAGCAGACTCAAGAGGAGAAAAATGCGCAGCAGCGGAGCGATACCGAAGCGTCACGGCTGAAATATACCGAAGAGGCGCAGAAGGCTNACGAACGGCTGCAGACGCCGCTGCAGAAATATACCGCCCGTCAGGAAGAACTGANCAAGGCACNGAAAGACGGGAAAATCCTGCAGGCGGATTACAACACGCTGATGGCGGCGGCGAAAAAGGATTATGAAGCGACGCTGTAAAAGCCGAAACAGTCCAGCGTGAAGGTGTCTGCGGGCGAT 7F$%6=$:9B@/F'>=?!D?@0(:A*)7/>9C>6#1<6:C(.CC;#.;>;2'$4D:?&B!>689?(0(G7+0=@37F)GG [...]
+ r4 0 gi|9626243|ref|NC_001416.1| 40075 42 184M * 0 0 GGGCCAATGCGCTTACTGATGCGGAATTACGCCGTAAGGCCGCAGATGAGCTTGTCCATATGACTGCGAGAATTAACNGTGGTGAGGCGATCCCTGAACCAGTAAAACAACTTCCTGTCATGGGCGGTAGACCTCTAAATCGTGCACAGGCTCTGGCGAAGATCGCAGAAATCAAAGCTAAGT(=8B)GD04*G%&4F,1'A>.C&7=F$,+#6!))43C,5/5+)?-/0>/D3=-,2/+.1?@->;)00!'3!7BH$G)HG+ADC'#-9F)7<7"$?&.>0)@5;4,!0-#C!15CF8&HB+B==H>7,/)C5)5*+(F5A%D,EA<(>G9E0>7&/E?4%;#'92)<5+ at 7:A.(BG at BG86@.G AS:i:-1 XN:i:0 XM:i:1 XO:i:0 XG:i:0 NM:i:1 MD:Z [...]
+ r5 0 gi|9626243|ref|NC_001416.1| 48010 42 138M * 0 0 GTCAGGAAAGTGGTAAAACTGCAACTCAATTACTGCAATGCCCTCGTAATTAAGTGAATTTACAATATCGTCCTGTTCGGAGGGAAGAACGCGGGATGTTCATTCTTCATCACTTTTAATTGATGTATATGCTCTCTT 9''%<D)A03E1-*7=),:F/0!6,D9:H,<9D%:0B(%'E,(8EFG$E89B$27G8F*2+4,-!,0D5()&=(FGG:5;3*@/.0F-G#5#3->('FDFEG?)5.!)"AGADB3?6(@H(:B<>6!>;>6>G,."?% AS:i:0 XN:i:0 XM:i:0 XO:i:0 XG:i:0 NM:i:0 MD:Z:138 YT:Z:UU
+ r6 16 gi|9626243|ref|NC_001416.1| 41607 42 72M2D119M * 0 0 TCGATTTGCAAATACCGGAACATCTCGGTAACTGCATATTCTGCATTAAAAAATCAACGCAAAAAATCGGACGCCTGCAAAGATGAGGAGGGATTGCAGCGTGTTTTTAATGAGGTCATCACGGGATNCCATGTGCGTGACGGNCATCGGGAAACGCCAAAGGAGATTATGTACCGAGGAAGAATGTCGCT 1H#G;H"$E*E#&"*)2%66?=9/9'=;4)4/>@%+5#@#$4A*!<D=="8#1*A9BA=:(1+#C&.#(3#H=9E)AC*5,AC#E'536*2?)H14?>9'B=7(3H/B:+A:8%1-+#(E%&$$&14"76D?>7(&20H5%*&CF8!G5B+A4F$7(:"'?0$?G+$)B-?2<0<F=D!38BH,%=8&5 at + AS:i:-13 XN:i:0 XM:i:2 XO:i [...]
+ r7 16 gi|9626243|ref|NC_001416.1| 4692 42 143M * 0 0 TCAGCCGGACGCGGGCGCTGCAGCCGTACTCGGGGATGACCGGTTACAACGGCATTATCGCCCGTCTGCAACAGGCTGCCAGCGATCCGATGGTGGACAGCATTCTGCTCGATATGGACANGCCCGGCGGGATGGTGGCGGGG -"/@*7A0)>2,AAH@&"%B)*5*23B/,)90.B@%=FE,E063C9?,:26$-0:,.,1849'4.;F>FA;76+5&$<C":$!A*,<B,<)@<'85D%C*:)30 at 85;?.B$05=@95DCDH<53!8G:F:B7/A.E':434> AS:i:-6 XN:i:0 XM:i:2 XO:i:0 XG:i:0 NM:i:2 MD:Z:98G21C22 YT:Z:UU
+
+The first few lines (beginning with @) are SAM header lines, and the
+rest of the lines are SAM alignments, one line per read or mate. See the
+Bowtie 2 manual section on SAM output and the SAM specification for
+details about how to interpret the SAM file format.
+
Paired-end example
-------------------
-To align paired-end reads included with Bowtie 2, stay in the same directory and
-run:
+To align paired-end reads included with Bowtie 2, stay in the same
+directory and run:
$BT2_HOME/bowtie2 -x lambda_virus -1 $BT2_HOME/example/reads/reads_1.fq -2 $BT2_HOME/example/reads/reads_2.fq -S eg2.sam
-This aligns a set of paired-end reads to the reference genome, with results
-written to the file `eg2.sam`.
+This aligns a set of paired-end reads to the reference genome, with
+results written to the file eg2.sam.
+
Local alignment example
------------------------
-To use [local alignment] to align some longer reads included with Bowtie 2, stay
-in the same directory and run:
+To use local alignment to align some longer reads included with Bowtie
+2, stay in the same directory and run:
$BT2_HOME/bowtie2 --local -x lambda_virus -U $BT2_HOME/example/reads/longreads.fq -S eg3.sam
-This aligns the long reads to the reference genome using local alignment, with
-results written to the file `eg3.sam`.
+This aligns the long reads to the reference genome using local
+alignment, with results written to the file eg3.sam.
+
Using SAMtools/BCFtools downstream
-----------------------------------
-[SAMtools] is a collection of tools for manipulating and analyzing SAM and BAM
-alignment files. [BCFtools] is a collection of tools for calling variants and
-manipulating VCF and BCF files, and it is typically distributed with [SAMtools].
-Using these tools together allows you to get from alignments in SAM format to
-variant calls in VCF format. This example assumes that `samtools` and
-`bcftools` are installed and that the directories containing these binaries are
-in your [PATH environment variable].
+SAMtools is a collection of tools for manipulating and analyzing SAM and
+BAM alignment files. BCFtools is a collection of tools for calling
+variants and manipulating VCF and BCF files, and it is typically
+distributed with SAMtools. Using these tools together allows you to get
+from alignments in SAM format to variant calls in VCF format. This
+example assumes that samtools and bcftools are installed and that the
+directories containing these binaries are in your PATH environment
+variable.
Run the paired-end example:
$BT2_HOME/bowtie2 -x $BT2_HOME/example/index/lambda_virus -1 $BT2_HOME/example/reads/reads_1.fq -2 $BT2_HOME/example/reads/reads_2.fq -S eg2.sam
-Use `samtools view` to convert the SAM file into a BAM file. BAM is the
-binary format corresponding to the SAM text format. Run:
+Use samtools view to convert the SAM file into a BAM file. BAM is the
+binary format corresponding to the SAM text format. Run:
samtools view -bS eg2.sam > eg2.bam
-Use `samtools sort` to convert the BAM file to a sorted BAM file.
+Use samtools sort to convert the BAM file to a sorted BAM file.
samtools sort eg2.bam -o eg2.sorted.bam
-We now have a sorted BAM file called `eg2.sorted.bam`. Sorted BAM is a useful
-format because the alignments are (a) compressed, which is convenient for
-long-term storage, and (b) sorted, which is conveneint for variant discovery.
-To generate variant calls in VCF format, run:
+We now have a sorted BAM file called eg2.sorted.bam. Sorted BAM is a
+useful format because the alignments are (a) compressed, which is
+convenient for long-term storage, and (b) sorted, which is conveneint
+for variant discovery. To generate variant calls in VCF format, run:
samtools mpileup -uf $BT2_HOME/example/reference/lambda_virus.fa eg2.sorted.bam | bcftools view -Ov - > eg2.raw.bcf
@@ -2018,10 +2050,5 @@ Then to view the variants, run:
bcftools view eg2.raw.bcf
-See the official SAMtools guide to [Calling SNPs/INDELs with SAMtools/BCFtools]
-for more details and variations on this process.
-
-[SAMtools]: http://samtools.sourceforge.net/
-[BCFtools]: http://samtools.sourceforge.net/mpileup.shtml
-[PATH environment variable]: http://en.wikipedia.org/wiki/PATH_(variable)
-[Calling SNPs/INDELs with SAMtools/BCFtools]: http://samtools.sourceforge.net/mpileup.shtml
+See the official SAMtools guide to Calling SNPs/INDELs with
+SAMtools/BCFtools for more details and variations on this process.
diff --git a/MANUAL.markdown b/MANUAL.markdown
index e3604a1..039118c 100644
--- a/MANUAL.markdown
+++ b/MANUAL.markdown
@@ -13,11 +13,12 @@ What is Bowtie 2?
[Bowtie 2] is an ultrafast and memory-efficient tool for aligning sequencing
reads to long reference sequences. It is particularly good at aligning reads of
about 50 up to 100s or 1,000s of characters to relatively long (e.g. mammalian)
-genomes. Bowtie 2 indexes the genome with an [FM Index] (based on the
+genomes. Bowtie 2 indexes the genome with an [FM Index][FM Index Wiki] (based on the
[Burrows-Wheeler Transform] or [BWT]) to keep its memory footprint small: for
the human genome, its memory footprint is typically around 3.2 gigabytes of RAM.
Bowtie 2 supports gapped, local, and paired-end alignment modes. Multiple
processors can be used simultaneously to achieve greater alignment speed.
+[Cufflinks][]: a tool for transcriptome assembly and isoform quantitiation from
Bowtie 2 outputs alignments in [SAM] format, enabling interoperation with a
large number of other tools (e.g. [SAMtools], [GATK]) that use SAM. Bowtie 2 is
distributed under the [GPLv3 license], and it runs on the command line under
@@ -27,7 +28,6 @@ Windows, Mac OS X and Linux.
including for variation calling, ChIP-seq, RNA-seq, BS-seq. [Bowtie 2] and
[Bowtie] (also called "[Bowtie 1]" here) are also tightly integrated into some
tools, including [TopHat]: a fast splice junction mapper for RNA-seq reads,
-[Cufflinks]: a tool for transcriptome assembly and isoform quantitiation from
RNA-seq reads, [Crossbow]: a cloud-enabled software tool for analyzing
resequencing data, and [Myrna]: a cloud-enabled software tool for aligning
RNA-seq reads and measuring differential gene expression.
@@ -35,21 +35,6 @@ RNA-seq reads and measuring differential gene expression.
If you use [Bowtie 2] for your published research, please cite the [Bowtie
paper]. Thank you!
-[Bowtie 2]: http://bowtie-bio.sf.net/bowtie2
-[Bowtie]: http://bowtie-bio.sf.net
-[Bowtie 1]: http://bowtie-bio.sf.net
-[Burrows-Wheeler Transform]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
-[BWT]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
-[FM Index]: http://en.wikipedia.org/wiki/FM-index
-[SAM]: http://samtools.sourceforge.net/SAM1.pdf
-[SAMtools]: http://samtools.sourceforge.net
-[GATK]: http://www.broadinstitute.org/gsa/wiki/index.php/The_Genome_Analysis_Toolkit
-[TopHat]: http://tophat.cbcb.umd.edu/
-[Cufflinks]: http://cufflinks.cbcb.umd.edu/
-[Crossbow]: http://bowtie-bio.sf.net/crossbow
-[Myrna]: http://bowtie-bio.sf.net/myrna
-[Bowtie paper]: http://genomebiology.com/2009/10/3/R25
-[GPLv3 license]: http://www.gnu.org/licenses/gpl-3.0.html
How is Bowtie 2 different from Bowtie 1?
----------------------------------------
@@ -64,7 +49,6 @@ The chief differences between Bowtie 1 and Bowtie 2 are:
1. For reads longer than about 50 bp Bowtie 2 is generally faster, more
sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g.
less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive.
-B
2. Bowtie 2 supports gapped alignment with affine gap penalties. Number of gaps
and gap lengths are not restricted, except by way of the configurable scoring
@@ -99,13 +83,6 @@ which reports either 0 or high.
Bowtie 2 is not a "drop-in" replacement for Bowtie 1. Bowtie 2's command-line
arguments and genome index format are both different from Bowtie 1's.
-[local alignment]: #end-to-end-alignment-versus-local-alignment
-[end-to-end alignment]: #end-to-end-alignment-versus-local-alignment
-[overlap ambiguous characters]: #ambiguous-characters
-[Needleman-Wunsch]: http://en.wikipedia.org/wiki/Needleman-Wunsch_algorithm
-[Smith-Waterman]: http://en.wikipedia.org/wiki/Smith_waterman
-[scoring scheme]: #scores-higher-more-similar
-[paired-end alignment]: #aligning-pairs
What isn't Bowtie 2?
--------------------
@@ -126,11 +103,6 @@ adequate when the reference is short.
Bowtie 2 does not support alignment of colorspace reads.
-[MUMmer]: http://mummer.sourceforge.net/
-[NUCmer]: http://mummer.sourceforge.net/manual/#nucmer
-[BLAST]: http://blast.ncbi.nlm.nih.gov/Blast.cgi
-[BLAT]: http://genome.ucsc.edu/cgi-bin/hgBlat?command=start
-[Vmatch]: http://www.vmatch.de/
What does it mean that some older Bowtie 2 versions are "beta"?
--------------------------------------------------------------
@@ -165,27 +137,19 @@ Bowtie 2 tools by running GNU `make` (usually with the command `make`, but
sometimes with `gmake`) with no arguments. If building with MinGW, run `make`
from the MSYS environment.
-+Bowtie 2 is using the multithreading software model in order to
-+speed up execution times on SMP architectures where this is possible.
-+The Threading Building Blocks library, TBB, is now the default
-+threading library in Bowtie 2. On POSIX platforms (like Linux, Mac
-+OS, etc.) if TBB is not available the pthread library will be used.
-+Although it is possible to use pthread library on Windows, a non-POSIX
-+platform, due to performance reasons Bowtie 2 will try to use Windows
-+native multithreading if possible. We recommend that you first
-+install the [Threading Building Blocks library], but if unable to
-+do so please specify `make NO_TBB=1`. TBB comes installed by default
-+on many popular Linux distros. Please note, packages built without
-+TBB will have _-legacy_ appended to the name.
-
-[MinGW]: http://www.mingw.org/
-[MSYS]: http://www.mingw.org/wiki/msys
-[pthreads]: http://sourceware.org/pthreads-win32/
-[GnuWin32]: http://gnuwin32.sf.net/packages/coreutils.htm
-[Threading Building Blocks library]: https://www.threadingbuildingblocks.org
-[Download]: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/
-[sourceforge site]: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/
-[Xcode]: http://developer.apple.com/xcode/
+Bowtie 2 is using the multithreading software model in order to
+speed up execution times on SMP architectures where this is possible.
+The Threading Building Blocks library, TBB, is now the default
+threading library in Bowtie 2. On POSIX platforms (like Linux, Mac
+OS, etc.) if TBB is not available the pthread library will be used.
+Although it is possible to use pthread library on Windows, a non-POSIX
+platform, due to performance reasons Bowtie 2 will try to use Windows
+native multithreading if possible. We recommend that you first
+install the [Threading Building Blocks library], but if unable to
+do so please specify `make NO_TBB=1`. TBB comes installed by default
+on many popular Linux distros. Please note, packages built without
+TBB will have _-legacy_ appended to the name.
+
Adding to PATH
--------------
@@ -203,8 +167,6 @@ executables, including `bowtie2`, `bowtie2-align-s`, `bowtie2-align-l`,
`bowtie2-build`, `bowtie2-build-s`, `bowtie2-build-l`, `bowtie2-inspect`,
`bowtie2-inspect-s` and `bowtie2-inspect-l`.
-[PATH environment variable]: http://en.wikipedia.org/wiki/PATH_(variable)
-[PATH]: http://en.wikipedia.org/wiki/PATH_(variable)
The `bowtie2` aligner
=====================
@@ -324,7 +286,6 @@ configured with the [`--score-min`] option. For details on how to set options
like `--score-min` that correspond to functions, see the section on [setting
function options].
-[setting function options]: #setting-function-options
Mapping quality: higher = more unique
-------------------------------------
@@ -352,7 +313,6 @@ with mapping quality less than, say, 10. A mapping quality of 10 or less
indicates that there is at least a 1 in 10 chance that the read truly originated
elsewhere.
-[SAM]: http://samtools.sourceforge.net/SAM1.pdf
Aligning pairs
--------------
@@ -408,8 +368,8 @@ instance, when seeking [structural variants].
The expected relative orientation of the mates is set using the [`--ff`],
[`--fr`], or [`--rf`] options. The expected range of inter-mates distances (as
measured from the furthest extremes of the mates; also called "outer distance")
-is set with the [`-I`] and [`-X`] options. Note that setting [`-I`] and [`-X`]
-far apart makes Bowtie 2 slower. See documentation for [`-I`] and [`-X`].
+is set with the [`-I`][`+I`] and [`-X`][`+X`] options. Note that setting [`-I`][`+I`] and [`-X`][`+X`]
+far apart makes Bowtie 2 slower. See documentation for [`-I`][`+I`] and [`-X`][`+X`].
To declare that a pair aligns discordantly, Bowtie 2 requires that both mates
align uniquely. This is a conservative threshold, but this is often desirable
@@ -419,7 +379,6 @@ By default, Bowtie 2 searches for both concordant and discordant alignments,
though searching for discordant alignments can be disabled with the
[`--no-discordant`] option.
-[structural variants]: http://www.ncbi.nlm.nih.gov/dbvar/content/overview/
### Mixed mode: paired where possible, unpaired otherwise
@@ -510,8 +469,6 @@ In general, when we say that a read has an alignment, we mean that it has a
[valid alignment]. When we say that a read has multiple alignments, we mean
that it has multiple alignments that are valid and distinct from one another.
-[valid alignment]: #valid-alignments-meet-or-exceed-the-minimum-score-threshold
-[BWA]: http://bio-bwa.sourceforge.net/
### Distinct alignments map a read to different places
@@ -535,7 +492,7 @@ distinct or both.
By default, Bowtie 2 searches for distinct, valid alignments for each read. When
it finds a valid alignment, it generally will continue to look for alignments
that are nearly as good or better. It will eventually stop looking, either
-because it exceeded a limit placed on search effort (see [`-D`] and [`-R`]) or
+because it exceeded a limit placed on search effort (see [`-D`] and [`-R`][`+R`]) or
because it already knows all it needs to know to report an alignment.
Information from the best alignments are used to estimate mapping quality (the
`MAPQ` [SAM] field) and to set SAM optional fields, such as [`AS:i`] and
@@ -548,8 +505,8 @@ searching. Increasing [`-D`] makes Bowtie 2 slower, but increases the
likelihood that it will report the correct alignment for a read that aligns many
places.
-See also: [`-R`], which sets the maximum number of times Bowtie 2 will "re-seed"
-when attempting to align a read with repetitive seeds. Increasing [`-R`] makes
+See also: [`-R`][`+R`], which sets the maximum number of times Bowtie 2 will "re-seed"
+when attempting to align a read with repetitive seeds. Increasing [`-R`][`+R`] makes
Bowtie 2 slower, but increases the likelihood that it will report the correct
alignment for a read that aligns many places.
@@ -571,7 +528,6 @@ Still, this mode can be effective and fast in situations where the user cares
more about whether a read aligns (or aligns a certain number of times) than
where exactly it originated.
-[SAM specification]: http://samtools.sourceforge.net/SAM1.pdf
### -a mode: search for and report all alignments
@@ -586,7 +542,6 @@ details.
Some tools are designed with this reporting mode in mind. Bowtie 2 is not! For
very large genomes, this mode is very slow.
-[SAM specification]: http://samtools.sourceforge.net/SAM1.pdf
### Randomness in Bowtie 2
@@ -618,8 +573,8 @@ Multiseed heuristic
To rapidly narrow the number of possible alignments that must be considered,
Bowtie 2 begins by extracting substrings ("seeds") from the read and its reverse
-complement and aligning them in an ungapped fashion with the help of the [FM
-Index]. This is "multiseed alignment" and it is similar to what [Bowtie 1
+complement and aligning them in an ungapped fashion with the help of the [FM Index][FM Index Paper].
+This is "multiseed alignment" and it is similar to what [Bowtie 1
does], except Bowtie 1 attempts to align the entire read this way.
This initial step makes Bowtie 2 much faster than it would be without such a
@@ -636,22 +591,17 @@ shorter, and/or (c) allow more mismatches. You can adjust these options
one-by-one, though Bowtie 2 comes with some useful combinations of options
prepackaged as "[preset options]."
-[`-D`] and [`-R`] are also options that adjust the trade-off between speed and
+[`-D`] and [`-R`][`+R`] are also options that adjust the trade-off between speed and
sensitivity/accuracy.
-[preset options]: #presets-setting-many-settings-at-once
### FM Index memory footprint
-Bowtie 2 uses the [FM Index] to find ungapped alignments for seeds. This step
-accounts for the bulk of Bowtie 2's memory footprint, as the [FM Index] itself
+Bowtie 2 uses the [FM Index][FM Index Paper] to find ungapped alignments for seeds. This step
+accounts for the bulk of Bowtie 2's memory footprint, as the [FM Index][FM Index Paper] itself
is typically the largest data structure used. For instance, the memory
-footprint of the [FM Index] for the human genome is about 3.2 gigabytes of RAM.
+footprint of the [FM Index][FM Index Paper] for the human genome is about 3.2 gigabytes of RAM.
-[Bowtie 1 does]: http://genomebiology.com/2009/10/3/R25
-[Bowtie 1 paper]: http://genomebiology.com/2009/10/3/R25
-[FM Index]: http://portal.acm.org/citation.cfm?id=796543
-[bi-directional BWT approach]: http://www.computer.org/portal/web/csdl/doi/10.1109/BIBM.2009.42
Ambiguous characters
--------------------
@@ -673,14 +623,12 @@ ambiguous reference characters. For an alignment overlapping an ambiguous
reference character to be found, it must have one or more seed alignments that
do not overlap ambiguous reference characters.
-[IUPAC nucleotide codes]: http://www.bioinformatics.org/sms/iupac.html
-[multiseed heuristic]: #multiseed-heuristic
Presets: setting many settings at once
--------------------------------------
Bowtie 2 comes with some useful combinations of parameters packaged into shorter
-"preset" parameters. For example, running Bowtie 2 with the `--very-sensitive`
+"preset" parameters. For example, running Bowtie 2 with the [`--very-sensitive`]
option is the same as running with options: `-D 20 -R 3 -N 0 -L 20 -i S,1,0.50`.
The preset options that come with Bowtie 2 are designed to cover a wide area of
the speed/sensitivity/accuracy trade-off space, with the presets ending in `fast`
@@ -688,7 +636,6 @@ generally being faster but less sensitive and less accurate, and the presets
ending in `sensitive` generally being slower but more sensitive and more
accurate. See the [documentation for the preset options] for details.
-[documentation for the preset options]: #preset-options-in---end-to-end-mode
Filtering
---------
@@ -791,7 +738,7 @@ Performance tuning
If you are using [`-k`] or [`-a`] options and Bowtie 2 is reporting many
alignments per read, using an index with a denser SA sample can speed
things up considerably. To do this, specify a smaller-than-default
- [`-o`/`--offrate`](#bowtie2-build-options-o) value when running `bowtie2-build`.
+ [`-o`/`--offrate`] value when running `bowtie2-build`.
A denser SA sample yields a larger index, but is also particularly
effective at speeding up alignment when many alignments are reported per
read.
@@ -832,14 +779,12 @@ considered valid, and `x` is the read length.
### Usage
- bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r>} -S [<hit>]
+ bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i>} -S [<sam>]
### Main arguments
<table><tr><td>
-[`-x`]: #bowtie2-options-x
-
-x <bt2-idx>
</td><td>
@@ -851,8 +796,6 @@ then in the directory specified in the `BOWTIE2_INDEXES` environment variable.
</td></tr><tr><td>
-[`-1`]: #bowtie2-options-1
-
-1 <m1>
</td><td>
@@ -865,8 +808,6 @@ mate 1s from the "standard in" or "stdin" filehandle.
</td></tr><tr><td>
-[`-2`]: #bowtie2-options-2
-
-2 <m2>
</td><td>
@@ -879,8 +820,6 @@ mate 2s from the "standard in" or "stdin" filehandle.
</td></tr><tr><td>
-[`-U`]: #bowtie2-options-U
-
-U <r>
</td><td>
@@ -892,9 +831,7 @@ filehandle.
</td></tr><tr><td>
-[`-S`]: #bowtie2-options-S
-
- -S <hit>
+ -S <sam>
</td><td>
@@ -910,8 +847,6 @@ File to write SAM alignments to. By default, alignments are written to the
<table>
<tr><td id="bowtie2-options-q">
-[`-q`]: #bowtie2-options-q
-
-q
</td><td>
@@ -923,8 +858,6 @@ also: [`--solexa-quals`] and [`--int-quals`].
</td></tr>
<tr><td id="bowtie2-options-interleaved">
-[`--interleaved`]: #bowtie2-options-interleaved
-
--interleaved
</td><td>
@@ -935,36 +868,30 @@ represent a mate pair.
</td></tr>
<tr><td id="bowtie2-options-tab5">
-[`--tab5`]: #bowtie2-options-tab5
-
--tab5
</td><td>
Each read or pair is on a single line. An unpaired read line is
-[name]\t[seq]\t[qual]\n. A paired-end read line is
-[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n. An input file can be a
+`[name]\t[seq]\t[qual]\n`. A paired-end read line is
+`[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n`. An input file can be a
mix of unpaired and paired-end reads and Bowtie 2 recognizes each
according to the number of fields, handling each as it should.
</td></tr>
<tr><td id="bowtie2-options-tab6">
-[`--tab6`]: #bowtie2-options-tab6
-
--tab6
</td><td>
Similar to [`--tab5`] except, for paired-end reads, the second end can have a
different name from the first:
-[name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n
+`[name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n`
</td></tr>
<tr><td id="bowtie2-options-qseq">
-[`--qseq`]: #bowtie2-options-qseq
-
--qseq
</td><td>
@@ -975,35 +902,29 @@ end in `_qseq.txt`. See also: [`--solexa-quals`] and [`--int-quals`].
</td></tr>
<tr><td id="bowtie2-options-f">
-[`-f`]: #bowtie2-options-f
-
-f
</td><td>
-Reads (specified with `<m1>`, `<m2>`, `<s>`) are FASTA files. FASTA files
-usually have extension `.fa`, `.fasta`, `.mfa`, `.fna` or similar. FASTA files
+Reads (specified with `<m1>`, `<m2>`, `<s>`) are [`FASTA`] files. [`FASTA`] files
+usually have extension `.fa`, `.fasta`, `.mfa`, `.fna` or similar. [`FASTA`] files
do not have a way of specifying quality values, so when `-f` is set, the result
-is as if `--ignore-quals` is also set.
+is as if [`--ignore-quals`] is also set.
</td></tr>
<tr><td id="bowtie2-options-r">
-[`-r`]: #bowtie2-options-r
-
-r
</td><td>
Reads (specified with `<m1>`, `<m2>`, `<s>`) are files with one input sequence
per line, without any other information (no read names, no qualities). When
-`-r` is set, the result is as if `--ignore-quals` is also set.
+`-r` is set, the result is as if [`--ignore-quals`] is also set.
</td></tr>
<tr><td id="bowtie2-options-c">
-[`-c`]: #bowtie2-options-c
-
-c
</td><td>
@@ -1011,14 +932,11 @@ per line, without any other information (no read names, no qualities). When
The read sequences are given on command line. I.e. `<m1>`, `<m2>` and
`<singles>` are comma-separated lists of reads rather than lists of read files.
There is no way to specify read names or qualities, so `-c` also implies
-`--ignore-quals`.
+[`--ignore-quals`].
</td></tr>
<tr><td id="bowtie2-options-s">
-[`-s`/`--skip`]: #bowtie2-options-s
-[`-s`]: #bowtie2-options-s
-
-s/--skip <int>
</td><td>
@@ -1028,9 +946,6 @@ Skip (i.e. do not align) the first `<int>` reads or pairs in the input.
</td></tr>
<tr><td id="bowtie2-options-u">
-[`-u`/`--qupto`]: #bowtie2-options-u
-[`-u`]: #bowtie2-options-u
-
-u/--qupto <int>
</td><td>
@@ -1041,9 +956,6 @@ Align the first `<int>` reads or read pairs from the input (after the
</td></tr>
<tr><td id="bowtie2-options-5">
-[`-5`/`--trim5`]: #bowtie2-options-5
-[`-5`]: #bowtie2-options-5
-
-5/--trim5 <int>
</td><td>
@@ -1053,20 +965,14 @@ Trim `<int>` bases from 5' (left) end of each read before alignment (default: 0)
</td></tr>
<tr><td id="bowtie2-options-3">
-[`-3`/`--trim3`]: #bowtie2-options-3
-[`-3`]: #bowtie2-options-3
-
-3/--trim3 <int>
</td><td>
-Trim `<int>` bases from 3' (right) end of each read before alignment (default:
-0).
+Trim `<int>` bases from 3' (right) end of each read before alignment (default: 0).
</td></tr><tr><td id="bowtie2-options-phred33-quals">
-[`--phred33`]: #bowtie2-options-phred33-quals
-
--phred33
</td><td>
@@ -1075,17 +981,11 @@ Input qualities are ASCII chars equal to the [Phred quality] plus 33. This is
also called the "Phred+33" encoding, which is used by the very latest Illumina
pipelines.
-[Phred quality]: http://en.wikipedia.org/wiki/Phred_quality_score
-
</td></tr>
<tr><td id="bowtie2-options-phred64-quals">
-[`--phred64`]: #bowtie2-options-phred64-quals
-
--phred64
-[Phred quality]: http://en.wikipedia.org/wiki/Phred_quality_score
-
</td><td>
Input qualities are ASCII chars equal to the [Phred quality] plus 64. This is
@@ -1094,8 +994,6 @@ also called the "Phred+64" encoding.
</td></tr>
<tr><td id="bowtie2-options-solexa-quals">
-[`--solexa-quals`]: #bowtie2-options-solexa-quals
-
--solexa-quals
</td><td>
@@ -1107,8 +1005,6 @@ Pipeline versions (prior to 1.3). Default: off.
</td></tr>
<tr><td id="bowtie2-options-int-quals">
-[`--int-quals`]: #bowtie2-options-int-quals
-
--int-quals
</td><td>
@@ -1125,8 +1021,6 @@ integers, e.g., `40 40 30 40`..., rather than ASCII characters, e.g., `II?I`....
<table>
<tr><td id="bowtie2-options-very-fast">
-[`--very-fast`]: #bowtie2-options-very-fast
-
--very-fast
</td><td>
@@ -1136,8 +1030,6 @@ Same as: `-D 5 -R 1 -N 0 -L 22 -i S,0,2.50`
</td></tr>
<tr><td id="bowtie2-options-fast">
-[`--fast`]: #bowtie2-options-fast
-
--fast
</td><td>
@@ -1147,8 +1039,6 @@ Same as: `-D 10 -R 2 -N 0 -L 22 -i S,0,2.50`
</td></tr>
<tr><td id="bowtie2-options-sensitive">
-[`--sensitive`]: #bowtie2-options-sensitive
-
--sensitive
</td><td>
@@ -1158,8 +1048,6 @@ Same as: `-D 15 -R 2 -L 22 -i S,1,1.15` (default in [`--end-to-end`] mode)
</td></tr>
<tr><td id="bowtie2-options-very-sensitive">
-[`--very-sensitive`]: #bowtie2-options-very-sensitive
-
--very-sensitive
</td><td>
@@ -1174,8 +1062,6 @@ Same as: `-D 20 -R 3 -N 0 -L 20 -i S,1,0.50`
<table>
<tr><td id="bowtie2-options-very-fast-local">
-[`--very-fast-local`]: #bowtie2-options-very-fast-local
-
--very-fast-local
</td><td>
@@ -1185,8 +1071,6 @@ Same as: `-D 5 -R 1 -N 0 -L 25 -i S,1,2.00`
</td></tr>
<tr><td id="bowtie2-options-fast-local">
-[`--fast-local`]: #bowtie2-options-fast-local
-
--fast-local
</td><td>
@@ -1196,8 +1080,6 @@ Same as: `-D 10 -R 2 -N 0 -L 22 -i S,1,1.75`
</td></tr>
<tr><td id="bowtie2-options-sensitive-local">
-[`--sensitive-local`]: #bowtie2-options-sensitive-local
-
--sensitive-local
</td><td>
@@ -1207,8 +1089,6 @@ Same as: `-D 15 -R 2 -N 0 -L 20 -i S,1,0.75` (default in [`--local`] mode)
</td></tr>
<tr><td id="bowtie2-options-very-sensitive-local">
-[`--very-sensitive-local`]: #bowtie2-options-very-sensitive-local
-
--very-sensitive-local
</td><td>
@@ -1224,8 +1104,6 @@ Same as: `-D 20 -R 3 -N 0 -L 20 -i S,1,0.50`
<tr><td id="bowtie2-options-N">
-[`-N`]: #bowtie2-options-N
-
-N <int>
</td><td>
@@ -1234,14 +1112,9 @@ Sets the number of mismatches to allowed in a seed alignment during [multiseed
alignment]. Can be set to 0 or 1. Setting this higher makes alignment slower
(often much slower) but increases sensitivity. Default: 0.
-[multiseed alignment]: #multiseed-heuristic
-[yields a larger memory footprint]: #fm-index-memory-footprint
-
</td></tr>
<tr><td id="bowtie2-options-L">
-[`-L`]: #bowtie2-options-L
-
-L <int>
</td><td>
@@ -1251,13 +1124,9 @@ Smaller values make alignment slower but more sensitive. Default: the
[`--sensitive`] preset is used by default, which sets `-L` to 20 both in
[`--end-to-end`] mode and in [`--local`] mode.
-[multiseed alignment]: #multiseed-heuristic
-
</td></tr>
<tr><td id="bowtie2-options-i">
-[`-i`]: #bowtie2-options-i
-
-i <func>
</td><td>
@@ -1285,14 +1154,9 @@ See also: [setting function options]. If the function returns a result less than
default, which sets `-i` to `S,1,1.15` in [`--end-to-end`] mode to `-i S,1,0.75`
in [`--local`] mode.
-[setting function options]: #setting-function-options
-[multiseed alignment]: #multiseed-heuristic
-
</td></tr>
<tr><td id="bowtie2-options-n-ceil">
-[`--n-ceil`]: #bowtie2-options-n-ceil
-
--n-ceil <func>
</td><td>
@@ -1303,13 +1167,9 @@ specifying `-L,0,0.15` sets the N-ceiling function `f` to `f(x) = 0 + 0.15 * x`,
where x is the read length. See also: [setting function options]. Reads
exceeding this ceiling are [filtered out]. Default: `L,0,0.15`.
-[filtered out]: #filtering
-
</td></tr>
<tr><td id="bowtie2-options-dpad">
-[`--dpad`]: #bowtie2-options-dpad
-
--dpad <int>
</td><td>
@@ -1320,8 +1180,6 @@ gaps. Default: 15.
</td></tr>
<tr><td id="bowtie2-options-gbar">
-[`--gbar`]: #bowtie2-options-gbar
-
--gbar <int>
</td><td>
@@ -1332,8 +1190,6 @@ Default: 4.
</td></tr>
<tr><td id="bowtie2-options-ignore-quals">
-[`--ignore-quals`]: #bowtie2-options-ignore-quals
-
--ignore-quals
</td><td>
@@ -1347,8 +1203,6 @@ default behavior when the input doesn't specify quality values (e.g. in [`-f`],
</td></tr>
<tr><td id="bowtie2-options-nofw">
-[`--nofw`]: #bowtie2-options-nofw
-
--nofw/--norc
</td><td>
@@ -1364,8 +1218,6 @@ paired-end configurations corresponding to fragments from the reverse-complement
</td></tr>
<tr><td id="bowtie2-options-no-1mm-upfront">
-[`--no-1mm-upfront`]: #bowtie2-options-no-1mm-upfront
-
--no-1mm-upfront
</td><td>
@@ -1384,8 +1236,6 @@ expense of speed.
</td></tr><tr><td id="bowtie2-options-end-to-end">
-[`--end-to-end`]: #bowtie2-options-end-to-end
-
--end-to-end
</td><td>
@@ -1399,8 +1249,6 @@ This is mutually exclusive with [`--local`]. `--end-to-end` is the default mode
</td></tr>
<tr><td id="bowtie2-options-local">
-[`--local`]: #bowtie2-options-local
-
--local
</td><td>
@@ -1423,8 +1271,6 @@ exclusive with [`--end-to-end`]. `--end-to-end` is the default mode.
<tr><td id="bowtie2-options-ma">
-[`--ma`]: #bowtie2-options-ma
-
--ma <int>
</td><td>
@@ -1436,8 +1282,6 @@ and the characters match. Not used in [`--end-to-end`] mode. Default: 2.
</td></tr>
<tr><td id="bowtie2-options-mp">
-[`--mp`]: #bowtie2-options-mp
-
--mp MX,MN
</td><td>
@@ -1453,8 +1297,6 @@ where Q is the Phred quality value. Default: `MX` = 6, `MN` = 2.
</td></tr>
<tr><td id="bowtie2-options-np">
-[`--np`]: #bowtie2-options-np
-
--np <int>
</td><td>
@@ -1465,8 +1307,6 @@ ambiguous character such as `N`. Default: 1.
</td></tr>
<tr><td id="bowtie2-options-rdg">
-[`--rdg`]: #bowtie2-options-rdg
-
--rdg <int1>,<int2>
</td><td>
@@ -1477,8 +1317,6 @@ length N gets a penalty of `<int1>` + N * `<int2>`. Default: 5, 3.
</td></tr>
<tr><td id="bowtie2-options-rfg">
-[`--rfg`]: #bowtie2-options-rfg
-
--rfg <int1>,<int2>
</td><td>
@@ -1490,8 +1328,6 @@ reference gap of length N gets a penalty of `<int1>` + N * `<int2>`. Default:
</td></tr>
<tr><td id="bowtie2-options-score-min">
-[`--score-min`]: #bowtie2-options-score-min
-
--score-min <func>
</td><td>
@@ -1512,8 +1348,6 @@ the default in [`--local`] mode is `G,20,8`.
<tr><td id="bowtie2-options-k">
-[`-k`]: #bowtie2-options-k
-
-k <int>
</td><td>
@@ -1543,8 +1377,6 @@ aligning reads to long, repetitive genomes large `-k` can be very, very slow.
</td></tr>
<tr><td id="bowtie2-options-a">
-[`-a`]: #bowtie2-options-a
-
-a
</td><td>
@@ -1555,8 +1387,6 @@ is mutually exclusive with [`-k`].
Note: Bowtie 2 is not designed with `-a` mode in mind, and when
aligning reads to long, repetitive genomes this mode can be very, very slow.
-[reporting]: #reporting
-
</td></tr>
</table>
@@ -1566,8 +1396,6 @@ aligning reads to long, repetitive genomes this mode can be very, very slow.
<tr><td id="bowtie2-options-D">
-[`-D`]: #bowtie2-options-D
-
-D <int>
</td><td>
@@ -1580,8 +1408,6 @@ automatically adjusted up when -k or -a are specified. Default: 15.
</td></tr>
<tr><td id="bowtie2-options-R">
-[`-R`]: #bowtie2-options-R
-
-R <int>
</td><td>
@@ -1602,9 +1428,6 @@ least once is greater than 300. Default: 2.
<tr><td id="bowtie2-options-I">
-[`-I`/`--minins`]: #bowtie2-options-I
-[`-I`]: #bowtie2-options-I
-
-I/--minins <int>
</td><td>
@@ -1612,12 +1435,12 @@ least once is greater than 300. Default: 2.
The minimum fragment length for valid paired-end alignments. E.g. if `-I 60` is
specified and a paired-end alignment consists of two 20-bp alignments in the
appropriate orientation with a 20-bp gap between them, that alignment is
-considered valid (as long as [`-X`] is also satisfied). A 19-bp gap would not
+considered valid (as long as [`-X`][`+X`] is also satisfied). A 19-bp gap would not
be valid in that case. If trimming options [`-3`] or [`-5`] are also used, the
-[`-I`] constraint is applied with respect to the untrimmed mates.
+[`-I`][`+I`] constraint is applied with respect to the untrimmed mates.
-The larger the difference between [`-I`] and [`-X`], the slower Bowtie 2 will
-run. This is because larger differences between [`-I`] and [`-X`] require that
+The larger the difference between [`-I`][`+I`] and [`-X`][`+X`], the slower Bowtie 2 will
+run. This is because larger differences between [`-I`][`+I`] and [`-X`][`+X`] require that
Bowtie 2 scan a larger window to determine if a concordant alignment exists.
For typical fragment length ranges (200 to 400 nucleotides), Bowtie 2 is very
efficient.
@@ -1627,9 +1450,6 @@ Default: 0 (essentially imposing no minimum)
</td></tr>
<tr><td id="bowtie2-options-X">
-[`-X`/`--maxins`]: #bowtie2-options-X
-[`-X`]: #bowtie2-options-X
-
-X/--maxins <int>
</td><td>
@@ -1637,13 +1457,13 @@ Default: 0 (essentially imposing no minimum)
The maximum fragment length for valid paired-end alignments. E.g. if `-X 100`
is specified and a paired-end alignment consists of two 20-bp alignments in the
proper orientation with a 60-bp gap between them, that alignment is considered
-valid (as long as [`-I`] is also satisfied). A 61-bp gap would not be valid in
+valid (as long as [`-I`][`+I`] is also satisfied). A 61-bp gap would not be valid in
that case. If trimming options [`-3`] or [`-5`] are also used, the `-X`
constraint is applied with respect to the untrimmed mates, not the trimmed
mates.
-The larger the difference between [`-I`] and [`-X`], the slower Bowtie 2 will
-run. This is because larger differences between [`-I`] and [`-X`] require that
+The larger the difference between [`-I`][`+I`] and [`-X`][`+X`], the slower Bowtie 2 will
+run. This is because larger differences between [`-I`][`+I`] and [`-X`][`+X`] require that
Bowtie 2 scan a larger window to determine if a concordant alignment exists.
For typical fragment length ranges (200 to 400 nucleotides), Bowtie 2 is very
efficient.
@@ -1653,11 +1473,6 @@ Default: 500.
</td></tr>
<tr><td id="bowtie2-options-fr">
-[`--fr`/`--rf`/`--ff`]: #bowtie2-options-fr
-[`--fr`]: #bowtie2-options-fr
-[`--rf`]: #bowtie2-options-fr
-[`--ff`]: #bowtie2-options-fr
-
--fr/--rf/--ff
</td><td>
@@ -1665,7 +1480,7 @@ Default: 500.
The upstream/downstream mate orientations for a valid paired-end alignment
against the forward reference strand. E.g., if `--fr` is specified and there is
a candidate paired-end alignment where mate 1 appears upstream of the reverse
-complement of mate 2 and the fragment length constraints ([`-I`] and [`-X`]) are
+complement of mate 2 and the fragment length constraints ([`-I`][`+I`] and [`-X`][`+X`]) are
met, that alignment is valid. Also, if mate 2 appears upstream of the reverse
complement of mate 1 and all other constraints are met, that too is valid.
`--rf` likewise requires that an upstream mate1 be reverse-complemented and a
@@ -1676,8 +1491,6 @@ for Illumina's Paired-end Sequencing Assay).
</td></tr>
<tr><td id="bowtie2-options-no-mixed">
-[`--no-mixed`]: #bowtie2-options-no-mixed
-
--no-mixed
</td><td>
@@ -1689,8 +1502,6 @@ disables that behavior.
</td></tr>
<tr><td id="bowtie2-options-no-discordant">
-[`--no-discordant`]: #bowtie2-options-no-discordant
-
--no-discordant
</td><td>
@@ -1698,13 +1509,11 @@ disables that behavior.
By default, `bowtie2` looks for discordant alignments if it cannot find any
concordant alignments. A discordant alignment is an alignment where both mates
align uniquely, but that does not satisfy the paired-end constraints
-([`--fr`/`--rf`/`--ff`], [`-I`], [`-X`]). This option disables that behavior.
+([`--fr`/`--rf`/`--ff`], [`-I`][`+I`], [`-X`][`+X`]). This option disables that behavior.
</td></tr>
<tr><td id="bowtie2-options-dovetail">
-[`--dovetail`]: #bowtie2-options-dovetail
-
--dovetail
</td><td>
@@ -1714,13 +1523,9 @@ beginning of the other such that the wrong mate begins upstream, consider that
to be concordant. See also: [Mates can overlap, contain or dovetail each
other]. Default: mates cannot dovetail in a concordant alignment.
-[Mates can overlap, contain or dovetail each other]: #mates-can-overlap-contain-or-dovetail-each-other
-
</td></tr>
<tr><td id="bowtie2-options-no-contain">
-[`--no-contain`]: #bowtie2-options-no-contain
-
--no-contain
</td><td>
@@ -1732,8 +1537,6 @@ can contain the other in a concordant alignment.
</td></tr>
<tr><td id="bowtie2-options-no-overlap">
-[`--no-overlap`]: #bowtie2-options-no-overlap
-
--no-overlap
</td><td>
@@ -1750,9 +1553,6 @@ Default: mates can overlap in a concordant alignment.
<tr><td id="bowtie2-options-t">
-[`-t`/`--time`]: #bowtie2-options-t
-[`-t`]: #bowtie2-options-t
-
-t/--time
</td><td>
@@ -1763,11 +1563,6 @@ This is printed to the "standard error" ("stderr") filehandle. Default: off.
</td></tr>
<tr><td id="bowtie2-options-un">
-[`--un`]: #bowtie2-options-un
-[`--un-gz`]: #bowtie2-options-un
-[`--un-bz2`]: #bowtie2-options-un
-[`--un-lz4`]: #bowtie2-options-un
-
--un <path>
--un-gz <path>
--un-bz2 <path>
@@ -1787,11 +1582,6 @@ order as they did in the input.
</td></tr>
<tr><td id="bowtie2-options-al">
-[`--al`]: #bowtie2-options-al
-[`--al-gz`]: #bowtie2-options-al
-[`--al-bz2`]: #bowtie2-options-al
-[`--al-lz4`]: #bowtie2-options-al
-
--al <path>
--al-gz <path>
--al-bz2 <path>
@@ -1811,11 +1601,6 @@ not necessarily appear in the same order as they did in the input.
</td></tr>
<tr><td id="bowtie2-options-un-conc">
-[`--un-conc`]: #bowtie2-options-un-conc
-[`--un-conc-gz`]: #bowtie2-options-un-conc
-[`--un-conc-bz2`]: #bowtie2-options-un-conc
-[`--un-conc-lz4`]: #bowtie2-options-un-conc
-
--un-conc <path>
--un-conc-gz <path>
--un-conc-bz2 <path>
@@ -1838,11 +1623,6 @@ the same order as they did in the inputs.
</td></tr>
<tr><td id="bowtie2-options-al-conc">
-[`--al-conc`]: #bowtie2-options-al-conc
-[`--al-conc-gz`]: #bowtie2-options-al-conc
-[`--al-conc-bz2`]: #bowtie2-options-al-conc
-[`--al-conc-lz4`]: #bowtie2-options-al-conc
-
--al-conc <path>
--al-conc-gz <path>
--al-conc-bz2 <path>
@@ -1865,8 +1645,6 @@ in the same order as they did in the inputs.
</td></tr>
<tr><td id="bowtie2-options-quiet">
-[`--quiet`]: #bowtie2-options-quiet
-
--quiet
</td><td>
@@ -1876,8 +1654,6 @@ Print nothing besides alignments and serious errors.
</td></tr>
<tr><td id="bowtie2-options-met-file">
-[`--met-file`]: #bowtie2-options-met-file
-
--met-file <path>
</td><td>
@@ -1889,8 +1665,6 @@ for debugging certain problems, especially performance issues. See also:
</td></tr>
<tr><td id="bowtie2-options-met-stderr">
-[`--met-stderr`]: #bowtie2-options-met-stderr
-
--met-stderr <path>
</td><td>
@@ -1903,8 +1677,6 @@ useful for debugging certain problems, especially performance issues. See also:
</td></tr>
<tr><td id="bowtie2-options-met">
-[`--met`]: #bowtie2-options-met
-
--met <int>
</td><td>
@@ -1921,8 +1693,6 @@ either [`--met-stderr`] or [`--met-file`] are specified. Default: 1.
<tr><td id="bowtie2-options-no-unal">
-[`--no-unal`]: #bowtie2-options-no-unal
-
--no-unal
</td><td>
@@ -1932,8 +1702,6 @@ Suppress SAM records for reads that failed to align.
</td></tr>
<tr><td id="bowtie2-options-no-hd">
-[`--no-hd`]: #bowtie2-options-no-hd
-
--no-hd
</td><td>
@@ -1943,8 +1711,6 @@ Suppress SAM header lines (starting with `@`).
</td></tr>
<tr><td id="bowtie2-options-no-sq">
-[`--no-sq`]: #bowtie2-options-no-sq
-
--no-sq
</td><td>
@@ -1954,8 +1720,6 @@ Suppress `@SQ` SAM header lines.
</td></tr>
<tr><td id="bowtie2-options-rg-id">
-[`--rg-id`]: #bowtie2-options-rg-id
-
--rg-id <text>
</td><td>
@@ -1968,8 +1732,6 @@ value set to `<text>`.
</td></tr>
<tr><td id="bowtie2-options-rg">
-[`--rg`]: #bowtie2-options-rg
-
--rg <text>
</td><td>
@@ -1980,12 +1742,9 @@ must also be specified. This is because the `ID` tag is required by the [SAM
Spec][SAM]. Specify `--rg` multiple times to set multiple fields. See the
[SAM Spec][SAM] for details about what fields are legal.
-
</td></tr>
<tr><td id="bowtie2-options-omit-sec-seq">
-[`--omit-sec-seq`]: #bowtie2-options-omit-sec-seq
-
--omit-sec-seq
</td><td>
@@ -1995,8 +1754,34 @@ and `QUAL` strings. Specifying this option causes Bowtie 2 to print an asterisk
in those fields instead.
</td></tr>
+<tr><td id="bowtie2-options-soft-clipped-unmapped-tlen-sec-seq">
+
+ --soft-clipped-unmapped-tlen
+
+</td><td>
+
+Consider soft-clipped bases unmapped when calculating `TLEN`.
+
+</td></tr>
+<tr><td id="bowtie2-options-sam-no-qname-trunc">
+
+ --sam-no-qname-trunc
+
+</td><td>
+
+Suppress standard behavior of truncating readname at first whitespace
+at the expense of generating non-standard SAM
+
+</td></tr>
+<tr><td id="bowtie2-options-xeq">
+ --xeq
+</td><td>
+
+Use `'='/'X'`, instead of `'M'`, to specify matches/mismatches in SAM record
+
+</td></tr>
</table>
#### Performance options
@@ -2005,10 +1790,6 @@ in those fields instead.
<td id="bowtie2-options-o">
-[`-o`/`--offrate`]: #bowtie2-options-o
-[`-o`]: #bowtie2-options-o
-[`--offrate`]: #bowtie2-options-o
-
-o/--offrate <int>
</td><td>
@@ -2023,9 +1804,6 @@ index.
</td></tr>
<tr><td id="bowtie2-options-p">
-[`-p`/`--threads`]: #bowtie2-options-p
-[`-p`]: #bowtie2-options-p
-
-p/--threads NTHREADS
</td><td>
@@ -2042,8 +1820,6 @@ not specified at build time).
</td></tr>
<tr><td id="bowtie2-options-reorder">
-[`--reorder`]: #bowtie2-options-reorder
-
--reorder
</td><td>
@@ -2058,8 +1834,6 @@ naturally correspond to input order in that case.
</td></tr>
<tr><td id="bowtie2-options-mm">
-[`--mm`]: #bowtie2-options-mm
-
--mm
</td><td>
@@ -2077,8 +1851,6 @@ situations where using [`-p`] is not possible or not preferable.
<table>
<tr><td id="bowtie2-options-qc-filter">
-[`--qc-filter`]: #bowtie2-options-qc-filter
-
--qc-filter
</td><td>
@@ -2089,8 +1861,6 @@ effect when read format is [`--qseq`]. Default: off.
</td></tr>
<tr><td id="bowtie2-options-seed">
-[`--seed`]: #bowtie2-options-seed
-
--seed <int>
</td><td>
@@ -2100,8 +1870,6 @@ Use `<int>` as the seed for pseudo-random number generator. Default: 0.
</td></tr>
<tr><td id="bowtie2-options-non-deterministic">
-[`--non-deterministic`]: #bowtie2-options-non-deterministic
-
--non-deterministic
</td><td>
@@ -2121,8 +1889,6 @@ identical reads.
</td></tr>
<tr><td id="bowtie2-options-version">
-[`--version`]: #bowtie2-options-version
-
--version
</td><td>
@@ -2269,133 +2035,120 @@ scale and the encoding is ASCII-offset by 33 (ASCII char `!`), similarly to a
of these optional fields for each alignment, depending on the type of the
alignment:
-<table>
-<tr><td id="bowtie2-build-opt-fields-as">
-[`AS:i`]: #bowtie2-build-opt-fields-as
+<table><tr><td id="bowtie2-build-opt-fields-as">
- AS:i:<N>
+ AS:i:<N>
-</td>
-<td>
- Alignment score. Can be negative. Can be greater than 0 in [`--local`]
- mode (but not in [`--end-to-end`] mode). Only present if SAM record is for
- an aligned read.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-xs">
-[`XS:i`]: #bowtie2-build-opt-fields-xs
+</td><td>
- XS:i:<N>
+Alignment score. Can be negative. Can be greater than 0 in [`--local`]
+mode (but not in [`--end-to-end`] mode). Only present if SAM record is for
+an aligned read.
-</td>
-<td>
- Alignment score for the best-scoring alignment found other than the
- alignment reported. Can be negative. Can be greater than 0 in [`--local`]
- mode (but not in [`--end-to-end`] mode). Only present if the SAM record is
- for an aligned read and more than one alignment was found for the read.
- Note that, when the read is part of a concordantly-aligned pair, this score
- could be greater than [`AS:i`].
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-ys">
-[`YS:i`]: #bowtie2-build-opt-fields-ys
+</td></tr><tr><td id="bowtie2-build-opt-fields-xs">
- YS:i:<N>
+ XS:i:<N>
-</td>
-<td>
- Alignment score for opposite mate in the paired-end alignment. Only present
- if the SAM record is for a read that aligned as part of a paired-end
- alignment.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-xn">
-[`XN:i`]: #bowtie2-build-opt-fields-xn
+</td><td>
- XN:i:<N>
+Alignment score for the best-scoring alignment found other than the
+alignment reported. Can be negative. Can be greater than 0 in [`--local`]
+mode (but not in [`--end-to-end`] mode). Only present if the SAM record is
+for an aligned read and more than one alignment was found for the read.
+Note that, when the read is part of a concordantly-aligned pair, this score
+could be greater than [`AS:i`].
-</td>
-<td>
- The number of ambiguous bases in the reference covering this alignment.
- Only present if SAM record is for an aligned read.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-xm">
-[`XM:i`]: #bowtie2-build-opt-fields-xm
+</td></tr><tr><td id="bowtie2-build-opt-fields-ys">
- XM:i:<N>
+ YS:i:<N>
-</td>
-<td>
- The number of mismatches in the alignment. Only present if SAM record is
- for an aligned read.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-xo">
-[`XO:i`]: #bowtie2-build-opt-fields-xo
+</td><td>
- XO:i:<N>
+Alignment score for opposite mate in the paired-end alignment. Only present
+if the SAM record is for a read that aligned as part of a paired-end
+alignment.
-</td>
-<td>
- The number of gap opens, for both read and reference gaps, in the alignment.
- Only present if SAM record is for an aligned read.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-xg">
-[`XG:i`]: #bowtie2-build-opt-fields-xg
+</td></tr><tr><td id="bowtie2-build-opt-fields-xn">
- XG:i:<N>
+ XN:i:<N>
-</td>
-<td>
- The number of gap extensions, for both read and reference gaps, in the
- alignment. Only present if SAM record is for an aligned read.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-nm">
-[`NM:i`]: #bowtie2-build-opt-fields-nm
+</td><td>
- NM:i:<N>
+The number of ambiguous bases in the reference covering this alignment.
+Only present if SAM record is for an aligned read.
-</td>
-<td>
- The edit distance; that is, the minimal number of one-nucleotide edits
- (substitutions, insertions and deletions) needed to transform the read
- string into the reference string. Only present if SAM record is for an
- aligned read.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-yf">
-[`YF:Z`]: #bowtie2-build-opt-fields-yf
+</td></tr><tr><td id="bowtie2-build-opt-fields-xm">
- YF:Z:<S>
+ XM:i:<N>
</td><td>
- String indicating reason why the read was filtered out. See also:
- [Filtering]. Only appears for reads that were filtered out.
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-yt">
-[`YT:Z`]: #bowtie2-build-opt-fields-yt
- YT:Z:<S>
+The number of mismatches in the alignment. Only present if SAM record is
+for an aligned read.
+
+</td></tr><tr><td id="bowtie2-build-opt-fields-xo">
+
+ XO:i:<N>
</td><td>
- Value of `UU` indicates the read was not part of a pair. Value of `CP`
- indicates the read was part of a pair and the pair aligned concordantly.
- Value of `DP` indicates the read was part of a pair and the pair aligned
- discordantly. Value of `UP` indicates the read was part of a pair but the
- pair failed to aligned either concordantly or discordantly.
-[Filtering]: #filtering
-</td></tr>
-<tr><td id="bowtie2-build-opt-fields-md">
-[`MD:Z`]: #bowtie2-build-opt-fields-md
- MD:Z:<S>
+The number of gap opens, for both read and reference gaps, in the alignment.
+Only present if SAM record is for an aligned read.
+
+</td></tr><tr><td id="bowtie2-build-opt-fields-xg">
+
+ XG:i:<N>
+
+</td><td>
+
+The number of gap extensions, for both read and reference gaps, in the
+alignment. Only present if SAM record is for an aligned read.
+
+</td></tr><tr><td id="bowtie2-build-opt-fields-nm">
+
+ NM:i:<N>
+
+</td><td>
+
+The edit distance; that is, the minimal number of one-nucleotide edits
+(substitutions, insertions and deletions) needed to transform the read
+string into the reference string. Only present if SAM record is for an
+aligned read.
+
+</td></tr><tr><td id="bowtie2-build-opt-fields-yf">
+
+ YF:Z:<S>
</td><td>
- A string representation of the mismatched reference bases in the alignment.
- See [SAM] format specification for details. Only present if SAM record is
- for an aligned read.
+
+String indicating reason why the read was filtered out. See also:
+[Filtering]. Only appears for reads that were filtered out.
+
+</td></tr><tr><td id="bowtie2-build-opt-fields-yt">
+
+ YT:Z:<S>
+
+</td><td>
+
+Value of `UU` indicates the read was not part of a pair. Value of `CP`
+indicates the read was part of a pair and the pair aligned concordantly.
+Value of `DP` indicates the read was part of a pair and the pair aligned
+discordantly. Value of `UP` indicates the read was part of a pair but the
+pair failed to aligned either concordantly or discordantly.
+
+</td></tr><tr><td id="bowtie2-build-opt-fields-md">
+
+ MD:Z:<S>
+
+</td><td>
+
+A string representation of the mismatched reference bases in the alignment.
+See [SAM Tags format specification][SAMTags] for details. Only present if SAM record is
+for an aligned read.
+
</td></tr>
</table>
-[SAM format specification]: http://samtools.sf.net/SAM1.pdf
-[FASTQ]: http://en.wikipedia.org/wiki/FASTQ_format
-[`-S`/`--sam`]: #bowtie2-options-S
-[`-m`]: #bowtie2-options-m
The `bowtie2-build` indexer
===========================
@@ -2405,7 +2158,7 @@ The `bowtie2-build` indexer
`.3.bt2`, `.4.bt2`, `.rev.1.bt2`, and `.rev.2.bt2`. In the case of a large
index these suffixes will have a `bt2l` termination. These files together
constitute the index: they are all that is needed to align reads to that
-reference. The original sequence FASTA files are no longer used by Bowtie 2
+reference. The original sequence [`FASTA`] files are no longer used by Bowtie 2
once the index is built.
Bowtie 2's `.bt2` index format is different from Bowtie 1's `.ebwt` format, and
@@ -2421,7 +2174,7 @@ behavior can be disabled using the [`-a`/`--noauto`] option.
The indexer provides options pertaining to the "shape" of the index, e.g.
[`--offrate`](#bowtie2-build-options-o) governs the fraction of [Burrows-Wheeler]
rows that are "marked" (i.e., the density of the suffix-array sample; see the
-original [FM Index] paper for details). All of these options are potentially
+original [FM Index][FM Index Paper] paper for details). All of these options are potentially
profitable trade-offs depending on the application. They have been set to
defaults that are reasonable for most cases according to our experiments. See
[Performance tuning] for details.
@@ -2432,15 +2185,10 @@ does not exceed 4 billion characters but a large index is preferred, the user
can specify [`--large-index`] to force `bowtie2-build` to build a large index
instead.
-The Bowtie 2 index is based on the [FM Index] of Ferragina and Manzini, which in
+The Bowtie 2 index is based on the [FM Index][FM Index Paper] of Ferragina and Manzini, which in
turn is based on the [Burrows-Wheeler] transform. The algorithm used to build
the index is based on the [blockwise algorithm] of Karkkainen.
-[Blockwise algorithm]: http://portal.acm.org/citation.cfm?id=1314852
-[blockwise algorithm]: http://portal.acm.org/citation.cfm?id=1314852
-[FM Index]: http://portal.acm.org/citation.cfm?id=796543
-[Burrows-Wheeler]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
-[Performance tuning]: #performance-tuning
Command Line
------------
@@ -2457,7 +2205,7 @@ Usage:
</td><td>
-A comma-separated list of FASTA files containing the reference sequences to be
+A comma-separated list of [`FASTA`] files containing the reference sequences to be
aligned to, or, if [`-c`](#bowtie2-build-options-c) is specified, the sequences
themselves. E.g., `<reference_in>` might be `chr1.fa,chr2.fa,chrX.fa,chrY.fa`,
or, if [`-c`](#bowtie2-build-options-c) is specified, this might be
@@ -2483,7 +2231,7 @@ files named `NAME.1.bt2`, `NAME.2.bt2`, `NAME.3.bt2`, `NAME.4.bt2`,
</td><td>
-The reference input files (specified as `<reference_in>`) are FASTA files
+The reference input files (specified as `<reference_in>`) are [`FASTA`] files
(usually having extension `.fa`, `.mfa`, `.fna` or similar).
</td></tr><tr><td id="bowtie2-build-options-c">
@@ -2493,13 +2241,10 @@ The reference input files (specified as `<reference_in>`) are FASTA files
</td><td>
The reference sequences are given on the command line. I.e. `<reference_in>` is
-a comma-separated list of sequences rather than a list of FASTA files.
+a comma-separated list of sequences rather than a list of [`FASTA`] files.
-</td></tr>
</td></tr><tr><td id="bowtie2-build-options-large-index">
-[`--large-index`]: #bowtie2-build-options-large-index
-
--large-index
</td><td>
@@ -2510,8 +2255,6 @@ than ~ 4 billion nucleotides inlong.
</td></tr>
<tr><td id="bowtie2-build-options-a">
-[`-a`/`--noauto`]: #bowtie2-build-options-a
-
-a/--noauto
</td><td>
@@ -2524,9 +2267,6 @@ to the user to try new parameters.
</td></tr><tr><td id="bowtie2-build-options-p">
-[`--packed`]: #bowtie2-build-options-p
-[`-p`/`--packed`]: #bowtie2-build-options-p
-
-p/--packed
</td><td>
@@ -2537,8 +2277,6 @@ automatically by default; use [`-a`/`--noauto`] to configure manually.
</td></tr><tr><td id="bowtie2-build-options-bmax">
-[`--bmax`]: #bowtie2-build-options-bmax
-
--bmax <int>
</td><td>
@@ -2551,8 +2289,6 @@ configured automatically by default; use [`-a`/`--noauto`] to configure manually
</td></tr><tr><td id="bowtie2-build-options-bmaxdivn">
-[`--bmaxdivn`]: #bowtie2-build-options-bmaxdivn
-
--bmaxdivn <int>
</td><td>
@@ -2564,8 +2300,6 @@ configured automatically by default; use [`-a`/`--noauto`] to configure manually
</td></tr><tr><td id="bowtie2-build-options-dcv">
-[`--dcv`]: #bowtie2-build-options-dcv
-
--dcv <int>
</td><td>
@@ -2578,8 +2312,6 @@ manually.
</td></tr><tr><td id="bowtie2-build-options-nodc">
-[`--nodc`]: #bowtie2-build-options-nodc
-
--nodc
</td><td>
@@ -2660,7 +2392,6 @@ sequences) and ignore the rest.
`bowtie2-build` is verbose by default. With this option `bowtie2-build` will
print only error messages.
-
</td></tr><tr><td>
--threads <int>
@@ -2693,7 +2424,7 @@ The `bowtie2-inspect` index inspector
`bowtie2-inspect` extracts information from a Bowtie index about what kind of
index it is and what reference sequences were used to build it. When run without
-any options, the tool will output a FASTA file containing the sequences of the
+any options, the tool will output a [`FASTA`] file containing the sequences of the
original references (with all non-`A`/`C`/`G`/`T` characters converted to `N`s).
It can also be used to extract just the reference sequence names using the
[`-n`/`--names`] option or a more verbose summary using the [`-s`/`--summary`]
@@ -2729,13 +2460,11 @@ in the directory specified in the `BOWTIE2_INDEXES` environment variable.
</td><td>
-When printing FASTA output, output a newline character every `<int>` bases
+When printing [`FASTA`] output, output a newline character every `<int>` bases
(default: 60).
</td></tr><tr><td id="bowtie2-inspect-options-n">
-[`-n`/`--names`]: #bowtie2-inspect-options-n
-
-n/--names
</td><td>
@@ -2744,8 +2473,6 @@ Print reference sequence names, one per line, and quit.
</td></tr><tr><td id="bowtie2-inspect-options-s">
-[`-s`/`--summary`]: #bowtie2-inspect-options-s
-
-s/--summary
</td><td>
@@ -2804,8 +2531,6 @@ environment variable to point to the new Bowtie 2 directory containing the
as the `BT2_HOME` variable is used in the commands below to refer to that
directory.
-[Lambda phage]: http://en.wikipedia.org/wiki/Lambda_phage
-[obtain Bowtie 2]: #obtaining-bowtie-2
Indexing a reference genome
---------------------------
@@ -2821,19 +2546,14 @@ completes, the current directory will contain four new files that all start with
`lambda_virus` and end with `.1.bt2`, `.2.bt2`, `.3.bt2`, `.4.bt2`,
`.rev.1.bt2`, and `.rev.2.bt2`. These files constitute the index - you're done!
-You can use `bowtie2-build` to create an index for a set of FASTA files obtained
+You can use `bowtie2-build` to create an index for a set of [`FASTA`] files obtained
from any source, including sites such as [UCSC], [NCBI], and [Ensembl]. When
-indexing multiple FASTA files, specify all the files using commas to separate
+indexing multiple [`FASTA`] files, specify all the files using commas to separate
file names. For more details on how to create an index with `bowtie2-build`,
see the [manual section on index building]. You may also want to bypass this
process by obtaining a pre-built index. See [using a pre-built index] below
for an example.
-[UCSC]: http://genome.ucsc.edu/cgi-bin/hgGateway
-[NCBI]: http://www.ncbi.nlm.nih.gov/sites/genome
-[Ensembl]: http://www.ensembl.org/
-[manual section on index building]: #the-bowtie2-build-indexer
-[using a pre-built index]: #using-a-pre-built-index
Aligning example reads
----------------------
@@ -2872,7 +2592,6 @@ the lines are SAM alignments, one line per read or mate. See the [Bowtie 2
manual section on SAM output] and the [SAM specification] for details about how
to interpret the SAM file format.
-[Bowtie 2 manual section on SAM output]: #sam-output
Paired-end example
------------------
@@ -2896,7 +2615,6 @@ in the same directory and run:
This aligns the long reads to the reference genome using local alignment, with
results written to the file `eg3.sam`.
-[local alignment]: #end-to-end-alignment-versus-local-alignment
Using SAMtools/BCFtools downstream
----------------------------------
@@ -2936,7 +2654,208 @@ Then to view the variants, run:
See the official SAMtools guide to [Calling SNPs/INDELs with SAMtools/BCFtools]
for more details and variations on this process.
-[SAMtools]: http://samtools.sourceforge.net/
-[BCFtools]: http://samtools.sourceforge.net/mpileup.shtml
-[PATH environment variable]: http://en.wikipedia.org/wiki/PATH_(variable)
-[Calling SNPs/INDELs with SAMtools/BCFtools]: http://samtools.sourceforge.net/mpileup.shtml
+[BCFtools]: http://samtools.sourceforge.net/mpileup.shtml
+[BLAST]: http://blast.ncbi.nlm.nih.gov/Blast.cgi
+[BLAT]: http://genome.ucsc.edu/cgi-bin/hgBlat?command=start
+[BWA]: http://bio-bwa.sourceforge.net/
+[BWT]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
+[Blockwise algorithm]: http://portal.acm.org/citation.cfm?id=1314852
+[Bowtie 1 does]: http://genomebiology.com/2009/10/3/R25
+[Bowtie 1 paper]: http://genomebiology.com/2009/10/3/R25
+[Bowtie 1]: http://bowtie-bio.sf.net
+[Bowtie 2 manual section on SAM output]: #sam-output
+[Bowtie 2]: http://bowtie-bio.sf.net/bowtie2
+[Bowtie paper]: http://genomebiology.com/2009/10/3/R25
+[Bowtie]: http://bowtie-bio.sf.net
+[Burrows-Wheeler Transform]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
+[Burrows-Wheeler]: http://en.wikipedia.org/wiki/Burrows-Wheeler_transform
+[Calling SNPs/INDELs with SAMtools/BCFtools]: http://samtools.sourceforge.net/mpileup.shtml
+[Crossbow]: http://bowtie-bio.sf.net/crossbow
+[Cufflinks]: http://cufflinks.cbcb.umd.edu/
+[Download]: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/
+[Ensembl]: http://www.ensembl.org/
+[`FASTA`]: https://en.wikipedia.org/wiki/FASTA
+[FASTQ]: http://en.wikipedia.org/wiki/FASTQ_format
+[FM Index Paper]: http://portal.acm.org/citation.cfm?id=796543
+[FM Index Wiki]: http://en.wikipedia.org/wiki/FM-index
+[Filtering]: #filtering
+[GATK]: http://www.broadinstitute.org/gsa/wiki/index.php/The_Genome_Analysis_Toolkit
+[GPLv3 license]: http://www.gnu.org/licenses/gpl-3.0.html
+[GnuWin32]: http://gnuwin32.sf.net/packages/coreutils.htm
+[IUPAC nucleotide codes]: http://www.bioinformatics.org/sms/iupac.html
+[Lambda phage]: http://en.wikipedia.org/wiki/Lambda_phage
+[MSYS]: http://www.mingw.org/wiki/msys
+[MUMmer]: http://mummer.sourceforge.net/
+[Mates can overlap, contain or dovetail each other]: #mates-can-overlap-contain-or-dovetail-each-other
+[MinGW]: http://www.mingw.org/
+[Myrna]: http://bowtie-bio.sf.net/myrna
+[NCBI]: http://www.ncbi.nlm.nih.gov/sites/genome
+[NUCmer]: http://mummer.sourceforge.net/manual/#nucmer
+[Needleman-Wunsch]: http://en.wikipedia.org/wiki/Needleman-Wunsch_algorithm
+[PATH environment variable]: http://en.wikipedia.org/wiki/PATH_(variable)
+[PATH]: http://en.wikipedia.org/wiki/PATH_(variable)
+[Performance tuning]: #performance-tuning
+[Phred quality]: http://en.wikipedia.org/wiki/Phred_quality_score
+[SAM format specification]: http://samtools.sf.net/SAM1.pdf
+[SAM specification]: http://samtools.sourceforge.net/SAM1.pdf
+[SAMTags]: https://samtools.github.io/hts-specs/SAMtags.pdf
+[SAM]: http://samtools.sourceforge.net/SAM1.pdf
+[SAMtools]: http://samtools.sourceforge.net
+[Smith-Waterman]: http://en.wikipedia.org/wiki/Smith_waterman
+[Threading Building Blocks library]: https://www.threadingbuildingblocks.org
+[TopHat]: http://tophat.cbcb.umd.edu/
+[UCSC]: http://genome.ucsc.edu/cgi-bin/hgGateway
+[Vmatch]: http://www.vmatch.de/
+[Xcode]: http://developer.apple.com/xcode/
+[`+I`/`--minins`]: #bowtie2-options-I
+[`+I`]: #bowtie2-options-I
+[`+R`]: #bowtie2-options-R
+[`+S`/`--sam`]: #bowtie2-options-S
+[`+S`]: #bowtie2-options-S
+[`+U`]: #bowtie2-options-U
+[`+X`/`--maxins`]: #bowtie2-options-X
+[`+X`]: #bowtie2-options-X
+[`--al-bz2`]: #bowtie2-options-al
+[`--al-conc-bz2`]: #bowtie2-options-al-conc
+[`--al-conc-gz`]: #bowtie2-options-al-conc
+[`--al-conc-lz4`]: #bowtie2-options-al-conc
+[`--al-conc`]: #bowtie2-options-al-conc
+[`--al-gz`]: #bowtie2-options-al
+[`--al-lz4`]: #bowtie2-options-al
+[`--al`]: #bowtie2-options-al
+[`--bmax`]: #bowtie2-build-options-bmax
+[`--bmaxdivn`]: #bowtie2-build-options-bmaxdivn
+[`--dcv`]: #bowtie2-build-options-dcv
+[`--dovetail`]: #bowtie2-options-dovetail
+[`--dpad`]: #bowtie2-options-dpad
+[`--end-to-end`]: #bowtie2-options-end-to-end
+[`--fast-local`]: #bowtie2-options-fast-local
+[`--fast`]: #bowtie2-options-fast
+[`--ff`]: #bowtie2-options-fr
+[`--fr`/`--rf`/`--ff`]: #bowtie2-options-fr
+[`--fr`]: #bowtie2-options-fr
+[`--gbar`]: #bowtie2-options-gbar
+[`--ignore-quals`]: #bowtie2-options-ignore-quals
+[`--int-quals`]: #bowtie2-options-int-quals
+[`--interleaved`]: #bowtie2-options-interleaved
+[`--large-index`]: #bowtie2-build-options-large-index
+[`--local`]: #bowtie2-options-local
+[`--ma`]: #bowtie2-options-ma
+[`--met-file`]: #bowtie2-options-met-file
+[`--met-stderr`]: #bowtie2-options-met-stderr
+[`--met`]: #bowtie2-options-met
+[`--mm`]: #bowtie2-options-mm
+[`--mp`]: #bowtie2-options-mp
+[`--n-ceil`]: #bowtie2-options-n-ceil
+[`--no-1mm-upfront`]: #bowtie2-options-no-1mm-upfront
+[`--no-contain`]: #bowtie2-options-no-contain
+[`--no-discordant`]: #bowtie2-options-no-discordant
+[`--no-hd`]: #bowtie2-options-no-hd
+[`--no-mixed`]: #bowtie2-options-no-mixed
+[`--no-overlap`]: #bowtie2-options-no-overlap
+[`--no-sq`]: #bowtie2-options-no-sq
+[`--no-unal`]: #bowtie2-options-no-unal
+[`--nodc`]: #bowtie2-build-options-nodc
+[`--nofw`]: #bowtie2-options-nofw
+[`--non-deterministic`]: #bowtie2-options-non-deterministic
+[`--np`]: #bowtie2-options-np
+[`--offrate`]: #bowtie2-options-o
+[`--omit-sec-seq`]: #bowtie2-options-omit-sec-seq
+[`--packed`]: #bowtie2-build-options-p
+[`--phred33`]: #bowtie2-options-phred33-quals
+[`--phred64`]: #bowtie2-options-phred64-quals
+[`--qc-filter`]: #bowtie2-options-qc-filter
+[`--qseq`]: #bowtie2-options-qseq
+[`--quiet`]: #bowtie2-options-quiet
+[`--rdg`]: #bowtie2-options-rdg
+[`--reorder`]: #bowtie2-options-reorder
+[`--rf`]: #bowtie2-options-fr
+[`--rfg`]: #bowtie2-options-rfg
+[`--rg-id`]: #bowtie2-options-rg-id
+[`--rg`]: #bowtie2-options-rg
+[`--score-min`]: #bowtie2-options-score-min
+[`--seed`]: #bowtie2-options-seed
+[`--sensitive-local`]: #bowtie2-options-sensitive-local
+[`--sensitive`]: #bowtie2-options-sensitive
+[`--soft-clipped-unmapped-tlen`]: #bowtie2-options-soft-clipped-unmapped-tlen
+[`--solexa-quals`]: #bowtie2-options-solexa-quals
+[`--tab5`]: #bowtie2-options-tab5
+[`--tab6`]: #bowtie2-options-tab6
+[`--un-bz2`]: #bowtie2-options-un
+[`--un-conc-bz2`]: #bowtie2-options-un-conc
+[`--un-conc-gz`]: #bowtie2-options-un-conc
+[`--un-conc-lz4`]: #bowtie2-options-un-conc
+[`--un-conc`]: #bowtie2-options-un-conc
+[`--un-gz`]: #bowtie2-options-un
+[`--un-lz4`]: #bowtie2-options-un
+[`--un`]: #bowtie2-options-un
+[`--version`]: #bowtie2-options-version
+[`--very-fast-local`]: #bowtie2-options-very-fast-local
+[`--very-fast`]: #bowtie2-options-very-fast
+[`--very-sensitive-local`]: #bowtie2-options-very-sensitive-local
+[`--very-sensitive`]: #bowtie2-options-very-sensitive
+[`--xeq`]: #bowtie2-options-xeq
+[`-1`]: #bowtie2-options-1
+[`-2`]: #bowtie2-options-2
+[`-3`/`--trim3`]: #bowtie2-options-3
+[`-3`]: #bowtie2-options-3
+[`-5`/`--trim5`]: #bowtie2-options-5
+[`-5`]: #bowtie2-options-5
+[`-D`]: #bowtie2-options-D
+[`-L`]: #bowtie2-options-L
+[`-N`]: #bowtie2-options-N
+[`-a`/`--noauto`]: #bowtie2-build-options-a
+[`-a`]: #bowtie2-options-a
+[`-c`]: #bowtie2-options-c
+[`-f`]: #bowtie2-options-f
+[`-i`]: #bowtie2-options-i
+[`-k`]: #bowtie2-options-k
+[`-m`]: #bowtie2-options-m
+[`-n`/`--names`]: #bowtie2-inspect-options-n
+[`-o`/`--offrate`]: #bowtie2-options-o
+[`-o`]: #bowtie2-options-o
+[`-p`/`--packed`]: #bowtie2-build-options-p
+[`-p`/`--threads`]: #bowtie2-options-p
+[`-p`]: #bowtie2-options-p
+[`-q`]: #bowtie2-options-q
+[`-r`]: #bowtie2-options-r
+[`-s`/`--skip`]: #bowtie2-options-s
+[`-s`/`--summary`]: #bowtie2-inspect-options-s
+[`-s`]: #bowtie2-options-s
+[`-t`/`--time`]: #bowtie2-options-t
+[`-t`]: #bowtie2-options-t
+[`-u`/`--qupto`]: #bowtie2-options-u
+[`-u`]: #bowtie2-options-u
+[`-x`]: #bowtie2-options-x
+[`AS:i`]: #bowtie2-build-opt-fields-as
+[`MD:Z`]: #bowtie2-build-opt-fields-md
+[`NM:i`]: #bowtie2-build-opt-fields-nm
+[`XG:i`]: #bowtie2-build-opt-fields-xg
+[`XM:i`]: #bowtie2-build-opt-fields-xm
+[`XN:i`]: #bowtie2-build-opt-fields-xn
+[`XO:i`]: #bowtie2-build-opt-fields-xo
+[`XS:i`]: #bowtie2-build-opt-fields-xs
+[`YF:Z`]: #bowtie2-build-opt-fields-yf
+[`YS:i`]: #bowtie2-build-opt-fields-ys
+[`YT:Z`]: #bowtie2-build-opt-fields-yt
+[bi-directional BWT approach]: http://www.computer.org/portal/web/csdl/doi/10.1109/BIBM.2009.42
+[documentation for the preset options]: #preset-options-in---end-to-end-mode
+[end-to-end alignment]: #end-to-end-alignment-versus-local-alignment
+[filtered out]: #filtering
+[local alignment]: #end-to-end-alignment-versus-local-alignment
+[manual section on index building]: #the-bowtie2-build-indexer
+[multiseed alignment]: #multiseed-heuristic
+[multiseed heuristic]: #multiseed-heuristic
+[obtain Bowtie 2]: #obtaining-bowtie-2
+[overlap ambiguous characters]: #ambiguous-characters
+[paired-end alignment]: #aligning-pairs
+[preset options]: #presets-setting-many-settings-at-once
+[pthreads]: http://sourceware.org/pthreads-win32/
+[reporting]: #reporting
+[scoring scheme]: #scores-higher-more-similar
+[setting function options]: #setting-function-options
+[sourceforge site]: https://sourceforge.net/projects/bowtie-bio/files/bowtie2/
+[structural variants]: http://www.ncbi.nlm.nih.gov/dbvar/content/overview/
+[using a pre-built index]: #using-a-pre-built-index
+[valid alignment]: #valid-alignments-meet-or-exceed-the-minimum-score-threshold
+[yields a larger memory footprint]: #fm-index-memory-footprint
diff --git a/Makefile b/Makefile
index f85df39..37c97e9 100644
--- a/Makefile
+++ b/Makefile
@@ -243,7 +243,6 @@ SRC_PKG_LIST = $(wildcard *.h) \
$(wildcard *.c) \
$(wildcard *.cpp) \
$(wildcard third_party/*) \
- doc/strip_markdown.pl \
Makefile \
$(GENERAL_LIST)
@@ -473,7 +472,7 @@ doc/manual.html: MANUAL.markdown
rm -f .tmp.head
MANUAL: MANUAL.markdown
- perl doc/strip_markdown.pl < $^ > $@
+ pandoc -f markdown -t plain $^ -o $@
.PHONY: install
install: all
diff --git a/NEWS b/NEWS
index 03506ba..4b0cd62 100644
--- a/NEWS
+++ b/NEWS
@@ -1,21 +1,44 @@
Bowtie 2 NEWS
=============
-Bowtie 2 is now available for download from the project website,
-http://bowtie-bio.sf.net/bowtie2. 2.0.0-beta1 is the first version released to
-the public and 2.3.2 is the latest version. Bowtie 2 is licensed under
-the GPLv3 license. See `LICENSE' file for details.
+Bowtie 2 is available for download from the project website,
+http://bowtie-bio.sf.net/bowtie2 and on Github,
+https://github.com/BenLangmead/bowtie2/releases. 2.0.0-beta1 is
+the first version released to the public and 2.3.3 is the latest
+version. Bowtie 2 is licensed under the GPLv3 license. See `LICENSE'
+file for details.
Reporting Issues
================
-Please report any issues using the Sourceforge bug tracker:
+Please report any issues to the Bowtie 2 Github page or using the Sourceforge bug tracker:
- https://sourceforge.net/tracker/?group_id=236897&atid=1101606
+ * https://github.com/BenLangmead/bowtie2/issues
+ * https://sourceforge.net/tracker/?group_id=236897&atid=1101606
Version Release History
=======================
+Version 2.3.3 - Sep 09, 2017
+From this release forward prepackaged bowtie2 binaries are now
+statically linked to the zlib compression library and, the recommended
+threading library, TBB. Users who rely on prepackaged builds are
+no longer required to have these packages pre-installed. As a result
+of the aforementioned changes legacy packages have been discontinued.
+
+ * bowtie2-build now supports gzip-compressed FASTA inputs
+ * New --xeq parameter for bowtie2 disambiguates the 'M' CIGAR
+ flag. When specified, matches are indicated with the '=' operation and
+ mismatches with 'X'
+ * Fixed a possible infinite loop during parallel index building due
+ to the compiler optimizing away a loop condition
+ * Added --soft-clipped-unmapped-tlen parameter for bowtie2 that
+ ignores soft-clipped bases when calculating template length (TLEN)
+ * Added support for multi-line sequences in FASTA read inputs
+ * Expanded explanation of MD:Z field in manual
+ * Fixed a crashing bug when output is redirected to a pipe
+ * Fixed ambiguity in the SEED alignment policy that sometimes caused -N parameter to be ignored
+
Version 2.3.2 - May 05, 2017
* Added support for interleaved paired-end FASTQ inputs
(--interleaved)
diff --git a/VERSION b/VERSION
index f90b1af..0bee604 100644
--- a/VERSION
+++ b/VERSION
@@ -1 +1 @@
-2.3.2
+2.3.3
diff --git a/aligner_cache.cpp b/aligner_cache.cpp
index 3930356..846a38a 100644
--- a/aligner_cache.cpp
+++ b/aligner_cache.cpp
@@ -51,16 +51,14 @@ bool SAVal::repOk(const AlignmentCache& ac) const {
* Add a new association between a read sequnce ('seq') and a
* reference sequence ('')
*/
-bool AlignmentCache::addOnTheFly(
+bool AlignmentCache::addOnTheFlyImpl(
QVal& qv, // qval that points to the range of reference substrings
const SAKey& sak, // the key holding the reference substring
TIndexOffU topf, // top range elt in BWT index
TIndexOffU botf, // bottom range elt in BWT index
TIndexOffU topb, // top range elt in BWT' index
- TIndexOffU botb, // bottom range elt in BWT' index
- bool getLock)
+ TIndexOffU botb) // bottom range elt in BWT' index
{
- ThreadSafe ts(lockPtr(), shared_ && getLock);
bool added = true;
// If this is the first reference sequence we're associating with
// the query sequence, initialize the QVal.
@@ -104,6 +102,23 @@ bool AlignmentCache::addOnTheFly(
return true;
}
+bool AlignmentCache::addOnTheFly(
+ QVal& qv, // qval that points to the range of reference substrings
+ const SAKey& sak, // the key holding the reference substring
+ TIndexOffU topf, // top range elt in BWT index
+ TIndexOffU botf, // bottom range elt in BWT index
+ TIndexOffU topb, // top range elt in BWT' index
+ TIndexOffU botb, // bottom range elt in BWT' index
+ bool getLock)
+{
+ if(shared_ && getLock) {
+ ThreadSafe ts(mutex_m);
+ return addOnTheFlyImpl(qv, sak, topf, botf, topb, botb);
+ } else {
+ return addOnTheFlyImpl(qv, sak, topf, botf, topb, botb);
+ }
+}
+
#ifdef ALIGNER_CACHE_MAIN
#include <iostream>
diff --git a/aligner_cache.h b/aligner_cache.h
index c5c15d7..770cc45 100644
--- a/aligner_cache.h
+++ b/aligner_cache.h
@@ -444,10 +444,8 @@ public:
samap_(CACHE_PAGE_SZ, CA_CAT),
salist_(CA_CAT),
shared_(shared),
- mutex_m(),
- version_(0)
- {
- }
+ mutex_m(),
+ version_(0) { }
/**
* Given a QVal, populate the given EList of SATuples with records
@@ -462,37 +460,11 @@ public:
size_t& nelt,
bool getLock = true)
{
- ThreadSafe ts(lockPtr(), shared_ && getLock);
- assert(qv.repOk(*this));
- const size_t refi = qv.offset();
- const size_t reff = refi + qv.numRanges();
- // For each reference sequence sufficiently similar to the
- // query sequence in the QKey...
- for(size_t i = refi; i < reff; i++) {
- // Get corresponding SAKey, containing similar reference
- // sequence & length
- SAKey sak = qlist_.get(i);
- // Shouldn't have identical keys in qlist_
- assert(i == refi || qlist_.get(i) != qlist_.get(i-1));
- // Get corresponding SANode
- SANode *n = samap_.lookup(sak);
- assert(n != NULL);
- const SAVal& sav = n->payload;
- assert(sav.repOk(*this));
- if(sav.len > 0) {
- nrange++;
- satups.expand();
- satups.back().init(sak, sav.topf, sav.topb, TSlice(salist_, sav.i, sav.len));
- nelt += sav.len;
-#ifndef NDEBUG
- // Shouldn't add consecutive identical entries too satups
- if(i > refi) {
- const SATuple b1 = satups.back();
- const SATuple b2 = satups[satups.size()-2];
- assert(b1.key != b2.key || b1.topf != b2.topf || b1.offs != b2.offs);
- }
-#endif
- }
+ if(shared_ && getLock) {
+ ThreadSafe ts(mutex_m);
+ queryQvalImpl(qv, satups, nrange, nelt);
+ } else {
+ queryQvalImpl(qv, satups, nrange, nelt);
}
}
@@ -522,12 +494,14 @@ public:
bool *added,
bool getLock = true)
{
- ThreadSafe ts(lockPtr(), shared_ && getLock);
- assert(qk.cacheable());
- QNode *n = qmap_.add(pool(), qk, added);
- return (n != NULL ? &n->payload : NULL);
+ if(shared_ && getLock) {
+ ThreadSafe ts(mutex_m);
+ return addImpl(qk, added);
+ } else {
+ return addImpl(qk, added);
+ }
}
-
+
/**
* Add a new association between a read sequnce ('seq') and a
* reference sequence ('')
@@ -546,8 +520,8 @@ public:
* ranges in this cache will become invalid and the corresponding
* reads will have to be re-aligned.
*/
- void clear(bool getLock = true) {
- ThreadSafe ts(lockPtr(), shared_ && getLock);
+ void clear() {
+ ThreadSafe ts(mutex_m);
pool_.clear();
qmap_.clear();
qlist_.clear();
@@ -585,18 +559,10 @@ public:
* Return the lock object.
*/
MUTEX_T& lock() {
- return mutex_m;
+ return mutex_m;
}
/**
- * Return a const pointer to the lock object. This allows us to
- * write const member functions that grab the lock.
- */
- MUTEX_T* lockPtr() const {
- return const_cast<MUTEX_T*>(&mutex_m);
- }
-
- /**
* Return true iff this cache is shared among threads.
*/
bool shared() const { return shared_; }
@@ -618,6 +584,79 @@ protected:
bool shared_; // true -> this cache is global
MUTEX_T mutex_m; // mutex used for syncronization in case the the cache is shared.
uint32_t version_; // cache version
+
+private:
+
+ template <int S>
+ void queryQvalImpl(
+ const QVal& qv,
+ EList<SATuple, S>& satups,
+ size_t& nrange,
+ size_t& nelt)
+ {
+ assert(qv.repOk(*this));
+ const size_t refi = qv.offset();
+ const size_t reff = refi + qv.numRanges();
+ // For each reference sequence sufficiently similar to the
+ // query sequence in the QKey...
+ for(size_t i = refi; i < reff; i++) {
+ // Get corresponding SAKey, containing similar reference
+ // sequence & length
+ SAKey sak = qlist_.get(i);
+ // Shouldn't have identical keys in qlist_
+ assert(i == refi || qlist_.get(i) != qlist_.get(i-1));
+ // Get corresponding SANode
+ SANode *n = samap_.lookup(sak);
+ assert(n != NULL);
+ const SAVal& sav = n->payload;
+ assert(sav.repOk(*this));
+ if(sav.len > 0) {
+ nrange++;
+ satups.expand();
+ satups.back().init(sak, sav.topf, sav.topb, TSlice(salist_, sav.i, sav.len));
+ nelt += sav.len;
+#ifndef NDEBUG
+ // Shouldn't add consecutive identical entries too satups
+ if(i > refi) {
+ const SATuple b1 = satups.back();
+ const SATuple b2 = satups[satups.size()-2];
+ assert(b1.key != b2.key || b1.topf != b2.topf || b1.offs != b2.offs);
+ }
+#endif
+ }
+ }
+ }
+
+ /**
+ * Add a new association between a read sequnce ('seq') and a
+ * reference sequence ('')
+ */
+ bool addOnTheFlyImpl(
+ QVal& qv, // qval that points to the range of reference substrings
+ const SAKey& sak, // the key holding the reference substring
+ TIndexOffU topf, // top range elt in BWT index
+ TIndexOffU botf, // bottom range elt in BWT index
+ TIndexOffU topb, // top range elt in BWT' index
+ TIndexOffU botb); // bottom range elt in BWT' index
+
+ /**
+ * Add a new query key ('qk'), usually a 2-bit encoded substring of
+ * the read) as the key in a new Red-Black node in the qmap and
+ * return a pointer to the node's QVal.
+ *
+ * The expectation is that the caller is about to set about finding
+ * associated reference substrings, and that there will be future
+ * calls to addOnTheFly to add associations to reference substrings
+ * found.
+ */
+ QVal* addImpl(
+ const QKey& qk,
+ bool *added)
+ {
+ assert(qk.cacheable());
+ QNode *n = qmap_.add(pool(), qk, added);
+ return (n != NULL ? &n->payload : NULL);
+ }
};
/**
diff --git a/aligner_result.h b/aligner_result.h
index ae47d94..6369ce6 100644
--- a/aligner_result.h
+++ b/aligner_result.h
@@ -295,7 +295,9 @@ public:
false, // mixedMode
false, // primary
false, // oppAligned
- false); // oppFw
+ false, // oppFw
+ false, // scUnMapped
+ false); // xeq
}
AlnFlags(
@@ -310,10 +312,13 @@ public:
bool mixedMode,
bool primary,
bool oppAligned, // opposite mate aligned?
- bool oppFw) // opposite mate aligned forward?
+ bool oppFw, // opposite mate aligned forward?
+ bool scUnMapped,
+ bool xeq)
{
init(pairing, canMax, maxed, maxedPair, nfilt, scfilt,
- lenfilt, qcfilt, mixedMode, primary, oppAligned, oppFw);
+ lenfilt, qcfilt, mixedMode, primary, oppAligned,
+ oppFw, scUnMapped, xeq);
}
/**
@@ -331,7 +336,9 @@ public:
bool mixedMode,
bool primary,
bool oppAligned,
- bool oppFw)
+ bool oppFw,
+ bool scUnMapped,
+ bool xeq)
{
assert_gt(pairing, 0);
assert_leq(pairing, ALN_FLAG_PAIR_UNPAIRED);
@@ -347,6 +354,8 @@ public:
primary_ = primary;
oppAligned_ = oppAligned;
oppFw_ = oppFw;
+ scUnMapped_ = scUnMapped;
+ xeq_ = xeq;
}
/**
@@ -495,10 +504,18 @@ public:
return oppAligned_;
}
- inline bool isOppFw() const {
+ bool isOppFw() const {
return oppFw_;
}
+ bool scUnMapped() const {
+ return scUnMapped_;
+ }
+
+ bool xeq() const {
+ return xeq_;
+ }
+
protected:
// See ALN_FLAG_PAIR_* above
@@ -533,6 +550,11 @@ protected:
// True if opposite mate aligned in the forward direction
bool oppFw_;
+
+ // True if soft clipped bases are considered unmapped w/r/t TLEN
+ bool scUnMapped_;
+
+ bool xeq_;
};
static inline ostream& operator<<(ostream& os, const AlnScore& o) {
@@ -870,17 +892,20 @@ public:
*/
inline void getExtendedCoords(
Coord& st, // out: install starting coordinate here
- Coord& en) // out: install ending coordinate here
+ Coord& en, // out: install ending coordinate here
+ const AlnFlags& flags)
const
{
getCoords(st, en);
// Take trimming into account
- int64_t trim_st = (fw() ? trim5p_ : trim3p_);
- int64_t trim_en = (fw() ? trim3p_ : trim5p_);
- trim_st += (fw() ? pretrim5p_ : pretrim3p_);
- trim_en += (fw() ? pretrim3p_ : pretrim5p_);
- st.adjustOff(-trim_st);
- en.adjustOff( trim_en);
+ if (!flags.scUnMapped()) {
+ int64_t trim_st = (fw() ? trim5p_ : trim3p_);
+ int64_t trim_en = (fw() ? trim3p_ : trim5p_);
+ trim_st += (fw() ? pretrim5p_ : pretrim3p_);
+ trim_en += (fw() ? pretrim3p_ : pretrim5p_);
+ st.adjustOff(-trim_st);
+ en.adjustOff( trim_en);
+ }
}
/**
@@ -1268,7 +1293,7 @@ public:
if((sameChr && refcoord_.ref() == omate->refcoord_.ref()) ||
flags.alignedConcordant())
{
- setFragmentLength(*omate);
+ setFragmentLength(*omate, flags);
} else {
assert(!isFraglenSet());
}
@@ -1283,12 +1308,12 @@ public:
* by the user in how they set the maximum and minimum fragment length
* settings.
*/
- int64_t setFragmentLength(const AlnRes& omate) {
+ int64_t setFragmentLength(const AlnRes& omate, const AlnFlags& flags) {
Coord st, en;
Coord ost, oen;
assert_eq(refid(), omate.refid());
- getExtendedCoords(st, en);
- omate.getExtendedCoords(ost, oen);
+ getExtendedCoords(st, en, flags);
+ omate.getExtendedCoords(ost, oen, flags);
bool imUpstream = st.off() < ost.off();
TRefOff up = std::min(st.off(), ost.off());
TRefOff dn = std::max(en.off(), oen.off());
diff --git a/aligner_seed.cpp b/aligner_seed.cpp
index 04006e0..c03d405 100644
--- a/aligner_seed.cpp
+++ b/aligner_seed.cpp
@@ -1526,7 +1526,7 @@ SeedAligner::searchSeedBi(
bool leaveZone = s.zones[i].first < 0;
//bool leaveZoneIns = zones_[i].second < 0;
Constraint& cons = *zones[abs(s.zones[i].first)];
- Constraint& insCons = *zones[abs(s.zones[i].second)];
+ //Constraint& insCons = *zones[abs(s.zones[i].second)];
int c = (*seq_)[off]; assert_range(0, 4, c);
int q = (*qual_)[off];
// Is it legal for us to advance on characters other than 'c'?
@@ -1598,14 +1598,14 @@ SeedAligner::searchSeedBi(
}
if(cons.canGap() && overall.canGap()) {
throw 1; // TODO
- int delEx = 0;
- if(cons.canDelete(delEx, *sc_) && overall.canDelete(delEx, *sc_)) {
- // Try delete
- }
- int insEx = 0;
- if(insCons.canInsert(insEx, *sc_) && overall.canInsert(insEx, *sc_)) {
- // Try insert
- }
+// int delEx = 0;
+// if(cons.canDelete(delEx, *sc_) && overall.canDelete(delEx, *sc_)) {
+// // Try delete
+// }
+// int insEx = 0;
+// if(insCons.canInsert(insEx, *sc_) && overall.canInsert(insEx, *sc_)) {
+// // Try insert
+// }
}
} // if(!bail)
}
diff --git a/aligner_seed.h b/aligner_seed.h
index 8d6a6c8..ed92616 100644
--- a/aligner_seed.h
+++ b/aligner_seed.h
@@ -841,7 +841,7 @@ public:
size_t numRepeatSeeds() const {
return repTot_;
}
-
+
/**
* Return fraction of seeds that align repetitively.
*/
@@ -1401,7 +1401,6 @@ struct SeedSearchMetrics {
* SeedSearchMetrics object shread by multiple threads.
*/
void merge(const SeedSearchMetrics& m, bool getLock = false) {
- ThreadSafe ts(&mutex_m, getLock);
seedsearch += m.seedsearch;
nrange += m.nrange;
nelt += m.nelt;
diff --git a/aligner_seed_policy.cpp b/aligner_seed_policy.cpp
index 32e4bb9..a74c3a4 100644
--- a/aligner_seed_policy.cpp
+++ b/aligner_seed_policy.cpp
@@ -534,9 +534,9 @@ void SeedAlignmentPolicy::parseString(
* interval is determined by IVAL.
*/
else if(tag == "SEED") {
- if(ctoks.size() > 2) {
+ if(ctoks.size() > 1) {
cerr << "Error parsing alignment policy setting "
- << "'" << tag.c_str() << "'; RHS must have 1 or 2 tokens, "
+ << "'" << tag.c_str() << "'; RHS must have 1 token, "
<< "had " << ctoks.size() << ". "
<< "Policy: '" << s.c_str() << "'" << endl;
assert(false); throw 1;
@@ -553,13 +553,6 @@ void SeedAlignmentPolicy::parseString(
throw 1;
}
}
- if(ctoks.size() >= 2) {
- istringstream tmpss(ctoks[1]);
- tmpss >> multiseedLen;
- } else {
- multiseedLen = gDefaultSeedLen;
-;
- }
}
else if(tag == "SEEDLEN") {
if(ctoks.size() > 1) {
diff --git a/aligner_sw_common.h b/aligner_sw_common.h
index be3f388..8fd91e8 100644
--- a/aligner_sw_common.h
+++ b/aligner_sw_common.h
@@ -209,8 +209,8 @@ struct SwMetrics {
* object. This is the only safe way to update a SwMetrics shared
* by multiple threads.
*/
- void merge(const SwMetrics& r, bool getLock = false) {
- ThreadSafe ts(&mutex_m, getLock);
+ void merge(const SwMetrics& r) {
+ ThreadSafe ts(mutex_m);
sws += r.sws;
sws10 += r.sws10;
sws5 += r.sws5;
diff --git a/aligner_sw_driver.cpp b/aligner_sw_driver.cpp
index c46db61..f0c0e0a 100644
--- a/aligner_sw_driver.cpp
+++ b/aligner_sw_driver.cpp
@@ -326,11 +326,11 @@ void SwDriver::extend(
// Have to do both because whether we can get through an N depends on
// which direction we're coming in
- bool fwContains = ebwtFw.contains(tmp_rdseq_);
- tmp_rdseq_.reverse();
- bool bwContains = ebwtBw != NULL && ebwtBw->contains(tmp_rdseq_);
- tmp_rdseq_.reverse();
- assert(fwContains || bwContains);
+// bool fwContains = ebwtFw.contains(tmp_rdseq_);
+// tmp_rdseq_.reverse();
+// bool bwContains = ebwtBw != NULL && ebwtBw->contains(tmp_rdseq_);
+// tmp_rdseq_.reverse();
+// assert(fwContains || bwContains);
}
#endif
ASSERT_ONLY(tmp_rdseq_.reverse());
@@ -471,11 +471,11 @@ void SwDriver::extend(
// Have to do both because whether we can get through an N depends on
// which direction we're coming in
- bool fwContains = ebwtFw.contains(tmp_rdseq_);
- tmp_rdseq_.reverse();
- bool bwContains = ebwtBw != NULL && ebwtBw->contains(tmp_rdseq_);
- tmp_rdseq_.reverse();
- assert(fwContains || bwContains);
+// bool fwContains = ebwtFw.contains(tmp_rdseq_);
+// tmp_rdseq_.reverse();
+// bool bwContains = ebwtBw != NULL && ebwtBw->contains(tmp_rdseq_);
+// tmp_rdseq_.reverse();
+// assert(fwContains || bwContains);
}
#endif
assert_lt(nlex, rdlen);
diff --git a/aligner_swsse.h b/aligner_swsse.h
index 8e5bbd3..7d8759c 100644
--- a/aligner_swsse.h
+++ b/aligner_swsse.h
@@ -41,8 +41,7 @@ struct SSEMetrics {
corerej = nrej = 0;
}
- void merge(const SSEMetrics& o, bool getLock = false) {
- ThreadSafe ts(&mutex_m, getLock);
+ void merge(const SSEMetrics& o) {
dp += o.dp;
dpsat += o.dpsat;
dpfail += o.dpfail;
diff --git a/aligner_swsse_ee_i16.cpp b/aligner_swsse_ee_i16.cpp
index c49199f..2e761be 100644
--- a/aligner_swsse_ee_i16.cpp
+++ b/aligner_swsse_ee_i16.cpp
@@ -1362,7 +1362,7 @@ bool SwAligner::backtraceNucleotidesEnd2EndSseI16(
int readq = (*qu_)[row];
assert_leq(col, origCol);
// Get score in this cell
- bool empty, reportedThru, canMoveThru, branch = false;
+ bool empty = false, reportedThru, canMoveThru, branch = false;
int cur = SSEMatrix::H;
if(!d.mat_.reset_[row]) {
d.mat_.resetRow(row);
diff --git a/aligner_swsse_loc_i16.cpp b/aligner_swsse_loc_i16.cpp
index 9330aa2..b0fa0c8 100644
--- a/aligner_swsse_loc_i16.cpp
+++ b/aligner_swsse_loc_i16.cpp
@@ -1721,7 +1721,7 @@ bool SwAligner::backtraceNucleotidesLocalSseI16(
int readq = (*qu_)[row];
assert_leq(col, origCol);
// Get score in this cell
- bool empty, reportedThru, canMoveThru, branch = false;
+ bool empty = false, reportedThru, canMoveThru, branch = false;
int cur = SSEMatrix::H;
if(!d.mat_.reset_[row]) {
d.mat_.resetRow(row);
diff --git a/aligner_swsse_loc_u8.cpp b/aligner_swsse_loc_u8.cpp
index b8d8360..790b561 100644
--- a/aligner_swsse_loc_u8.cpp
+++ b/aligner_swsse_loc_u8.cpp
@@ -1719,7 +1719,7 @@ bool SwAligner::backtraceNucleotidesLocalSseU8(
int readq = (*qu_)[row];
assert_leq(col, origCol);
// Get score in this cell
- bool empty, reportedThru, canMoveThru, branch = false;
+ bool empty = false, reportedThru, canMoveThru, branch = false;
int cur = SSEMatrix::H;
if(!d.mat_.reset_[row]) {
d.mat_.resetRow(row);
diff --git a/aln_sink.cpp b/aln_sink.cpp
index de65b64..c34a377 100644
--- a/aln_sink.cpp
+++ b/aln_sink.cpp
@@ -658,7 +658,9 @@ void AlnSinkWrap::finishRead(
const PerReadMetrics& prm, // per-read metrics
const Scoring& sc, // scoring scheme
bool suppressSeedSummary, // = true
- bool suppressAlignments) // = false
+ bool suppressAlignments, // = false
+ bool scUnMapped, // = false
+ bool xeq) // = false
{
obuf_.clear();
OutputQueueMark qqm(g_.outq(), obuf_, rdid_, threadid_);
@@ -776,7 +778,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
true, // primary
true, // opp aligned
- rs2->fw()); // opp fw
+ rs2->fw(), // opp fw
+ scUnMapped,
+ xeq);
AlnFlags flags2(
ALN_FLAG_PAIR_CONCORD_MATE2,
st_.params().mhitsSet(),
@@ -789,7 +793,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
false, // primary
true, // opp aligned
- rs1->fw()); // opp fw
+ rs1->fw(), // opp fw
+ scUnMapped,
+ xeq);
// Issue: we only set the flags once, but some of the flags might
// vary from pair to pair among the pairs we're reporting. For
// instance, whether the a given mate aligns to the forward strand.
@@ -838,7 +844,7 @@ void AlnSinkWrap::finishRead(
//g_.outq().finishRead(obuf_, rdid_, threadid_);
return;
}
- // Report concordant paired-end alignments if possible
+ // Report disconcordant paired-end alignments if possible
else if(ndiscord > 0) {
ASSERT_ONLY(bool ret =) prepareDiscordants();
assert(ret);
@@ -861,7 +867,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
true, // primary
true, // opp aligned
- rs2->fw()); // opp fw
+ rs2->fw(), // opp fw
+ scUnMapped,
+ xeq);
AlnFlags flags2(
ALN_FLAG_PAIR_DISCORD_MATE2,
st_.params().mhitsSet(),
@@ -874,7 +882,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
false, // primary
true, // opp aligned
- rs1->fw()); // opp fw
+ rs1->fw(), // opp fw
+ scUnMapped,
+ xeq);
SeedAlSumm ssm1, ssm2;
sr1->toSeedAlSumm(ssm1);
sr2->toSeedAlSumm(ssm2);
@@ -1174,7 +1184,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
true, // primary
repRs2 != NULL, // opp aligned
- repRs2 == NULL || repRs2->fw()); // opp fw
+ repRs2 == NULL || repRs2->fw(), // opp fw
+ scUnMapped,
+ xeq);
for(size_t i = 0; i < rs1u_.size(); i++) {
rs1u_[i].setMateParams(ALN_RES_TYPE_UNPAIRED_MATE1, NULL, flags1);
}
@@ -1196,7 +1208,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
true, // primary
repRs1 != NULL, // opp aligned
- repRs1 == NULL || repRs1->fw()); // opp fw
+ repRs1 == NULL || repRs1->fw(), // opp fw
+ scUnMapped,
+ xeq);
for(size_t i = 0; i < rs2u_.size(); i++) {
rs2u_[i].setMateParams(ALN_RES_TYPE_UNPAIRED_MATE2, NULL, flags2);
}
@@ -1295,7 +1309,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
true, // primary
repRs2 != NULL, // opp aligned
- (repRs2 != NULL) ? repRs2->fw() : false); // opp fw
+ (repRs2 != NULL) ? repRs2->fw() : false, // opp fw
+ scUnMapped,
+ xeq);
g_.reportUnaligned(
obuf_, // string to write output to
staln_,
@@ -1341,7 +1357,9 @@ void AlnSinkWrap::finishRead(
st_.params().mixed,
true, // primary
repRs1 != NULL, // opp aligned
- (repRs1 != NULL) ? repRs1->fw() : false); // opp fw
+ (repRs1 != NULL) ? repRs1->fw() : false, // opp fw
+ scUnMapped,
+ xeq);
g_.reportUnaligned(
obuf_, // string to write output to
staln_,
@@ -1979,7 +1997,7 @@ void AlnSinkSam::appendMate(
}
// CIGAR
if(rs != NULL) {
- staln.buildCigar(false);
+ staln.buildCigar(flags.xeq());
staln.writeCigar(&o, NULL);
o.append('\t');
} else {
diff --git a/aln_sink.h b/aln_sink.h
index 64a26fa..bd42ec6 100644
--- a/aln_sink.h
+++ b/aln_sink.h
@@ -121,8 +121,8 @@ struct ReportingMetrics {
* into this object. This is the only safe way to update a
* ReportingMetrics shared by multiple threads.
*/
- void merge(const ReportingMetrics& met, bool getLock = false) {
- ThreadSafe ts(&mutex_m, getLock);
+ void merge(const ReportingMetrics& met) {
+ ThreadSafe ts(mutex_m);
nread += met.nread;
npaired += met.npaired;
@@ -847,8 +847,8 @@ public:
/**
* Merge given metrics in with ours by summing all individual metrics.
*/
- void mergeMetrics(const ReportingMetrics& met, bool getLock = true) {
- met_.merge(met, getLock);
+ void mergeMetrics(const ReportingMetrics& met) {
+ met_.merge(met);
}
/**
@@ -1029,7 +1029,9 @@ public:
const PerReadMetrics& prm, // per-read metrics
const Scoring& sc, // scoring scheme
bool suppressSeedSummary = true,
- bool suppressAlignments = false);
+ bool suppressAlignments = false,
+ bool scUnMapped = false,
+ bool xeq = false);
/**
* Called by the aligner when a new unpaired or paired alignment is
diff --git a/blockwise_sa.h b/blockwise_sa.h
index 2990e2c..e64172b 100644
--- a/blockwise_sa.h
+++ b/blockwise_sa.h
@@ -29,6 +29,7 @@
#include <stdlib.h>
#include <iostream>
#include <sstream>
+#include <memory>
#include <stdexcept>
#include "assert_helpers.h"
#include "diff_sample.h"
@@ -215,7 +216,7 @@ public:
#endif
{ _randomSrc.init(__seed); reset(); }
- ~KarkkainenBlockwiseSA()
+ ~KarkkainenBlockwiseSA() throw()
{
#ifdef WITH_TBB
tbb_grp.wait();
@@ -244,89 +245,79 @@ public:
return bsz;
}
-
- /**
- * Get the next suffix; compute the next bucket if necessary.
- */
- virtual TIndexOffU nextSuffix()
- {
- // Launch threads if not
- if(this->_nthreads > 1)
- {
+
+ /**
+ * Get the next suffix; compute the next bucket if necessary.
+ */
+ virtual TIndexOffU nextSuffix()
+ {
+ // Launch threads if not
+ if(this->_nthreads > 1) {
#ifdef WITH_TBB
- if(!thread_group_started)
- {
+ if(!thread_group_started) {
#else
- if(_threads.size() == 0)
- {
+ if(_threads.size() == 0) {
#endif
- _done.resize(_sampleSuffs.size() + 1);
- _done.fill(false);
- _itrBuckets.resize(this->_nthreads);
- _tparams.resize(this->_nthreads);
- for(int tid = 0; tid < this->_nthreads; tid++) {
- _tparams[tid].first = this;
- _tparams[tid].second = tid;
-#ifdef WITH_TBB
- tbb_grp.run(nextBlock_Worker((void*)&_tparams[tid]));
- }
- thread_group_started = true;
- }
-#else
- _threads.push_back(new tthread::thread(nextBlock_Worker, (void*)&_tparams[tid]));
+ _done = std::auto_ptr<volatile bool>(new volatile bool[_sampleSuffs.size() + 1]);
+ for (int i = 0; i < _sampleSuffs.size() + 1; i++) {
+ _done.get()[i] = false;
}
- assert_eq(_threads.size(), (size_t)this->_nthreads);
- }
-#endif
- }
- if(this->_itrPushedBackSuffix != OFF_MASK) {
- TIndexOffU tmp = this->_itrPushedBackSuffix;
- this->_itrPushedBackSuffix = OFF_MASK;
- return tmp;
- }
- while(this->_itrBucketPos >= this->_itrBucket.size() ||
- this->_itrBucket.size() == 0)
- {
- if(!hasMoreBlocks()) {
- throw out_of_range("No more suffixes");
- }
- if(this->_nthreads == 1) {
- nextBlock((int)_cur);
- _cur++;
- }
- else
- {
- while(!_done[this->_itrBucketIdx])
- {
-#if defined(_TTHREAD_WIN32_)
- Sleep(1);
+ _itrBuckets.resize(this->_nthreads);
+ _tparams.resize(this->_nthreads);
+ for(int tid = 0; tid < this->_nthreads; tid++) {
+ _tparams[tid].first = this;
+ _tparams[tid].second = tid;
+#ifdef WITH_TBB
+ tbb_grp.run(nextBlock_Worker((void*)&_tparams[tid]));
+ }
+ thread_group_started = true;
#else
- const static timespec ts = {0, 1000000}; // 1 millisecond
- nanosleep(&ts, NULL);
+ _threads.push_back(new tthread::thread(nextBlock_Worker, (void*)&_tparams[tid]));
+ }
+ assert_eq(_threads.size(), (size_t)this->_nthreads);
#endif
- }
- // Read suffixes from a file
- std::ostringstream number; number << this->_itrBucketIdx;
- const string fname = _base_fname + "." + number.str() + ".sa";
- ifstream sa_file(fname.c_str(), ios::binary);
- if(!sa_file.good()) {
- cerr << "Could not open file for reading a suffix array: \"" << fname << "\"" << endl;
- throw 1;
- }
- size_t numSAs = readU<TIndexOffU>(sa_file, _bigEndian);
- this->_itrBucket.resizeExact(numSAs);
- for(size_t i = 0; i < numSAs; i++) {
- this->_itrBucket[i] = readU<TIndexOffU>(sa_file, _bigEndian);
- }
- sa_file.close();
- std::remove(fname.c_str());
- }
- this->_itrBucketIdx++;
- this->_itrBucketPos = 0;
- }
- return this->_itrBucket[this->_itrBucketPos++];
- }
-
+ }
+ }
+ if(this->_itrPushedBackSuffix != OFF_MASK) {
+ TIndexOffU tmp = this->_itrPushedBackSuffix;
+ this->_itrPushedBackSuffix = OFF_MASK;
+ return tmp;
+ }
+ while(this->_itrBucketPos >= this->_itrBucket.size() ||
+ this->_itrBucket.size() == 0)
+ {
+ if(!hasMoreBlocks()) {
+ throw out_of_range("No more suffixes");
+ }
+ if(this->_nthreads == 1) {
+ nextBlock((int)_cur);
+ _cur++;
+ } else {
+ while(!_done.get()[this->_itrBucketIdx]) {
+ SLEEP(1);
+ }
+ // Read suffixes from a file
+ std::ostringstream number; number << this->_itrBucketIdx;
+ const string fname = _base_fname + "." + number.str() + ".sa";
+ ifstream sa_file(fname.c_str(), ios::binary);
+ if(!sa_file.good()) {
+ cerr << "Could not open file for reading a suffix array: \"" << fname << "\"" << endl;
+ throw 1;
+ }
+ size_t numSAs = readU<TIndexOffU>(sa_file, _bigEndian);
+ this->_itrBucket.resizeExact(numSAs);
+ for(size_t i = 0; i < numSAs; i++) {
+ this->_itrBucket[i] = readU<TIndexOffU>(sa_file, _bigEndian);
+ }
+ sa_file.close();
+ std::remove(fname.c_str());
+ }
+ this->_itrBucketIdx++;
+ this->_itrBucketPos = 0;
+ }
+ return this->_itrBucket[this->_itrBucketPos++];
+ }
+
/// Defined in blockwise_sa.cpp
virtual void nextBlock(int cur_block, int tid = 0);
@@ -361,7 +352,7 @@ public:
while(true) {
size_t cur = 0;
{
- ThreadSafe ts(&sa->_mutex, sa->_nthreads > 1);
+ ThreadSafe ts(sa->_mutex);
cur = sa->_cur;
if(cur > sa->_sampleSuffs.size()) break;
sa->_cur++;
@@ -382,7 +373,7 @@ public:
}
sa_file.close();
sa->_itrBuckets[tid].clear();
- sa->_done[cur] = true;
+ sa->_done.get()[cur] = true;
}
}
#ifdef WITH_TBB
@@ -458,27 +449,27 @@ private:
void buildSamples();
- EList<TIndexOffU> _sampleSuffs; /// sample suffixes
- int _nthreads; /// # of threads
- TIndexOffU _itrBucketIdx;
- TIndexOffU _cur; /// offset to 1st elt of next block
- const uint32_t _dcV; /// difference-cover periodicity
- PtrWrap<TDC> _dc; /// queryable difference-cover data
- bool _built; /// whether samples/DC have been built
- RandomSource _randomSrc; /// source of pseudo-randoms
-
- MUTEX_T _mutex; /// synchronization of output message
- string _base_fname; /// base file name for storing SA blocks
- bool _bigEndian; /// bigEndian?
+ EList<TIndexOffU> _sampleSuffs; /// sample suffixes
+ int _nthreads; /// # of threads
+ TIndexOffU _itrBucketIdx;
+ TIndexOffU _cur; /// offset to 1st elt of next block
+ const uint32_t _dcV; /// difference-cover periodicity
+ PtrWrap<TDC> _dc; /// queryable difference-cover data
+ bool _built; /// whether samples/DC have been built
+ RandomSource _randomSrc; /// source of pseudo-randoms
+
+ MUTEX_T _mutex; /// synchronization of output message
+ string _base_fname; /// base file name for storing SA blocks
+ bool _bigEndian; /// bigEndian?
#ifdef WITH_TBB
- tbb::task_group tbb_grp; /// thread "list" via Intel TBB
- bool thread_group_started;
+ tbb::task_group tbb_grp; /// thread "list" via Intel TBB
+ bool thread_group_started;
#else
- EList<tthread::thread*> _threads; /// thread list
+ EList<tthread::thread*> _threads; /// thread list
#endif
- EList<pair<KarkkainenBlockwiseSA*, int> > _tparams;
- ELList<TIndexOffU> _itrBuckets; /// buckets
- EList<bool> _done; /// is a block processed?
+ EList<pair<KarkkainenBlockwiseSA*, int> > _tparams;
+ ELList<TIndexOffU> _itrBuckets; /// buckets
+ std::auto_ptr<volatile bool> _done; /// is a block processed?
};
@@ -1007,7 +998,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
#endif
EList<TIndexOffU>& bucket = (this->_nthreads > 1 ? this->_itrBuckets[tid] : this->_itrBucket);
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL("Getting block " << (cur_block+1) << " of " << _sampleSuffs.size()+1);
}
assert(_built);
@@ -1022,7 +1013,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
// Special case: if _sampleSuffs is 0, then multikey-quicksort
// everything
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL(" No samples; assembling all-inclusive block");
}
assert_eq(0, cur_block);
@@ -1047,7 +1038,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
} else {
try {
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL(" Reserving size (" << this->bucketSz() << ") for bucket " << (cur_block+1));
}
// BTL: Add a +100 fudge factor; there seem to be instances
@@ -1076,7 +1067,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
try {
// Timer timer(cout, " Calculating Z arrays time: ", this->verbose());
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL(" Calculating Z arrays for bucket " << (cur_block+1));
}
if(!last) {
@@ -1124,14 +1115,14 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
{
// Timer timer(cout, " Block accumulator loop time: ", this->verbose());
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL(" Entering block accumulator loop for bucket " << (cur_block+1) << ":");
}
TIndexOffU lenDiv10 = (len + 9) / 10;
for(TIndexOffU iten = 0, ten = 0; iten < len; iten += lenDiv10, ten++) {
TIndexOffU itenNext = iten + lenDiv10;
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
if(ten > 0) VMSG_NL(" bucket " << (cur_block+1) << ": " << (ten * 10) << "%");
}
for(TIndexOffU i = iten; i < itenNext && i < len; i++) {
@@ -1165,7 +1156,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
}
} // end loop over all suffixes of t
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL(" bucket " << (cur_block+1) << ": 100%");
}
}
@@ -1174,7 +1165,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
if(bucket.size() > 0) {
Timer timer(cout, " Sorting block time: ", this->verbose());
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL(" Sorting block of length " << bucket.size() << " for bucket " << (cur_block+1));
}
this->qsort(bucket);
@@ -1187,7 +1178,7 @@ void KarkkainenBlockwiseSA<TStr>::nextBlock(int cur_block, int tid) {
bucket.push_back(len);
}
{
- ThreadSafe ts(&_mutex, this->_nthreads > 1);
+ ThreadSafe ts(_mutex);
VMSG_NL("Returning block of " << bucket.size() << " for bucket " << (cur_block+1));
}
}
diff --git a/bowtie2 b/bowtie2
index 7f25561..8476ad3 100755
--- a/bowtie2
+++ b/bowtie2
@@ -342,7 +342,12 @@ sub Extract_IndexName_From {
my $index_opt = $ref_str ? '--index' : '-x';
for (my $i=0; $i<@_; $i++) {
if ($_[$i] eq $index_opt){
- return $_[$i+1];
+ my $idx_basename = $_[$i+1];
+ my @idx_filenames = glob($idx_basename . "*.bt2{,l}");
+ unless (@idx_filenames) {
+ Fail("\"" . $idx_basename . "\" is not a Bowtie 2 index\n");
+ }
+ return $idx_basename;
}
}
Info("Cannot find any index option (--reference-string, --ref-string or -x) in the given command line.\n");
diff --git a/bowtie_main.cpp b/bowtie_main.cpp
index 844c8ca..60a9cd7 100644
--- a/bowtie_main.cpp
+++ b/bowtie_main.cpp
@@ -43,7 +43,7 @@ int main(int argc, const char **argv) {
if(argc > 2 && strcmp(argv[1], "-A") == 0) {
const char *file = argv[2];
ifstream in;
- in.open(file);
+ in.open(file);
char buf[4096];
int lastret = -1;
while(in.getline(buf, 4095)) {
diff --git a/bt2_build.cpp b/bt2_build.cpp
index 0497f98..3389193 100644
--- a/bt2_build.cpp
+++ b/bt2_build.cpp
@@ -17,6 +17,7 @@
* along with Bowtie 2. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <zlib.h>
#include <iostream>
#include <fstream>
#include <string>
@@ -381,12 +382,26 @@ static void driver(
} else {
// Adapt sequence files to ifstreams
for(size_t i = 0; i < infiles.size(); i++) {
- FILE *f = fopen(infiles[i].c_str(), "rb");
- if (f == NULL) {
- cerr << "Error: could not open "<< infiles[i].c_str() << endl;
- throw 1;
+ FileBuf *fb;
+
+ size_t idx = infiles[i].find_last_of(".");
+ std::string ext = (idx == std::string::npos) ? "" : infiles[i].substr(idx + 1);
+ if (ext == "" || ext == "gz" || ext == "Z") {
+ gzFile zFp = gzopen(infiles[i].c_str(), "rb");
+ if (zFp == NULL) {
+ cerr << "Error: could not open "<< infiles[i].c_str() << endl;
+ throw 1;
+ }
+ fb = new FileBuf(zFp);
+ }
+ else {
+ FILE *f = fopen(infiles[i].c_str(), "rb");
+ if (f == NULL) {
+ cerr << "Error: could not open "<< infiles[i].c_str() << endl;
+ throw 1;
+ }
+ fb = new FileBuf(f);
}
- FileBuf *fb = new FileBuf(f);
assert(fb != NULL);
if(fb->peek() == -1 || fb->eof()) {
cerr << "Warning: Empty fasta file: '" << infile.c_str() << "'" << endl;
diff --git a/bt2_search.cpp b/bt2_search.cpp
index 7decf9b..7214f57 100644
--- a/bt2_search.cpp
+++ b/bt2_search.cpp
@@ -27,6 +27,9 @@
#include <math.h>
#include <utility>
#include <limits>
+#include <time.h>
+#include <dirent.h>
+#include <signal.h>
#include "alphabet.h"
#include "assert_helpers.h"
#include "endian_swap.h"
@@ -54,9 +57,20 @@
#include "outq.h"
#include "aligner_seed2.h"
#include "bt2_search.h"
+#ifdef WITH_TBB
+ #include <tbb/compat/thread>
+#endif
using namespace std;
+static int FNAME_SIZE;
+#ifdef WITH_TBB
+static tbb::atomic<int> thread_counter;
+#else
+static int thread_counter;
+static MUTEX_T thread_counter_mutex;
+#endif
+
static EList<string> mates1; // mated reads (first mate)
static EList<string> mates2; // mated reads (second mate)
static EList<string> mates12; // mated reads (1st/2nd interleaved in 1 file)
@@ -77,13 +91,16 @@ static bool allHits; // for multihits, report just one
static bool showVersion; // just print version and quit?
static int ipause; // pause before maching?
static uint32_t qUpto; // max # of queries to read
-int gTrim5; // amount to trim from 5' end
-int gTrim3; // amount to trim from 3' end
+static int gTrim5; // amount to trim from 5' end
+static int gTrim3; // amount to trim from 3' end
static int offRate; // keep default offRate
static bool solexaQuals; // quality strings are solexa quals, not phred, and subtract 64 (not 33)
static bool phred64Quals; // quality chars are phred, but must subtract 64 (not 33)
static bool integerQuals; // quality strings are space-separated strings of integers, not ASCII
static int nthreads; // number of pthreads operating concurrently
+static int thread_ceiling;// maximum number of threads user wants bowtie to use
+static string thread_stealing_dir; // keep track of pids in this directory
+static bool thread_stealing;// true iff thread stealing is in use
static int outType; // style of output
static bool noRefNames; // true -> print reference indexes; not names
static uint32_t khits; // number of hits per read; >1 is much slower
@@ -202,7 +219,9 @@ static uint32_t seedCacheCurrentMB; // # MB to use for current-read seed hit cac
static uint32_t exactCacheCurrentMB; // # MB to use for current-read seed hit cacheing
static size_t maxhalf; // max width on one side of DP table
static bool seedSumm; // print summary information about seed hits, not alignments
+static bool scUnMapped; // consider soft-clipped bases unmapped when calculating TLEN
static bool doUngapped; // do ungapped alignment
+static bool xeq; // use X/= instead of M in CIGAR string
static size_t maxIters; // stop after this many extend loop iterations
static size_t maxUg; // stop after this many ungap extends
static size_t maxDp; // stop after this many DPs
@@ -270,6 +289,10 @@ static void resetOptions() {
phred64Quals = false; // quality chars are phred, but must subtract 64 (not 33)
integerQuals = false; // quality strings are space-separated strings of integers, not ASCII
nthreads = 1; // number of pthreads operating concurrently
+ thread_ceiling = 0; // max # threads user asked for
+ thread_stealing_dir = ""; // keep track of pids in this directory
+ thread_stealing = false; // true iff thread stealing is in use
+ FNAME_SIZE = 4096;
outType = OUTPUT_SAM; // style of output
noRefNames = false; // true -> print reference indexes; not names
khits = 1; // number of hits per read; >1 is much slower
@@ -388,6 +411,8 @@ static void resetOptions() {
exactCacheCurrentMB = 20; // # MB to use for current-read seed hit cacheing
maxhalf = 15; // max width on one side of DP table
seedSumm = false; // print summary information about seed hits, not alignments
+ scUnMapped = false; // consider soft clipped bases unmapped when calculating TLEN
+ xeq = false; // use =/X instead of M in CIGAR string
doUngapped = true; // do ungapped alignment
maxIters = 400; // max iterations of extend loop
maxUg = 300; // stop after this many ungap extends
@@ -429,188 +454,192 @@ static void resetOptions() {
static const char *short_options = "fF:qbzhcu:rv:s:aP:t3:5:w:p:k:M:1:2:I:X:CQ:N:i:L:U:x:S:g:O:D:R:";
static struct option long_options[] = {
- {(char*)"verbose", no_argument, 0, ARG_VERBOSE},
- {(char*)"startverbose", no_argument, 0, ARG_STARTVERBOSE},
- {(char*)"quiet", no_argument, 0, ARG_QUIET},
- {(char*)"sanity", no_argument, 0, ARG_SANITY},
- {(char*)"pause", no_argument, &ipause, 1},
- {(char*)"orig", required_argument, 0, ARG_ORIG},
- {(char*)"all", no_argument, 0, 'a'},
- {(char*)"solexa-quals", no_argument, 0, ARG_SOLEXA_QUALS},
- {(char*)"integer-quals",no_argument, 0, ARG_INTEGER_QUALS},
- {(char*)"int-quals", no_argument, 0, ARG_INTEGER_QUALS},
- {(char*)"metrics", required_argument, 0, ARG_METRIC_IVAL},
- {(char*)"metrics-file", required_argument, 0, ARG_METRIC_FILE},
- {(char*)"metrics-stderr",no_argument, 0, ARG_METRIC_STDERR},
- {(char*)"metrics-per-read", no_argument, 0, ARG_METRIC_PER_READ},
- {(char*)"met-read", no_argument, 0, ARG_METRIC_PER_READ},
- {(char*)"met", required_argument, 0, ARG_METRIC_IVAL},
- {(char*)"met-file", required_argument, 0, ARG_METRIC_FILE},
- {(char*)"met-stderr", no_argument, 0, ARG_METRIC_STDERR},
- {(char*)"time", no_argument, 0, 't'},
- {(char*)"trim3", required_argument, 0, '3'},
- {(char*)"trim5", required_argument, 0, '5'},
- {(char*)"seed", required_argument, 0, ARG_SEED},
- {(char*)"qupto", required_argument, 0, 'u'},
- {(char*)"upto", required_argument, 0, 'u'},
- {(char*)"version", no_argument, 0, ARG_VERSION},
- {(char*)"reads-per-batch", required_argument, 0, ARG_READS_PER_BATCH},
- {(char*)"filepar", no_argument, 0, ARG_FILEPAR},
- {(char*)"help", no_argument, 0, 'h'},
- {(char*)"threads", required_argument, 0, 'p'},
- {(char*)"khits", required_argument, 0, 'k'},
- {(char*)"minins", required_argument, 0, 'I'},
- {(char*)"maxins", required_argument, 0, 'X'},
- {(char*)"quals", required_argument, 0, 'Q'},
- {(char*)"Q1", required_argument, 0, ARG_QUALS1},
- {(char*)"Q2", required_argument, 0, ARG_QUALS2},
- {(char*)"refidx", no_argument, 0, ARG_REFIDX},
- {(char*)"partition", required_argument, 0, ARG_PARTITION},
- {(char*)"ff", no_argument, 0, ARG_FF},
- {(char*)"fr", no_argument, 0, ARG_FR},
- {(char*)"rf", no_argument, 0, ARG_RF},
- {(char*)"cachelim", required_argument, 0, ARG_CACHE_LIM},
- {(char*)"cachesz", required_argument, 0, ARG_CACHE_SZ},
- {(char*)"nofw", no_argument, 0, ARG_NO_FW},
- {(char*)"norc", no_argument, 0, ARG_NO_RC},
- {(char*)"skip", required_argument, 0, 's'},
- {(char*)"12", required_argument, 0, ARG_ONETWO},
- {(char*)"tab5", required_argument, 0, ARG_TAB5},
- {(char*)"tab6", required_argument, 0, ARG_TAB6},
- {(char*)"interleaved", required_argument, 0, ARG_INTERLEAVED_FASTQ},
- {(char*)"phred33-quals", no_argument, 0, ARG_PHRED33},
- {(char*)"phred64-quals", no_argument, 0, ARG_PHRED64},
- {(char*)"phred33", no_argument, 0, ARG_PHRED33},
- {(char*)"phred64", no_argument, 0, ARG_PHRED64},
- {(char*)"solexa1.3-quals", no_argument, 0, ARG_PHRED64},
- {(char*)"mm", no_argument, 0, ARG_MM},
- {(char*)"shmem", no_argument, 0, ARG_SHMEM},
- {(char*)"mmsweep", no_argument, 0, ARG_MMSWEEP},
- {(char*)"hadoopout", no_argument, 0, ARG_HADOOPOUT},
- {(char*)"fullref", no_argument, 0, ARG_FULLREF},
- {(char*)"usage", no_argument, 0, ARG_USAGE},
- {(char*)"sam-no-qname-trunc", no_argument, 0, ARG_SAM_NO_QNAME_TRUNC},
- {(char*)"sam-omit-sec-seq", no_argument, 0, ARG_SAM_OMIT_SEC_SEQ},
- {(char*)"omit-sec-seq", no_argument, 0, ARG_SAM_OMIT_SEC_SEQ},
- {(char*)"sam-no-head", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"sam-nohead", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"sam-noHD", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"sam-no-hd", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"sam-nosq", no_argument, 0, ARG_SAM_NOSQ},
- {(char*)"sam-no-sq", no_argument, 0, ARG_SAM_NOSQ},
- {(char*)"sam-noSQ", no_argument, 0, ARG_SAM_NOSQ},
- {(char*)"no-head", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"no-hd", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"no-sq", no_argument, 0, ARG_SAM_NOSQ},
- {(char*)"no-HD", no_argument, 0, ARG_SAM_NOHEAD},
- {(char*)"no-SQ", no_argument, 0, ARG_SAM_NOSQ},
- {(char*)"no-unal", no_argument, 0, ARG_SAM_NO_UNAL},
- {(char*)"sam-RG", required_argument, 0, ARG_SAM_RG},
- {(char*)"sam-rg", required_argument, 0, ARG_SAM_RG},
- {(char*)"sam-rg-id", required_argument, 0, ARG_SAM_RGID},
- {(char*)"RG", required_argument, 0, ARG_SAM_RG},
- {(char*)"rg", required_argument, 0, ARG_SAM_RG},
- {(char*)"rg-id", required_argument, 0, ARG_SAM_RGID},
- {(char*)"snpphred", required_argument, 0, ARG_SNPPHRED},
- {(char*)"snpfrac", required_argument, 0, ARG_SNPFRAC},
- {(char*)"gbar", required_argument, 0, ARG_GAP_BAR},
- {(char*)"qseq", no_argument, 0, ARG_QSEQ},
- {(char*)"policy", required_argument, 0, ARG_ALIGN_POLICY},
- {(char*)"preset", required_argument, 0, 'P'},
- {(char*)"seed-summ", no_argument, 0, ARG_SEED_SUMM},
- {(char*)"seed-summary", no_argument, 0, ARG_SEED_SUMM},
- {(char*)"overhang", no_argument, 0, ARG_OVERHANG},
- {(char*)"no-cache", no_argument, 0, ARG_NO_CACHE},
- {(char*)"cache", no_argument, 0, ARG_USE_CACHE},
- {(char*)"454", no_argument, 0, ARG_NOISY_HPOLY},
- {(char*)"ion-torrent", no_argument, 0, ARG_NOISY_HPOLY},
- {(char*)"no-mixed", no_argument, 0, ARG_NO_MIXED},
- {(char*)"no-discordant",no_argument, 0, ARG_NO_DISCORDANT},
- {(char*)"local", no_argument, 0, ARG_LOCAL},
- {(char*)"end-to-end", no_argument, 0, ARG_END_TO_END},
- {(char*)"ungapped", no_argument, 0, ARG_UNGAPPED},
- {(char*)"no-ungapped", no_argument, 0, ARG_UNGAPPED_NO},
- {(char*)"sse8", no_argument, 0, ARG_SSE8},
- {(char*)"no-sse8", no_argument, 0, ARG_SSE8_NO},
- {(char*)"scan-narrowed",no_argument, 0, ARG_SCAN_NARROWED},
- {(char*)"qc-filter", no_argument, 0, ARG_QC_FILTER},
- {(char*)"bwa-sw-like", no_argument, 0, ARG_BWA_SW_LIKE},
- {(char*)"multiseed", required_argument, 0, ARG_MULTISEED_IVAL},
- {(char*)"ma", required_argument, 0, ARG_SCORE_MA},
- {(char*)"mp", required_argument, 0, ARG_SCORE_MMP},
- {(char*)"np", required_argument, 0, ARG_SCORE_NP},
- {(char*)"rdg", required_argument, 0, ARG_SCORE_RDG},
- {(char*)"rfg", required_argument, 0, ARG_SCORE_RFG},
- {(char*)"score-min", required_argument, 0, ARG_SCORE_MIN},
- {(char*)"min-score", required_argument, 0, ARG_SCORE_MIN},
- {(char*)"n-ceil", required_argument, 0, ARG_N_CEIL},
- {(char*)"dpad", required_argument, 0, ARG_DPAD},
- {(char*)"mapq-print-inputs",no_argument, 0, ARG_SAM_PRINT_YI},
- {(char*)"very-fast", no_argument, 0, ARG_PRESET_VERY_FAST},
- {(char*)"fast", no_argument, 0, ARG_PRESET_FAST},
- {(char*)"sensitive", no_argument, 0, ARG_PRESET_SENSITIVE},
- {(char*)"very-sensitive", no_argument, 0, ARG_PRESET_VERY_SENSITIVE},
- {(char*)"very-fast-local", no_argument, 0, ARG_PRESET_VERY_FAST_LOCAL},
- {(char*)"fast-local", no_argument, 0, ARG_PRESET_FAST_LOCAL},
- {(char*)"sensitive-local", no_argument, 0, ARG_PRESET_SENSITIVE_LOCAL},
- {(char*)"very-sensitive-local", no_argument, 0, ARG_PRESET_VERY_SENSITIVE_LOCAL},
- {(char*)"seedlen", required_argument, 0, 'L'},
- {(char*)"seedmms", required_argument, 0, 'N'},
- {(char*)"seedival", required_argument, 0, 'i'},
- {(char*)"ignore-quals", no_argument, 0, ARG_IGNORE_QUALS},
- {(char*)"index", required_argument, 0, 'x'},
- {(char*)"arg-desc", no_argument, 0, ARG_DESC},
- {(char*)"wrapper", required_argument, 0, ARG_WRAPPER},
- {(char*)"unpaired", required_argument, 0, 'U'},
- {(char*)"output", required_argument, 0, 'S'},
- {(char*)"mapq-v", required_argument, 0, ARG_MAPQ_V},
- {(char*)"dovetail", no_argument, 0, ARG_DOVETAIL},
- {(char*)"no-dovetail", no_argument, 0, ARG_NO_DOVETAIL},
- {(char*)"contain", no_argument, 0, ARG_CONTAIN},
- {(char*)"no-contain", no_argument, 0, ARG_NO_CONTAIN},
- {(char*)"overlap", no_argument, 0, ARG_OVERLAP},
- {(char*)"no-overlap", no_argument, 0, ARG_NO_OVERLAP},
- {(char*)"tighten", required_argument, 0, ARG_TIGHTEN},
- {(char*)"exact-upfront", no_argument, 0, ARG_EXACT_UPFRONT},
- {(char*)"1mm-upfront", no_argument, 0, ARG_1MM_UPFRONT},
- {(char*)"no-exact-upfront", no_argument, 0, ARG_EXACT_UPFRONT_NO},
- {(char*)"no-1mm-upfront", no_argument, 0, ARG_1MM_UPFRONT_NO},
- {(char*)"1mm-minlen", required_argument, 0, ARG_1MM_MINLEN},
- {(char*)"seed-off", required_argument, 0, 'O'},
- {(char*)"seed-boost", required_argument, 0, ARG_SEED_BOOST_THRESH},
- {(char*)"read-times", no_argument, 0, ARG_READ_TIMES},
- {(char*)"show-rand-seed", no_argument, 0, ARG_SHOW_RAND_SEED},
- {(char*)"dp-fail-streak", required_argument, 0, ARG_DP_FAIL_STREAK_THRESH},
- {(char*)"ee-fail-streak", required_argument, 0, ARG_EE_FAIL_STREAK_THRESH},
- {(char*)"ug-fail-streak", required_argument, 0, ARG_UG_FAIL_STREAK_THRESH},
- {(char*)"fail-streak", required_argument, 0, 'D'},
- {(char*)"dp-fails", required_argument, 0, ARG_DP_FAIL_THRESH},
- {(char*)"ug-fails", required_argument, 0, ARG_UG_FAIL_THRESH},
- {(char*)"extends", required_argument, 0, ARG_EXTEND_ITERS},
- {(char*)"no-extend", no_argument, 0, ARG_NO_EXTEND},
- {(char*)"mapq-extra", no_argument, 0, ARG_MAPQ_EX},
- {(char*)"seed-rounds", required_argument, 0, 'R'},
- {(char*)"reorder", no_argument, 0, ARG_REORDER},
- {(char*)"passthrough", no_argument, 0, ARG_READ_PASSTHRU},
- {(char*)"sample", required_argument, 0, ARG_SAMPLE},
- {(char*)"cp-min", required_argument, 0, ARG_CP_MIN},
- {(char*)"cp-ival", required_argument, 0, ARG_CP_IVAL},
- {(char*)"tri", no_argument, 0, ARG_TRI},
- {(char*)"nondeterministic", no_argument, 0, ARG_NON_DETERMINISTIC},
- {(char*)"non-deterministic", no_argument, 0, ARG_NON_DETERMINISTIC},
- {(char*)"local-seed-cache-sz", required_argument, 0, ARG_LOCAL_SEED_CACHE_SZ},
- {(char*)"seed-cache-sz", required_argument, 0, ARG_CURRENT_SEED_CACHE_SZ},
- {(char*)"no-unal", no_argument, 0, ARG_SAM_NO_UNAL},
- {(char*)"test-25", no_argument, 0, ARG_TEST_25},
- // TODO: following should be a function of read length?
- {(char*)"desc-kb", required_argument, 0, ARG_DESC_KB},
- {(char*)"desc-landing", required_argument, 0, ARG_DESC_LANDING},
- {(char*)"desc-exp", required_argument, 0, ARG_DESC_EXP},
- {(char*)"desc-prioritize", no_argument, 0, ARG_DESC_PRIORITIZE},
- {(char*)"desc-fmops", required_argument, 0, ARG_DESC_FMOPS},
- {(char*)"log-dp", required_argument, 0, ARG_LOG_DP},
- {(char*)"log-dp-opp", required_argument, 0, ARG_LOG_DP_OPP},
- {(char*)0, 0, 0, 0} // terminator
+{(char*)"verbose", no_argument, 0, ARG_VERBOSE},
+{(char*)"startverbose", no_argument, 0, ARG_STARTVERBOSE},
+{(char*)"quiet", no_argument, 0, ARG_QUIET},
+{(char*)"sanity", no_argument, 0, ARG_SANITY},
+{(char*)"pause", no_argument, &ipause, 1},
+{(char*)"orig", required_argument, 0, ARG_ORIG},
+{(char*)"all", no_argument, 0, 'a'},
+{(char*)"solexa-quals", no_argument, 0, ARG_SOLEXA_QUALS},
+{(char*)"integer-quals", no_argument, 0, ARG_INTEGER_QUALS},
+{(char*)"int-quals", no_argument, 0, ARG_INTEGER_QUALS},
+{(char*)"metrics", required_argument, 0, ARG_METRIC_IVAL},
+{(char*)"metrics-file", required_argument, 0, ARG_METRIC_FILE},
+{(char*)"metrics-stderr", no_argument, 0, ARG_METRIC_STDERR},
+{(char*)"metrics-per-read", no_argument, 0, ARG_METRIC_PER_READ},
+{(char*)"met-read", no_argument, 0, ARG_METRIC_PER_READ},
+{(char*)"met", required_argument, 0, ARG_METRIC_IVAL},
+{(char*)"met-file", required_argument, 0, ARG_METRIC_FILE},
+{(char*)"met-stderr", no_argument, 0, ARG_METRIC_STDERR},
+{(char*)"time", no_argument, 0, 't'},
+{(char*)"trim3", required_argument, 0, '3'},
+{(char*)"trim5", required_argument, 0, '5'},
+{(char*)"seed", required_argument, 0, ARG_SEED},
+{(char*)"qupto", required_argument, 0, 'u'},
+{(char*)"upto", required_argument, 0, 'u'},
+{(char*)"version", no_argument, 0, ARG_VERSION},
+{(char*)"reads-per-batch", required_argument, 0, ARG_READS_PER_BATCH},
+{(char*)"filepar", no_argument, 0, ARG_FILEPAR},
+{(char*)"help", no_argument, 0, 'h'},
+{(char*)"threads", required_argument, 0, 'p'},
+{(char*)"khits", required_argument, 0, 'k'},
+{(char*)"minins", required_argument, 0, 'I'},
+{(char*)"maxins", required_argument, 0, 'X'},
+{(char*)"quals", required_argument, 0, 'Q'},
+{(char*)"Q1", required_argument, 0, ARG_QUALS1},
+{(char*)"Q2", required_argument, 0, ARG_QUALS2},
+{(char*)"refidx", no_argument, 0, ARG_REFIDX},
+{(char*)"partition", required_argument, 0, ARG_PARTITION},
+{(char*)"ff", no_argument, 0, ARG_FF},
+{(char*)"fr", no_argument, 0, ARG_FR},
+{(char*)"rf", no_argument, 0, ARG_RF},
+{(char*)"cachelim", required_argument, 0, ARG_CACHE_LIM},
+{(char*)"cachesz", required_argument, 0, ARG_CACHE_SZ},
+{(char*)"nofw", no_argument, 0, ARG_NO_FW},
+{(char*)"norc", no_argument, 0, ARG_NO_RC},
+{(char*)"skip", required_argument, 0, 's'},
+{(char*)"12", required_argument, 0, ARG_ONETWO},
+{(char*)"tab5", required_argument, 0, ARG_TAB5},
+{(char*)"tab6", required_argument, 0, ARG_TAB6},
+{(char*)"interleaved", required_argument, 0, ARG_INTERLEAVED_FASTQ},
+{(char*)"phred33-quals", no_argument, 0, ARG_PHRED33},
+{(char*)"phred64-quals", no_argument, 0, ARG_PHRED64},
+{(char*)"phred33", no_argument, 0, ARG_PHRED33},
+{(char*)"phred64", no_argument, 0, ARG_PHRED64},
+{(char*)"solexa1.3-quals", no_argument, 0, ARG_PHRED64},
+{(char*)"mm", no_argument, 0, ARG_MM},
+{(char*)"shmem", no_argument, 0, ARG_SHMEM},
+{(char*)"mmsweep", no_argument, 0, ARG_MMSWEEP},
+{(char*)"hadoopout", no_argument, 0, ARG_HADOOPOUT},
+{(char*)"fullref", no_argument, 0, ARG_FULLREF},
+{(char*)"usage", no_argument, 0, ARG_USAGE},
+{(char*)"sam-no-qname-trunc", no_argument, 0, ARG_SAM_NO_QNAME_TRUNC},
+{(char*)"sam-omit-sec-seq", no_argument, 0, ARG_SAM_OMIT_SEC_SEQ},
+{(char*)"omit-sec-seq", no_argument, 0, ARG_SAM_OMIT_SEC_SEQ},
+{(char*)"sam-no-head", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"sam-nohead", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"sam-noHD", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"sam-no-hd", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"sam-nosq", no_argument, 0, ARG_SAM_NOSQ},
+{(char*)"sam-no-sq", no_argument, 0, ARG_SAM_NOSQ},
+{(char*)"sam-noSQ", no_argument, 0, ARG_SAM_NOSQ},
+{(char*)"no-head", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"no-hd", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"no-sq", no_argument, 0, ARG_SAM_NOSQ},
+{(char*)"no-HD", no_argument, 0, ARG_SAM_NOHEAD},
+{(char*)"no-SQ", no_argument, 0, ARG_SAM_NOSQ},
+{(char*)"no-unal", no_argument, 0, ARG_SAM_NO_UNAL},
+{(char*)"sam-RG", required_argument, 0, ARG_SAM_RG},
+{(char*)"sam-rg", required_argument, 0, ARG_SAM_RG},
+{(char*)"sam-rg-id", required_argument, 0, ARG_SAM_RGID},
+{(char*)"RG", required_argument, 0, ARG_SAM_RG},
+{(char*)"rg", required_argument, 0, ARG_SAM_RG},
+{(char*)"rg-id", required_argument, 0, ARG_SAM_RGID},
+{(char*)"snpphred", required_argument, 0, ARG_SNPPHRED},
+{(char*)"snpfrac", required_argument, 0, ARG_SNPFRAC},
+{(char*)"gbar", required_argument, 0, ARG_GAP_BAR},
+{(char*)"qseq", no_argument, 0, ARG_QSEQ},
+{(char*)"policy", required_argument, 0, ARG_ALIGN_POLICY},
+{(char*)"preset", required_argument, 0, 'P'},
+{(char*)"seed-summ", no_argument, 0, ARG_SEED_SUMM},
+{(char*)"seed-summary", no_argument, 0, ARG_SEED_SUMM},
+{(char*)"overhang", no_argument, 0, ARG_OVERHANG},
+{(char*)"no-cache", no_argument, 0, ARG_NO_CACHE},
+{(char*)"cache", no_argument, 0, ARG_USE_CACHE},
+{(char*)"454", no_argument, 0, ARG_NOISY_HPOLY},
+{(char*)"ion-torrent", no_argument, 0, ARG_NOISY_HPOLY},
+{(char*)"no-mixed", no_argument, 0, ARG_NO_MIXED},
+{(char*)"no-discordant", no_argument, 0, ARG_NO_DISCORDANT},
+{(char*)"local", no_argument, 0, ARG_LOCAL},
+{(char*)"end-to-end", no_argument, 0, ARG_END_TO_END},
+{(char*)"ungapped", no_argument, 0, ARG_UNGAPPED},
+{(char*)"no-ungapped", no_argument, 0, ARG_UNGAPPED_NO},
+{(char*)"sse8", no_argument, 0, ARG_SSE8},
+{(char*)"no-sse8", no_argument, 0, ARG_SSE8_NO},
+{(char*)"scan-narrowed", no_argument, 0, ARG_SCAN_NARROWED},
+{(char*)"qc-filter", no_argument, 0, ARG_QC_FILTER},
+{(char*)"bwa-sw-like", no_argument, 0, ARG_BWA_SW_LIKE},
+{(char*)"multiseed", required_argument, 0, ARG_MULTISEED_IVAL},
+{(char*)"ma", required_argument, 0, ARG_SCORE_MA},
+{(char*)"mp", required_argument, 0, ARG_SCORE_MMP},
+{(char*)"np", required_argument, 0, ARG_SCORE_NP},
+{(char*)"rdg", required_argument, 0, ARG_SCORE_RDG},
+{(char*)"rfg", required_argument, 0, ARG_SCORE_RFG},
+{(char*)"score-min", required_argument, 0, ARG_SCORE_MIN},
+{(char*)"min-score", required_argument, 0, ARG_SCORE_MIN},
+{(char*)"n-ceil", required_argument, 0, ARG_N_CEIL},
+{(char*)"dpad", required_argument, 0, ARG_DPAD},
+{(char*)"mapq-print-inputs", no_argument, 0, ARG_SAM_PRINT_YI},
+{(char*)"very-fast", no_argument, 0, ARG_PRESET_VERY_FAST},
+{(char*)"fast", no_argument, 0, ARG_PRESET_FAST},
+{(char*)"sensitive", no_argument, 0, ARG_PRESET_SENSITIVE},
+{(char*)"very-sensitive", no_argument, 0, ARG_PRESET_VERY_SENSITIVE},
+{(char*)"very-fast-local", no_argument, 0, ARG_PRESET_VERY_FAST_LOCAL},
+{(char*)"fast-local", no_argument, 0, ARG_PRESET_FAST_LOCAL},
+{(char*)"sensitive-local", no_argument, 0, ARG_PRESET_SENSITIVE_LOCAL},
+{(char*)"very-sensitive-local", no_argument, 0, ARG_PRESET_VERY_SENSITIVE_LOCAL},
+{(char*)"seedlen", required_argument, 0, 'L'},
+{(char*)"seedmms", required_argument, 0, 'N'},
+{(char*)"seedival", required_argument, 0, 'i'},
+{(char*)"ignore-quals", no_argument, 0, ARG_IGNORE_QUALS},
+{(char*)"index", required_argument, 0, 'x'},
+{(char*)"arg-desc", no_argument, 0, ARG_DESC},
+{(char*)"wrapper", required_argument, 0, ARG_WRAPPER},
+{(char*)"unpaired", required_argument, 0, 'U'},
+{(char*)"output", required_argument, 0, 'S'},
+{(char*)"mapq-v", required_argument, 0, ARG_MAPQ_V},
+{(char*)"dovetail", no_argument, 0, ARG_DOVETAIL},
+{(char*)"no-dovetail", no_argument, 0, ARG_NO_DOVETAIL},
+{(char*)"contain", no_argument, 0, ARG_CONTAIN},
+{(char*)"no-contain", no_argument, 0, ARG_NO_CONTAIN},
+{(char*)"overlap", no_argument, 0, ARG_OVERLAP},
+{(char*)"no-overlap", no_argument, 0, ARG_NO_OVERLAP},
+{(char*)"tighten", required_argument, 0, ARG_TIGHTEN},
+{(char*)"exact-upfront", no_argument, 0, ARG_EXACT_UPFRONT},
+{(char*)"1mm-upfront", no_argument, 0, ARG_1MM_UPFRONT},
+{(char*)"no-exact-upfront", no_argument, 0, ARG_EXACT_UPFRONT_NO},
+{(char*)"no-1mm-upfront", no_argument, 0, ARG_1MM_UPFRONT_NO},
+{(char*)"1mm-minlen", required_argument, 0, ARG_1MM_MINLEN},
+{(char*)"seed-off", required_argument, 0, 'O'},
+{(char*)"seed-boost", required_argument, 0, ARG_SEED_BOOST_THRESH},
+{(char*)"read-times", no_argument, 0, ARG_READ_TIMES},
+{(char*)"show-rand-seed", no_argument, 0, ARG_SHOW_RAND_SEED},
+{(char*)"dp-fail-streak", required_argument, 0, ARG_DP_FAIL_STREAK_THRESH},
+{(char*)"ee-fail-streak", required_argument, 0, ARG_EE_FAIL_STREAK_THRESH},
+{(char*)"ug-fail-streak", required_argument, 0, ARG_UG_FAIL_STREAK_THRESH},
+{(char*)"fail-streak", required_argument, 0, 'D'},
+{(char*)"dp-fails", required_argument, 0, ARG_DP_FAIL_THRESH},
+{(char*)"ug-fails", required_argument, 0, ARG_UG_FAIL_THRESH},
+{(char*)"extends", required_argument, 0, ARG_EXTEND_ITERS},
+{(char*)"no-extend", no_argument, 0, ARG_NO_EXTEND},
+{(char*)"mapq-extra", no_argument, 0, ARG_MAPQ_EX},
+{(char*)"seed-rounds", required_argument, 0, 'R'},
+{(char*)"reorder", no_argument, 0, ARG_REORDER},
+{(char*)"passthrough", no_argument, 0, ARG_READ_PASSTHRU},
+{(char*)"sample", required_argument, 0, ARG_SAMPLE},
+{(char*)"cp-min", required_argument, 0, ARG_CP_MIN},
+{(char*)"cp-ival", required_argument, 0, ARG_CP_IVAL},
+{(char*)"tri", no_argument, 0, ARG_TRI},
+{(char*)"nondeterministic", no_argument, 0, ARG_NON_DETERMINISTIC},
+{(char*)"non-deterministic", no_argument, 0, ARG_NON_DETERMINISTIC},
+{(char*)"local-seed-cache-sz", required_argument, 0, ARG_LOCAL_SEED_CACHE_SZ},
+{(char*)"seed-cache-sz", required_argument, 0, ARG_CURRENT_SEED_CACHE_SZ},
+{(char*)"no-unal", no_argument, 0, ARG_SAM_NO_UNAL},
+{(char*)"test-25", no_argument, 0, ARG_TEST_25},
+// TODO: following should be a function of read length?
+{(char*)"desc-kb", required_argument, 0, ARG_DESC_KB},
+{(char*)"desc-landing", required_argument, 0, ARG_DESC_LANDING},
+{(char*)"desc-exp", required_argument, 0, ARG_DESC_EXP},
+{(char*)"desc-prioritize", no_argument, 0, ARG_DESC_PRIORITIZE},
+{(char*)"desc-fmops", required_argument, 0, ARG_DESC_FMOPS},
+{(char*)"log-dp", required_argument, 0, ARG_LOG_DP},
+{(char*)"log-dp-opp", required_argument, 0, ARG_LOG_DP_OPP},
+{(char*)"soft-clipped-unmapped-tlen", no_argument, 0, ARG_SC_UNMAPPED},
+{(char*)"xeq", no_argument, 0, ARG_XEQ},
+{(char*)"thread-ceiling", required_argument, 0, ARG_THREAD_CEILING},
+{(char*)"thread-piddir", required_argument, 0, ARG_THREAD_PIDDIR},
+{(char*)0, 0, 0, 0} // terminator
};
/**
@@ -770,10 +799,10 @@ static void printUsage(ostream& out) {
//}
out << " -t/--time print wall-clock time taken by search phases" << endl;
if(wrapper == "basic-0") {
- out << " --un <path> write unpaired reads that didn't align to <path>" << endl
- << " --al <path> write unpaired reads that aligned at least once to <path>" << endl
- << " --un-conc <path> write pairs that didn't align concordantly to <path>" << endl
- << " --al-conc <path> write pairs that aligned concordantly at least once to <path>" << endl
+ out << " --un <path> write unpaired reads that didn't align to <path>" << endl
+ << " --al <path> write unpaired reads that aligned at least once to <path>" << endl
+ << " --un-conc <path> write pairs that didn't align concordantly to <path>" << endl
+ << " --al-conc <path> write pairs that aligned concordantly at least once to <path>" << endl
<< " (Note: for --un, --al, --un-conc, or --al-conc, add '-gz' to the option name, e.g." << endl
<< " --un-gz <path>, to gzip compress output, or add '-bz2' to bzip2 compress output.)" << endl;
}
@@ -792,6 +821,8 @@ static void printUsage(ostream& out) {
<< " --omit-sec-seq put '*' in SEQ and QUAL fields for secondary alignments." << endl
<< " --sam-noqname-trunc Suppress standard behavior of truncating readname at first whitespace " << endl
<< " at the expense of generating non-standard SAM." << endl
+ << " --xeq Use '='/'X', instead of 'M,' to specify matches/mismatches in SAM record." << endl
+ << " --soft-clipped-unmapped-tlen Exclude soft-clipped bases when reporting TLEN" << endl
<< endl
<< " Performance:" << endl
// << " -o/--offrate <int> override offrate of index; must be >= index's offrate" << endl
@@ -986,6 +1017,8 @@ static void parseOption(int next_option, const char *arg) {
case ARG_FR: gMate1fw = true; gMate2fw = false; break;
case ARG_SHMEM: useShmem = true; break;
case ARG_SEED_SUMM: seedSumm = true; break;
+ case ARG_SC_UNMAPPED: scUnMapped = true; break;
+ case ARG_XEQ: xeq = true; break;
case ARG_MM: {
#ifdef BOWTIE_MM
useMm = true;
@@ -1050,6 +1083,12 @@ static void parseOption(int next_option, const char *arg) {
case 'p':
nthreads = parseInt(1, "-p/--threads arg must be at least 1", arg);
break;
+ case ARG_THREAD_CEILING:
+ thread_ceiling = parseInt(0, "--thread-ceiling must be at least 0", arg);
+ break;
+ case ARG_THREAD_PIDDIR:
+ thread_stealing_dir = arg;
+ break;
case ARG_FILEPAR:
fileParallel = true;
break;
@@ -1280,25 +1319,23 @@ static void parseOption(int next_option, const char *arg) {
break;
}
case 'N': {
- if (!gSeedLenIsSet){
- polstr += ";SEED=";
- polstr += arg;
+ int64_t len = parse<size_t>(arg);
+ if (len < 0 || len > 1) {
+ cerr << "Error: -N argument must be within the interval [0,1]; was " << arg << endl;
+ throw 1;
}
+ polstr += ";SEED=";
+ polstr += arg;
break;
}
case 'L': {
int64_t len = parse<size_t>(arg);
- if(len < 0) {
- cerr << "Error: -L argument must be >= 0; was " << arg << endl;
- throw 1;
- }
- if(len > 32) {
- cerr << "Error: -L argument must be <= 32; was " << arg << endl;
+ if(len < 1 || len > 32) {
+ cerr << "Error: -L argument must be within the interval [1,32]; was " << arg << endl;
throw 1;
}
polstr += ";SEEDLEN=";
polstr += arg;
- gSeedLenIsSet = true;
break;
}
case 'O':
@@ -1337,7 +1374,7 @@ static void parseOption(int next_option, const char *arg) {
// Seed mm and length arguments
polstr += "SEED=";
polstr += (args[0]); // # mismatches
- if(args.size() > 1) polstr += ("," + args[ 1]); // length
+ if(args.size() > 1) polstr += (";SEEDLEN=" + args[1]); // length
if(args.size() > 2) polstr += (";IVAL=" + args[2]); // Func type
if(args.size() > 3) polstr += ("," + args[ 3]); // Constant term
if(args.size() > 4) polstr += ("," + args[ 4]); // Coefficient
@@ -1424,6 +1461,12 @@ static void parseOption(int next_option, const char *arg) {
printUsage(cerr);
throw 1;
}
+ if (!localAlign && scUnMapped) {
+ scUnMapped = false;
+ cerr << "WARNING: --soft-clipped-unmapped-tlen can only be set for "
+ << "local alignment... ignoring" << endl;
+ }
+
}
/**
@@ -1652,11 +1695,7 @@ struct OuterLoopMetrics {
* is the only safe way to update an OuterLoopMetrics that's shared
* by multiple threads.
*/
- void merge(
- const OuterLoopMetrics& m,
- bool getLock = false)
- {
- ThreadSafe ts(&mutex_m, getLock);
+ void merge(const OuterLoopMetrics& m) {
reads += m.reads;
bases += m.bases;
srreads += m.srreads;
@@ -1735,39 +1774,38 @@ struct PerfMetrics {
const SSEMetrics *dpSse16Ma,
uint64_t nbtfiltst_,
uint64_t nbtfiltsc_,
- uint64_t nbtfiltdo_,
- bool getLock)
+ uint64_t nbtfiltdo_)
{
- ThreadSafe ts(&mutex_m, getLock);
+ ThreadSafe ts(mutex_m);
if(ol != NULL) {
- olmu.merge(*ol, false);
+ olmu.merge(*ol);
}
if(sd != NULL) {
- sdmu.merge(*sd, false);
+ sdmu.merge(*sd);
}
if(wl != NULL) {
- wlmu.merge(*wl, false);
+ wlmu.merge(*wl);
}
if(swSeed != NULL) {
- swmuSeed.merge(*swSeed, false);
+ swmuSeed.merge(*swSeed);
}
if(swMate != NULL) {
- swmuMate.merge(*swMate, false);
+ swmuMate.merge(*swMate);
}
if(rm != NULL) {
- rpmu.merge(*rm, false);
+ rpmu.merge(*rm);
}
if(dpSse8Ex != NULL) {
- dpSse8uSeed.merge(*dpSse8Ex, false);
+ dpSse8uSeed.merge(*dpSse8Ex);
}
if(dpSse8Ma != NULL) {
- dpSse8uMate.merge(*dpSse8Ma, false);
+ dpSse8uMate.merge(*dpSse8Ma);
}
if(dpSse16Ex != NULL) {
- dpSse16uSeed.merge(*dpSse16Ex, false);
+ dpSse16uSeed.merge(*dpSse16Ex);
}
if(dpSse16Ma != NULL) {
- dpSse16uMate.merge(*dpSse16Ma, false);
+ dpSse16uMate.merge(*dpSse16Ma);
}
nbtfiltst_u += nbtfiltst_;
nbtfiltsc_u += nbtfiltsc_;
@@ -1783,10 +1821,9 @@ struct PerfMetrics {
OutFileBuf* o, // file to send output to
bool metricsStderr, // additionally output to stderr?
bool total, // true -> report total, otherwise incremental
- bool sync, // synchronize output
const BTString *name) // non-NULL name pointer if is per-read record
{
- ThreadSafe ts(&mutex_m, sync);
+ ThreadSafe ts(mutex_m);
ostringstream stderrSs;
time_t curtime = time(0);
char buf[1024];
@@ -2519,15 +2556,15 @@ struct PerfMetrics {
}
void mergeIncrementals() {
- olm.merge(olmu, false);
- sdm.merge(sdmu, false);
- wlm.merge(wlmu, false);
- swmSeed.merge(swmuSeed, false);
- swmMate.merge(swmuMate, false);
- dpSse8Seed.merge(dpSse8uSeed, false);
- dpSse8Mate.merge(dpSse8uMate, false);
- dpSse16Seed.merge(dpSse16uSeed, false);
- dpSse16Mate.merge(dpSse16uMate, false);
+ olm.merge(olmu);
+ sdm.merge(sdmu);
+ wlm.merge(wlmu);
+ swmSeed.merge(swmuSeed);
+ swmMate.merge(swmuMate);
+ dpSse8Seed.merge(dpSse8uSeed);
+ dpSse8Mate.merge(dpSse8uMate);
+ dpSse16Seed.merge(dpSse16uSeed);
+ dpSse16Mate.merge(dpSse16uMate);
nbtfiltst_u += nbtfiltst;
nbtfiltsc_u += nbtfiltsc;
nbtfiltdo_u += nbtfiltdo;
@@ -2733,7 +2770,7 @@ static void setupMinScores(
}
}
-#define MERGE_METRICS(met, sync) { \
+#define MERGE_METRICS(met) { \
msink.mergeMetrics(rpm); \
met.merge( \
&olm, \
@@ -2748,8 +2785,7 @@ static void setupMinScores(
&sseI16MateMet, \
nbtfiltst, \
nbtfiltsc, \
- nbtfiltdo, \
- sync); \
+ nbtfiltdo); \
olm.reset(); \
sdm.reset(); \
wlm.reset(); \
@@ -2784,6 +2820,27 @@ void get_cpu_and_node(int& cpu, int& node) {
}
#endif
+class ThreadCounter {
+public:
+ ThreadCounter() {
+#ifdef WITH_TBB
+ thread_counter.fetch_and_increment();
+#else
+ ThreadSafe ts(thread_counter_mutex);
+ thread_counter++;
+#endif
+ }
+
+ ~ThreadCounter() {
+#ifdef WITH_TBB
+ thread_counter.fetch_and_decrement();
+#else
+ ThreadSafe ts(thread_counter_mutex);
+ thread_counter--;
+#endif
+ }
+};
+
/**
* Called once per thread. Sets up per-thread pointers to the shared global
* data structures, creates per-thread structures, then enters the alignment
@@ -2800,7 +2857,11 @@ void get_cpu_and_node(int& cpu, int& node) {
* -
*/
#ifdef WITH_TBB
-void multiseedSearchWorker::operator()() const {
+//void multiseedSearchWorker::operator()() const {
+static void multiseedSearchWorker(void *vp) {
+ //int tid = *((int*)vp);
+ thread_tracking_pair *p = (thread_tracking_pair*) vp;
+ int tid = p->tid;
#else
static void multiseedSearchWorker(void *vp) {
int tid = *((int*)vp);
@@ -2817,900 +2878,625 @@ static void multiseedSearchWorker(void *vp) {
AlnSink& msink = *multiseed_msink;
OutFileBuf* metricsOfb = multiseed_metricsOfb;
+ {
#ifdef PER_THREAD_TIMING
- uint64_t ncpu_changeovers = 0;
- uint64_t nnuma_changeovers = 0;
-
- int current_cpu = 0, current_node = 0;
- get_cpu_and_node(current_cpu, current_node);
-
- std::stringstream ss;
- std::string msg;
- ss << "thread: " << tid << " time: ";
- msg = ss.str();
- Timer timer(std::cout, msg.c_str());
+ uint64_t ncpu_changeovers = 0;
+ uint64_t nnuma_changeovers = 0;
+
+ int current_cpu = 0, current_node = 0;
+ get_cpu_and_node(current_cpu, current_node);
+
+ std::stringstream ss;
+ std::string msg;
+ ss << "thread: " << tid << " time: ";
+ msg = ss.str();
+ Timer timer(std::cout, msg.c_str());
#endif
- // Sinks: these are so that we can print tables encoding counts for
- // events of interest on a per-read, per-seed, per-join, or per-SW
- // level. These in turn can be used to diagnose performance
- // problems, or generally characterize performance.
-
- //const BitPairReference& refs = *multiseed_refs;
- auto_ptr<PatternSourcePerThreadFactory> patsrcFact(createPatsrcFactory(patsrc, pp, tid));
- auto_ptr<PatternSourcePerThread> ps(patsrcFact->create());
-
- // Thread-local cache for seed alignments
- PtrWrap<AlignmentCache> scLocal;
- if(!msNoCache) {
- scLocal.init(new AlignmentCache(seedCacheLocalMB * 1024 * 1024, false));
- }
- AlignmentCache scCurrent(seedCacheCurrentMB * 1024 * 1024, false);
- // Thread-local cache for current seed alignments
-
- // Interfaces for alignment and seed caches
- AlignmentCacheIface ca(
- &scCurrent,
- scLocal.get(),
- msNoCache ? NULL : &scShared);
-
- // Instantiate an object for holding reporting-related parameters.
- ReportingParams rp(
- (allHits ? std::numeric_limits<THitInt>::max() : khits), // -k
- mhits, // -m/-M
- 0, // penalty gap (not used now)
- msample, // true -> -M was specified, otherwise assume -m
- gReportDiscordant, // report discordang paired-end alignments?
- gReportMixed); // report unpaired alignments for paired reads?
+ // Sinks: these are so that we can print tables encoding counts for
+ // events of interest on a per-read, per-seed, per-join, or per-SW
+ // level. These in turn can be used to diagnose performance
+ // problems, or generally characterize performance.
+
+ //const BitPairReference& refs = *multiseed_refs;
+ auto_ptr<PatternSourcePerThreadFactory> patsrcFact(createPatsrcFactory(patsrc, pp, tid));
+ auto_ptr<PatternSourcePerThread> ps(patsrcFact->create());
+
+ // Thread-local cache for seed alignments
+ PtrWrap<AlignmentCache> scLocal;
+ if(!msNoCache) {
+ scLocal.init(new AlignmentCache(seedCacheLocalMB * 1024 * 1024, false));
+ }
+ AlignmentCache scCurrent(seedCacheCurrentMB * 1024 * 1024, false);
+ // Thread-local cache for current seed alignments
+
+ // Interfaces for alignment and seed caches
+ AlignmentCacheIface ca(
+ &scCurrent,
+ scLocal.get(),
+ msNoCache ? NULL : &scShared);
+
+ // Instantiate an object for holding reporting-related parameters.
+ ReportingParams rp(
+ (allHits ? std::numeric_limits<THitInt>::max() : khits), // -k
+ mhits, // -m/-M
+ 0, // penalty gap (not used now)
+ msample, // true -> -M was specified, otherwise assume -m
+ gReportDiscordant, // report discordang paired-end alignments?
+ gReportMixed); // report unpaired alignments for paired reads?
- // Instantiate a mapping quality calculator
- auto_ptr<Mapq> bmapq(new_mapq(mapqv, scoreMin, sc));
-
- // Make a per-thread wrapper for the global MHitSink object.
- AlnSinkWrap msinkwrap(
- msink, // global sink
- rp, // reporting parameters
- *bmapq, // MAPQ calculator
- (size_t)tid); // thread id
-
- // Write dynamic-programming problem descriptions here
- ofstream *dpLog = NULL, *dpLogOpp = NULL;
- if(!logDps.empty()) {
- dpLog = new ofstream(logDps.c_str(), ofstream::out);
- dpLog->sync_with_stdio(false);
- }
- if(!logDpsOpp.empty()) {
- dpLogOpp = new ofstream(logDpsOpp.c_str(), ofstream::out);
- dpLogOpp->sync_with_stdio(false);
- }
-
- SeedAligner al;
- SwDriver sd(exactCacheCurrentMB * 1024 * 1024);
- SwAligner sw(dpLog), osw(dpLogOpp);
- SeedResults shs[2];
- OuterLoopMetrics olm;
- SeedSearchMetrics sdm;
- WalkMetrics wlm;
- SwMetrics swmSeed, swmMate;
- ReportingMetrics rpm;
- RandomSource rnd, rndArb;
- SSEMetrics sseU8ExtendMet;
- SSEMetrics sseU8MateMet;
- SSEMetrics sseI16ExtendMet;
- SSEMetrics sseI16MateMet;
- uint64_t nbtfiltst = 0; // TODO: find a new home for these
- uint64_t nbtfiltsc = 0; // TODO: find a new home for these
- uint64_t nbtfiltdo = 0; // TODO: find a new home for these
+ // Instantiate a mapping quality calculator
+ auto_ptr<Mapq> bmapq(new_mapq(mapqv, scoreMin, sc));
+
+ // Make a per-thread wrapper for the global MHitSink object.
+ AlnSinkWrap msinkwrap(
+ msink, // global sink
+ rp, // reporting parameters
+ *bmapq, // MAPQ calculator
+ (size_t)tid); // thread id
+
+ // Write dynamic-programming problem descriptions here
+ ofstream *dpLog = NULL, *dpLogOpp = NULL;
+ if(!logDps.empty()) {
+ dpLog = new ofstream(logDps.c_str(), ofstream::out);
+ dpLog->sync_with_stdio(false);
+ }
+ if(!logDpsOpp.empty()) {
+ dpLogOpp = new ofstream(logDpsOpp.c_str(), ofstream::out);
+ dpLogOpp->sync_with_stdio(false);
+ }
+
+ SeedAligner al;
+ SwDriver sd(exactCacheCurrentMB * 1024 * 1024);
+ SwAligner sw(dpLog), osw(dpLogOpp);
+ SeedResults shs[2];
+ OuterLoopMetrics olm;
+ SeedSearchMetrics sdm;
+ WalkMetrics wlm;
+ SwMetrics swmSeed, swmMate;
+ ReportingMetrics rpm;
+ RandomSource rnd, rndArb;
+ SSEMetrics sseU8ExtendMet;
+ SSEMetrics sseU8MateMet;
+ SSEMetrics sseI16ExtendMet;
+ SSEMetrics sseI16MateMet;
+ uint64_t nbtfiltst = 0; // TODO: find a new home for these
+ uint64_t nbtfiltsc = 0; // TODO: find a new home for these
+ uint64_t nbtfiltdo = 0; // TODO: find a new home for these
- ASSERT_ONLY(BTDnaString tmp);
+ ASSERT_ONLY(BTDnaString tmp);
- int pepolFlag;
- if(gMate1fw && gMate2fw) {
- pepolFlag = PE_POLICY_FF;
- } else if(gMate1fw && !gMate2fw) {
- pepolFlag = PE_POLICY_FR;
- } else if(!gMate1fw && gMate2fw) {
- pepolFlag = PE_POLICY_RF;
- } else {
- pepolFlag = PE_POLICY_RR;
- }
- assert_geq(gMaxInsert, gMinInsert);
- assert_geq(gMinInsert, 0);
- PairedEndPolicy pepol(
- pepolFlag,
- gMaxInsert,
- gMinInsert,
- localAlign,
- gFlippedMatesOK,
- gDovetailMatesOK,
- gContainMatesOK,
- gOlapMatesOK,
- gExpandToFrag);
-
- PerfMetrics metricsPt; // per-thread metrics object; for read-level metrics
- BTString nametmp;
- EList<Seed> seeds1, seeds2;
- EList<Seed> *seeds[2] = { &seeds1, &seeds2 };
-
- PerReadMetrics prm;
+ int pepolFlag;
+ if(gMate1fw && gMate2fw) {
+ pepolFlag = PE_POLICY_FF;
+ } else if(gMate1fw && !gMate2fw) {
+ pepolFlag = PE_POLICY_FR;
+ } else if(!gMate1fw && gMate2fw) {
+ pepolFlag = PE_POLICY_RF;
+ } else {
+ pepolFlag = PE_POLICY_RR;
+ }
+ assert_geq(gMaxInsert, gMinInsert);
+ assert_geq(gMinInsert, 0);
+ PairedEndPolicy pepol(
+ pepolFlag,
+ gMaxInsert,
+ gMinInsert,
+ localAlign,
+ gFlippedMatesOK,
+ gDovetailMatesOK,
+ gContainMatesOK,
+ gOlapMatesOK,
+ gExpandToFrag);
+
+ PerfMetrics metricsPt; // per-thread metrics object; for read-level metrics
+ BTString nametmp;
+ EList<Seed> seeds1, seeds2;
+ EList<Seed> *seeds[2] = { &seeds1, &seeds2 };
+
+ PerReadMetrics prm;
- // Used by thread with threadid == 1 to measure time elapsed
- time_t iTime = time(0);
+ // Used by thread with threadid == 1 to measure time elapsed
+ time_t iTime = time(0);
- // Keep track of whether last search was exhaustive for mates 1 and 2
- bool exhaustive[2] = { false, false };
- // Keep track of whether mates 1/2 were filtered out last time through
- bool filt[2] = { true, true };
- // Keep track of whether mates 1/2 were filtered out due Ns last time
- bool nfilt[2] = { true, true };
- // Keep track of whether mates 1/2 were filtered out due to not having
- // enough characters to rise about the score threshold.
- bool scfilt[2] = { true, true };
- // Keep track of whether mates 1/2 were filtered out due to not having
- // more characters than the number of mismatches permitted in a seed.
- bool lenfilt[2] = { true, true };
- // Keep track of whether mates 1/2 were filtered out by upstream qc
- bool qcfilt[2] = { true, true };
+ // Keep track of whether last search was exhaustive for mates 1 and 2
+ bool exhaustive[2] = { false, false };
+ // Keep track of whether mates 1/2 were filtered out last time through
+ bool filt[2] = { true, true };
+ // Keep track of whether mates 1/2 were filtered out due Ns last time
+ bool nfilt[2] = { true, true };
+ // Keep track of whether mates 1/2 were filtered out due to not having
+ // enough characters to rise about the score threshold.
+ bool scfilt[2] = { true, true };
+ // Keep track of whether mates 1/2 were filtered out due to not having
+ // more characters than the number of mismatches permitted in a seed.
+ bool lenfilt[2] = { true, true };
+ // Keep track of whether mates 1/2 were filtered out by upstream qc
+ bool qcfilt[2] = { true, true };
- rndArb.init((uint32_t)time(0));
- int mergei = 0;
- int mergeival = 16;
- bool done = false;
- while(!done) {
- pair<bool, bool> ret = ps->nextReadPair();
- bool success = ret.first;
- done = ret.second;
- if(!success && done) {
- break;
- } else if(!success) {
- continue;
- }
- TReadId rdid = ps->read_a().rdid;
- bool sample = true;
- if(arbitraryRandom) {
- ps->read_a().seed = rndArb.nextU32();
- ps->read_b().seed = rndArb.nextU32();
- }
- if(sampleFrac < 1.0f) {
- rnd.init(ROTL(ps->read_a().seed, 2));
- sample = rnd.nextFloat() < sampleFrac;
- }
- if(rdid >= skipReads && rdid < qUpto && sample) {
- // Align this read/pair
- bool retry = true;
- //
- // Check if there is metrics reporting for us to do.
- //
- if(metricsIval > 0 &&
- (metricsOfb != NULL || metricsStderr) &&
- !metricsPerRead &&
- ++mergei == mergeival)
- {
- // Do a periodic merge. Update global metrics, in a
- // synchronized manner if needed.
- MERGE_METRICS(metrics, nthreads > 1);
- mergei = 0;
- // Check if a progress message should be printed
- if(tid == 0) {
- // Only thread 1 prints progress messages
- time_t curTime = time(0);
- if(curTime - iTime >= metricsIval) {
- metrics.reportInterval(metricsOfb, metricsStderr, false, true, NULL);
- iTime = curTime;
- }
- }
- }
- prm.reset(); // per-read metrics
- prm.doFmString = false;
- if(sam_print_xt) {
- gettimeofday(&prm.tv_beg, &prm.tz_beg);
+ rndArb.init((uint32_t)time(0));
+ int mergei = 0;
+ int mergeival = 16;
+ bool done = false;
+ while(!done) {
+ pair<bool, bool> ret = ps->nextReadPair();
+ bool success = ret.first;
+ done = ret.second;
+ if(!success && done) {
+ break;
+ } else if(!success) {
+ continue;
}
-#ifdef PER_THREAD_TIMING
- int cpu = 0, node = 0;
- get_cpu_and_node(cpu, node);
- if(cpu != current_cpu) {
- ncpu_changeovers++;
- current_cpu = cpu;
+ TReadId rdid = ps->read_a().rdid;
+ bool sample = true;
+ if(arbitraryRandom) {
+ ps->read_a().seed = rndArb.nextU32();
+ ps->read_b().seed = rndArb.nextU32();
}
- if(node != current_node) {
- nnuma_changeovers++;
- current_node = node;
+ if(sampleFrac < 1.0f) {
+ rnd.init(ROTL(ps->read_a().seed, 2));
+ sample = rnd.nextFloat() < sampleFrac;
}
-#endif
- // Try to align this read
- while(retry) {
- retry = false;
- ca.nextRead(); // clear the cache
- olm.reads++;
- assert(!ca.aligning());
- bool paired = !ps->read_b().empty();
- const size_t rdlen1 = ps->read_a().length();
- const size_t rdlen2 = paired ? ps->read_b().length() : 0;
- olm.bases += (rdlen1 + rdlen2);
- msinkwrap.nextRead(
- &ps->read_a(),
- paired ? &ps->read_b() : NULL,
- rdid,
- sc.qualitiesMatter());
- assert(msinkwrap.inited());
- size_t rdlens[2] = { rdlen1, rdlen2 };
- size_t rdrows[2] = { rdlen1, rdlen2 };
- // Calculate the minimum valid score threshold for the read
- TAlScore minsc[2];
- minsc[0] = minsc[1] = std::numeric_limits<TAlScore>::max();
- if(bwaSwLike) {
- // From BWA-SW manual: "Given an l-long query, the
- // threshold for a hit to be retained is
- // a*max{T,c*log(l)}." We try to recreate that here.
- float a = (float)sc.match(30);
- float T = bwaSwLikeT, c = bwaSwLikeC;
- minsc[0] = (TAlScore)max<float>(a*T, a*c*log(rdlens[0]));
- if(paired) {
- minsc[1] = (TAlScore)max<float>(a*T, a*c*log(rdlens[1]));
- }
- } else {
- minsc[0] = scoreMin.f<TAlScore>(rdlens[0]);
- if(paired) minsc[1] = scoreMin.f<TAlScore>(rdlens[1]);
- if(localAlign) {
- if(minsc[0] < 0) {
- if(!gQuiet) printLocalScoreMsg(*ps, paired, true);
- minsc[0] = 0;
- }
- if(paired && minsc[1] < 0) {
- if(!gQuiet) printLocalScoreMsg(*ps, paired, false);
- minsc[1] = 0;
- }
- } else {
- if(minsc[0] > 0) {
- if(!gQuiet) printEEScoreMsg(*ps, paired, true);
- minsc[0] = 0;
- }
- if(paired && minsc[1] > 0) {
- if(!gQuiet) printEEScoreMsg(*ps, paired, false);
- minsc[1] = 0;
+ if(rdid >= skipReads && rdid < qUpto && sample) {
+ // Align this read/pair
+ bool retry = true;
+ //
+ // Check if there is metrics reporting for us to do.
+ //
+ if(metricsIval > 0 &&
+ (metricsOfb != NULL || metricsStderr) &&
+ !metricsPerRead &&
+ ++mergei == mergeival)
+ {
+ // Do a periodic merge. Update global metrics, in a
+ // synchronized manner if needed.
+ MERGE_METRICS(metrics);
+ mergei = 0;
+ // Check if a progress message should be printed
+ if(tid == 0) {
+ // Only thread 1 prints progress messages
+ time_t curTime = time(0);
+ if(curTime - iTime >= metricsIval) {
+ metrics.reportInterval(metricsOfb, metricsStderr, false, NULL);
+ iTime = curTime;
}
}
}
- // N filter; does the read have too many Ns?
- size_t readns[2] = {0, 0};
- sc.nFilterPair(
- &ps->read_a().patFw,
- paired ? &ps->read_b().patFw : NULL,
- readns[0],
- readns[1],
- nfilt[0],
- nfilt[1]);
- // Score filter; does the read enough character to rise above
- // the score threshold?
- scfilt[0] = sc.scoreFilter(minsc[0], rdlens[0]);
- scfilt[1] = sc.scoreFilter(minsc[1], rdlens[1]);
- lenfilt[0] = lenfilt[1] = true;
- if(rdlens[0] <= (size_t)multiseedMms || rdlens[0] < 2) {
- if(!gQuiet) printMmsSkipMsg(*ps, paired, true, multiseedMms);
- lenfilt[0] = false;
- }
- if((rdlens[1] <= (size_t)multiseedMms || rdlens[1] < 2) && paired) {
- if(!gQuiet) printMmsSkipMsg(*ps, paired, false, multiseedMms);
- lenfilt[1] = false;
- }
- if(rdlens[0] < 2) {
- if(!gQuiet) printLenSkipMsg(*ps, paired, true);
- lenfilt[0] = false;
+ prm.reset(); // per-read metrics
+ prm.doFmString = false;
+ if(sam_print_xt) {
+ gettimeofday(&prm.tv_beg, &prm.tz_beg);
}
- if(rdlens[1] < 2 && paired) {
- if(!gQuiet) printLenSkipMsg(*ps, paired, false);
- lenfilt[1] = false;
- }
- qcfilt[0] = qcfilt[1] = true;
- if(qcFilter) {
- qcfilt[0] = (ps->read_a().filter != '0');
- qcfilt[1] = (ps->read_b().filter != '0');
- }
- filt[0] = (nfilt[0] && scfilt[0] && lenfilt[0] && qcfilt[0]);
- filt[1] = (nfilt[1] && scfilt[1] && lenfilt[1] && qcfilt[1]);
- prm.nFilt += (filt[0] ? 0 : 1) + (filt[1] ? 0 : 1);
- Read* rds[2] = { &ps->read_a(), &ps->read_b() };
- // For each mate...
- assert(msinkwrap.empty());
- sd.nextRead(paired, rdrows[0], rdrows[1]); // SwDriver
- size_t minedfw[2] = { 0, 0 };
- size_t minedrc[2] = { 0, 0 };
- // Calcualte nofw / no rc
- bool nofw[2] = { false, false };
- bool norc[2] = { false, false };
- nofw[0] = paired ? (gMate1fw ? gNofw : gNorc) : gNofw;
- norc[0] = paired ? (gMate1fw ? gNorc : gNofw) : gNorc;
- nofw[1] = paired ? (gMate2fw ? gNofw : gNorc) : gNofw;
- norc[1] = paired ? (gMate2fw ? gNorc : gNofw) : gNorc;
- // Calculate nceil
- int nceil[2] = { 0, 0 };
- nceil[0] = nCeil.f<int>((double)rdlens[0]);
- nceil[0] = min(nceil[0], (int)rdlens[0]);
- if(paired) {
- nceil[1] = nCeil.f<int>((double)rdlens[1]);
- nceil[1] = min(nceil[1], (int)rdlens[1]);
+#ifdef PER_THREAD_TIMING
+ int cpu = 0, node = 0;
+ get_cpu_and_node(cpu, node);
+ if(cpu != current_cpu) {
+ ncpu_changeovers++;
+ current_cpu = cpu;
}
- exhaustive[0] = exhaustive[1] = false;
- size_t matemap[2] = { 0, 1 };
- bool pairPostFilt = filt[0] && filt[1];
- if(pairPostFilt) {
- rnd.init(ps->read_a().seed ^ ps->read_b().seed);
- } else {
- rnd.init(ps->read_a().seed);
+ if(node != current_node) {
+ nnuma_changeovers++;
+ current_node = node;
}
- // Calculate interval length for both mates
- int interval[2] = { 0, 0 };
- for(size_t mate = 0; mate < (paired ? 2:1); mate++) {
- interval[mate] = msIval.f<int>((double)rdlens[mate]);
- if(filt[0] && filt[1]) {
- // Boost interval length by 20% for paired-end reads
- interval[mate] = (int)(interval[mate] * 1.2 + 0.5);
+#endif
+ // Try to align this read
+ while(retry) {
+ retry = false;
+ ca.nextRead(); // clear the cache
+ olm.reads++;
+ assert(!ca.aligning());
+ bool paired = !ps->read_b().empty();
+ const size_t rdlen1 = ps->read_a().length();
+ const size_t rdlen2 = paired ? ps->read_b().length() : 0;
+ olm.bases += (rdlen1 + rdlen2);
+ msinkwrap.nextRead(
+ &ps->read_a(),
+ paired ? &ps->read_b() : NULL,
+ rdid,
+ sc.qualitiesMatter());
+ assert(msinkwrap.inited());
+ size_t rdlens[2] = { rdlen1, rdlen2 };
+ size_t rdrows[2] = { rdlen1, rdlen2 };
+ // Calculate the minimum valid score threshold for the read
+ TAlScore minsc[2];
+ minsc[0] = minsc[1] = std::numeric_limits<TAlScore>::max();
+ if(bwaSwLike) {
+ // From BWA-SW manual: "Given an l-long query, the
+ // threshold for a hit to be retained is
+ // a*max{T,c*log(l)}." We try to recreate that here.
+ float a = (float)sc.match(30);
+ float T = bwaSwLikeT, c = bwaSwLikeC;
+ minsc[0] = (TAlScore)max<float>(a*T, a*c*log(rdlens[0]));
+ if(paired) {
+ minsc[1] = (TAlScore)max<float>(a*T, a*c*log(rdlens[1]));
+ }
+ } else {
+ minsc[0] = scoreMin.f<TAlScore>(rdlens[0]);
+ if(paired) minsc[1] = scoreMin.f<TAlScore>(rdlens[1]);
+ if(localAlign) {
+ if(minsc[0] < 0) {
+ if(!gQuiet) printLocalScoreMsg(*ps, paired, true);
+ minsc[0] = 0;
+ }
+ if(paired && minsc[1] < 0) {
+ if(!gQuiet) printLocalScoreMsg(*ps, paired, false);
+ minsc[1] = 0;
+ }
+ } else {
+ if(minsc[0] > 0) {
+ if(!gQuiet) printEEScoreMsg(*ps, paired, true);
+ minsc[0] = 0;
+ }
+ if(paired && minsc[1] > 0) {
+ if(!gQuiet) printEEScoreMsg(*ps, paired, false);
+ minsc[1] = 0;
+ }
+ }
}
- interval[mate] = max(interval[mate], 1);
- }
- // Calculate streak length
- size_t streak[2] = { maxDpStreak, maxDpStreak };
- size_t mtStreak[2] = { maxMateStreak, maxMateStreak };
- size_t mxDp[2] = { maxDp, maxDp };
- size_t mxUg[2] = { maxUg, maxUg };
- size_t mxIter[2] = { maxIters, maxIters };
- if(allHits) {
- streak[0] = streak[1] = std::numeric_limits<size_t>::max();
- mtStreak[0] = mtStreak[1] = std::numeric_limits<size_t>::max();
- mxDp[0] = mxDp[1] = std::numeric_limits<size_t>::max();
- mxUg[0] = mxUg[1] = std::numeric_limits<size_t>::max();
- mxIter[0] = mxIter[1] = std::numeric_limits<size_t>::max();
- } else if(khits > 1) {
- for(size_t mate = 0; mate < 2; mate++) {
- streak[mate] += (khits-1) * maxStreakIncr;
- mtStreak[mate] += (khits-1) * maxStreakIncr;
- mxDp[mate] += (khits-1) * maxItersIncr;
- mxUg[mate] += (khits-1) * maxItersIncr;
- mxIter[mate] += (khits-1) * maxItersIncr;
+ // N filter; does the read have too many Ns?
+ size_t readns[2] = {0, 0};
+ sc.nFilterPair(
+ &ps->read_a().patFw,
+ paired ? &ps->read_b().patFw : NULL,
+ readns[0],
+ readns[1],
+ nfilt[0],
+ nfilt[1]);
+ // Score filter; does the read enough character to rise above
+ // the score threshold?
+ scfilt[0] = sc.scoreFilter(minsc[0], rdlens[0]);
+ scfilt[1] = sc.scoreFilter(minsc[1], rdlens[1]);
+ lenfilt[0] = lenfilt[1] = true;
+ if(rdlens[0] <= (size_t)multiseedMms || rdlens[0] < 2) {
+ if(!gQuiet) printMmsSkipMsg(*ps, paired, true, multiseedMms);
+ lenfilt[0] = false;
}
- }
- if(filt[0] && filt[1]) {
- streak[0] = (size_t)ceil((double)streak[0] / 2.0);
- streak[1] = (size_t)ceil((double)streak[1] / 2.0);
- assert_gt(streak[1], 0);
- }
- assert_gt(streak[0], 0);
- // Calculate # seed rounds for each mate
- size_t nrounds[2] = { nSeedRounds, nSeedRounds };
- if(filt[0] && filt[1]) {
- nrounds[0] = (size_t)ceil((double)nrounds[0] / 2.0);
- nrounds[1] = (size_t)ceil((double)nrounds[1] / 2.0);
- assert_gt(nrounds[1], 0);
- }
- assert_gt(nrounds[0], 0);
- // Increment counters according to what got filtered
- for(size_t mate = 0; mate < (paired ? 2:1); mate++) {
- if(!filt[mate]) {
- // Mate was rejected by N filter
- olm.freads++; // reads filtered out
- olm.fbases += rdlens[mate]; // bases filtered out
+ if((rdlens[1] <= (size_t)multiseedMms || rdlens[1] < 2) && paired) {
+ if(!gQuiet) printMmsSkipMsg(*ps, paired, false, multiseedMms);
+ lenfilt[1] = false;
+ }
+ if(rdlens[0] < 2) {
+ if(!gQuiet) printLenSkipMsg(*ps, paired, true);
+ lenfilt[0] = false;
+ }
+ if(rdlens[1] < 2 && paired) {
+ if(!gQuiet) printLenSkipMsg(*ps, paired, false);
+ lenfilt[1] = false;
+ }
+ qcfilt[0] = qcfilt[1] = true;
+ if(qcFilter) {
+ qcfilt[0] = (ps->read_a().filter != '0');
+ qcfilt[1] = (ps->read_b().filter != '0');
+ }
+ filt[0] = (nfilt[0] && scfilt[0] && lenfilt[0] && qcfilt[0]);
+ filt[1] = (nfilt[1] && scfilt[1] && lenfilt[1] && qcfilt[1]);
+ prm.nFilt += (filt[0] ? 0 : 1) + (filt[1] ? 0 : 1);
+ Read* rds[2] = { &ps->read_a(), &ps->read_b() };
+ // For each mate...
+ assert(msinkwrap.empty());
+ sd.nextRead(paired, rdrows[0], rdrows[1]); // SwDriver
+ size_t minedfw[2] = { 0, 0 };
+ size_t minedrc[2] = { 0, 0 };
+ // Calcualte nofw / no rc
+ bool nofw[2] = { false, false };
+ bool norc[2] = { false, false };
+ nofw[0] = paired ? (gMate1fw ? gNofw : gNorc) : gNofw;
+ norc[0] = paired ? (gMate1fw ? gNorc : gNofw) : gNorc;
+ nofw[1] = paired ? (gMate2fw ? gNofw : gNorc) : gNofw;
+ norc[1] = paired ? (gMate2fw ? gNorc : gNofw) : gNorc;
+ // Calculate nceil
+ int nceil[2] = { 0, 0 };
+ nceil[0] = nCeil.f<int>((double)rdlens[0]);
+ nceil[0] = min(nceil[0], (int)rdlens[0]);
+ if(paired) {
+ nceil[1] = nCeil.f<int>((double)rdlens[1]);
+ nceil[1] = min(nceil[1], (int)rdlens[1]);
+ }
+ exhaustive[0] = exhaustive[1] = false;
+ size_t matemap[2] = { 0, 1 };
+ bool pairPostFilt = filt[0] && filt[1];
+ if(pairPostFilt) {
+ rnd.init(ps->read_a().seed ^ ps->read_b().seed);
} else {
- shs[mate].clear();
- shs[mate].nextRead(mate == 0 ? ps->read_a() : ps->read_b());
- assert(shs[mate].empty());
- olm.ureads++; // reads passing filter
- olm.ubases += rdlens[mate]; // bases passing filter
+ rnd.init(ps->read_a().seed);
}
- }
- size_t eePeEeltLimit = std::numeric_limits<size_t>::max();
- // Whether we're done with mate1 / mate2
- bool done[2] = { !filt[0], !filt[1] };
- size_t nelt[2] = {0, 0};
-
- // Find end-to-end exact alignments for each read
- if(doExactUpFront) {
- for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
- size_t mate = matemap[matei];
- if(!filt[mate] || done[mate] || msinkwrap.state().doneWithMate(mate == 0)) {
- continue;
- }
- swmSeed.exatts++;
- nelt[mate] = al.exactSweep(
- ebwtFw, // index
- *rds[mate], // read
- sc, // scoring scheme
- nofw[mate], // nofw?
- norc[mate], // norc?
- 2, // max # edits we care about
- minedfw[mate], // minimum # edits for fw mate
- minedrc[mate], // minimum # edits for rc mate
- true, // report 0mm hits
- shs[mate], // put end-to-end results here
- sdm); // metrics
- size_t bestmin = min(minedfw[mate], minedrc[mate]);
- if(bestmin == 0) {
- sdm.bestmin0++;
- } else if(bestmin == 1) {
- sdm.bestmin1++;
- } else {
- assert_eq(2, bestmin);
- sdm.bestmin2++;
- }
+ // Calculate interval length for both mates
+ int interval[2] = { 0, 0 };
+ for(size_t mate = 0; mate < (paired ? 2:1); mate++) {
+ interval[mate] = msIval.f<int>((double)rdlens[mate]);
+ if(filt[0] && filt[1]) {
+ // Boost interval length by 20% for paired-end reads
+ interval[mate] = (int)(interval[mate] * 1.2 + 0.5);
}
- matemap[0] = 0; matemap[1] = 1;
- if(nelt[0] > 0 && nelt[1] > 0 && nelt[0] > nelt[1]) {
- // Do the mate with fewer exact hits first
- // TODO: Consider mates & orientations separately?
- matemap[0] = 1; matemap[1] = 0;
+ interval[mate] = max(interval[mate], 1);
+ }
+ // Calculate streak length
+ size_t streak[2] = { maxDpStreak, maxDpStreak };
+ size_t mtStreak[2] = { maxMateStreak, maxMateStreak };
+ size_t mxDp[2] = { maxDp, maxDp };
+ size_t mxUg[2] = { maxUg, maxUg };
+ size_t mxIter[2] = { maxIters, maxIters };
+ if(allHits) {
+ streak[0] = streak[1] = std::numeric_limits<size_t>::max();
+ mtStreak[0] = mtStreak[1] = std::numeric_limits<size_t>::max();
+ mxDp[0] = mxDp[1] = std::numeric_limits<size_t>::max();
+ mxUg[0] = mxUg[1] = std::numeric_limits<size_t>::max();
+ mxIter[0] = mxIter[1] = std::numeric_limits<size_t>::max();
+ } else if(khits > 1) {
+ for(size_t mate = 0; mate < 2; mate++) {
+ streak[mate] += (khits-1) * maxStreakIncr;
+ mtStreak[mate] += (khits-1) * maxStreakIncr;
+ mxDp[mate] += (khits-1) * maxItersIncr;
+ mxUg[mate] += (khits-1) * maxItersIncr;
+ mxIter[mate] += (khits-1) * maxItersIncr;
}
- for(size_t matei = 0; matei < (seedSumm ? 0:2); matei++) {
- size_t mate = matemap[matei];
- if(nelt[mate] == 0 || nelt[mate] > eePeEeltLimit) {
- shs[mate].clearExactE2eHits();
- continue;
- }
- if(msinkwrap.state().doneWithMate(mate == 0)) {
- shs[mate].clearExactE2eHits();
- done[mate] = true;
- continue;
- }
- assert(filt[mate]);
- assert(matei == 0 || paired);
- assert(!msinkwrap.maxed());
- assert(msinkwrap.repOk());
- int ret = 0;
- if(paired) {
- // Paired-end dynamic programming driver
- ret = sd.extendSeedsPaired(
- *rds[mate], // mate to align as anchor
- *rds[mate ^ 1], // mate to align as opp.
- mate == 0, // anchor is mate 1?
- !filt[mate ^ 1],// opposite mate filtered out?
- shs[mate], // seed hits for anchor
- ebwtFw, // bowtie index
- &ebwtBw, // rev bowtie index
- ref, // packed reference strings
- sw, // dyn prog aligner, anchor
- osw, // dyn prog aligner, opposite
- sc, // scoring scheme
- pepol, // paired-end policy
- -1, // # mms allowed in a seed
- 0, // length of a seed
- 0, // interval between seeds
- minsc[mate], // min score for anchor
- minsc[mate^1], // min score for opp.
- nceil[mate], // N ceil for anchor
- nceil[mate^1], // N ceil for opp.
- nofw[mate], // don't align forward read
- norc[mate], // don't align revcomp read
- maxhalf, // max width on one DP side
- doUngapped, // do ungapped alignment
- mxIter[mate], // max extend loop iters
- mxUg[mate], // max # ungapped extends
- mxDp[mate], // max # DPs
- streak[mate], // stop after streak of this many end-to-end fails
- streak[mate], // stop after streak of this many ungap fails
- streak[mate], // stop after streak of this many dp fails
- mtStreak[mate], // max mate fails per seed range
- doExtend, // extend seed hits
- enable8, // use 8-bit SSE where possible
- cminlen, // checkpoint if read is longer
- cpow2, // checkpointer interval, log2
- doTri, // triangular mini-fills?
- tighten, // -M score tightening mode
- ca, // seed alignment cache
- rnd, // pseudo-random source
- wlm, // group walk left metrics
- swmSeed, // DP metrics, seed extend
- swmMate, // DP metrics, mate finding
- prm, // per-read metrics
- &msinkwrap, // for organizing hits
- true, // seek mate immediately
- true, // report hits once found
- gReportDiscordant,// look for discordant alns?
- gReportMixed, // look for unpaired alns?
- exhaustive[mate]);
- // Might be done, but just with this mate
- } else {
- // Unpaired dynamic programming driver
- ret = sd.extendSeeds(
- *rds[mate], // read
- mate == 0, // mate #1?
- shs[mate], // seed hits
- ebwtFw, // bowtie index
- &ebwtBw, // rev bowtie index
- ref, // packed reference strings
- sw, // dynamic prog aligner
- sc, // scoring scheme
- -1, // # mms allowed in a seed
- 0, // length of a seed
- 0, // interval between seeds
- minsc[mate], // minimum score for valid
- nceil[mate], // N ceil for anchor
- maxhalf, // max width on one DP side
- doUngapped, // do ungapped alignment
- mxIter[mate], // max extend loop iters
- mxUg[mate], // max # ungapped extends
- mxDp[mate], // max # DPs
- streak[mate], // stop after streak of this many end-to-end fails
- streak[mate], // stop after streak of this many ungap fails
- doExtend, // extend seed hits
- enable8, // use 8-bit SSE where possible
- cminlen, // checkpoint if read is longer
- cpow2, // checkpointer interval, log2
- doTri, // triangular mini-fills
- tighten, // -M score tightening mode
- ca, // seed alignment cache
- rnd, // pseudo-random source
- wlm, // group walk left metrics
- swmSeed, // DP metrics, seed extend
- prm, // per-read metrics
- &msinkwrap, // for organizing hits
- true, // report hits once found
- exhaustive[mate]);
- }
- assert_gt(ret, 0);
- MERGE_SW(sw);
- MERGE_SW(osw);
- // Clear out the exact hits so that we don't try to
- // extend them again later!
- shs[mate].clearExactE2eHits();
- if(ret == EXTEND_EXHAUSTED_CANDIDATES) {
- // Not done yet
- } else if(ret == EXTEND_POLICY_FULFILLED) {
- // Policy is satisfied for this mate at least
- if(msinkwrap.state().doneWithMate(mate == 0)) {
- done[mate] = true;
+ }
+ if(filt[0] && filt[1]) {
+ streak[0] = (size_t)ceil((double)streak[0] / 2.0);
+ streak[1] = (size_t)ceil((double)streak[1] / 2.0);
+ assert_gt(streak[1], 0);
+ }
+ assert_gt(streak[0], 0);
+ // Calculate # seed rounds for each mate
+ size_t nrounds[2] = { nSeedRounds, nSeedRounds };
+ if(filt[0] && filt[1]) {
+ nrounds[0] = (size_t)ceil((double)nrounds[0] / 2.0);
+ nrounds[1] = (size_t)ceil((double)nrounds[1] / 2.0);
+ assert_gt(nrounds[1], 0);
+ }
+ assert_gt(nrounds[0], 0);
+ // Increment counters according to what got filtered
+ for(size_t mate = 0; mate < (paired ? 2:1); mate++) {
+ if(!filt[mate]) {
+ // Mate was rejected by N filter
+ olm.freads++; // reads filtered out
+ olm.fbases += rdlens[mate]; // bases filtered out
+ } else {
+ shs[mate].clear();
+ shs[mate].nextRead(mate == 0 ? ps->read_a() : ps->read_b());
+ assert(shs[mate].empty());
+ olm.ureads++; // reads passing filter
+ olm.ubases += rdlens[mate]; // bases passing filter
+ }
+ }
+ size_t eePeEeltLimit = std::numeric_limits<size_t>::max();
+ // Whether we're done with mate1 / mate2
+ bool done[2] = { !filt[0], !filt[1] };
+ size_t nelt[2] = {0, 0};
+
+ // Find end-to-end exact alignments for each read
+ if(doExactUpFront) {
+ for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
+ size_t mate = matemap[matei];
+ if(!filt[mate] || done[mate] || msinkwrap.state().doneWithMate(mate == 0)) {
+ continue;
}
- if(msinkwrap.state().doneWithMate(mate == 1)) {
- done[mate^1] = true;
+ swmSeed.exatts++;
+ nelt[mate] = al.exactSweep(
+ ebwtFw, // index
+ *rds[mate], // read
+ sc, // scoring scheme
+ nofw[mate], // nofw?
+ norc[mate], // norc?
+ 2, // max # edits we care about
+ minedfw[mate], // minimum # edits for fw mate
+ minedrc[mate], // minimum # edits for rc mate
+ true, // report 0mm hits
+ shs[mate], // put end-to-end results here
+ sdm); // metrics
+ size_t bestmin = min(minedfw[mate], minedrc[mate]);
+ if(bestmin == 0) {
+ sdm.bestmin0++;
+ } else if(bestmin == 1) {
+ sdm.bestmin1++;
+ } else {
+ assert_eq(2, bestmin);
+ sdm.bestmin2++;
}
- } else if(ret == EXTEND_PERFECT_SCORE) {
- // We exhausted this mode at least
- done[mate] = true;
- } else if(ret == EXTEND_EXCEEDED_HARD_LIMIT) {
- // We exceeded a per-read limit
- done[mate] = true;
- } else if(ret == EXTEND_EXCEEDED_SOFT_LIMIT) {
- // Not done yet
- } else {
- //
- cerr << "Bad return value: " << ret << endl;
- throw 1;
}
- if(!done[mate]) {
- TAlScore perfectScore = sc.perfectScore(rdlens[mate]);
- if(!done[mate] && minsc[mate] == perfectScore) {
+ matemap[0] = 0; matemap[1] = 1;
+ if(nelt[0] > 0 && nelt[1] > 0 && nelt[0] > nelt[1]) {
+ // Do the mate with fewer exact hits first
+ // TODO: Consider mates & orientations separately?
+ matemap[0] = 1; matemap[1] = 0;
+ }
+ for(size_t matei = 0; matei < (seedSumm ? 0:2); matei++) {
+ size_t mate = matemap[matei];
+ if(nelt[mate] == 0 || nelt[mate] > eePeEeltLimit) {
+ shs[mate].clearExactE2eHits();
+ continue;
+ }
+ if(msinkwrap.state().doneWithMate(mate == 0)) {
+ shs[mate].clearExactE2eHits();
done[mate] = true;
+ continue;
}
- }
- }
- }
-
- // 1-mismatch
- if(do1mmUpFront && !seedSumm) {
- for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
- size_t mate = matemap[matei];
- if(!filt[mate] || done[mate] || nelt[mate] > eePeEeltLimit) {
- // Done with this mate
- shs[mate].clear1mmE2eHits();
- nelt[mate] = 0;
- continue;
- }
- nelt[mate] = 0;
- assert(!msinkwrap.maxed());
- assert(msinkwrap.repOk());
- //rnd.init(ROTL(rds[mate]->seed, 10));
- assert(shs[mate].empty());
- assert(shs[mate].repOk(&ca.current()));
- bool yfw = minedfw[mate] <= 1 && !nofw[mate];
- bool yrc = minedrc[mate] <= 1 && !norc[mate];
- if(yfw || yrc) {
- // Clear out the exact hits
- swmSeed.mm1atts++;
- al.oneMmSearch(
- &ebwtFw, // BWT index
- &ebwtBw, // BWT' index
- *rds[mate], // read
- sc, // scoring scheme
- minsc[mate], // minimum score
- !yfw, // don't align forward read
- !yrc, // don't align revcomp read
- localAlign, // must be legal local alns?
- false, // do exact match
- true, // do 1mm
- shs[mate], // seed hits (hits installed here)
- sdm); // metrics
- nelt[mate] = shs[mate].num1mmE2eHits();
- }
- }
- // Possibly reorder the mates
- matemap[0] = 0; matemap[1] = 1;
- if(nelt[0] > 0 && nelt[1] > 0 && nelt[0] > nelt[1]) {
- // Do the mate with fewer exact hits first
- // TODO: Consider mates & orientations separately?
- matemap[0] = 1; matemap[1] = 0;
- }
- for(size_t matei = 0; matei < (seedSumm ? 0:2); matei++) {
- size_t mate = matemap[matei];
- if(nelt[mate] == 0 || nelt[mate] > eePeEeltLimit) {
- continue;
- }
- if(msinkwrap.state().doneWithMate(mate == 0)) {
- done[mate] = true;
- continue;
- }
- int ret = 0;
- if(paired) {
- // Paired-end dynamic programming driver
- ret = sd.extendSeedsPaired(
- *rds[mate], // mate to align as anchor
- *rds[mate ^ 1], // mate to align as opp.
- mate == 0, // anchor is mate 1?
- !filt[mate ^ 1],// opposite mate filtered out?
- shs[mate], // seed hits for anchor
- ebwtFw, // bowtie index
- &ebwtBw, // rev bowtie index
- ref, // packed reference strings
- sw, // dyn prog aligner, anchor
- osw, // dyn prog aligner, opposite
- sc, // scoring scheme
- pepol, // paired-end policy
- -1, // # mms allowed in a seed
- 0, // length of a seed
- 0, // interval between seeds
- minsc[mate], // min score for anchor
- minsc[mate^1], // min score for opp.
- nceil[mate], // N ceil for anchor
- nceil[mate^1], // N ceil for opp.
- nofw[mate], // don't align forward read
- norc[mate], // don't align revcomp read
- maxhalf, // max width on one DP side
- doUngapped, // do ungapped alignment
- mxIter[mate], // max extend loop iters
- mxUg[mate], // max # ungapped extends
- mxDp[mate], // max # DPs
- streak[mate], // stop after streak of this many end-to-end fails
- streak[mate], // stop after streak of this many ungap fails
- streak[mate], // stop after streak of this many dp fails
- mtStreak[mate], // max mate fails per seed range
- doExtend, // extend seed hits
- enable8, // use 8-bit SSE where possible
- cminlen, // checkpoint if read is longer
- cpow2, // checkpointer interval, log2
- doTri, // triangular mini-fills?
- tighten, // -M score tightening mode
- ca, // seed alignment cache
- rnd, // pseudo-random source
- wlm, // group walk left metrics
- swmSeed, // DP metrics, seed extend
- swmMate, // DP metrics, mate finding
- prm, // per-read metrics
- &msinkwrap, // for organizing hits
- true, // seek mate immediately
- true, // report hits once found
- gReportDiscordant,// look for discordant alns?
- gReportMixed, // look for unpaired alns?
- exhaustive[mate]);
- // Might be done, but just with this mate
- } else {
- // Unpaired dynamic programming driver
- ret = sd.extendSeeds(
- *rds[mate], // read
- mate == 0, // mate #1?
- shs[mate], // seed hits
- ebwtFw, // bowtie index
- &ebwtBw, // rev bowtie index
- ref, // packed reference strings
- sw, // dynamic prog aligner
- sc, // scoring scheme
- -1, // # mms allowed in a seed
- 0, // length of a seed
- 0, // interval between seeds
- minsc[mate], // minimum score for valid
- nceil[mate], // N ceil for anchor
- maxhalf, // max width on one DP side
- doUngapped, // do ungapped alignment
- mxIter[mate], // max extend loop iters
- mxUg[mate], // max # ungapped extends
- mxDp[mate], // max # DPs
- streak[mate], // stop after streak of this many end-to-end fails
- streak[mate], // stop after streak of this many ungap fails
- doExtend, // extend seed hits
- enable8, // use 8-bit SSE where possible
- cminlen, // checkpoint if read is longer
- cpow2, // checkpointer interval, log2
- doTri, // triangular mini-fills?
- tighten, // -M score tightening mode
- ca, // seed alignment cache
- rnd, // pseudo-random source
- wlm, // group walk left metrics
- swmSeed, // DP metrics, seed extend
- prm, // per-read metrics
- &msinkwrap, // for organizing hits
- true, // report hits once found
- exhaustive[mate]);
- }
- assert_gt(ret, 0);
- MERGE_SW(sw);
- MERGE_SW(osw);
- // Clear out the 1mm hits so that we don't try to
- // extend them again later!
- shs[mate].clear1mmE2eHits();
- if(ret == EXTEND_EXHAUSTED_CANDIDATES) {
- // Not done yet
- } else if(ret == EXTEND_POLICY_FULFILLED) {
- // Policy is satisfied for this mate at least
- if(msinkwrap.state().doneWithMate(mate == 0)) {
- done[mate] = true;
- }
- if(msinkwrap.state().doneWithMate(mate == 1)) {
- done[mate^1] = true;
+ assert(filt[mate]);
+ assert(matei == 0 || paired);
+ assert(!msinkwrap.maxed());
+ assert(msinkwrap.repOk());
+ int ret = 0;
+ if(paired) {
+ // Paired-end dynamic programming driver
+ ret = sd.extendSeedsPaired(
+ *rds[mate], // mate to align as anchor
+ *rds[mate ^ 1], // mate to align as opp.
+ mate == 0, // anchor is mate 1?
+ !filt[mate ^ 1],// opposite mate filtered out?
+ shs[mate], // seed hits for anchor
+ ebwtFw, // bowtie index
+ &ebwtBw, // rev bowtie index
+ ref, // packed reference strings
+ sw, // dyn prog aligner, anchor
+ osw, // dyn prog aligner, opposite
+ sc, // scoring scheme
+ pepol, // paired-end policy
+ -1, // # mms allowed in a seed
+ 0, // length of a seed
+ 0, // interval between seeds
+ minsc[mate], // min score for anchor
+ minsc[mate^1], // min score for opp.
+ nceil[mate], // N ceil for anchor
+ nceil[mate^1], // N ceil for opp.
+ nofw[mate], // don't align forward read
+ norc[mate], // don't align revcomp read
+ maxhalf, // max width on one DP side
+ doUngapped, // do ungapped alignment
+ mxIter[mate], // max extend loop iters
+ mxUg[mate], // max # ungapped extends
+ mxDp[mate], // max # DPs
+ streak[mate], // stop after streak of this many end-to-end fails
+ streak[mate], // stop after streak of this many ungap fails
+ streak[mate], // stop after streak of this many dp fails
+ mtStreak[mate], // max mate fails per seed range
+ doExtend, // extend seed hits
+ enable8, // use 8-bit SSE where possible
+ cminlen, // checkpoint if read is longer
+ cpow2, // checkpointer interval, log2
+ doTri, // triangular mini-fills?
+ tighten, // -M score tightening mode
+ ca, // seed alignment cache
+ rnd, // pseudo-random source
+ wlm, // group walk left metrics
+ swmSeed, // DP metrics, seed extend
+ swmMate, // DP metrics, mate finding
+ prm, // per-read metrics
+ &msinkwrap, // for organizing hits
+ true, // seek mate immediately
+ true, // report hits once found
+ gReportDiscordant,// look for discordant alns?
+ gReportMixed, // look for unpaired alns?
+ exhaustive[mate]);
+ // Might be done, but just with this mate
+ } else {
+ // Unpaired dynamic programming driver
+ ret = sd.extendSeeds(
+ *rds[mate], // read
+ mate == 0, // mate #1?
+ shs[mate], // seed hits
+ ebwtFw, // bowtie index
+ &ebwtBw, // rev bowtie index
+ ref, // packed reference strings
+ sw, // dynamic prog aligner
+ sc, // scoring scheme
+ -1, // # mms allowed in a seed
+ 0, // length of a seed
+ 0, // interval between seeds
+ minsc[mate], // minimum score for valid
+ nceil[mate], // N ceil for anchor
+ maxhalf, // max width on one DP side
+ doUngapped, // do ungapped alignment
+ mxIter[mate], // max extend loop iters
+ mxUg[mate], // max # ungapped extends
+ mxDp[mate], // max # DPs
+ streak[mate], // stop after streak of this many end-to-end fails
+ streak[mate], // stop after streak of this many ungap fails
+ doExtend, // extend seed hits
+ enable8, // use 8-bit SSE where possible
+ cminlen, // checkpoint if read is longer
+ cpow2, // checkpointer interval, log2
+ doTri, // triangular mini-fills
+ tighten, // -M score tightening mode
+ ca, // seed alignment cache
+ rnd, // pseudo-random source
+ wlm, // group walk left metrics
+ swmSeed, // DP metrics, seed extend
+ prm, // per-read metrics
+ &msinkwrap, // for organizing hits
+ true, // report hits once found
+ exhaustive[mate]);
}
- } else if(ret == EXTEND_PERFECT_SCORE) {
- // We exhausted this mode at least
- done[mate] = true;
- } else if(ret == EXTEND_EXCEEDED_HARD_LIMIT) {
- // We exceeded a per-read limit
- done[mate] = true;
- } else if(ret == EXTEND_EXCEEDED_SOFT_LIMIT) {
- // Not done yet
- } else {
- //
- cerr << "Bad return value: " << ret << endl;
- throw 1;
- }
- if(!done[mate]) {
- TAlScore perfectScore = sc.perfectScore(rdlens[mate]);
- if(!done[mate] && minsc[mate] == perfectScore) {
+ assert_gt(ret, 0);
+ MERGE_SW(sw);
+ MERGE_SW(osw);
+ // Clear out the exact hits so that we don't try to
+ // extend them again later!
+ shs[mate].clearExactE2eHits();
+ if(ret == EXTEND_EXHAUSTED_CANDIDATES) {
+ // Not done yet
+ } else if(ret == EXTEND_POLICY_FULFILLED) {
+ // Policy is satisfied for this mate at least
+ if(msinkwrap.state().doneWithMate(mate == 0)) {
+ done[mate] = true;
+ }
+ if(msinkwrap.state().doneWithMate(mate == 1)) {
+ done[mate^1] = true;
+ }
+ } else if(ret == EXTEND_PERFECT_SCORE) {
+ // We exhausted this mode at least
+ done[mate] = true;
+ } else if(ret == EXTEND_EXCEEDED_HARD_LIMIT) {
+ // We exceeded a per-read limit
done[mate] = true;
+ } else if(ret == EXTEND_EXCEEDED_SOFT_LIMIT) {
+ // Not done yet
+ } else {
+ //
+ cerr << "Bad return value: " << ret << endl;
+ throw 1;
+ }
+ if(!done[mate]) {
+ TAlScore perfectScore = sc.perfectScore(rdlens[mate]);
+ if(!done[mate] && minsc[mate] == perfectScore) {
+ done[mate] = true;
+ }
}
}
}
- }
- int seedlens[2] = { multiseedLen, multiseedLen };
- nrounds[0] = min<size_t>(nrounds[0], interval[0]);
- nrounds[1] = min<size_t>(nrounds[1], interval[1]);
- Constraint gc = Constraint::penaltyFuncBased(scoreMin);
- size_t seedsTried = 0;
- size_t seedsTriedMS[] = {0, 0, 0, 0};
- size_t nUniqueSeeds = 0, nRepeatSeeds = 0, seedHitTot = 0;
- size_t nUniqueSeedsMS[] = {0, 0, 0, 0};
- size_t nRepeatSeedsMS[] = {0, 0, 0, 0};
- size_t seedHitTotMS[] = {0, 0, 0, 0};
- for(size_t roundi = 0; roundi < nSeedRounds; roundi++) {
- ca.nextRead(); // Clear cache in preparation for new search
- shs[0].clearSeeds();
- shs[1].clearSeeds();
- assert(shs[0].empty());
- assert(shs[1].empty());
- assert(shs[0].repOk(&ca.current()));
- assert(shs[1].repOk(&ca.current()));
- //if(roundi > 0) {
- // if(seedlens[0] > 8) seedlens[0]--;
- // if(seedlens[1] > 8) seedlens[1]--;
- //}
- for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
- size_t mate = matemap[matei];
- if(done[mate] || msinkwrap.state().doneWithMate(mate == 0)) {
- // Done with this mate
- done[mate] = true;
- continue;
- }
- if(roundi >= nrounds[mate]) {
- // Not doing this round for this mate
- continue;
- }
- // Figure out the seed offset
- if(interval[mate] <= (int)roundi) {
- // Can't do this round, seeds already packed as
- // tight as possible
- continue;
- }
- size_t offset = (interval[mate] * roundi) / nrounds[mate];
- assert(roundi == 0 || offset > 0);
- assert(!msinkwrap.maxed());
- assert(msinkwrap.repOk());
- //rnd.init(ROTL(rds[mate]->seed, 10));
- assert(shs[mate].repOk(&ca.current()));
- swmSeed.sdatts++;
- // Set up seeds
- seeds[mate]->clear();
- Seed::mmSeeds(
- multiseedMms, // max # mms per seed
- seedlens[mate], // length of a multiseed seed
- *seeds[mate], // seeds
- gc); // global constraint
- // Check whether the offset would drive the first seed
- // off the end
- if(offset > 0 && (*seeds[mate])[0].len + offset > rds[mate]->length()) {
- continue;
- }
- // Instantiate the seeds
- std::pair<int, int> instFw, instRc;
- std::pair<int, int> inst = al.instantiateSeeds(
- *seeds[mate], // search seeds
- offset, // offset to begin extracting
- interval[mate], // interval between seeds
- *rds[mate], // read to align
- sc, // scoring scheme
- nofw[mate], // don't align forward read
- norc[mate], // don't align revcomp read
- ca, // holds some seed hits from previous reads
- shs[mate], // holds all the seed hits
- sdm, // metrics
- instFw,
- instRc);
- assert(shs[mate].repOk(&ca.current()));
- if(inst.first + inst.second == 0) {
- // No seed hits! Done with this mate.
+
+ // 1-mismatch
+ if(do1mmUpFront && !seedSumm) {
+ for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
+ size_t mate = matemap[matei];
+ if(!filt[mate] || done[mate] || nelt[mate] > eePeEeltLimit) {
+ // Done with this mate
+ shs[mate].clear1mmE2eHits();
+ nelt[mate] = 0;
+ continue;
+ }
+ nelt[mate] = 0;
+ assert(!msinkwrap.maxed());
+ assert(msinkwrap.repOk());
+ //rnd.init(ROTL(rds[mate]->seed, 10));
assert(shs[mate].empty());
- done[mate] = true;
- break;
- }
- seedsTried += (inst.first + inst.second);
- seedsTriedMS[mate * 2 + 0] = instFw.first + instFw.second;
- seedsTriedMS[mate * 2 + 1] = instRc.first + instRc.second;
- // Align seeds
- al.searchAllSeeds(
- *seeds[mate], // search seeds
- &ebwtFw, // BWT index
- &ebwtBw, // BWT' index
- *rds[mate], // read
- sc, // scoring scheme
- ca, // alignment cache
- shs[mate], // store seed hits here
- sdm, // metrics
- prm); // per-read metrics
- assert(shs[mate].repOk(&ca.current()));
- if(shs[mate].empty()) {
- // No seed alignments! Done with this mate.
- done[mate] = true;
- break;
- }
- }
- // shs contain what we need to know to update our seed
- // summaries for this seeding
- for(size_t mate = 0; mate < 2; mate++) {
- if(!shs[mate].empty()) {
- nUniqueSeeds += shs[mate].numUniqueSeeds();
- nUniqueSeedsMS[mate * 2 + 0] += shs[mate].numUniqueSeedsStrand(true);
- nUniqueSeedsMS[mate * 2 + 1] += shs[mate].numUniqueSeedsStrand(false);
- nRepeatSeeds += shs[mate].numRepeatSeeds();
- nRepeatSeedsMS[mate * 2 + 0] += shs[mate].numRepeatSeedsStrand(true);
- nRepeatSeedsMS[mate * 2 + 1] += shs[mate].numRepeatSeedsStrand(false);
- seedHitTot += shs[mate].numElts();
- seedHitTotMS[mate * 2 + 0] += shs[mate].numEltsFw();
- seedHitTotMS[mate * 2 + 1] += shs[mate].numEltsRc();
- }
- }
- double uniqFactor[2] = { 0.0f, 0.0f };
- for(size_t i = 0; i < 2; i++) {
- if(!shs[i].empty()) {
- swmSeed.sdsucc++;
- uniqFactor[i] = shs[i].uniquenessFactor();
+ assert(shs[mate].repOk(&ca.current()));
+ bool yfw = minedfw[mate] <= 1 && !nofw[mate];
+ bool yrc = minedrc[mate] <= 1 && !norc[mate];
+ if(yfw || yrc) {
+ // Clear out the exact hits
+ swmSeed.mm1atts++;
+ al.oneMmSearch(
+ &ebwtFw, // BWT index
+ &ebwtBw, // BWT' index
+ *rds[mate], // read
+ sc, // scoring scheme
+ minsc[mate], // minimum score
+ !yfw, // don't align forward read
+ !yrc, // don't align revcomp read
+ localAlign, // must be legal local alns?
+ false, // do exact match
+ true, // do 1mm
+ shs[mate], // seed hits (hits installed here)
+ sdm); // metrics
+ nelt[mate] = shs[mate].num1mmE2eHits();
+ }
}
- }
- // Possibly reorder the mates
- matemap[0] = 0; matemap[1] = 1;
- if(!shs[0].empty() && !shs[1].empty() && uniqFactor[1] > uniqFactor[0]) {
- // Do the mate with fewer exact hits first
- // TODO: Consider mates & orientations separately?
- matemap[0] = 1; matemap[1] = 0;
- }
- for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
- size_t mate = matemap[matei];
- if(done[mate] || msinkwrap.state().doneWithMate(mate == 0)) {
- // Done with this mate
- done[mate] = true;
- continue;
+ // Possibly reorder the mates
+ matemap[0] = 0; matemap[1] = 1;
+ if(nelt[0] > 0 && nelt[1] > 0 && nelt[0] > nelt[1]) {
+ // Do the mate with fewer exact hits first
+ // TODO: Consider mates & orientations separately?
+ matemap[0] = 1; matemap[1] = 0;
}
- assert(!msinkwrap.maxed());
- assert(msinkwrap.repOk());
- //rnd.init(ROTL(rds[mate]->seed, 10));
- assert(shs[mate].repOk(&ca.current()));
- if(!seedSumm) {
- // If there aren't any seed hits...
- if(shs[mate].empty()) {
- continue; // on to the next mate
+ for(size_t matei = 0; matei < (seedSumm ? 0:2); matei++) {
+ size_t mate = matemap[matei];
+ if(nelt[mate] == 0 || nelt[mate] > eePeEeltLimit) {
+ continue;
+ }
+ if(msinkwrap.state().doneWithMate(mate == 0)) {
+ done[mate] = true;
+ continue;
}
- // Sort seed hits into ranks
- shs[mate].rankSeedHits(rnd, msinkwrap.allHits());
int ret = 0;
if(paired) {
// Paired-end dynamic programming driver
@@ -3727,9 +3513,9 @@ static void multiseedSearchWorker(void *vp) {
osw, // dyn prog aligner, opposite
sc, // scoring scheme
pepol, // paired-end policy
- multiseedMms, // # mms allowed in a seed
- seedlens[mate], // length of a seed
- interval[mate], // interval between seeds
+ -1, // # mms allowed in a seed
+ 0, // length of a seed
+ 0, // interval between seeds
minsc[mate], // min score for anchor
minsc[mate^1], // min score for opp.
nceil[mate], // N ceil for anchor
@@ -3775,9 +3561,9 @@ static void multiseedSearchWorker(void *vp) {
ref, // packed reference strings
sw, // dynamic prog aligner
sc, // scoring scheme
- multiseedMms, // # mms allowed in a seed
- seedlens[mate], // length of a seed
- interval[mate], // interval between seeds
+ -1, // # mms allowed in a seed
+ 0, // length of a seed
+ 0, // interval between seeds
minsc[mate], // minimum score for valid
nceil[mate], // N ceil for anchor
maxhalf, // max width on one DP side
@@ -3805,6 +3591,9 @@ static void multiseedSearchWorker(void *vp) {
assert_gt(ret, 0);
MERGE_SW(sw);
MERGE_SW(osw);
+ // Clear out the 1mm hits so that we don't try to
+ // extend them again later!
+ shs[mate].clear1mmE2eHits();
if(ret == EXTEND_EXHAUSTED_CANDIDATES) {
// Not done yet
} else if(ret == EXTEND_POLICY_FULFILLED) {
@@ -3816,7 +3605,7 @@ static void multiseedSearchWorker(void *vp) {
done[mate^1] = true;
}
} else if(ret == EXTEND_PERFECT_SCORE) {
- // We exhausted this made at least
+ // We exhausted this mode at least
done[mate] = true;
} else if(ret == EXTEND_EXCEEDED_HARD_LIMIT) {
// We exceeded a per-read limit
@@ -3828,22 +3617,295 @@ static void multiseedSearchWorker(void *vp) {
cerr << "Bad return value: " << ret << endl;
throw 1;
}
- } // if(!seedSumm)
- } // for(size_t matei = 0; matei < 2; matei++)
-
- // We don't necessarily have to continue investigating both
- // mates. We continue on a mate only if its average
- // interval length is high (> 1000)
- for(size_t mate = 0; mate < 2; mate++) {
- if(!done[mate] && shs[mate].averageHitsPerSeed() < seedBoostThresh) {
- done[mate] = true;
+ if(!done[mate]) {
+ TAlScore perfectScore = sc.perfectScore(rdlens[mate]);
+ if(!done[mate] && minsc[mate] == perfectScore) {
+ done[mate] = true;
+ }
+ }
}
}
- } // end loop over reseeding rounds
+ int seedlens[2] = { multiseedLen, multiseedLen };
+ nrounds[0] = min<size_t>(nrounds[0], interval[0]);
+ nrounds[1] = min<size_t>(nrounds[1], interval[1]);
+ Constraint gc = Constraint::penaltyFuncBased(scoreMin);
+ size_t seedsTried = 0;
+ size_t seedsTriedMS[] = {0, 0, 0, 0};
+ size_t nUniqueSeeds = 0, nRepeatSeeds = 0, seedHitTot = 0;
+ size_t nUniqueSeedsMS[] = {0, 0, 0, 0};
+ size_t nRepeatSeedsMS[] = {0, 0, 0, 0};
+ size_t seedHitTotMS[] = {0, 0, 0, 0};
+ for(size_t roundi = 0; roundi < nSeedRounds; roundi++) {
+ ca.nextRead(); // Clear cache in preparation for new search
+ shs[0].clearSeeds();
+ shs[1].clearSeeds();
+ assert(shs[0].empty());
+ assert(shs[1].empty());
+ assert(shs[0].repOk(&ca.current()));
+ assert(shs[1].repOk(&ca.current()));
+ //if(roundi > 0) {
+ // if(seedlens[0] > 8) seedlens[0]--;
+ // if(seedlens[1] > 8) seedlens[1]--;
+ //}
+ for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
+ size_t mate = matemap[matei];
+ if(done[mate] || msinkwrap.state().doneWithMate(mate == 0)) {
+ // Done with this mate
+ done[mate] = true;
+ continue;
+ }
+ if(roundi >= nrounds[mate]) {
+ // Not doing this round for this mate
+ continue;
+ }
+ // Figure out the seed offset
+ if(interval[mate] <= (int)roundi) {
+ // Can't do this round, seeds already packed as
+ // tight as possible
+ continue;
+ }
+ size_t offset = (interval[mate] * roundi) / nrounds[mate];
+ assert(roundi == 0 || offset > 0);
+ assert(!msinkwrap.maxed());
+ assert(msinkwrap.repOk());
+ //rnd.init(ROTL(rds[mate]->seed, 10));
+ assert(shs[mate].repOk(&ca.current()));
+ swmSeed.sdatts++;
+ // Set up seeds
+ seeds[mate]->clear();
+ Seed::mmSeeds(
+ multiseedMms, // max # mms per seed
+ seedlens[mate], // length of a multiseed seed
+ *seeds[mate], // seeds
+ gc); // global constraint
+ // Check whether the offset would drive the first seed
+ // off the end
+ if(offset > 0 && (*seeds[mate])[0].len + offset > rds[mate]->length()) {
+ continue;
+ }
+ // Instantiate the seeds
+ std::pair<int, int> instFw, instRc;
+ std::pair<int, int> inst = al.instantiateSeeds(
+ *seeds[mate], // search seeds
+ offset, // offset to begin extracting
+ interval[mate], // interval between seeds
+ *rds[mate], // read to align
+ sc, // scoring scheme
+ nofw[mate], // don't align forward read
+ norc[mate], // don't align revcomp read
+ ca, // holds some seed hits from previous reads
+ shs[mate], // holds all the seed hits
+ sdm, // metrics
+ instFw,
+ instRc);
+ assert(shs[mate].repOk(&ca.current()));
+ if(inst.first + inst.second == 0) {
+ // No seed hits! Done with this mate.
+ assert(shs[mate].empty());
+ done[mate] = true;
+ break;
+ }
+ seedsTried += (inst.first + inst.second);
+ seedsTriedMS[mate * 2 + 0] = instFw.first + instFw.second;
+ seedsTriedMS[mate * 2 + 1] = instRc.first + instRc.second;
+ // Align seeds
+ al.searchAllSeeds(
+ *seeds[mate], // search seeds
+ &ebwtFw, // BWT index
+ &ebwtBw, // BWT' index
+ *rds[mate], // read
+ sc, // scoring scheme
+ ca, // alignment cache
+ shs[mate], // store seed hits here
+ sdm, // metrics
+ prm); // per-read metrics
+ assert(shs[mate].repOk(&ca.current()));
+ if(shs[mate].empty()) {
+ // No seed alignments! Done with this mate.
+ done[mate] = true;
+ break;
+ }
+ }
+ // shs contain what we need to know to update our seed
+ // summaries for this seeding
+ for(size_t mate = 0; mate < 2; mate++) {
+ if(!shs[mate].empty()) {
+ nUniqueSeeds += shs[mate].numUniqueSeeds();
+ nUniqueSeedsMS[mate * 2 + 0] += shs[mate].numUniqueSeedsStrand(true);
+ nUniqueSeedsMS[mate * 2 + 1] += shs[mate].numUniqueSeedsStrand(false);
+ nRepeatSeeds += shs[mate].numRepeatSeeds();
+ nRepeatSeedsMS[mate * 2 + 0] += shs[mate].numRepeatSeedsStrand(true);
+ nRepeatSeedsMS[mate * 2 + 1] += shs[mate].numRepeatSeedsStrand(false);
+ seedHitTot += shs[mate].numElts();
+ seedHitTotMS[mate * 2 + 0] += shs[mate].numEltsFw();
+ seedHitTotMS[mate * 2 + 1] += shs[mate].numEltsRc();
+ }
+ }
+ double uniqFactor[2] = { 0.0f, 0.0f };
+ for(size_t i = 0; i < 2; i++) {
+ if(!shs[i].empty()) {
+ swmSeed.sdsucc++;
+ uniqFactor[i] = shs[i].uniquenessFactor();
+ }
+ }
+ // Possibly reorder the mates
+ matemap[0] = 0; matemap[1] = 1;
+ if(!shs[0].empty() && !shs[1].empty() && uniqFactor[1] > uniqFactor[0]) {
+ // Do the mate with fewer exact hits first
+ // TODO: Consider mates & orientations separately?
+ matemap[0] = 1; matemap[1] = 0;
+ }
+ for(size_t matei = 0; matei < (paired ? 2:1); matei++) {
+ size_t mate = matemap[matei];
+ if(done[mate] || msinkwrap.state().doneWithMate(mate == 0)) {
+ // Done with this mate
+ done[mate] = true;
+ continue;
+ }
+ assert(!msinkwrap.maxed());
+ assert(msinkwrap.repOk());
+ //rnd.init(ROTL(rds[mate]->seed, 10));
+ assert(shs[mate].repOk(&ca.current()));
+ if(!seedSumm) {
+ // If there aren't any seed hits...
+ if(shs[mate].empty()) {
+ continue; // on to the next mate
+ }
+ // Sort seed hits into ranks
+ shs[mate].rankSeedHits(rnd, msinkwrap.allHits());
+ int ret = 0;
+ if(paired) {
+ // Paired-end dynamic programming driver
+ ret = sd.extendSeedsPaired(
+ *rds[mate], // mate to align as anchor
+ *rds[mate ^ 1], // mate to align as opp.
+ mate == 0, // anchor is mate 1?
+ !filt[mate ^ 1],// opposite mate filtered out?
+ shs[mate], // seed hits for anchor
+ ebwtFw, // bowtie index
+ &ebwtBw, // rev bowtie index
+ ref, // packed reference strings
+ sw, // dyn prog aligner, anchor
+ osw, // dyn prog aligner, opposite
+ sc, // scoring scheme
+ pepol, // paired-end policy
+ multiseedMms, // # mms allowed in a seed
+ seedlens[mate], // length of a seed
+ interval[mate], // interval between seeds
+ minsc[mate], // min score for anchor
+ minsc[mate^1], // min score for opp.
+ nceil[mate], // N ceil for anchor
+ nceil[mate^1], // N ceil for opp.
+ nofw[mate], // don't align forward read
+ norc[mate], // don't align revcomp read
+ maxhalf, // max width on one DP side
+ doUngapped, // do ungapped alignment
+ mxIter[mate], // max extend loop iters
+ mxUg[mate], // max # ungapped extends
+ mxDp[mate], // max # DPs
+ streak[mate], // stop after streak of this many end-to-end fails
+ streak[mate], // stop after streak of this many ungap fails
+ streak[mate], // stop after streak of this many dp fails
+ mtStreak[mate], // max mate fails per seed range
+ doExtend, // extend seed hits
+ enable8, // use 8-bit SSE where possible
+ cminlen, // checkpoint if read is longer
+ cpow2, // checkpointer interval, log2
+ doTri, // triangular mini-fills?
+ tighten, // -M score tightening mode
+ ca, // seed alignment cache
+ rnd, // pseudo-random source
+ wlm, // group walk left metrics
+ swmSeed, // DP metrics, seed extend
+ swmMate, // DP metrics, mate finding
+ prm, // per-read metrics
+ &msinkwrap, // for organizing hits
+ true, // seek mate immediately
+ true, // report hits once found
+ gReportDiscordant,// look for discordant alns?
+ gReportMixed, // look for unpaired alns?
+ exhaustive[mate]);
+ // Might be done, but just with this mate
+ } else {
+ // Unpaired dynamic programming driver
+ ret = sd.extendSeeds(
+ *rds[mate], // read
+ mate == 0, // mate #1?
+ shs[mate], // seed hits
+ ebwtFw, // bowtie index
+ &ebwtBw, // rev bowtie index
+ ref, // packed reference strings
+ sw, // dynamic prog aligner
+ sc, // scoring scheme
+ multiseedMms, // # mms allowed in a seed
+ seedlens[mate], // length of a seed
+ interval[mate], // interval between seeds
+ minsc[mate], // minimum score for valid
+ nceil[mate], // N ceil for anchor
+ maxhalf, // max width on one DP side
+ doUngapped, // do ungapped alignment
+ mxIter[mate], // max extend loop iters
+ mxUg[mate], // max # ungapped extends
+ mxDp[mate], // max # DPs
+ streak[mate], // stop after streak of this many end-to-end fails
+ streak[mate], // stop after streak of this many ungap fails
+ doExtend, // extend seed hits
+ enable8, // use 8-bit SSE where possible
+ cminlen, // checkpoint if read is longer
+ cpow2, // checkpointer interval, log2
+ doTri, // triangular mini-fills?
+ tighten, // -M score tightening mode
+ ca, // seed alignment cache
+ rnd, // pseudo-random source
+ wlm, // group walk left metrics
+ swmSeed, // DP metrics, seed extend
+ prm, // per-read metrics
+ &msinkwrap, // for organizing hits
+ true, // report hits once found
+ exhaustive[mate]);
+ }
+ assert_gt(ret, 0);
+ MERGE_SW(sw);
+ MERGE_SW(osw);
+ if(ret == EXTEND_EXHAUSTED_CANDIDATES) {
+ // Not done yet
+ } else if(ret == EXTEND_POLICY_FULFILLED) {
+ // Policy is satisfied for this mate at least
+ if(msinkwrap.state().doneWithMate(mate == 0)) {
+ done[mate] = true;
+ }
+ if(msinkwrap.state().doneWithMate(mate == 1)) {
+ done[mate^1] = true;
+ }
+ } else if(ret == EXTEND_PERFECT_SCORE) {
+ // We exhausted this made at least
+ done[mate] = true;
+ } else if(ret == EXTEND_EXCEEDED_HARD_LIMIT) {
+ // We exceeded a per-read limit
+ done[mate] = true;
+ } else if(ret == EXTEND_EXCEEDED_SOFT_LIMIT) {
+ // Not done yet
+ } else {
+ //
+ cerr << "Bad return value: " << ret << endl;
+ throw 1;
+ }
+ } // if(!seedSumm)
+ } // for(size_t matei = 0; matei < 2; matei++)
+
+ // We don't necessarily have to continue investigating both
+ // mates. We continue on a mate only if its average
+ // interval length is high (> 1000)
+ for(size_t mate = 0; mate < 2; mate++) {
+ if(!done[mate] && shs[mate].averageHitsPerSeed() < seedBoostThresh) {
+ done[mate] = true;
+ }
+ }
+ } // end loop over reseeding rounds
if(seedsTried > 0) {
- prm.seedPctUnique = (float)nUniqueSeeds / seedsTried;
- prm.seedPctRep = (float)nRepeatSeeds / seedsTried;
- prm.seedHitAvg = (float)seedHitTot / seedsTried;
+ prm.seedPctUnique = (float)nUniqueSeeds / seedsTried;
+ prm.seedPctRep = (float)nRepeatSeeds / seedsTried;
+ prm.seedHitAvg = (float)seedHitTot / seedsTried;
} else {
prm.seedPctUnique = -1.0f;
prm.seedPctRep = -1.0f;
@@ -3859,31 +3921,31 @@ static void multiseedSearchWorker(void *vp) {
prm.seedPctRepMS[i] = -1.0f;
prm.seedHitAvgMS[i] = -1.0f;
}
- }
- size_t totnucs = 0;
- for(size_t mate = 0; mate < (paired ? 2:1); mate++) {
- if(filt[mate]) {
- size_t len = rdlens[mate];
- if(!nofw[mate] && !norc[mate]) {
- len *= 2;
+ }
+ size_t totnucs = 0;
+ for(size_t mate = 0; mate < (paired ? 2:1); mate++) {
+ if(filt[mate]) {
+ size_t len = rdlens[mate];
+ if(!nofw[mate] && !norc[mate]) {
+ len *= 2;
+ }
+ totnucs += len;
}
- totnucs += len;
}
- }
prm.seedsPerNuc = totnucs > 0 ? ((float)seedsTried / totnucs) : -1;
for(int i = 0; i < 4; i++) {
prm.seedsPerNucMS[i] = totnucs > 0 ? ((float)seedsTriedMS[i] / totnucs) : -1;
}
- for(size_t i = 0; i < 2; i++) {
- assert_leq(prm.nExIters, mxIter[i]);
- assert_leq(prm.nExDps, mxDp[i]);
- assert_leq(prm.nMateDps, mxDp[i]);
- assert_leq(prm.nExUgs, mxUg[i]);
- assert_leq(prm.nMateUgs, mxUg[i]);
- assert_leq(prm.nDpFail, streak[i]);
- assert_leq(prm.nUgFail, streak[i]);
- assert_leq(prm.nEeFail, streak[i]);
- }
+ for(size_t i = 0; i < 2; i++) {
+ assert_leq(prm.nExIters, mxIter[i]);
+ assert_leq(prm.nExDps, mxDp[i]);
+ assert_leq(prm.nMateDps, mxDp[i]);
+ assert_leq(prm.nExUgs, mxUg[i]);
+ assert_leq(prm.nMateUgs, mxUg[i]);
+ assert_leq(prm.nDpFail, streak[i]);
+ assert_leq(prm.nUgFail, streak[i]);
+ assert_leq(prm.nEeFail, streak[i]);
+ }
// Commit and report paired-end/unpaired alignments
//uint32_t sd = rds[0]->seed ^ rds[1]->seed;
@@ -3906,7 +3968,9 @@ static void multiseedSearchWorker(void *vp) {
prm, // per-read metrics
sc, // scoring scheme
!seedSumm, // suppress seed summaries?
- seedSumm); // suppress alignments?
+ seedSumm, // suppress alignments?
+ scUnMapped, // Consider soft-clipped bases unmapped when calculating TLEN
+ xeq);
assert(!retry || msinkwrap.empty());
} // while(retry)
} // if(rdid >= skipReads && rdid < qUpto)
@@ -3914,33 +3978,41 @@ static void multiseedSearchWorker(void *vp) {
break;
}
if(metricsPerRead) {
- MERGE_METRICS(metricsPt, nthreads > 1);
+ MERGE_METRICS(metricsPt);
nametmp = ps->read_a().name;
metricsPt.reportInterval(
- metricsOfb, metricsStderr, true, true, &nametmp);
+ metricsOfb, metricsStderr, true, &nametmp);
metricsPt.reset();
}
} // while(true)
// One last metrics merge
- MERGE_METRICS(metrics, nthreads > 1);
+ MERGE_METRICS(metrics);
if(dpLog != NULL) dpLog->close();
if(dpLogOpp != NULL) dpLogOpp->close();
#ifdef PER_THREAD_TIMING
- ss.str("");
- ss.clear();
- ss << "thread: " << tid << " cpu_changeovers: " << ncpu_changeovers << std::endl
- << "thread: " << tid << " node_changeovers: " << nnuma_changeovers << std::endl;
- std::cout << ss.str();
+ ss.str("");
+ ss.clear();
+ ss << "thread: " << tid << " cpu_changeovers: " << ncpu_changeovers << std::endl
+ << "thread: " << tid << " node_changeovers: " << nnuma_changeovers << std::endl;
+ std::cout << ss.str();
+#endif
+ }
+#ifdef WITH_TBB
+ p->done->fetch_and_add(1);
#endif
return;
}
#ifdef WITH_TBB
-void multiseedSearchWorker_2p5::operator()() const {
+//void multiseedSearchWorker_2p5::operator()() const {
+static void multiseedSearchWorker_2p5(void *vp) {
+ //int tid = *((int*)vp);
+ thread_tracking_pair *p = (thread_tracking_pair*) vp;
+ int tid = p->tid;
#else
static void multiseedSearchWorker_2p5(void *vp) {
int tid = *((int*)vp);
@@ -3961,6 +4033,7 @@ static void multiseedSearchWorker_2p5(void *vp) {
// level. These in turn can be used to diagnose performance
// problems, or generally characterize performance.
+ ThreadCounter tc;
auto_ptr<PatternSourcePerThreadFactory> patsrcFact(createPatsrcFactory(patsrc, pp, tid));
auto_ptr<PatternSourcePerThread> ps(patsrcFact->create());
@@ -4088,14 +4161,14 @@ static void multiseedSearchWorker_2p5(void *vp) {
{
// Do a periodic merge. Update global metrics, in a
// synchronized manner if needed.
- MERGE_METRICS(metrics, nthreads > 1);
+ MERGE_METRICS(metrics);
mergei = 0;
// Check if a progress message should be printed
if(tid == 0) {
// Only thread 1 prints progress messages
time_t curTime = time(0);
if(curTime - iTime >= metricsIval) {
- metrics.reportInterval(metricsOfb, metricsStderr, false, true, NULL);
+ metrics.reportInterval(metricsOfb, metricsStderr, false, NULL);
iTime = curTime;
}
}
@@ -4255,26 +4328,203 @@ static void multiseedSearchWorker_2p5(void *vp) {
prm, // per-read metrics
sc, // scoring scheme
!seedSumm, // suppress seed summaries?
- seedSumm); // suppress alignments?
+ seedSumm, // suppress alignments?
+ scUnMapped, // Consider soft-clipped bases unmapped when calculating TLEN
+ xeq);
} // if(rdid >= skipReads && rdid < qUpto)
else if(rdid >= qUpto) {
break;
}
if(metricsPerRead) {
- MERGE_METRICS(metricsPt, nthreads > 1);
+ MERGE_METRICS(metricsPt);
nametmp = ps->read_a().name;
metricsPt.reportInterval(
- metricsOfb, metricsStderr, true, true, &nametmp);
+ metricsOfb, metricsStderr, true, &nametmp);
metricsPt.reset();
}
} // while(true)
// One last metrics merge
- MERGE_METRICS(metrics, nthreads > 1);
+ MERGE_METRICS(metrics);
+#ifdef WITH_TBB
+ p->done->fetch_and_add(1);
+#endif
return;
}
+
+/**
+ * Print friendly-ish message pertaining to failed system call.
+ */
+static void errno_message() {
+ int errnum = errno;
+ cerr << "errno is " << errnum << endl;
+ perror("perror error: ");
+}
+
+/**
+ * Delete PID file. Raise error if the file doesn't exist or if
+ * we fail to delete it.
+ */
+void del_pid(const char* dirname,int pid) {
+ char* fname = (char*)calloc(FNAME_SIZE, sizeof(char));
+ if(fname == NULL) {
+ errno_message();
+ cerr << "del_pid: could not allocate buffer" << endl;
+ throw 1;
+ }
+ snprintf(fname, FNAME_SIZE, "%s/%d", dirname, pid);
+ if(unlink(fname) != 0) {
+ if(errno != ENOENT) {
+ errno_message();
+ cerr << "del_pid: could not delete PID file " << fname << endl;
+ free(fname);
+ throw 1;
+ } else {
+ // Probably just a race between processes
+ }
+ }
+ free(fname);
+}
+
+/**
+ * Write PID file.
+ */
+static void write_pid(const char* dirname,int pid) {
+ struct stat dinfo;
+ if(stat(dirname, &dinfo) != 0) {
+ if(mkdir(dirname, 0755) != 0) {
+ if(errno != EEXIST) {
+ errno_message();
+ cerr << "write_pid: could not create PID directory " << dirname << endl;
+ throw 1;
+ }
+ }
+ }
+ char* fname = (char*)calloc(FNAME_SIZE, sizeof(char));
+ if(fname == NULL) {
+ errno_message();
+ cerr << "write_pid: could not allocate buffer" << endl;
+ throw 1;
+ }
+ snprintf(fname, FNAME_SIZE, "%s/%d", dirname, pid);
+ FILE *f = fopen(fname, "w");
+ if(f == NULL) {
+ errno_message();
+ cerr << "write_pid: could not open PID file " << fname << endl;
+ throw 1;
+ }
+ if(fclose(f) != 0) {
+ errno_message();
+ cerr << "write_pid: could not close PID file " << fname << endl;
+ throw 1;
+ }
+ free(fname);
+}
+
+/**
+ * Read all the PID files in the given PID directory. If the
+ * process corresponding to a PID file seems to have expired,
+ * delete the PID file. Return the lowest PID encountered for
+ * a still-valid process.
+ */
+static int read_dir(const char* dirname, int* num_pids) {
+ DIR *dir;
+ struct dirent *ent;
+ char* fname = (char*)calloc(FNAME_SIZE, sizeof(char));
+ if(fname == NULL) {
+ errno_message();
+ cerr << "read_dir: could not allocate buffer" << endl;
+ throw 1;
+ }
+ dir = opendir(dirname);
+ if(dir == NULL) {
+ errno_message();
+ cerr << "read_dir: could not open directory " << dirname << endl;
+ free(fname);
+ throw 1;
+ }
+ int lowest_pid = -1;
+ while(true) {
+ errno = 0;
+ ent = readdir(dir);
+ if(ent == NULL) {
+ if(errno != 0) {
+ errno_message();
+ cerr << "read_dir: could not read directory " << dirname << endl;
+ free(fname);
+ throw 1;
+ }
+ break;
+ }
+ if(ent->d_name[0] == '.') {
+ continue;
+ }
+ int pid = atoi(ent->d_name);
+ if(kill(pid, 0) != 0) {
+ if(errno == ESRCH) {
+ del_pid(dirname, pid);
+ continue;
+ } else {
+ errno_message();
+ cerr << "read_dir: could not interrogate pid " << pid << endl;
+ free(fname);
+ throw 1;
+ }
+ }
+ (*num_pids)++;
+ if(pid < lowest_pid || lowest_pid == -1) {
+ lowest_pid = pid;
+ }
+ }
+ if(closedir(dir) != 0) {
+ errno_message();
+ cerr << "read_dir: could not close directory " << dir << endl;
+ free(fname);
+ throw 1;
+ }
+ free(fname);
+ return lowest_pid;
+}
+
+template<typename T>
+static void steal_threads(int pid, int orig_nthreads, EList<int>& tids, EList<T*>& threads)
+{
+ int ncpu = thread_ceiling;
+ if(thread_ceiling <= nthreads) {
+ return;
+ }
+ int num_pids = 0;
+ int lowest_pid = read_dir(thread_stealing_dir.c_str(), &num_pids);
+ if(lowest_pid != pid) {
+ return;
+ }
+ int in_use = ((num_pids-1) * orig_nthreads) + nthreads;
+ if(in_use < ncpu) {
+ nthreads++;
+ tids.push_back(nthreads);
+ threads.push_back(new T(multiseedSearchWorker, (void*)&tids.back()));
+ cerr << "pid " << pid << " started new worker # " << nthreads << endl;
+ }
+}
+
+template<typename T>
+static void thread_monitor(int pid, int orig_threads, EList<int>& tids, EList<T*>& threads)
+{
+ for(int j = 0; j < 10; j++) {
+ sleep(1);
+ }
+ int steal_ctr = 1;
+ while(thread_counter > 0) {
+ steal_threads(pid, orig_threads, tids, threads);
+ steal_ctr++;
+ for(int j = 0; j < 10; j++) {
+ sleep(1);
+ }
+ }
+}
+
/**
* Called once per alignment job. Sets up global pointers to the
* shared global data structures, creates per-thread structures, then
@@ -4314,12 +4564,19 @@ static void multiseedSearch(
delete _t;
if(!refs->loaded()) throw 1;
multiseed_refs = refs.get();
+ sigset_t set;
+ sigemptyset(&set);
+ sigaddset(&set, SIGPIPE);
+ pthread_sigmask(SIG_BLOCK, &set, NULL);
+ EList<int> tids;
#ifdef WITH_TBB
- tbb::task_group tbb_grp;
+ //tbb::task_group tbb_grp;
+ EList<std::thread*> threads(nthreads);
#else
- AutoArray<tthread::thread*> threads(nthreads+1);
- AutoArray<int> tids(nthreads+1);
+ EList<tthread::thread*> threads;
#endif
+ threads.reserveExact(std::max(nthreads, thread_ceiling));
+ tids.reserveExact(std::max(nthreads, thread_ceiling));
{
// Load the other half of the index into memory
assert(!ebwtFw.isInMemory());
@@ -4349,34 +4606,65 @@ static void multiseedSearch(
startVerbose);
}
// Start the metrics thread
+
+#ifdef WITH_TBB
+ tbb::atomic<int> all_threads_done;
+ all_threads_done = 0;
+#endif
{
Timer _t(cerr, "Multiseed full-index search: ", timing);
- for(int i = 1; i <= nthreads; i++) {
+ int pid = 0;
+ if(thread_stealing) {
+ pid = getpid();
+ write_pid(thread_stealing_dir.c_str(), pid);
+ thread_counter = 0;
+ }
+
+ for(int i = 0; i < nthreads; i++) {
#ifdef WITH_TBB
+ thread_tracking_pair tp;
+ tp.tid = i;
+ tp.done = &all_threads_done;
if(bowtie2p5) {
- tbb_grp.run(multiseedSearchWorker_2p5(i));
+ threads.push_back(new std::thread(multiseedSearchWorker_2p5, (void*) &tp));
} else {
- tbb_grp.run(multiseedSearchWorker(i));
+ threads.push_back(new std::thread(multiseedSearchWorker, (void*) &tp));
}
- }
- tbb_grp.wait();
+ threads[i]->detach();
+ SLEEP(10);
#else
// Thread IDs start at 1
- tids[i] = i;
+ tids.push_back(i);
if(bowtie2p5) {
- threads[i] = new tthread::thread(multiseedSearchWorker_2p5, (void*)&tids[i]);
+ threads.push_back(new tthread::thread(multiseedSearchWorker_2p5, (void*)&tids.back()));
} else {
- threads[i] = new tthread::thread(multiseedSearchWorker, (void*)&tids[i]);
+ threads.push_back(new tthread::thread(multiseedSearchWorker, (void*)&tids.back()));
}
+#endif
}
- for (int i = 1; i <= nthreads; i++) {
+
+ if(thread_stealing) {
+ int orig_threads = nthreads;
+ thread_monitor(pid, orig_threads, tids, threads);
+ }
+
+#ifdef WITH_TBB
+ while(all_threads_done < nthreads) {
+ SLEEP(10);
+ }
+#else
+ for (int i = 0; i < nthreads; i++) {
threads[i]->join();
}
#endif
+
+ if(thread_stealing) {
+ del_pid(thread_stealing_dir.c_str(), pid);
+ }
}
if(!metricsPerRead && (metricsOfb != NULL || metricsStderr)) {
- metrics.reportInterval(metricsOfb, metricsStderr, true, false, NULL);
+ metrics.reportInterval(metricsOfb, metricsStderr, true, NULL);
}
}
@@ -4409,6 +4697,8 @@ static void driver(
solexaQuals, // true -> qualities are on solexa64 scale
phred64Quals, // true -> qualities are on phred64 scale
integerQuals, // true -> qualities are space-separated numbers
+ gTrim5, // amt to hard clip from 5' end
+ gTrim3, // amt to hard clip from 3' end
fastaContLen, // length of sampled reads for FastaContinuous...
fastaContFreq, // frequency of sampled reads for FastaContinuous...
skipReads, // skip the first 'skip' patterns
@@ -4508,11 +4798,12 @@ static void driver(
ebwt.evictFromMemory();
}
OutputQueue oq(
- *fout, // out file buffer
- reorder && nthreads > 1, // whether to reorder when there's >1 thread
- nthreads, // # threads
- nthreads > 1, // whether to be thread-safe
- skipReads); // first read will have this rdid
+ *fout, // out file buffer
+ reorder && (nthreads > 1 || thread_stealing), // whether to reorder
+ nthreads, // # threads
+ nthreads > 1 || thread_stealing, // whether to be thread-safe
+ readsPerBatch, // size of output buffer of reads
+ skipReads); // first read will have this rdid
{
Timer _t(cerr, "Time searching: ", timing);
// Set up penalities
@@ -4720,6 +5011,13 @@ int bowtie(int argc, const char **argv) {
printUsage(cerr);
return 1;
}
+
+ thread_stealing = thread_ceiling > nthreads;
+ if(thread_stealing && thread_stealing_dir.empty()) {
+ cerr << "When --thread-ceiling is specified, must also specify --thread-piddir" << endl;
+ printUsage(cerr);
+ return 1;
+ }
// Get query filename
bool got_reads = !queries.empty() || !mates1.empty() || !mates12.empty();
diff --git a/diff_sample.h b/diff_sample.h
index 91acc2f..5943163 100644
--- a/diff_sample.h
+++ b/diff_sample.h
@@ -710,7 +710,7 @@ static void VSorting_worker(void *vp)
while(true) {
size_t cur = 0;
{
- ThreadSafe ts(param->mutex, true);
+ ThreadSafe ts(*param->mutex);
cur = *(param->cur);
(*param->cur)++;
}
@@ -743,177 +743,177 @@ static void VSorting_worker(void *vp)
*/
template <typename TStr>
void DifferenceCoverSample<TStr>::build(int nthreads) {
- // Local names for relevant types
- VMSG_NL("Building DifferenceCoverSample");
- // Local names for relevant data
- const TStr& t = this->text();
- uint32_t v = this->v();
- assert_gt(v, 2);
- // Build s'
- EList<TIndexOffU> sPrime;
- // Need to allocate 2 extra elements at the end of the sPrime and _isaPrime
- // arrays. One element that's less than all others, and another that acts
- // as needed padding for the Larsson-Sadakane sorting code.
- size_t padding = 1;
- VMSG_NL(" Building sPrime");
- buildSPrime(sPrime, padding);
- size_t sPrimeSz = sPrime.size() - padding;
- assert_gt(sPrime.size(), padding);
- assert_leq(sPrime.size(), t.length() + padding + 1);
- TIndexOffU nextRank = 0;
- {
- VMSG_NL(" Building sPrimeOrder");
- EList<TIndexOffU> sPrimeOrder;
- sPrimeOrder.resizeExact(sPrimeSz);
- for(TIndexOffU i = 0; i < sPrimeSz; i++) {
- sPrimeOrder[i] = i;
- }
- // sPrime now holds suffix-offsets for DC samples.
- {
- Timer timer(cout, " V-Sorting samples time: ", this->verbose());
- VMSG_NL(" V-Sorting samples");
- // Extract backing-store array from sPrime and sPrimeOrder;
- // the mkeyQSortSuf2 routine works on the array for maximum
- // efficiency
- TIndexOffU *sPrimeArr = (TIndexOffU*)sPrime.ptr();
- assert_eq(sPrimeArr[0], sPrime[0]);
- assert_eq(sPrimeArr[sPrimeSz-1], sPrime[sPrimeSz-1]);
- TIndexOffU *sPrimeOrderArr = (TIndexOffU*)sPrimeOrder.ptr();
- assert_eq(sPrimeOrderArr[0], sPrimeOrder[0]);
- assert_eq(sPrimeOrderArr[sPrimeSz-1], sPrimeOrder[sPrimeSz-1]);
- // Sort sample suffixes up to the vth character using a
- // multikey quicksort. Sort time is proportional to the
- // number of samples times v. It isn't quadratic.
- // sPrimeOrder is passed in as a swapping partner for
- // sPrimeArr, i.e., every time the multikey qsort swaps
- // elements in sPrime, it swaps the same elements in
- // sPrimeOrder too. This allows us to easily reconstruct
- // what the sort did.
- if(nthreads == 1) {
- mkeyQSortSuf2(t, sPrimeArr, sPrimeSz, sPrimeOrderArr, 4,
- this->verbose(), this->sanityCheck(), v);
- } else {
- int query_depth = 0;
- int tmp_nthreads = nthreads;
- while(tmp_nthreads > 0) {
- query_depth++;
- tmp_nthreads >>= 1;
- }
- EList<size_t> boundaries; // bucket boundaries for parallelization
- TIndexOffU *sOrig = NULL;
- if(this->sanityCheck()) {
- sOrig = new TIndexOffU[sPrimeSz];
- memcpy(sOrig, sPrimeArr, OFF_SIZE * sPrimeSz);
- }
- mkeyQSortSuf2(t, sPrimeArr, sPrimeSz, sPrimeOrderArr, 4,
- this->verbose(), false, query_depth, &boundaries);
- if(boundaries.size() > 0) {
+ // Local names for relevant types
+ VMSG_NL("Building DifferenceCoverSample");
+ // Local names for relevant data
+ const TStr& t = this->text();
+ uint32_t v = this->v();
+ assert_gt(v, 2);
+ // Build s'
+ EList<TIndexOffU> sPrime;
+ // Need to allocate 2 extra elements at the end of the sPrime and _isaPrime
+ // arrays. One element that's less than all others, and another that acts
+ // as needed padding for the Larsson-Sadakane sorting code.
+ size_t padding = 1;
+ VMSG_NL(" Building sPrime");
+ buildSPrime(sPrime, padding);
+ size_t sPrimeSz = sPrime.size() - padding;
+ assert_gt(sPrime.size(), padding);
+ assert_leq(sPrime.size(), t.length() + padding + 1);
+ TIndexOffU nextRank = 0;
+ {
+ VMSG_NL(" Building sPrimeOrder");
+ EList<TIndexOffU> sPrimeOrder;
+ sPrimeOrder.resizeExact(sPrimeSz);
+ for(TIndexOffU i = 0; i < sPrimeSz; i++) {
+ sPrimeOrder[i] = i;
+ }
+ // sPrime now holds suffix-offsets for DC samples.
+ {
+ Timer timer(cout, " V-Sorting samples time: ", this->verbose());
+ VMSG_NL(" V-Sorting samples");
+ // Extract backing-store array from sPrime and sPrimeOrder;
+ // the mkeyQSortSuf2 routine works on the array for maximum
+ // efficiency
+ TIndexOffU *sPrimeArr = (TIndexOffU*)sPrime.ptr();
+ assert_eq(sPrimeArr[0], sPrime[0]);
+ assert_eq(sPrimeArr[sPrimeSz-1], sPrime[sPrimeSz-1]);
+ TIndexOffU *sPrimeOrderArr = (TIndexOffU*)sPrimeOrder.ptr();
+ assert_eq(sPrimeOrderArr[0], sPrimeOrder[0]);
+ assert_eq(sPrimeOrderArr[sPrimeSz-1], sPrimeOrder[sPrimeSz-1]);
+ // Sort sample suffixes up to the vth character using a
+ // multikey quicksort. Sort time is proportional to the
+ // number of samples times v. It isn't quadratic.
+ // sPrimeOrder is passed in as a swapping partner for
+ // sPrimeArr, i.e., every time the multikey qsort swaps
+ // elements in sPrime, it swaps the same elements in
+ // sPrimeOrder too. This allows us to easily reconstruct
+ // what the sort did.
+ if(nthreads == 1) {
+ mkeyQSortSuf2(t, sPrimeArr, sPrimeSz, sPrimeOrderArr, 4,
+ this->verbose(), this->sanityCheck(), v);
+ } else {
+ int query_depth = 0;
+ int tmp_nthreads = nthreads;
+ while(tmp_nthreads > 0) {
+ query_depth++;
+ tmp_nthreads >>= 1;
+ }
+ EList<size_t> boundaries; // bucket boundaries for parallelization
+ TIndexOffU *sOrig = NULL;
+ if(this->sanityCheck()) {
+ sOrig = new TIndexOffU[sPrimeSz];
+ memcpy(sOrig, sPrimeArr, OFF_SIZE * sPrimeSz);
+ }
+ mkeyQSortSuf2(t, sPrimeArr, sPrimeSz, sPrimeOrderArr, 4,
+ this->verbose(), false, query_depth, &boundaries);
+ if(boundaries.size() > 0) {
#ifdef WITH_TBB
- tbb::task_group tbb_grp;
+ tbb::task_group tbb_grp;
#else
- AutoArray<tthread::thread*> threads(nthreads);
+ AutoArray<tthread::thread*> threads(nthreads);
#endif
- EList<VSortingParam<TStr> > tparams;
- size_t cur = 0;
- MUTEX_T mutex;
- tparams.resize(nthreads);
- for(int tid = 0; tid < nthreads; tid++) {
- // Calculate bucket sizes by doing a binary search for each
- // suffix and noting where it lands
- tparams[tid].dcs = this;
- tparams[tid].sPrimeArr = sPrimeArr;
- tparams[tid].sPrimeSz = sPrimeSz;
- tparams[tid].sPrimeOrderArr = sPrimeOrderArr;
- tparams[tid].depth = query_depth;
- tparams[tid].boundaries = &boundaries;
- tparams[tid].cur = &cur;
- tparams[tid].mutex = &mutex;
+ EList<VSortingParam<TStr> > tparams;
+ size_t cur = 0;
+ MUTEX_T mutex;
+ tparams.resize(nthreads);
+ for(int tid = 0; tid < nthreads; tid++) {
+ // Calculate bucket sizes by doing a binary search for each
+ // suffix and noting where it lands
+ tparams[tid].dcs = this;
+ tparams[tid].sPrimeArr = sPrimeArr;
+ tparams[tid].sPrimeSz = sPrimeSz;
+ tparams[tid].sPrimeOrderArr = sPrimeOrderArr;
+ tparams[tid].depth = query_depth;
+ tparams[tid].boundaries = &boundaries;
+ tparams[tid].cur = &cur;
+ tparams[tid].mutex = &mutex;
#ifdef WITH_TBB
- tbb_grp.run(VSorting_worker<TStr>(((void*)&tparams[tid])));
- }
- tbb_grp.wait();
+ tbb_grp.run(VSorting_worker<TStr>(((void*)&tparams[tid])));
+ }
+ tbb_grp.wait();
#else
- threads[tid] = new tthread::thread(VSorting_worker<TStr>, (void*)&tparams[tid]);
- }
- for (int tid = 0; tid < nthreads; tid++) {
- threads[tid]->join();
- }
+ threads[tid] = new tthread::thread(VSorting_worker<TStr>, (void*)&tparams[tid]);
+ }
+ for (int tid = 0; tid < nthreads; tid++) {
+ threads[tid]->join();
+ }
#endif
- }
- if(this->sanityCheck()) {
- sanityCheckOrderedSufs(t, t.length(), sPrimeArr, sPrimeSz, v);
- for(size_t i = 0; i < sPrimeSz; i++) {
- assert_eq(sPrimeArr[i], sOrig[sPrimeOrderArr[i]]);
- }
- delete[] sOrig;
- }
- }
- // Make sure sPrime and sPrimeOrder are consistent with
- // their respective backing-store arrays
- assert_eq(sPrimeArr[0], sPrime[0]);
- assert_eq(sPrimeArr[sPrimeSz-1], sPrime[sPrimeSz-1]);
- assert_eq(sPrimeOrderArr[0], sPrimeOrder[0]);
- assert_eq(sPrimeOrderArr[sPrimeSz-1], sPrimeOrder[sPrimeSz-1]);
- }
- // Now assign the ranking implied by the sorted sPrime/sPrimeOrder
- // arrays back into sPrime.
- VMSG_NL(" Allocating rank array");
- _isaPrime.resizeExact(sPrime.size());
- ASSERT_ONLY(_isaPrime.fill(OFF_MASK));
- assert_gt(_isaPrime.size(), 0);
- {
- Timer timer(cout, " Ranking v-sort output time: ", this->verbose());
- VMSG_NL(" Ranking v-sort output");
- for(size_t i = 0; i < sPrimeSz-1; i++) {
- // Place the appropriate ranking
- _isaPrime[sPrimeOrder[i]] = nextRank;
- // If sPrime[i] and sPrime[i+1] are identical up to v, then we
- // should give the next suffix the same rank
- if(!suffixSameUpTo(t, sPrime[i], sPrime[i+1], v)) nextRank++;
- }
- _isaPrime[sPrimeOrder[sPrimeSz-1]] = nextRank; // finish off
+ }
+ if(this->sanityCheck()) {
+ sanityCheckOrderedSufs(t, t.length(), sPrimeArr, sPrimeSz, v);
+ for(size_t i = 0; i < sPrimeSz; i++) {
+ assert_eq(sPrimeArr[i], sOrig[sPrimeOrderArr[i]]);
+ }
+ delete[] sOrig;
+ }
+ }
+ // Make sure sPrime and sPrimeOrder are consistent with
+ // their respective backing-store arrays
+ assert_eq(sPrimeArr[0], sPrime[0]);
+ assert_eq(sPrimeArr[sPrimeSz-1], sPrime[sPrimeSz-1]);
+ assert_eq(sPrimeOrderArr[0], sPrimeOrder[0]);
+ assert_eq(sPrimeOrderArr[sPrimeSz-1], sPrimeOrder[sPrimeSz-1]);
+ }
+ // Now assign the ranking implied by the sorted sPrime/sPrimeOrder
+ // arrays back into sPrime.
+ VMSG_NL(" Allocating rank array");
+ _isaPrime.resizeExact(sPrime.size());
+ ASSERT_ONLY(_isaPrime.fill(OFF_MASK));
+ assert_gt(_isaPrime.size(), 0);
+ {
+ Timer timer(cout, " Ranking v-sort output time: ", this->verbose());
+ VMSG_NL(" Ranking v-sort output");
+ for(size_t i = 0; i < sPrimeSz-1; i++) {
+ // Place the appropriate ranking
+ _isaPrime[sPrimeOrder[i]] = nextRank;
+ // If sPrime[i] and sPrime[i+1] are identical up to v, then we
+ // should give the next suffix the same rank
+ if(!suffixSameUpTo(t, sPrime[i], sPrime[i+1], v)) nextRank++;
+ }
+ _isaPrime[sPrimeOrder[sPrimeSz-1]] = nextRank; // finish off
#ifndef NDEBUG
- for(size_t i = 0; i < sPrimeSz; i++) {
- assert_neq(OFF_MASK, _isaPrime[i]);
- assert_lt(_isaPrime[i], sPrimeSz);
- }
+ for(size_t i = 0; i < sPrimeSz; i++) {
+ assert_neq(OFF_MASK, _isaPrime[i]);
+ assert_lt(_isaPrime[i], sPrimeSz);
+ }
#endif
- }
- // sPrimeOrder is destroyed
- // All the information we need is now in _isaPrime
- }
- _isaPrime[_isaPrime.size()-1] = (TIndexOffU)sPrimeSz;
- sPrime[sPrime.size()-1] = (TIndexOffU)sPrimeSz;
- // _isaPrime[_isaPrime.size()-1] and sPrime[sPrime.size()-1] are just
- // spacer for the Larsson-Sadakane routine to use
- {
- Timer timer(cout, " Invoking Larsson-Sadakane on ranks time: ", this->verbose());
- VMSG_NL(" Invoking Larsson-Sadakane on ranks");
- if(sPrime.size() >= LS_SIZE) {
- cerr << "Error; sPrime array has so many elements that it can't be converted to a signed array without overflow." << endl;
- throw 1;
- }
- LarssonSadakane<TIndexOff> ls;
- ls.suffixsort(
- (TIndexOff*)_isaPrime.ptr(),
- (TIndexOff*)sPrime.ptr(),
- (TIndexOff)sPrimeSz,
- (TIndexOff)sPrime.size(),
- 0);
- }
- // chop off final character of _isaPrime
- _isaPrime.resizeExact(sPrimeSz);
- for(size_t i = 0; i < _isaPrime.size(); i++) {
- _isaPrime[i]--;
- }
+ }
+ // sPrimeOrder is destroyed
+ // All the information we need is now in _isaPrime
+ }
+ _isaPrime[_isaPrime.size()-1] = (TIndexOffU)sPrimeSz;
+ sPrime[sPrime.size()-1] = (TIndexOffU)sPrimeSz;
+ // _isaPrime[_isaPrime.size()-1] and sPrime[sPrime.size()-1] are just
+ // spacer for the Larsson-Sadakane routine to use
+ {
+ Timer timer(cout, " Invoking Larsson-Sadakane on ranks time: ", this->verbose());
+ VMSG_NL(" Invoking Larsson-Sadakane on ranks");
+ if(sPrime.size() >= LS_SIZE) {
+ cerr << "Error; sPrime array has so many elements that it can't be converted to a signed array without overflow." << endl;
+ throw 1;
+ }
+ LarssonSadakane<TIndexOff> ls;
+ ls.suffixsort(
+ (TIndexOff*)_isaPrime.ptr(),
+ (TIndexOff*)sPrime.ptr(),
+ (TIndexOff)sPrimeSz,
+ (TIndexOff)sPrime.size(),
+ 0);
+ }
+ // chop off final character of _isaPrime
+ _isaPrime.resizeExact(sPrimeSz);
+ for(size_t i = 0; i < _isaPrime.size(); i++) {
+ _isaPrime[i]--;
+ }
#ifndef NDEBUG
- for(size_t i = 0; i < sPrimeSz-1; i++) {
- assert_lt(_isaPrime[i], sPrimeSz);
- assert(i == 0 || _isaPrime[i] != _isaPrime[i-1]);
- }
+ for(size_t i = 0; i < sPrimeSz-1; i++) {
+ assert_lt(_isaPrime[i], sPrimeSz);
+ assert(i == 0 || _isaPrime[i] != _isaPrime[i-1]);
+ }
#endif
- VMSG_NL(" Sanity-checking and returning");
- if(this->sanityCheck()) doBuiltSanityCheck();
+ VMSG_NL(" Sanity-checking and returning");
+ if(this->sanityCheck()) doBuiltSanityCheck();
}
/**
diff --git a/doc/manual.html b/doc/manual.html
index 4685a18..77d2adb 100644
--- a/doc/manual.html
+++ b/doc/manual.html
@@ -95,14 +95,14 @@
! -->
<h1 id="introduction">Introduction</h1>
<h2 id="what-is-bowtie-2">What is Bowtie 2?</h2>
-<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> (based on the <a href="http://en.wikipedia.org/wiki/Burrows-Wheeler_transform">Burrows-Wheeler [...]
-<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is often the first step in pipelines for comparative genomics, including for variation calling, ChIP-seq, RNA-seq, BS-seq. <a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> and <a href="http://bowtie-bio.sf.net">Bowtie</a> (also called "<a href="http://bowtie-bio.sf.net">Bowtie 1</a>" here) are also tightly integrated into some tools, including <a href="http://tophat.cbcb.umd.edu/">TopHat</a>: a fast splice juncti [...]
+<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an <a href="http://en.wikipedia.org/wiki/FM-index">FM Index</a> (based on the <a href="http://en.wikipedia.org/wiki/Burrows-Wheeler_transform">Burrows-Wheeler Transf [...]
+<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is often the first step in pipelines for comparative genomics, including for variation calling, ChIP-seq, RNA-seq, BS-seq. <a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> and <a href="http://bowtie-bio.sf.net">Bowtie</a> (also called "<a href="http://bowtie-bio.sf.net">Bowtie 1</a>" here) are also tightly integrated into some tools, including <a href="http://tophat.cbcb.umd.edu/">TopHat</a>: a fast splice juncti [...]
<p>If you use <a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> for your published research, please cite the <a href="http://genomebiology.com/2009/10/3/R25">Bowtie paper</a>. Thank you!</p>
<h2 id="how-is-bowtie-2-different-from-bowtie-1">How is Bowtie 2 different from Bowtie 1?</h2>
<p>Bowtie 1 was released in 2009 and was geared toward aligning the relatively short sequencing reads (up to 25-50 nucleotides) prevalent at the time. Since then, technology has improved both sequencing throughput (more nucleotides produced per sequencer per day) and read length (more nucleotides per read).</p>
<p>The chief differences between Bowtie 1 and Bowtie 2 are:</p>
<ol style="list-style-type: decimal">
-<li><p>For reads longer than about 50 bp Bowtie 2 is generally faster, more sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive. B</p></li>
+<li><p>For reads longer than about 50 bp Bowtie 2 is generally faster, more sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive.</p></li>
<li><p>Bowtie 2 supports gapped alignment with affine gap penalties. Number of gaps and gap lengths are not restricted, except by way of the configurable scoring scheme. Bowtie 1 finds just ungapped alignments.</p></li>
<li><p>Bowtie 2 supports <a href="#end-to-end-alignment-versus-local-alignment">local alignment</a>, which doesn't require reads to align end-to-end. Local alignments might be "trimmed" ("soft clipped") at one or both extremes in a way that optimizes alignment score. Bowtie 2 also supports <a href="#end-to-end-alignment-versus-local-alignment">end-to-end alignment</a> which, like Bowtie 1, requires that the read align entirely.</p></li>
<li><p>There is no upper limit on read length in Bowtie 2. Bowtie 1 had an upper limit of around 1000 bp.</p></li>
@@ -124,7 +124,7 @@
<h2 id="building-from-source">Building from source</h2>
<p>Building Bowtie 2 from source requires a GNU-like environment with GCC, GNU Make and other basics. It should be possible to build Bowtie 2 on most vanilla Linux installations or on a Mac installation with <a href="http://developer.apple.com/xcode/">Xcode</a> installed. Bowtie 2 can also be built on Windows using a 64-bit MinGW distribution and MSYS. In order to simplify the MinGW setup it might be worth investigating popular MinGW personal builds since these are coming already prepare [...]
<p>First, download the source package from the <a href="https://sourceforge.net/projects/bowtie-bio/files/bowtie2/">sourceforge site</a>. Make sure you're getting the source package; the file downloaded should end in <code>-source.zip</code>. Unzip the file, change to the unzipped directory, and build the Bowtie 2 tools by running GNU <code>make</code> (usually with the command <code>make</code>, but sometimes with <code>gmake</code>) with no arguments. If building with MinGW, run <code> [...]
-<p>+Bowtie 2 is using the multithreading software model in order to +speed up execution times on SMP architectures where this is possible. +The Threading Building Blocks library, TBB, is now the default +threading library in Bowtie 2. On POSIX platforms (like Linux, Mac +OS, etc.) if TBB is not available the pthread library will be used. +Although it is possible to use pthread library on Windows, a non-POSIX +platform, due to performance reasons Bowtie 2 will try to use Windows +native m [...]
+<p>Bowtie 2 is using the multithreading software model in order to speed up execution times on SMP architectures where this is possible. The Threading Building Blocks library, TBB, is now the default threading library in Bowtie 2. On POSIX platforms (like Linux, Mac OS, etc.) if TBB is not available the pthread library will be used. Although it is possible to use pthread library on Windows, a non-POSIX platform, due to performance reasons Bowtie 2 will try to use Windows native multithre [...]
<h2 id="adding-to-path">Adding to PATH</h2>
<p>By adding your new Bowtie 2 directory to your <a href="http://en.wikipedia.org/wiki/PATH_(variable)">PATH environment variable</a>, you ensure that whenever you run <code>bowtie2</code>, <code>bowtie2-build</code> or <code>bowtie2-inspect</code> from the command line, you will get the version you just installed without having to specify the entire path. This is recommended for most users. To do this, follow your operating system's instructions for adding the directory to your <a href= [...]
<p>If you would like to install Bowtie 2 by copying the Bowtie 2 executable files to an existing directory in your <a href="http://en.wikipedia.org/wiki/PATH_(variable)">PATH</a>, make sure that you copy all the executables, including <code>bowtie2</code>, <code>bowtie2-align-s</code>, <code>bowtie2-align-l</code>, <code>bowtie2-build</code>, <code>bowtie2-build-s</code>, <code>bowtie2-build-l</code>, <code>bowtie2-inspect</code>, <code>bowtie2-inspect-s</code> and <code>bowtie2-inspect- [...]
@@ -235,7 +235,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<h2 id="multiseed-heuristic">Multiseed heuristic</h2>
<p>To rapidly narrow the number of possible alignments that must be considered, Bowtie 2 begins by extracting substrings ("seeds") from the read and its reverse complement and aligning them in an ungapped fashion with the help of the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a>. This is "multiseed alignment" and it is similar to what <a href="http://genomebiology.com/2009/10/3/R25">Bowtie 1 does</a>, except Bowtie 1 attempts to align the entire [...]
<p>This initial step makes Bowtie 2 much faster than it would be without such a filter, but at the expense of missing some valid alignments. For instance, it is possible for a read to have a valid overall alignment but to have no valid seed alignments because each potential seed alignment is interrupted by too many mismatches or gaps.</p>
-<p>The trade-off between speed and sensitivity/accuracy can be adjusted by setting the seed length (<a href="#bowtie2-options-L"><code>-L</code></a>), the interval between extracted seeds (<a href="#bowtie2-options-I"><code>-i</code></a>), and the number of mismatches permitted per seed (<a href="#bowtie2-options-N"><code>-N</code></a>). For more sensitive alignment, set these parameters to (a) make the seeds closer together, (b) make the seeds shorter, and/or (c) allow more mismatches. [...]
+<p>The trade-off between speed and sensitivity/accuracy can be adjusted by setting the seed length (<a href="#bowtie2-options-L"><code>-L</code></a>), the interval between extracted seeds (<a href="#bowtie2-options-i"><code>-i</code></a>), and the number of mismatches permitted per seed (<a href="#bowtie2-options-N"><code>-N</code></a>). For more sensitive alignment, set these parameters to (a) make the seeds closer together, (b) make the seeds shorter, and/or (c) allow more mismatches. [...]
<p><a href="#bowtie2-options-D"><code>-D</code></a> and <a href="#bowtie2-options-R"><code>-R</code></a> are also options that adjust the trade-off between speed and sensitivity/accuracy.</p>
<h3 id="fm-index-memory-footprint">FM Index memory footprint</h3>
<p>Bowtie 2 uses the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> to find ungapped alignments for seeds. This step accounts for the bulk of Bowtie 2's memory footprint, as the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> itself is typically the largest data structure used. For instance, the memory footprint of the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> for the human genome is about 3.2 gigabytes of RAM.</p>
@@ -244,7 +244,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<p>Bowtie 2 allows alignments to overlap ambiguous characters in the reference. An alignment position that contains an ambiguous character in the read, reference, or both, is penalized according to <a href="#bowtie2-options-np"><code>--np</code></a>. <a href="#bowtie2-options-n-ceil"><code>--n-ceil</code></a> sets an upper limit on the number of positions that may contain ambiguous reference characters in a valid alignment. The optional field <a href="#bowtie2-build-opt-fields-xn"><code> [...]
<p>Note that the <a href="#multiseed-heuristic">multiseed heuristic</a> cannot find <em>seed</em> alignments that overlap ambiguous reference characters. For an alignment overlapping an ambiguous reference character to be found, it must have one or more seed alignments that do not overlap ambiguous reference characters.</p>
<h2 id="presets-setting-many-settings-at-once">Presets: setting many settings at once</h2>
-<p>Bowtie 2 comes with some useful combinations of parameters packaged into shorter "preset" parameters. For example, running Bowtie 2 with the <code>--very-sensitive</code> option is the same as running with options: <code>-D 20 -R 3 -N 0 -L 20 -i S,1,0.50</code>. The preset options that come with Bowtie 2 are designed to cover a wide area of the speed/sensitivity/accuracy trade-off space, with the presets ending in <code>fast</code> generally being faster but less sensitive a [...]
+<p>Bowtie 2 comes with some useful combinations of parameters packaged into shorter "preset" parameters. For example, running Bowtie 2 with the <a href="#bowtie2-options-very-sensitive"><code>--very-sensitive</code></a> option is the same as running with options: <code>-D 20 -R 3 -N 0 -L 20 -i S,1,0.50</code>. The preset options that come with Bowtie 2 are designed to cover a wide area of the speed/sensitivity/accuracy trade-off space, with the presets ending in <code>fast</cod [...]
<h2 id="filtering">Filtering</h2>
<p>Some reads are skipped or "filtered out" by Bowtie 2. For example, reads may be filtered out because they are extremely short or have a high proportion of ambiguous nucleotides. Bowtie 2 will still print a SAM record for such a read, but no alignment will be reported and the <code>YF:i</code> SAM optional field will be set to indicate the reason the read was filtered.</p>
<ul>
@@ -289,7 +289,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<li><p>If your computer has multiple processors/cores, use <code>-p</code></p>
<p>The <a href="#bowtie2-options-p"><code>-p</code></a> option causes Bowtie 2 to launch a specified number of parallel search threads. Each thread runs on a different processor/core and all threads find alignments in parallel, increasing alignment throughput by approximately a multiple of the number of threads (though in practice, speedup is somewhat worse than linear).</p></li>
<li><p>If reporting many alignments per read, try reducing <code>bowtie2-build --offrate</code></p>
-<p>If you are using <a href="#bowtie2-options-k"><code>-k</code></a> or <a href="#bowtie2-options-a"><code>-a</code></a> options and Bowtie 2 is reporting many alignments per read, using an index with a denser SA sample can speed things up considerably. To do this, specify a smaller-than-default <a href="#bowtie2-build-options-o"><code>-o</code>/<code>--offrate</code></a> value when running <code>bowtie2-build</code>. A denser SA sample yields a larger index, but is also particularly eff [...]
+<p>If you are using <a href="#bowtie2-options-k"><code>-k</code></a> or <a href="#bowtie2-options-a"><code>-a</code></a> options and Bowtie 2 is reporting many alignments per read, using an index with a denser SA sample can speed things up considerably. To do this, specify a smaller-than-default <a href="#bowtie2-options-o"><code>-o</code>/<code>--offrate</code></a> value when running <code>bowtie2-build</code>. A denser SA sample yields a larger index, but is also particularly effective [...]
<li><p>If <code>bowtie2</code> "thrashes", try increasing <code>bowtie2-build --offrate</code></p>
<p>If <code>bowtie2</code> runs very slowly on a relatively low-memory computer, try setting <a href="#bowtie2-options-o"><code>-o</code>/<code>--offrate</code></a> to a <em>larger</em> value when building the index. This decreases the memory footprint of the index.</p></li>
</ol>
@@ -302,7 +302,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>f(x) = 1.0 + 5.4 * ln(x)</code></pre>
<p>See the documentation for the option in question to learn what the parameter <code>x</code> is for. For example, in the case if the <a href="#bowtie2-options-score-min"><code>--score-min</code></a> option, the function <code>f(x)</code> sets the minimum alignment score necessary for an alignment to be considered valid, and <code>x</code> is the read length.</p>
<h3 id="usage">Usage</h3>
-<pre><code>bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r>} -S [<hit>]</code></pre>
+<pre><code>bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i>} -S [<sam>]</code></pre>
<h3 id="main-arguments">Main arguments</h3>
<table>
<tr>
@@ -339,7 +339,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
</tr>
<tr>
<td>
-<pre><code>-S <hit></code></pre>
+<pre><code>-S <sam></code></pre>
</td>
<td>
<p>File to write SAM alignments to. By default, alignments are written to the "standard out" or "stdout" filehandle (i.e. the console).</p>
@@ -370,7 +370,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>--tab5</code></pre>
</td>
<td>
-<p>Each read or pair is on a single line. An unpaired read line is [name]\t[seq]\t[qual]\n. A paired-end read line is [name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n. An input file can be a mix of unpaired and paired-end reads and Bowtie 2 recognizes each according to the number of fields, handling each as it should.</p>
+<p>Each read or pair is on a single line. An unpaired read line is <code>[name]\t[seq]\t[qual]\n</code>. A paired-end read line is <code>[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n</code>. An input file can be a mix of unpaired and paired-end reads and Bowtie 2 recognizes each according to the number of fields, handling each as it should.</p>
</td>
</tr>
<tr>
@@ -378,7 +378,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>--tab6</code></pre>
</td>
<td>
-<p>Similar to <a href="#bowtie2-options-tab5"><code>--tab5</code></a> except, for paired-end reads, the second end can have a different name from the first: [name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n</p>
+<p>Similar to <a href="#bowtie2-options-tab5"><code>--tab5</code></a> except, for paired-end reads, the second end can have a different name from the first: <code>[name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n</code></p>
</td>
</tr>
<tr>
@@ -394,7 +394,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>-f</code></pre>
</td>
<td>
-<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are FASTA files. FASTA files usually have extension <code>.fa</code>, <code>.fasta</code>, <code>.mfa</code>, <code>.fna</code> or similar. FASTA files do not have a way of specifying quality values, so when <code>-f</code> is set, the result is as if <code>--ignore-quals</code> is also set.</p>
+<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files. <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files usually have extension <code>.fa</code>, <code>.fasta</code>, <code>.mfa</code>, <code>.fna</code> or similar. <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files do not have a way of specifying quality values, so w [...]
</td>
</tr>
<tr>
@@ -402,7 +402,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>-r</code></pre>
</td>
<td>
-<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are files with one input sequence per line, without any other information (no read names, no qualities). When <code>-r</code> is set, the result is as if <code>--ignore-quals</code> is also set.</p>
+<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are files with one input sequence per line, without any other information (no read names, no qualities). When <code>-r</code> is set, the result is as if <a href="#bowtie2-options-ignore-quals"><code>--ignore-quals</code></a> is also set.</p>
</td>
</tr>
<tr>
@@ -410,7 +410,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>-c</code></pre>
</td>
<td>
-<p>The read sequences are given on command line. I.e. <code><m1></code>, <code><m2></code> and <code><singles></code> are comma-separated lists of reads rather than lists of read files. There is no way to specify read names or qualities, so <code>-c</code> also implies <code>--ignore-quals</code>.</p>
+<p>The read sequences are given on command line. I.e. <code><m1></code>, <code><m2></code> and <code><singles></code> are comma-separated lists of reads rather than lists of read files. There is no way to specify read names or qualities, so <code>-c</code> also implies <a href="#bowtie2-options-ignore-quals"><code>--ignore-quals</code></a>.</p>
</td>
</tr>
<tr>
@@ -613,7 +613,7 @@ Seed 4 rc: TTATGCATGA</code></pre>
<pre><code>--ignore-quals</code></pre>
</td>
<td>
-<p>When calculating a mismatch penalty, always consider the quality value at the mismatched position to be the highest possible, regardless of the actual value. I.e. input is treated as though all quality values are high. This is also the default behavior when the input doesn't specify quality values (e.g. in <a href="#bowtie2-options-f"><code>-f</code></a>, <a href="#bowtie2-options-R"><code>-r</code></a>, or <a href="#bowtie2-options-c"><code>-c</code></a> modes).</p>
+<p>When calculating a mismatch penalty, always consider the quality value at the mismatched position to be the highest possible, regardless of the actual value. I.e. input is treated as though all quality values are high. This is also the default behavior when the input doesn't specify quality values (e.g. in <a href="#bowtie2-options-f"><code>-f</code></a>, <a href="#bowtie2-options-r"><code>-r</code></a>, or <a href="#bowtie2-options-c"><code>-c</code></a> modes).</p>
</td>
</tr>
<tr>
@@ -949,6 +949,30 @@ Seed 4 rc: TTATGCATGA</code></pre>
<p>When printing secondary alignments, Bowtie 2 by default will write out the <code>SEQ</code> and <code>QUAL</code> strings. Specifying this option causes Bowtie 2 to print an asterisk in those fields instead.</p>
</td>
</tr>
+<tr>
+<td id="bowtie2-options-soft-clipped-unmapped-tlen-sec-seq">
+<pre><code>--soft-clipped-unmapped-tlen</code></pre>
+</td>
+<td>
+<p>Consider soft-clipped bases unmapped when calculating <code>TLEN</code>.</p>
+</td>
+</tr>
+<tr>
+<td id="bowtie2-options-sam-no-qname-trunc">
+<pre><code>--sam-no-qname-trunc</code></pre>
+</td>
+<td>
+<p>Suppress standard behavior of truncating readname at first whitespace at the expense of generating non-standard SAM</p>
+</td>
+</tr>
+<tr>
+<td id="bowtie2-options-xeq">
+<pre><code>--xeq</code></pre>
+</td>
+<td>
+<p>Use <code>'='/'X'</code>, instead of <code>'M'</code>, to specify matches/mismatches in SAM record</p>
+</td>
+</tr>
</table>
<h4 id="performance-options">Performance options</h4>
<table>
@@ -1117,118 +1141,95 @@ Seed 4 rc: TTATGCATGA</code></pre>
<table>
<tr>
<td id="bowtie2-build-opt-fields-as">
-<pre><code> AS:i:<N></code></pre>
+<pre><code>AS:i:<N></code></pre>
</td>
<td>
-<pre><code>Alignment score. Can be negative. Can be greater than 0 in [`--local`]
-mode (but not in [`--end-to-end`] mode). Only present if SAM record is for
-an aligned read.</code></pre>
+<p>Alignment score. Can be negative. Can be greater than 0 in <a href="#bowtie2-options-local"><code>--local</code></a> mode (but not in <a href="#bowtie2-options-end-to-end"><code>--end-to-end</code></a> mode). Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xs">
-<pre><code> XS:i:<N></code></pre>
+<pre><code>XS:i:<N></code></pre>
</td>
<td>
-<pre><code>Alignment score for the best-scoring alignment found other than the
-alignment reported. Can be negative. Can be greater than 0 in [`--local`]
-mode (but not in [`--end-to-end`] mode). Only present if the SAM record is
-for an aligned read and more than one alignment was found for the read.
-Note that, when the read is part of a concordantly-aligned pair, this score
-could be greater than [`AS:i`].</code></pre>
+<p>Alignment score for the best-scoring alignment found other than the alignment reported. Can be negative. Can be greater than 0 in <a href="#bowtie2-options-local"><code>--local</code></a> mode (but not in <a href="#bowtie2-options-end-to-end"><code>--end-to-end</code></a> mode). Only present if the SAM record is for an aligned read and more than one alignment was found for the read. Note that, when the read is part of a concordantly-aligned pair, this score could be greater than <a hr [...]
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-ys">
-<pre><code> YS:i:<N></code></pre>
+<pre><code>YS:i:<N></code></pre>
</td>
<td>
-<pre><code>Alignment score for opposite mate in the paired-end alignment. Only present
-if the SAM record is for a read that aligned as part of a paired-end
-alignment.</code></pre>
+<p>Alignment score for opposite mate in the paired-end alignment. Only present if the SAM record is for a read that aligned as part of a paired-end alignment.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xn">
-<pre><code> XN:i:<N></code></pre>
+<pre><code>XN:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of ambiguous bases in the reference covering this alignment.
-Only present if SAM record is for an aligned read.</code></pre>
+<p>The number of ambiguous bases in the reference covering this alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xm">
-<pre><code> XM:i:<N></code></pre>
+<pre><code>XM:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of mismatches in the alignment. Only present if SAM record is
-for an aligned read.</code></pre>
+<p>The number of mismatches in the alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xo">
-<pre><code> XO:i:<N></code></pre>
+<pre><code>XO:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of gap opens, for both read and reference gaps, in the alignment.
-Only present if SAM record is for an aligned read.</code></pre>
+<p>The number of gap opens, for both read and reference gaps, in the alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xg">
-<pre><code> XG:i:<N></code></pre>
+<pre><code>XG:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of gap extensions, for both read and reference gaps, in the
-alignment. Only present if SAM record is for an aligned read.</code></pre>
+<p>The number of gap extensions, for both read and reference gaps, in the alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-nm">
-<pre><code> NM:i:<N></code></pre>
+<pre><code>NM:i:<N></code></pre>
</td>
<td>
-<pre><code>The edit distance; that is, the minimal number of one-nucleotide edits
-(substitutions, insertions and deletions) needed to transform the read
-string into the reference string. Only present if SAM record is for an
-aligned read.</code></pre>
+<p>The edit distance; that is, the minimal number of one-nucleotide edits (substitutions, insertions and deletions) needed to transform the read string into the reference string. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-yf">
-<pre><code> YF:Z:<S></code></pre>
+<pre><code>YF:Z:<S></code></pre>
</td>
<td>
-<pre><code>String indicating reason why the read was filtered out. See also:
-[Filtering]. Only appears for reads that were filtered out.</code></pre>
+<p>String indicating reason why the read was filtered out. See also: <a href="#filtering">Filtering</a>. Only appears for reads that were filtered out.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-yt">
-<pre><code> YT:Z:<S></code></pre>
+<pre><code>YT:Z:<S></code></pre>
</td>
<td>
-<pre><code>Value of `UU` indicates the read was not part of a pair. Value of `CP`
-indicates the read was part of a pair and the pair aligned concordantly.
-Value of `DP` indicates the read was part of a pair and the pair aligned
-discordantly. Value of `UP` indicates the read was part of a pair but the
-pair failed to aligned either concordantly or discordantly.</code></pre>
+<p>Value of <code>UU</code> indicates the read was not part of a pair. Value of <code>CP</code> indicates the read was part of a pair and the pair aligned concordantly. Value of <code>DP</code> indicates the read was part of a pair and the pair aligned discordantly. Value of <code>UP</code> indicates the read was part of a pair but the pair failed to aligned either concordantly or discordantly.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-md">
-<pre><code> MD:Z:<S></code></pre>
+<pre><code>MD:Z:<S></code></pre>
</td>
<td>
-<pre><code>A string representation of the mismatched reference bases in the alignment.
-See [SAM] format specification for details. Only present if SAM record is
-for an aligned read.</code></pre>
+<p>A string representation of the mismatched reference bases in the alignment. See <a href="https://samtools.github.io/hts-specs/SAMtags.pdf">SAM Tags format specification</a> for details. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
</table>
<h1 id="the-bowtie2-build-indexer">The <code>bowtie2-build</code> indexer</h1>
-<p><code>bowtie2-build</code> builds a Bowtie index from a set of DNA sequences. <code>bowtie2-build</code> outputs a set of 6 files with suffixes <code>.1.bt2</code>, <code>.2.bt2</code>, <code>.3.bt2</code>, <code>.4.bt2</code>, <code>.rev.1.bt2</code>, and <code>.rev.2.bt2</code>. In the case of a large index these suffixes will have a <code>bt2l</code> termination. These files together constitute the index: they are all that is needed to align reads to that reference. The original se [...]
+<p><code>bowtie2-build</code> builds a Bowtie index from a set of DNA sequences. <code>bowtie2-build</code> outputs a set of 6 files with suffixes <code>.1.bt2</code>, <code>.2.bt2</code>, <code>.3.bt2</code>, <code>.4.bt2</code>, <code>.rev.1.bt2</code>, and <code>.rev.2.bt2</code>. In the case of a large index these suffixes will have a <code>bt2l</code> termination. These files together constitute the index: they are all that is needed to align reads to that reference. The original se [...]
<p>Bowtie 2's <code>.bt2</code> index format is different from Bowtie 1's <code>.ebwt</code> format, and they are not compatible with each other.</p>
<p>Use of Karkkainen's <a href="http://portal.acm.org/citation.cfm?id=1314852">blockwise algorithm</a> allows <code>bowtie2-build</code> to trade off between running time and memory usage. <code>bowtie2-build</code> has three options governing how it makes this trade: <a href="#bowtie2-build-options-p"><code>-p</code>/<code>--packed</code></a>, <a href="#bowtie2-build-options-bmax"><code>--bmax</code></a>/<a href="#bowtie2-build-options-bmaxdivn"><code>--bmaxdivn</code></a>, and <a href= [...]
<p>The indexer provides options pertaining to the "shape" of the index, e.g. <a href="#bowtie2-build-options-o"><code>--offrate</code></a> governs the fraction of <a href="http://en.wikipedia.org/wiki/Burrows-Wheeler_transform">Burrows-Wheeler</a> rows that are "marked" (i.e., the density of the suffix-array sample; see the original <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> paper for details). All of these options are potentially profitable t [...]
@@ -1244,7 +1245,7 @@ for an aligned read.</code></pre>
<pre><code><reference_in></code></pre>
</td>
<td>
-<p>A comma-separated list of FASTA files containing the reference sequences to be aligned to, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, the sequences themselves. E.g., <code><reference_in></code> might be <code>chr1.fa,chr2.fa,chrX.fa,chrY.fa</code>, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, this might be <code>GGTCATCCT,ACGGGTCGT,CCGTTCTATGCGGCTTA</code>.</p>
+<p>A comma-separated list of <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files containing the reference sequences to be aligned to, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, the sequences themselves. E.g., <code><reference_in></code> might be <code>chr1.fa,chr2.fa,chrX.fa,chrY.fa</code>, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, this might be <code>GGTCATCCT,ACGGGTCGT,CCGTTCTATGCGGCTTA</code>.</p>
</td>
</tr>
<tr>
@@ -1263,7 +1264,7 @@ for an aligned read.</code></pre>
<pre><code>-f</code></pre>
</td>
<td>
-<p>The reference input files (specified as <code><reference_in></code>) are FASTA files (usually having extension <code>.fa</code>, <code>.mfa</code>, <code>.fna</code> or similar).</p>
+<p>The reference input files (specified as <code><reference_in></code>) are <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files (usually having extension <code>.fa</code>, <code>.mfa</code>, <code>.fna</code> or similar).</p>
</td>
</tr>
<tr>
@@ -1271,9 +1272,7 @@ for an aligned read.</code></pre>
<pre><code>-c</code></pre>
</td>
<td>
-<p>The reference sequences are given on the command line. I.e. <code><reference_in></code> is a comma-separated list of sequences rather than a list of FASTA files.</p>
-</td>
-</tr>
+<p>The reference sequences are given on the command line. I.e. <code><reference_in></code> is a comma-separated list of sequences rather than a list of <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files.</p>
</td>
</tr>
<tr>
@@ -1414,7 +1413,7 @@ for an aligned read.</code></pre>
</tr>
</table>
<h1 id="the-bowtie2-inspect-index-inspector">The <code>bowtie2-inspect</code> index inspector</h1>
-<p><code>bowtie2-inspect</code> extracts information from a Bowtie index about what kind of index it is and what reference sequences were used to build it. When run without any options, the tool will output a FASTA file containing the sequences of the original references (with all non-<code>A</code>/<code>C</code>/<code>G</code>/<code>T</code> characters converted to <code>N</code>s). It can also be used to extract just the reference sequence names using the <a href="#bowtie2-inspect-opt [...]
+<p><code>bowtie2-inspect</code> extracts information from a Bowtie index about what kind of index it is and what reference sequences were used to build it. When run without any options, the tool will output a <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> file containing the sequences of the original references (with all non-<code>A</code>/<code>C</code>/<code>G</code>/<code>T</code> characters converted to <code>N</code>s). It can also be used to extract just the r [...]
<h2 id="command-line-2">Command Line</h2>
<p>Usage:</p>
<pre><code>bowtie2-inspect [options]* <bt2_base></code></pre>
@@ -1436,7 +1435,7 @@ for an aligned read.</code></pre>
<pre><code>-a/--across <int></code></pre>
</td>
<td>
-<p>When printing FASTA output, output a newline character every <code><int></code> bases (default: 60).</p>
+<p>When printing <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> output, output a newline character every <code><int></code> bases (default: 60).</p>
</td>
</tr>
<tr>
@@ -1495,7 +1494,7 @@ Sequence-N <name> <len></code></pre>
<p>To create an index for the <a href="http://en.wikipedia.org/wiki/Lambda_phage">Lambda phage</a> reference genome included with Bowtie 2, create a new temporary directory (it doesn't matter where), change into that directory, and run:</p>
<pre><code>$BT2_HOME/bowtie2-build $BT2_HOME/example/reference/lambda_virus.fa lambda_virus</code></pre>
<p>The command should print many lines of output then quit. When the command completes, the current directory will contain four new files that all start with <code>lambda_virus</code> and end with <code>.1.bt2</code>, <code>.2.bt2</code>, <code>.3.bt2</code>, <code>.4.bt2</code>, <code>.rev.1.bt2</code>, and <code>.rev.2.bt2</code>. These files constitute the index - you're done!</p>
-<p>You can use <code>bowtie2-build</code> to create an index for a set of FASTA files obtained from any source, including sites such as <a href="http://genome.ucsc.edu/cgi-bin/hgGateway">UCSC</a>, <a href="http://www.ncbi.nlm.nih.gov/sites/genome">NCBI</a>, and <a href="http://www.ensembl.org/">Ensembl</a>. When indexing multiple FASTA files, specify all the files using commas to separate file names. For more details on how to create an index with <code>bowtie2-build</code>, see the <a h [...]
+<p>You can use <code>bowtie2-build</code> to create an index for a set of <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files obtained from any source, including sites such as <a href="http://genome.ucsc.edu/cgi-bin/hgGateway">UCSC</a>, <a href="http://www.ncbi.nlm.nih.gov/sites/genome">NCBI</a>, and <a href="http://www.ensembl.org/">Ensembl</a>. When indexing multiple <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files, specify all the files [...]
<h2 id="aligning-example-reads">Aligning example reads</h2>
<p>Stay in the directory created in the previous step, which now contains the <code>lambda_virus</code> index files. Next, run:</p>
<pre><code>$BT2_HOME/bowtie2 -x lambda_virus -U $BT2_HOME/example/reads/reads_1.fq -S eg1.sam</code></pre>
@@ -1523,7 +1522,7 @@ r7 16 gi|9626243|ref|NC_001416.1| 4692 42 143M * 0 0 TCAGCCGGACGC
<pre><code>$BT2_HOME/bowtie2 --local -x lambda_virus -U $BT2_HOME/example/reads/longreads.fq -S eg3.sam</code></pre>
<p>This aligns the long reads to the reference genome using local alignment, with results written to the file <code>eg3.sam</code>.</p>
<h2 id="using-samtoolsbcftools-downstream">Using SAMtools/BCFtools downstream</h2>
-<p><a href="http://samtools.sourceforge.net/">SAMtools</a> is a collection of tools for manipulating and analyzing SAM and BAM alignment files. <a href="http://samtools.sourceforge.net/mpileup.shtml">BCFtools</a> is a collection of tools for calling variants and manipulating VCF and BCF files, and it is typically distributed with <a href="http://samtools.sourceforge.net/">SAMtools</a>. Using these tools together allows you to get from alignments in SAM format to variant calls in VCF form [...]
+<p><a href="http://samtools.sourceforge.net">SAMtools</a> is a collection of tools for manipulating and analyzing SAM and BAM alignment files. <a href="http://samtools.sourceforge.net/mpileup.shtml">BCFtools</a> is a collection of tools for calling variants and manipulating VCF and BCF files, and it is typically distributed with <a href="http://samtools.sourceforge.net">SAMtools</a>. Using these tools together allows you to get from alignments in SAM format to variant calls in VCF format [...]
<p>Run the paired-end example:</p>
<pre><code>$BT2_HOME/bowtie2 -x $BT2_HOME/example/index/lambda_virus -1 $BT2_HOME/example/reads/reads_1.fq -2 $BT2_HOME/example/reads/reads_2.fq -S eg2.sam</code></pre>
<p>Use <code>samtools view</code> to convert the SAM file into a BAM file. BAM is the binary format corresponding to the SAM text format. Run:</p>
diff --git a/doc/strip_markdown.pl b/doc/strip_markdown.pl
deleted file mode 100644
index 0ecc595..0000000
--- a/doc/strip_markdown.pl
+++ /dev/null
@@ -1,45 +0,0 @@
-#!/usr/bin/perl -w
-
-##
-# strip_markdown.pl
-#
-# Used to convert MANUAL.markdown to MANUAL. Leaves all manual content, but
-# strips away some of the clutter that makes it hard to read the markdown.
-#
-
-use strict;
-use warnings;
-
-my $lastBlank = 0;
-
-while(<>) {
- # Skip comments
- next if /^\s*<!--/;
- next if /^\s*!/;
- next if /^\s*-->/;
- # Skip internal links
- next if /\[.*\]: #/;
- # Skip HTML
- next if /^\s?\s?\s?<.*>\s*$/;
- # Skip HTML
- next if /^\s*<table/;
- next if /^\s*<\/td/;
- next if /^\s*<.*>\s*$/;
- # Strip [`...`]
- s/\[`/`/g;
- s/`\]/`/g;
- # Strip [#...]
- #s/\[#[^\]]*\]//g;
- # Strip (#...)
- s/\(#[^\)]*\)//g;
- # Turn hashes into spaces
- #s/^####/ /;
- #s/^###/ /;
- if(/^\s*$/) {
- next if $lastBlank;
- $lastBlank = 1;
- } else {
- $lastBlank = 0;
- }
- print $_;
-}
diff --git a/doc/website/manual.ssi b/doc/website/manual.ssi
index 5891c95..d9c222d 100644
--- a/doc/website/manual.ssi
+++ b/doc/website/manual.ssi
@@ -1,5 +1,5 @@
<h1>Table of Contents</h1>
-<p> Version <b>2.3.2</b></p>
+<p> Version <b>2.3.3</b></p>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
@@ -95,14 +95,14 @@
! -->
<h1 id="introduction">Introduction</h1>
<h2 id="what-is-bowtie-2">What is Bowtie 2?</h2>
-<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> (based on the <a href="http://en.wikipedia.org/wiki/Burrows-Wheeler_transform">Burrows-Wheeler [...]
-<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is often the first step in pipelines for comparative genomics, including for variation calling, ChIP-seq, RNA-seq, BS-seq. <a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> and <a href="http://bowtie-bio.sf.net">Bowtie</a> (also called "<a href="http://bowtie-bio.sf.net">Bowtie 1</a>" here) are also tightly integrated into some tools, including <a href="http://tophat.cbcb.umd.edu/">TopHat</a>: a fast splice juncti [...]
+<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is an ultrafast and memory-efficient tool for aligning sequencing reads to long reference sequences. It is particularly good at aligning reads of about 50 up to 100s or 1,000s of characters to relatively long (e.g. mammalian) genomes. Bowtie 2 indexes the genome with an <a href="http://en.wikipedia.org/wiki/FM-index">FM Index</a> (based on the <a href="http://en.wikipedia.org/wiki/Burrows-Wheeler_transform">Burrows-Wheeler Transf [...]
+<p><a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> is often the first step in pipelines for comparative genomics, including for variation calling, ChIP-seq, RNA-seq, BS-seq. <a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> and <a href="http://bowtie-bio.sf.net">Bowtie</a> (also called "<a href="http://bowtie-bio.sf.net">Bowtie 1</a>" here) are also tightly integrated into some tools, including <a href="http://tophat.cbcb.umd.edu/">TopHat</a>: a fast splice juncti [...]
<p>If you use <a href="http://bowtie-bio.sf.net/bowtie2">Bowtie 2</a> for your published research, please cite the <a href="http://genomebiology.com/2009/10/3/R25">Bowtie paper</a>. Thank you!</p>
<h2 id="how-is-bowtie-2-different-from-bowtie-1">How is Bowtie 2 different from Bowtie 1?</h2>
<p>Bowtie 1 was released in 2009 and was geared toward aligning the relatively short sequencing reads (up to 25-50 nucleotides) prevalent at the time. Since then, technology has improved both sequencing throughput (more nucleotides produced per sequencer per day) and read length (more nucleotides per read).</p>
<p>The chief differences between Bowtie 1 and Bowtie 2 are:</p>
<ol style="list-style-type: decimal">
-<li><p>For reads longer than about 50 bp Bowtie 2 is generally faster, more sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive. B</p></li>
+<li><p>For reads longer than about 50 bp Bowtie 2 is generally faster, more sensitive, and uses less memory than Bowtie 1. For relatively short reads (e.g. less than 50 bp) Bowtie 1 is sometimes faster and/or more sensitive.</p></li>
<li><p>Bowtie 2 supports gapped alignment with affine gap penalties. Number of gaps and gap lengths are not restricted, except by way of the configurable scoring scheme. Bowtie 1 finds just ungapped alignments.</p></li>
<li><p>Bowtie 2 supports <a href="#end-to-end-alignment-versus-local-alignment">local alignment</a>, which doesn't require reads to align end-to-end. Local alignments might be "trimmed" ("soft clipped") at one or both extremes in a way that optimizes alignment score. Bowtie 2 also supports <a href="#end-to-end-alignment-versus-local-alignment">end-to-end alignment</a> which, like Bowtie 1, requires that the read align entirely.</p></li>
<li><p>There is no upper limit on read length in Bowtie 2. Bowtie 1 had an upper limit of around 1000 bp.</p></li>
@@ -124,7 +124,7 @@
<h2 id="building-from-source">Building from source</h2>
<p>Building Bowtie 2 from source requires a GNU-like environment with GCC, GNU Make and other basics. It should be possible to build Bowtie 2 on most vanilla Linux installations or on a Mac installation with <a href="http://developer.apple.com/xcode/">Xcode</a> installed. Bowtie 2 can also be built on Windows using a 64-bit MinGW distribution and MSYS. In order to simplify the MinGW setup it might be worth investigating popular MinGW personal builds since these are coming already prepare [...]
<p>First, download the source package from the <a href="https://sourceforge.net/projects/bowtie-bio/files/bowtie2/">sourceforge site</a>. Make sure you're getting the source package; the file downloaded should end in <code>-source.zip</code>. Unzip the file, change to the unzipped directory, and build the Bowtie 2 tools by running GNU <code>make</code> (usually with the command <code>make</code>, but sometimes with <code>gmake</code>) with no arguments. If building with MinGW, run <code> [...]
-<p>+Bowtie 2 is using the multithreading software model in order to +speed up execution times on SMP architectures where this is possible. +The Threading Building Blocks library, TBB, is now the default +threading library in Bowtie 2. On POSIX platforms (like Linux, Mac +OS, etc.) if TBB is not available the pthread library will be used. +Although it is possible to use pthread library on Windows, a non-POSIX +platform, due to performance reasons Bowtie 2 will try to use Windows +native m [...]
+<p>Bowtie 2 is using the multithreading software model in order to speed up execution times on SMP architectures where this is possible. The Threading Building Blocks library, TBB, is now the default threading library in Bowtie 2. On POSIX platforms (like Linux, Mac OS, etc.) if TBB is not available the pthread library will be used. Although it is possible to use pthread library on Windows, a non-POSIX platform, due to performance reasons Bowtie 2 will try to use Windows native multithre [...]
<h2 id="adding-to-path">Adding to PATH</h2>
<p>By adding your new Bowtie 2 directory to your <a href="http://en.wikipedia.org/wiki/PATH_(variable)">PATH environment variable</a>, you ensure that whenever you run <code>bowtie2</code>, <code>bowtie2-build</code> or <code>bowtie2-inspect</code> from the command line, you will get the version you just installed without having to specify the entire path. This is recommended for most users. To do this, follow your operating system's instructions for adding the directory to your <a href= [...]
<p>If you would like to install Bowtie 2 by copying the Bowtie 2 executable files to an existing directory in your <a href="http://en.wikipedia.org/wiki/PATH_(variable)">PATH</a>, make sure that you copy all the executables, including <code>bowtie2</code>, <code>bowtie2-align-s</code>, <code>bowtie2-align-l</code>, <code>bowtie2-build</code>, <code>bowtie2-build-s</code>, <code>bowtie2-build-l</code>, <code>bowtie2-inspect</code>, <code>bowtie2-inspect-s</code> and <code>bowtie2-inspect- [...]
@@ -235,7 +235,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<h2 id="multiseed-heuristic">Multiseed heuristic</h2>
<p>To rapidly narrow the number of possible alignments that must be considered, Bowtie 2 begins by extracting substrings ("seeds") from the read and its reverse complement and aligning them in an ungapped fashion with the help of the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a>. This is "multiseed alignment" and it is similar to what <a href="http://genomebiology.com/2009/10/3/R25">Bowtie 1 does</a>, except Bowtie 1 attempts to align the entire [...]
<p>This initial step makes Bowtie 2 much faster than it would be without such a filter, but at the expense of missing some valid alignments. For instance, it is possible for a read to have a valid overall alignment but to have no valid seed alignments because each potential seed alignment is interrupted by too many mismatches or gaps.</p>
-<p>The trade-off between speed and sensitivity/accuracy can be adjusted by setting the seed length (<a href="#bowtie2-options-L"><code>-L</code></a>), the interval between extracted seeds (<a href="#bowtie2-options-I"><code>-i</code></a>), and the number of mismatches permitted per seed (<a href="#bowtie2-options-N"><code>-N</code></a>). For more sensitive alignment, set these parameters to (a) make the seeds closer together, (b) make the seeds shorter, and/or (c) allow more mismatches. [...]
+<p>The trade-off between speed and sensitivity/accuracy can be adjusted by setting the seed length (<a href="#bowtie2-options-L"><code>-L</code></a>), the interval between extracted seeds (<a href="#bowtie2-options-i"><code>-i</code></a>), and the number of mismatches permitted per seed (<a href="#bowtie2-options-N"><code>-N</code></a>). For more sensitive alignment, set these parameters to (a) make the seeds closer together, (b) make the seeds shorter, and/or (c) allow more mismatches. [...]
<p><a href="#bowtie2-options-D"><code>-D</code></a> and <a href="#bowtie2-options-R"><code>-R</code></a> are also options that adjust the trade-off between speed and sensitivity/accuracy.</p>
<h3 id="fm-index-memory-footprint">FM Index memory footprint</h3>
<p>Bowtie 2 uses the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> to find ungapped alignments for seeds. This step accounts for the bulk of Bowtie 2's memory footprint, as the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> itself is typically the largest data structure used. For instance, the memory footprint of the <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> for the human genome is about 3.2 gigabytes of RAM.</p>
@@ -244,7 +244,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<p>Bowtie 2 allows alignments to overlap ambiguous characters in the reference. An alignment position that contains an ambiguous character in the read, reference, or both, is penalized according to <a href="#bowtie2-options-np"><code>--np</code></a>. <a href="#bowtie2-options-n-ceil"><code>--n-ceil</code></a> sets an upper limit on the number of positions that may contain ambiguous reference characters in a valid alignment. The optional field <a href="#bowtie2-build-opt-fields-xn"><code> [...]
<p>Note that the <a href="#multiseed-heuristic">multiseed heuristic</a> cannot find <em>seed</em> alignments that overlap ambiguous reference characters. For an alignment overlapping an ambiguous reference character to be found, it must have one or more seed alignments that do not overlap ambiguous reference characters.</p>
<h2 id="presets-setting-many-settings-at-once">Presets: setting many settings at once</h2>
-<p>Bowtie 2 comes with some useful combinations of parameters packaged into shorter "preset" parameters. For example, running Bowtie 2 with the <code>--very-sensitive</code> option is the same as running with options: <code>-D 20 -R 3 -N 0 -L 20 -i S,1,0.50</code>. The preset options that come with Bowtie 2 are designed to cover a wide area of the speed/sensitivity/accuracy trade-off space, with the presets ending in <code>fast</code> generally being faster but less sensitive a [...]
+<p>Bowtie 2 comes with some useful combinations of parameters packaged into shorter "preset" parameters. For example, running Bowtie 2 with the <a href="#bowtie2-options-very-sensitive"><code>--very-sensitive</code></a> option is the same as running with options: <code>-D 20 -R 3 -N 0 -L 20 -i S,1,0.50</code>. The preset options that come with Bowtie 2 are designed to cover a wide area of the speed/sensitivity/accuracy trade-off space, with the presets ending in <code>fast</cod [...]
<h2 id="filtering">Filtering</h2>
<p>Some reads are skipped or "filtered out" by Bowtie 2. For example, reads may be filtered out because they are extremely short or have a high proportion of ambiguous nucleotides. Bowtie 2 will still print a SAM record for such a read, but no alignment will be reported and the <code>YF:i</code> SAM optional field will be set to indicate the reason the read was filtered.</p>
<ul>
@@ -289,7 +289,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<li><p>If your computer has multiple processors/cores, use <code>-p</code></p>
<p>The <a href="#bowtie2-options-p"><code>-p</code></a> option causes Bowtie 2 to launch a specified number of parallel search threads. Each thread runs on a different processor/core and all threads find alignments in parallel, increasing alignment throughput by approximately a multiple of the number of threads (though in practice, speedup is somewhat worse than linear).</p></li>
<li><p>If reporting many alignments per read, try reducing <code>bowtie2-build --offrate</code></p>
-<p>If you are using <a href="#bowtie2-options-k"><code>-k</code></a> or <a href="#bowtie2-options-a"><code>-a</code></a> options and Bowtie 2 is reporting many alignments per read, using an index with a denser SA sample can speed things up considerably. To do this, specify a smaller-than-default <a href="#bowtie2-build-options-o"><code>-o</code>/<code>--offrate</code></a> value when running <code>bowtie2-build</code>. A denser SA sample yields a larger index, but is also particularly eff [...]
+<p>If you are using <a href="#bowtie2-options-k"><code>-k</code></a> or <a href="#bowtie2-options-a"><code>-a</code></a> options and Bowtie 2 is reporting many alignments per read, using an index with a denser SA sample can speed things up considerably. To do this, specify a smaller-than-default <a href="#bowtie2-options-o"><code>-o</code>/<code>--offrate</code></a> value when running <code>bowtie2-build</code>. A denser SA sample yields a larger index, but is also particularly effective [...]
<li><p>If <code>bowtie2</code> "thrashes", try increasing <code>bowtie2-build --offrate</code></p>
<p>If <code>bowtie2</code> runs very slowly on a relatively low-memory computer, try setting <a href="#bowtie2-options-o"><code>-o</code>/<code>--offrate</code></a> to a <em>larger</em> value when building the index. This decreases the memory footprint of the index.</p></li>
</ol>
@@ -302,7 +302,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>f(x) = 1.0 + 5.4 * ln(x)</code></pre>
<p>See the documentation for the option in question to learn what the parameter <code>x</code> is for. For example, in the case if the <a href="#bowtie2-options-score-min"><code>--score-min</code></a> option, the function <code>f(x)</code> sets the minimum alignment score necessary for an alignment to be considered valid, and <code>x</code> is the read length.</p>
<h3 id="usage">Usage</h3>
-<pre><code>bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r>} -S [<hit>]</code></pre>
+<pre><code>bowtie2 [options]* -x <bt2-idx> {-1 <m1> -2 <m2> | -U <r> | --interleaved <i>} -S [<sam>]</code></pre>
<h3 id="main-arguments">Main arguments</h3>
<table>
<tr>
@@ -339,7 +339,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
</tr>
<tr>
<td>
-<pre><code>-S <hit></code></pre>
+<pre><code>-S <sam></code></pre>
</td>
<td>
<p>File to write SAM alignments to. By default, alignments are written to the "standard out" or "stdout" filehandle (i.e. the console).</p>
@@ -370,7 +370,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>--tab5</code></pre>
</td>
<td>
-<p>Each read or pair is on a single line. An unpaired read line is [name]\t[seq]\t[qual]\n. A paired-end read line is [name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n. An input file can be a mix of unpaired and paired-end reads and Bowtie 2 recognizes each according to the number of fields, handling each as it should.</p>
+<p>Each read or pair is on a single line. An unpaired read line is <code>[name]\t[seq]\t[qual]\n</code>. A paired-end read line is <code>[name]\t[seq1]\t[qual1]\t[seq2]\t[qual2]\n</code>. An input file can be a mix of unpaired and paired-end reads and Bowtie 2 recognizes each according to the number of fields, handling each as it should.</p>
</td>
</tr>
<tr>
@@ -378,7 +378,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>--tab6</code></pre>
</td>
<td>
-<p>Similar to <a href="#bowtie2-options-tab5"><code>--tab5</code></a> except, for paired-end reads, the second end can have a different name from the first: [name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n</p>
+<p>Similar to <a href="#bowtie2-options-tab5"><code>--tab5</code></a> except, for paired-end reads, the second end can have a different name from the first: <code>[name1]\t[seq1]\t[qual1]\t[name2]\t[seq2]\t[qual2]\n</code></p>
</td>
</tr>
<tr>
@@ -394,7 +394,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>-f</code></pre>
</td>
<td>
-<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are FASTA files. FASTA files usually have extension <code>.fa</code>, <code>.fasta</code>, <code>.mfa</code>, <code>.fna</code> or similar. FASTA files do not have a way of specifying quality values, so when <code>-f</code> is set, the result is as if <code>--ignore-quals</code> is also set.</p>
+<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files. <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files usually have extension <code>.fa</code>, <code>.fasta</code>, <code>.mfa</code>, <code>.fna</code> or similar. <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files do not have a way of specifying quality values, so w [...]
</td>
</tr>
<tr>
@@ -402,7 +402,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>-r</code></pre>
</td>
<td>
-<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are files with one input sequence per line, without any other information (no read names, no qualities). When <code>-r</code> is set, the result is as if <code>--ignore-quals</code> is also set.</p>
+<p>Reads (specified with <code><m1></code>, <code><m2></code>, <code><s></code>) are files with one input sequence per line, without any other information (no read names, no qualities). When <code>-r</code> is set, the result is as if <a href="#bowtie2-options-ignore-quals"><code>--ignore-quals</code></a> is also set.</p>
</td>
</tr>
<tr>
@@ -410,7 +410,7 @@ Reference: GCAGATTATATGAGTCAGCTACGATATTGTTTGGGGTGACACATTACGCGTCTTTGAC</code></pr
<pre><code>-c</code></pre>
</td>
<td>
-<p>The read sequences are given on command line. I.e. <code><m1></code>, <code><m2></code> and <code><singles></code> are comma-separated lists of reads rather than lists of read files. There is no way to specify read names or qualities, so <code>-c</code> also implies <code>--ignore-quals</code>.</p>
+<p>The read sequences are given on command line. I.e. <code><m1></code>, <code><m2></code> and <code><singles></code> are comma-separated lists of reads rather than lists of read files. There is no way to specify read names or qualities, so <code>-c</code> also implies <a href="#bowtie2-options-ignore-quals"><code>--ignore-quals</code></a>.</p>
</td>
</tr>
<tr>
@@ -613,7 +613,7 @@ Seed 4 rc: TTATGCATGA</code></pre>
<pre><code>--ignore-quals</code></pre>
</td>
<td>
-<p>When calculating a mismatch penalty, always consider the quality value at the mismatched position to be the highest possible, regardless of the actual value. I.e. input is treated as though all quality values are high. This is also the default behavior when the input doesn't specify quality values (e.g. in <a href="#bowtie2-options-f"><code>-f</code></a>, <a href="#bowtie2-options-R"><code>-r</code></a>, or <a href="#bowtie2-options-c"><code>-c</code></a> modes).</p>
+<p>When calculating a mismatch penalty, always consider the quality value at the mismatched position to be the highest possible, regardless of the actual value. I.e. input is treated as though all quality values are high. This is also the default behavior when the input doesn't specify quality values (e.g. in <a href="#bowtie2-options-f"><code>-f</code></a>, <a href="#bowtie2-options-r"><code>-r</code></a>, or <a href="#bowtie2-options-c"><code>-c</code></a> modes).</p>
</td>
</tr>
<tr>
@@ -949,6 +949,30 @@ Seed 4 rc: TTATGCATGA</code></pre>
<p>When printing secondary alignments, Bowtie 2 by default will write out the <code>SEQ</code> and <code>QUAL</code> strings. Specifying this option causes Bowtie 2 to print an asterisk in those fields instead.</p>
</td>
</tr>
+<tr>
+<td id="bowtie2-options-soft-clipped-unmapped-tlen-sec-seq">
+<pre><code>--soft-clipped-unmapped-tlen</code></pre>
+</td>
+<td>
+<p>Consider soft-clipped bases unmapped when calculating <code>TLEN</code>.</p>
+</td>
+</tr>
+<tr>
+<td id="bowtie2-options-sam-no-qname-trunc">
+<pre><code>--sam-no-qname-trunc</code></pre>
+</td>
+<td>
+<p>Suppress standard behavior of truncating readname at first whitespace at the expense of generating non-standard SAM</p>
+</td>
+</tr>
+<tr>
+<td id="bowtie2-options-xeq">
+<pre><code>--xeq</code></pre>
+</td>
+<td>
+<p>Use <code>'='/'X'</code>, instead of <code>'M'</code>, to specify matches/mismatches in SAM record</p>
+</td>
+</tr>
</table>
<h4 id="performance-options">Performance options</h4>
<table>
@@ -1117,118 +1141,95 @@ Seed 4 rc: TTATGCATGA</code></pre>
<table>
<tr>
<td id="bowtie2-build-opt-fields-as">
-<pre><code> AS:i:<N></code></pre>
+<pre><code>AS:i:<N></code></pre>
</td>
<td>
-<pre><code>Alignment score. Can be negative. Can be greater than 0 in [`--local`]
-mode (but not in [`--end-to-end`] mode). Only present if SAM record is for
-an aligned read.</code></pre>
+<p>Alignment score. Can be negative. Can be greater than 0 in <a href="#bowtie2-options-local"><code>--local</code></a> mode (but not in <a href="#bowtie2-options-end-to-end"><code>--end-to-end</code></a> mode). Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xs">
-<pre><code> XS:i:<N></code></pre>
+<pre><code>XS:i:<N></code></pre>
</td>
<td>
-<pre><code>Alignment score for the best-scoring alignment found other than the
-alignment reported. Can be negative. Can be greater than 0 in [`--local`]
-mode (but not in [`--end-to-end`] mode). Only present if the SAM record is
-for an aligned read and more than one alignment was found for the read.
-Note that, when the read is part of a concordantly-aligned pair, this score
-could be greater than [`AS:i`].</code></pre>
+<p>Alignment score for the best-scoring alignment found other than the alignment reported. Can be negative. Can be greater than 0 in <a href="#bowtie2-options-local"><code>--local</code></a> mode (but not in <a href="#bowtie2-options-end-to-end"><code>--end-to-end</code></a> mode). Only present if the SAM record is for an aligned read and more than one alignment was found for the read. Note that, when the read is part of a concordantly-aligned pair, this score could be greater than <a hr [...]
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-ys">
-<pre><code> YS:i:<N></code></pre>
+<pre><code>YS:i:<N></code></pre>
</td>
<td>
-<pre><code>Alignment score for opposite mate in the paired-end alignment. Only present
-if the SAM record is for a read that aligned as part of a paired-end
-alignment.</code></pre>
+<p>Alignment score for opposite mate in the paired-end alignment. Only present if the SAM record is for a read that aligned as part of a paired-end alignment.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xn">
-<pre><code> XN:i:<N></code></pre>
+<pre><code>XN:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of ambiguous bases in the reference covering this alignment.
-Only present if SAM record is for an aligned read.</code></pre>
+<p>The number of ambiguous bases in the reference covering this alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xm">
-<pre><code> XM:i:<N></code></pre>
+<pre><code>XM:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of mismatches in the alignment. Only present if SAM record is
-for an aligned read.</code></pre>
+<p>The number of mismatches in the alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xo">
-<pre><code> XO:i:<N></code></pre>
+<pre><code>XO:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of gap opens, for both read and reference gaps, in the alignment.
-Only present if SAM record is for an aligned read.</code></pre>
+<p>The number of gap opens, for both read and reference gaps, in the alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-xg">
-<pre><code> XG:i:<N></code></pre>
+<pre><code>XG:i:<N></code></pre>
</td>
<td>
-<pre><code>The number of gap extensions, for both read and reference gaps, in the
-alignment. Only present if SAM record is for an aligned read.</code></pre>
+<p>The number of gap extensions, for both read and reference gaps, in the alignment. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-nm">
-<pre><code> NM:i:<N></code></pre>
+<pre><code>NM:i:<N></code></pre>
</td>
<td>
-<pre><code>The edit distance; that is, the minimal number of one-nucleotide edits
-(substitutions, insertions and deletions) needed to transform the read
-string into the reference string. Only present if SAM record is for an
-aligned read.</code></pre>
+<p>The edit distance; that is, the minimal number of one-nucleotide edits (substitutions, insertions and deletions) needed to transform the read string into the reference string. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-yf">
-<pre><code> YF:Z:<S></code></pre>
+<pre><code>YF:Z:<S></code></pre>
</td>
<td>
-<pre><code>String indicating reason why the read was filtered out. See also:
-[Filtering]. Only appears for reads that were filtered out.</code></pre>
+<p>String indicating reason why the read was filtered out. See also: <a href="#filtering">Filtering</a>. Only appears for reads that were filtered out.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-yt">
-<pre><code> YT:Z:<S></code></pre>
+<pre><code>YT:Z:<S></code></pre>
</td>
<td>
-<pre><code>Value of `UU` indicates the read was not part of a pair. Value of `CP`
-indicates the read was part of a pair and the pair aligned concordantly.
-Value of `DP` indicates the read was part of a pair and the pair aligned
-discordantly. Value of `UP` indicates the read was part of a pair but the
-pair failed to aligned either concordantly or discordantly.</code></pre>
+<p>Value of <code>UU</code> indicates the read was not part of a pair. Value of <code>CP</code> indicates the read was part of a pair and the pair aligned concordantly. Value of <code>DP</code> indicates the read was part of a pair and the pair aligned discordantly. Value of <code>UP</code> indicates the read was part of a pair but the pair failed to aligned either concordantly or discordantly.</p>
</td>
</tr>
<tr>
<td id="bowtie2-build-opt-fields-md">
-<pre><code> MD:Z:<S></code></pre>
+<pre><code>MD:Z:<S></code></pre>
</td>
<td>
-<pre><code>A string representation of the mismatched reference bases in the alignment.
-See [SAM] format specification for details. Only present if SAM record is
-for an aligned read.</code></pre>
+<p>A string representation of the mismatched reference bases in the alignment. See <a href="https://samtools.github.io/hts-specs/SAMtags.pdf">SAM Tags format specification</a> for details. Only present if SAM record is for an aligned read.</p>
</td>
</tr>
</table>
<h1 id="the-bowtie2-build-indexer">The <code>bowtie2-build</code> indexer</h1>
-<p><code>bowtie2-build</code> builds a Bowtie index from a set of DNA sequences. <code>bowtie2-build</code> outputs a set of 6 files with suffixes <code>.1.bt2</code>, <code>.2.bt2</code>, <code>.3.bt2</code>, <code>.4.bt2</code>, <code>.rev.1.bt2</code>, and <code>.rev.2.bt2</code>. In the case of a large index these suffixes will have a <code>bt2l</code> termination. These files together constitute the index: they are all that is needed to align reads to that reference. The original se [...]
+<p><code>bowtie2-build</code> builds a Bowtie index from a set of DNA sequences. <code>bowtie2-build</code> outputs a set of 6 files with suffixes <code>.1.bt2</code>, <code>.2.bt2</code>, <code>.3.bt2</code>, <code>.4.bt2</code>, <code>.rev.1.bt2</code>, and <code>.rev.2.bt2</code>. In the case of a large index these suffixes will have a <code>bt2l</code> termination. These files together constitute the index: they are all that is needed to align reads to that reference. The original se [...]
<p>Bowtie 2's <code>.bt2</code> index format is different from Bowtie 1's <code>.ebwt</code> format, and they are not compatible with each other.</p>
<p>Use of Karkkainen's <a href="http://portal.acm.org/citation.cfm?id=1314852">blockwise algorithm</a> allows <code>bowtie2-build</code> to trade off between running time and memory usage. <code>bowtie2-build</code> has three options governing how it makes this trade: <a href="#bowtie2-build-options-p"><code>-p</code>/<code>--packed</code></a>, <a href="#bowtie2-build-options-bmax"><code>--bmax</code></a>/<a href="#bowtie2-build-options-bmaxdivn"><code>--bmaxdivn</code></a>, and <a href= [...]
<p>The indexer provides options pertaining to the "shape" of the index, e.g. <a href="#bowtie2-build-options-o"><code>--offrate</code></a> governs the fraction of <a href="http://en.wikipedia.org/wiki/Burrows-Wheeler_transform">Burrows-Wheeler</a> rows that are "marked" (i.e., the density of the suffix-array sample; see the original <a href="http://portal.acm.org/citation.cfm?id=796543">FM Index</a> paper for details). All of these options are potentially profitable t [...]
@@ -1244,7 +1245,7 @@ for an aligned read.</code></pre>
<pre><code><reference_in></code></pre>
</td>
<td>
-<p>A comma-separated list of FASTA files containing the reference sequences to be aligned to, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, the sequences themselves. E.g., <code><reference_in></code> might be <code>chr1.fa,chr2.fa,chrX.fa,chrY.fa</code>, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, this might be <code>GGTCATCCT,ACGGGTCGT,CCGTTCTATGCGGCTTA</code>.</p>
+<p>A comma-separated list of <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files containing the reference sequences to be aligned to, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, the sequences themselves. E.g., <code><reference_in></code> might be <code>chr1.fa,chr2.fa,chrX.fa,chrY.fa</code>, or, if <a href="#bowtie2-build-options-c"><code>-c</code></a> is specified, this might be <code>GGTCATCCT,ACGGGTCGT,CCGTTCTATGCGGCTTA</code>.</p>
</td>
</tr>
<tr>
@@ -1263,7 +1264,7 @@ for an aligned read.</code></pre>
<pre><code>-f</code></pre>
</td>
<td>
-<p>The reference input files (specified as <code><reference_in></code>) are FASTA files (usually having extension <code>.fa</code>, <code>.mfa</code>, <code>.fna</code> or similar).</p>
+<p>The reference input files (specified as <code><reference_in></code>) are <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files (usually having extension <code>.fa</code>, <code>.mfa</code>, <code>.fna</code> or similar).</p>
</td>
</tr>
<tr>
@@ -1271,9 +1272,7 @@ for an aligned read.</code></pre>
<pre><code>-c</code></pre>
</td>
<td>
-<p>The reference sequences are given on the command line. I.e. <code><reference_in></code> is a comma-separated list of sequences rather than a list of FASTA files.</p>
-</td>
-</tr>
+<p>The reference sequences are given on the command line. I.e. <code><reference_in></code> is a comma-separated list of sequences rather than a list of <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files.</p>
</td>
</tr>
<tr>
@@ -1414,7 +1413,7 @@ for an aligned read.</code></pre>
</tr>
</table>
<h1 id="the-bowtie2-inspect-index-inspector">The <code>bowtie2-inspect</code> index inspector</h1>
-<p><code>bowtie2-inspect</code> extracts information from a Bowtie index about what kind of index it is and what reference sequences were used to build it. When run without any options, the tool will output a FASTA file containing the sequences of the original references (with all non-<code>A</code>/<code>C</code>/<code>G</code>/<code>T</code> characters converted to <code>N</code>s). It can also be used to extract just the reference sequence names using the <a href="#bowtie2-inspect-opt [...]
+<p><code>bowtie2-inspect</code> extracts information from a Bowtie index about what kind of index it is and what reference sequences were used to build it. When run without any options, the tool will output a <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> file containing the sequences of the original references (with all non-<code>A</code>/<code>C</code>/<code>G</code>/<code>T</code> characters converted to <code>N</code>s). It can also be used to extract just the r [...]
<h2 id="command-line-2">Command Line</h2>
<p>Usage:</p>
<pre><code>bowtie2-inspect [options]* <bt2_base></code></pre>
@@ -1436,7 +1435,7 @@ for an aligned read.</code></pre>
<pre><code>-a/--across <int></code></pre>
</td>
<td>
-<p>When printing FASTA output, output a newline character every <code><int></code> bases (default: 60).</p>
+<p>When printing <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> output, output a newline character every <code><int></code> bases (default: 60).</p>
</td>
</tr>
<tr>
@@ -1495,7 +1494,7 @@ Sequence-N <name> <len></code></pre>
<p>To create an index for the <a href="http://en.wikipedia.org/wiki/Lambda_phage">Lambda phage</a> reference genome included with Bowtie 2, create a new temporary directory (it doesn't matter where), change into that directory, and run:</p>
<pre><code>$BT2_HOME/bowtie2-build $BT2_HOME/example/reference/lambda_virus.fa lambda_virus</code></pre>
<p>The command should print many lines of output then quit. When the command completes, the current directory will contain four new files that all start with <code>lambda_virus</code> and end with <code>.1.bt2</code>, <code>.2.bt2</code>, <code>.3.bt2</code>, <code>.4.bt2</code>, <code>.rev.1.bt2</code>, and <code>.rev.2.bt2</code>. These files constitute the index - you're done!</p>
-<p>You can use <code>bowtie2-build</code> to create an index for a set of FASTA files obtained from any source, including sites such as <a href="http://genome.ucsc.edu/cgi-bin/hgGateway">UCSC</a>, <a href="http://www.ncbi.nlm.nih.gov/sites/genome">NCBI</a>, and <a href="http://www.ensembl.org/">Ensembl</a>. When indexing multiple FASTA files, specify all the files using commas to separate file names. For more details on how to create an index with <code>bowtie2-build</code>, see the <a h [...]
+<p>You can use <code>bowtie2-build</code> to create an index for a set of <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files obtained from any source, including sites such as <a href="http://genome.ucsc.edu/cgi-bin/hgGateway">UCSC</a>, <a href="http://www.ncbi.nlm.nih.gov/sites/genome">NCBI</a>, and <a href="http://www.ensembl.org/">Ensembl</a>. When indexing multiple <a href="https://en.wikipedia.org/wiki/FASTA"><code>FASTA</code></a> files, specify all the files [...]
<h2 id="aligning-example-reads">Aligning example reads</h2>
<p>Stay in the directory created in the previous step, which now contains the <code>lambda_virus</code> index files. Next, run:</p>
<pre><code>$BT2_HOME/bowtie2 -x lambda_virus -U $BT2_HOME/example/reads/reads_1.fq -S eg1.sam</code></pre>
@@ -1523,7 +1522,7 @@ r7 16 gi|9626243|ref|NC_001416.1| 4692 42 143M * 0 0 TCAGCCGGACGC
<pre><code>$BT2_HOME/bowtie2 --local -x lambda_virus -U $BT2_HOME/example/reads/longreads.fq -S eg3.sam</code></pre>
<p>This aligns the long reads to the reference genome using local alignment, with results written to the file <code>eg3.sam</code>.</p>
<h2 id="using-samtoolsbcftools-downstream">Using SAMtools/BCFtools downstream</h2>
-<p><a href="http://samtools.sourceforge.net/">SAMtools</a> is a collection of tools for manipulating and analyzing SAM and BAM alignment files. <a href="http://samtools.sourceforge.net/mpileup.shtml">BCFtools</a> is a collection of tools for calling variants and manipulating VCF and BCF files, and it is typically distributed with <a href="http://samtools.sourceforge.net/">SAMtools</a>. Using these tools together allows you to get from alignments in SAM format to variant calls in VCF form [...]
+<p><a href="http://samtools.sourceforge.net">SAMtools</a> is a collection of tools for manipulating and analyzing SAM and BAM alignment files. <a href="http://samtools.sourceforge.net/mpileup.shtml">BCFtools</a> is a collection of tools for calling variants and manipulating VCF and BCF files, and it is typically distributed with <a href="http://samtools.sourceforge.net">SAMtools</a>. Using these tools together allows you to get from alignments in SAM format to variant calls in VCF format [...]
<p>Run the paired-end example:</p>
<pre><code>$BT2_HOME/bowtie2 -x $BT2_HOME/example/index/lambda_virus -1 $BT2_HOME/example/reads/reads_1.fq -2 $BT2_HOME/example/reads/reads_2.fq -S eg2.sam</code></pre>
<p>Use <code>samtools view</code> to convert the SAM file into a BAM file. BAM is the binary format corresponding to the SAM text format. Run:</p>
diff --git a/doc/website/recent_news.ssi b/doc/website/recent_news.ssi
index 8283835..ed82ee7 100644
--- a/doc/website/recent_news.ssi
+++ b/doc/website/recent_news.ssi
@@ -1,9 +1,21 @@
-<h2>Version 2.3.2 - May 05, 2017</h2>
+<h2>Version 2.3.3 - September 06, 2017</h2>
+<p>From this release forward prepackaged bowtie2 binaries are now statically linked to the zlib compression library and, the recommended threading library, TBB. Users who rely on prepackaged builds are no longer required to have these packages pre-installed. As a result of the aforementioned changes legacy packages have been discontinued.</p>
+<ul>
+ <li><tt>bowtie2-build</tt> now supports gzip-compressed FASTA inputs</li>
+ <li>New <tt><a href="manual.shtml#bowtie2-options-xeq">--xeq</a></tt> parameter for <tt>bowtie2</tt> disambiguates the 'M' CIGAR flag. When specified, matches are indicated with the <tt>=</tt> operation and mismatches with <tt>X</tt></li>
+ <li>Fixed a possible infinite loop during parallel index building due to the compiler optimizing away a loop condition</li>
+ <li>Added <tt><a href="manual.shtml#bowtie2-options-soft-clipped-unmapped-tlen">--soft-clipped-unmapped-tlen</a></tt> parameter for <tt>bowtie2</tt> that ignores soft-clipped bases when calculating template length (TLEN)</li>
+ <li>Added support for multi-line sequences in FASTA read inputs</li>
+ <li>Expanded explanation of <tt><a href="manual.shtml#bowtie2-build-opt-fields-md">MD:Z</a></tt> field in manual</li>
+ <li>Fixed a crashing bug when output is redirected to a pipe</li>
+ <li>Fixed ambiguity in the <tt>SEED</tt> alignment policy that sometimes caused <tt><a href="manual.shtml#bowtie2-options-N">-N</a></tt> parameter to be ignored</li>
</ul>
- <li>Added support for interleaved paired-end FASTQ inputs (--interleaved)</li>
+
+<h2>Version 2.3.2 - May 05, 2017</h2>
+<ul>
<li>Now reports MREVERSE SAM flag for unaligned end when only one end of a pair aligns</li>
<li>Fixed issue where first character of some read names was omitted from SAM output when using tabbed input formats</li>
- <li>Added --sam-no-qname-trunc option, which causes entire read name, including spaces, to be written to SAM output. This violates SAM specification, but can be useful in applications that immediately postprocess the SAM.</li>
+ <li>Added <tt><a href="manual.shtml#bowtie2-options-sam-no-qname-trunc">--sam-no-qname-trunc</a></tt> option, which causes entire read name, including spaces, to be written to SAM output. This violates SAM specification, but can be useful in applications that immediately postprocess the SAM.</li>
<li>Fixed compilation error caused by pointer comparison issue in aligner_result.cpp</li>
<li>Removed termcap and readline dependencies introduced in v2.3.1</li>
<li>Fixed compilation issues caused by gzbuffer function when compiling with zlib v1.2.3.5 and earlier. Users compiling against these libraries will use the zlib default buffer size of 8Kb when decompressing read files.</li>
@@ -13,18 +25,18 @@
<p>Please note that as of this release Bowtie 2 now has dependencies on zlib and readline libraries. Make sure that all dependencies are met before attempting to build from source.</p>
<ul>
<li>Added native support for gzipped read files. The wrapper script is no longer responsible for decompression. This simplifies the wrapper and improves speed and thread scalability for gzipped inputs.</li>
- <li>Fixed a bug that caused <tt>'bowtie2-build'</tt> to crash when the first FASTA sequence contains all Ns.</li>
+ <li>Fixed a bug that caused <tt>bowtie2-build</tt> to crash when the first FASTA sequence contains all Ns.</li>
<li>Add support for interleaved FASTQ format <tt><a href="manual.shtml#bowtie2-options-interleaved">-—interleaved</a></tt>.</li>
- <li>Empty FASTQ inputs would yield an error in Bowtie 2 2.3.0, whereas previous versions would simply align 0 reads and report the SAM header as usual. This version returns to the pre-2.3.0 behavior, resolving a compatibility issue between TopHat2 and Bowtie 2 2.3.0.</li>
- <li>Fixed a bug whereby combining <tt>'-—un-conc*</tt> with <tt>'-k'</tt> or <tt>'-a'</tt> would cause <tt>'bowtie2'</tt> to print duplicate reads in one or both of the <tt>'--un-conc*'</tt> output files, causing the ends to be misaligned.</li>
- <li>The default <tt>'--score-min'</tt> for <tt>'--local'</tt> mode is now <tt>'G,20,8'</tt>. That was the stated default in the documentation for a while, but the actual default was <tt>'G,0,10'</tt> for many versions. Now the default matches the documentation and, we find, yields more accurate alignments than <tt>'G,0,10'</tt></li>
+ <li>Empty FASTQ inputs would yield an error in Bowtie 2 2.3.0, whereas previous versions would simply align 0 reads and report the SAM header as usual. This version returns to the pre-2.3.0 behavior, resolving a compatibility issue between TopHat2 and Bowtie 2 2.3.0.</li>
+ <li>Fixed a bug whereby combining <tt><a href="manual.shtml#bowtie2-options-un-conc">-—un-conc</a></tt> with <tt><a href="manual.shtml#bowtie2-options-k">-k</a></tt> or <tt><a href="manual.shtml#bowtie2-build-options-a">-a</a></tt> would cause <tt>bowtie2</tt> to print duplicate reads in one or both of the <tt><a href="manual.shtml#bowtie2-options-un-conc">--un-conc*</a></tt> output files, causing the ends to be misaligned.</li>
+ <li>The default <tt><a href="manual.shtml#bowtie2-options-score-min">--score-min</a></tt> for <tt><a href="manual.shtml#bowtie2-options-local">--local</a></tt> mode is now <tt>'G,20,8'</tt>. That was the stated default in the documentation for a while, but the actual default was <tt>'G,0,10'</tt> for many versions. Now the default matches the documentation and, we find, yields more accurate alignments than <tt>'G,0,10'</tt></li>
</ul>
<h2>Version 2.3.0 - Dec 13, 2016</h2>
-<p>This is a major release with some larger and many smaller changes. These notes emphasize the large changes. See commit history for details.</p>
+<p>This is a major release with some larger and many smaller changes. These notes emphasize the large changes. See commit history for details.</p>
<ul>
- <li>Code related to read parsing was completely rewritten to improve scalability to many threads. In short, the critical section is simpler and parses input reads in batches rather than one at a time. The improvement applies to all read formats.</li>
- <li>TBB is now the default threading library. We consistently found TBB to give superior thread scaling. It is widely available and widely installed. That said, we are also preserving a "legacy" version of Bowtie that, like previous releases, does not use TBB. To compile Bowtie source in legacy mode use <tt>NO_TBB=1</tt>. To use legacy binaries, download the appropriate binary archive with "legacy" in the name.</li>
- <li>Bowtie now uses a queue-based lock rather than a spin or heavyweight lock. We find this gives superior thread scaling; we saw an order-of-magnitude throughput improvements at 120 threads in one experiment, for example.</li>
+ <li>Code related to read parsing was completely rewritten to improve scalability to many threads. In short, the critical section is simpler and parses input reads in batches rather than one at a time. The improvement applies to all read formats.</li>
+ <li>TBB is now the default threading library. We consistently found TBB to give superior thread scaling. It is widely available and widely installed. That said, we are also preserving a "legacy" version of Bowtie that, like previous releases, does not use TBB. To compile Bowtie source in legacy mode use <tt>NO_TBB=1</tt>. To use legacy binaries, download the appropriate binary archive with "legacy" in the name.</li>
+ <li>Bowtie now uses a queue-based lock rather than a spin or heavyweight lock. We find this gives superior thread scaling; we saw an order-of-magnitude throughput improvements at 120 threads in one experiment, for example.</li>
<li>Unnecessary thread synchronization removed</li>
<li>Fixed issue with parsing FASTA records with greater-than symbol in the name</li>
<li>Now detects and reports inconsistencies between <tt><a href="manual.shtml#bowtie2-options-score-min">--score-min</a></tt> and <tt><a href="manual.shtml#bowtie2-options-ma">--ma</a></tt></li>
@@ -45,7 +57,7 @@
</ul>
<h2>Version 2.2.7 - Feb 10, 2016</h2>
<ul>
- <li>Added a parallel index build option: bowtie2-build --threads <# threads>.</li>
+ <li>Added a parallel index build option: bowtie2-build --threads <# threads>.</li>
<li>Fixed an issue whereby IUPAC codes (other than A/C/G/T/N) in reads were converted to As. Now all non-A/C/G/T characters in reads become Ns.</li>
<li>Fixed some compilation issues, including for the Intel C++ Compiler.</li>
<li>Removed debugging code that could impede performance for many alignment threads.</li>
@@ -77,5 +89,5 @@
<h2>Lighter released</h2>
<ul>
- <li>Lighter is an extremely fast and memory-efficient program for correcting sequencing errors in DNA sequencing data. For details on how error correction can help improve the speed and accuracy of downstream analysis tools, see the <a href="http://genomebiology.com/2014/15/11/509">paper in Genome Biology</a>. Source and software <a href="https://github.com/mourisl/Lighter">available at GitHub</a>.
+ <li>Lighter is an extremely fast and memory-efficient program for correcting sequencing errors in DNA sequencing data. For details on how error correction can help improve the speed and accuracy of downstream analysis tools, see the <a href="http://genomebiology.com/2014/15/11/509">paper in Genome Biology</a>. Source and software <a href="https://github.com/mourisl/Lighter">available at GitHub</a></li>.
</ul>
diff --git a/doc/website/rhsidebar.ssi b/doc/website/rhsidebar.ssi
index 67b76b4..a490f3e 100644
--- a/doc/website/rhsidebar.ssi
+++ b/doc/website/rhsidebar.ssi
@@ -18,10 +18,10 @@
</tr>
<tr>
<td>
- <a href="https://sourceforge.net/projects/bowtie-bio/files/bowtie2/2.3.2">Bowtie2 2.3.2</a>
+ <a href="https://sourceforge.net/projects/bowtie-bio/files/bowtie2/2.3.3">Bowtie2 2.3.3</a>
</td>
<td align="right">
- 05/05/17
+ 09/06/17
</td>
</tr>
<tr>
diff --git a/ds.cpp b/ds.cpp
index b98eb95..6955536 100644
--- a/ds.cpp
+++ b/ds.cpp
@@ -25,7 +25,7 @@ MemoryTally gMemTally;
* Tally a memory allocation of size amt bytes.
*/
void MemoryTally::add(int cat, uint64_t amt) {
- ThreadSafe ts(&mutex_m);
+ ThreadSafe ts(mutex_m);
tots_[cat] += amt;
tot_ += amt;
if(tots_[cat] > peaks_[cat]) {
@@ -40,7 +40,7 @@ void MemoryTally::add(int cat, uint64_t amt) {
* Tally a memory free of size amt bytes.
*/
void MemoryTally::del(int cat, uint64_t amt) {
- ThreadSafe ts(&mutex_m);
+ ThreadSafe ts(mutex_m);
assert_geq(tots_[cat], amt);
assert_geq(tot_, amt);
tots_[cat] -= amt;
diff --git a/ds.h b/ds.h
index 8e8045a..d4a9737 100644
--- a/ds.h
+++ b/ds.h
@@ -3074,8 +3074,10 @@ public:
pagesz_(pagesz),
pages_(cat)
{
- for(size_t i = 0; i < ((bytes+pagesz-1)/pagesz); i++) {
- pages_.push_back(new uint8_t[pagesz]);
+ size_t super_page_num = ((bytes+pagesz-1)/pagesz + 1);
+ super_pages = new uint8_t[pagesz*super_page_num];
+ for(size_t i = 0; i < super_page_num ; i++) {
+ pages_.push_back(&super_pages[i*pagesz]);
#ifdef USE_MEM_TALLY
gMemTally.add(cat, pagesz);
#else
@@ -3091,9 +3093,8 @@ public:
* Free each page.
*/
~Pool() {
+ delete super_pages;
for(size_t i = 0; i < pages_.size(); i++) {
- assert(pages_[i] != NULL);
- delete[] pages_[i];
#ifdef USE_MEM_TALLY
gMemTally.del(cat_, pagesz_);
#else
@@ -3140,6 +3141,7 @@ public:
#endif
private:
+ uint8_t* super_pages; // for deleting of the one large chunk
int cat_; // memory category, for accounting purposes
size_t cur_; // next page to hand out
const size_t pagesz_; // size of a single page
diff --git a/endian_swap.h b/endian_swap.h
index 64c5889..bcd9d40 100644
--- a/endian_swap.h
+++ b/endian_swap.h
@@ -107,9 +107,9 @@ static inline T endianizeU(T u, bool toBig) {
return u;
}
if(sizeof(T) == 4) {
- return endianSwapU32((uint32_t)u);
+ return (T)endianSwapU32((uint32_t)u);
} else if(sizeof(T) == 8) {
- return endianSwapU64((uint64_t)u);
+ return (T)endianSwapU64((uint64_t)u);
} else {
assert(false);
}
diff --git a/filebuf.h b/filebuf.h
index f3e7b9c..964db80 100644
--- a/filebuf.h
+++ b/filebuf.h
@@ -28,6 +28,9 @@
#include <stdint.h>
#include <stdexcept>
#include "assert_helpers.h"
+#include <errno.h>
+#include <stdlib.h>
+#include <zlib.h>
/**
* Simple, fast helper for determining if a character is a newline.
@@ -66,6 +69,12 @@ public:
assert(_in != NULL);
}
+ FileBuf(gzFile in) {
+ init();
+ _zIn = in;
+ assert(_zIn != NULL);
+ }
+
FileBuf(std::ifstream *inf) {
init();
_inf = inf;
@@ -93,6 +102,8 @@ public:
fclose(_in);
} else if(_inf != NULL) {
_inf->close();
+ } else if(_zIn != NULL) {
+ gzclose(_zIn);
} else {
// can't close _ins
}
@@ -102,7 +113,7 @@ public:
* Get the next character of input and advance.
*/
int get() {
- assert(_in != NULL || _inf != NULL || _ins != NULL);
+ assert(_in != NULL || _zIn != NULL || _inf != NULL || _ins != NULL);
int c = peek();
if(c != -1) {
_cur++;
@@ -123,6 +134,20 @@ public:
*/
void newFile(FILE *in) {
_in = in;
+ _zIn = NULL;
+ _inf = NULL;
+ _ins = NULL;
+ _cur = BUF_SZ;
+ _buf_sz = BUF_SZ;
+ _done = false;
+ }
+
+ /**
+ * Initialize the buffer with a new gz file.
+ */
+ void newFile(gzFile in) {
+ _in = NULL;
+ _zIn = in;
_inf = NULL;
_ins = NULL;
_cur = BUF_SZ;
@@ -135,6 +160,7 @@ public:
*/
void newFile(std::ifstream *__inf) {
_in = NULL;
+ _zIn = NULL;
_inf = __inf;
_ins = NULL;
_cur = BUF_SZ;
@@ -147,6 +173,7 @@ public:
*/
void newFile(std::istream *__ins) {
_in = NULL;
+ _zIn = NULL;
_inf = NULL;
_ins = __ins;
_cur = BUF_SZ;
@@ -165,6 +192,8 @@ public:
} else if(_ins != NULL) {
_ins->clear();
_ins->seekg(0, std::ios::beg);
+ } else if (_zIn != NULL) {
+ gzrewind(_zIn);
} else {
rewind(_in);
}
@@ -179,7 +208,7 @@ public:
* Occasionally we'll need to read in a new buffer's worth of data.
*/
int peek() {
- assert(_in != NULL || _inf != NULL || _ins != NULL);
+ assert(_in != NULL || _zIn != NULL || _inf != NULL || _ins != NULL);
assert_leq(_cur, _buf_sz);
if(_cur == _buf_sz) {
if(_done) {
@@ -192,6 +221,8 @@ public:
if(_inf != NULL) {
_inf->read((char*)_buf, BUF_SZ);
_buf_sz = _inf->gcount();
+ } else if(_zIn != NULL) {
+ _buf_sz = gzread(_zIn, (void *)_buf, BUF_SZ);
} else if(_ins != NULL) {
_ins->read((char*)_buf, BUF_SZ);
_buf_sz = _ins->gcount();
@@ -431,6 +462,7 @@ private:
void init() {
_in = NULL;
+ _zIn = NULL;
_inf = NULL;
_ins = NULL;
_cur = _buf_sz = BUF_SZ;
@@ -441,6 +473,7 @@ private:
static const size_t BUF_SZ = 256 * 1024;
FILE *_in;
+ gzFile _zIn;
std::ifstream *_inf;
std::istream *_ins;
size_t _cur;
@@ -685,6 +718,9 @@ public:
void flush() {
if(!fwrite((const void *)buf_, cur_, 1, out_)) {
+ if (errno == EPIPE) {
+ exit(EXIT_SUCCESS);
+ }
std::cerr << "Error while flushing and closing output" << std::endl;
throw 1;
}
diff --git a/group_walk.h b/group_walk.h
index 2309bb1..63e2aa6 100644
--- a/group_walk.h
+++ b/group_walk.h
@@ -154,7 +154,7 @@ struct GroupWalkState {
struct WalkMetrics {
WalkMetrics() {
- reset();
+ reset();
}
/**
@@ -162,12 +162,12 @@ struct WalkMetrics {
* to update a WalkMetrics shared by many threads.
*/
void merge(const WalkMetrics& m, bool getLock = false) {
- ThreadSafe ts(&mutex_m, getLock);
- bwops += m.bwops;
- branches += m.branches;
- resolves += m.resolves;
- refresolves += m.refresolves;
- reports += m.reports;
+ if(getLock) {
+ ThreadSafe ts(mutex_m);
+ mergeImpl(m);
+ } else {
+ mergeImpl(m);
+ }
}
/**
@@ -183,6 +183,19 @@ struct WalkMetrics {
uint64_t refresolves; // # resolutions caused by reference scanning
uint64_t reports; // # offs reported (1 can be reported many times)
MUTEX_T mutex_m;
+
+private:
+ /**
+ * Sum each across this object and 'm'. This is the only safe way
+ * to update a WalkMetrics shared by many threads.
+ */
+ void mergeImpl(const WalkMetrics& m) {
+ bwops += m.bwops;
+ branches += m.branches;
+ resolves += m.resolves;
+ refresolves += m.refresolves;
+ reports += m.reports;
+ }
};
/**
diff --git a/multikey_qsort.h b/multikey_qsort.h
index eea455f..0fce1c9 100644
--- a/multikey_qsort.h
+++ b/multikey_qsort.h
@@ -610,14 +610,14 @@ void mkeyQSortSuf2(
}
// Do not recurse on ='s if the pivot was the off-the-end value;
// they're already fully sorted
- if(v != hi) { // recurse on ='s
+ //if(v != hi) { // recurse on ='s
block_list.back().expand();
block_list.back().back().begin = begin + r;
block_list.back().back().end = begin + r + (a-begin) + (end-d-1);
block_list.back().back().depth = depth + 1;
- }
+ //}
r = d-c; // r <- # of >'s excluding those exhausted
- if(r > 0 && v < hi-1) { // recurse on >'s
+ if(r > 0 /*&& v < hi-1*/) { // recurse on >'s
block_list.back().expand();
block_list.back().back().begin = end - r;
block_list.back().back().end = end;
diff --git a/opts.h b/opts.h
index e57aa26..c9f58fe 100644
--- a/opts.h
+++ b/opts.h
@@ -69,6 +69,7 @@ enum {
ARG_QUALS2, // --Q2
ARG_QSEQ, // --qseq
ARG_SEED_SUMM, // --seed-summary
+ ARG_SC_UNMAPPED, // --soft-clipped-unmapped-tlen
ARG_OVERHANG, // --overhang
ARG_NO_CACHE, // --no-cache
ARG_USE_CACHE, // --cache
@@ -152,6 +153,9 @@ enum {
ARG_DESC_FMOPS, // --desc-fmops
ARG_LOG_DP, // --log-dp
ARG_LOG_DP_OPP, // --log-dp-opp
+ ARG_XEQ, // --xeq
+ ARG_THREAD_CEILING, // --thread-ceiling
+ ARG_THREAD_PIDDIR, // --thread-piddir
ARG_INTERLEAVED_FASTQ // --interleaved
};
diff --git a/outq.cpp b/outq.cpp
index 1141675..947e2b4 100644
--- a/outq.cpp
+++ b/outq.cpp
@@ -23,9 +23,12 @@
* Caller is telling us that they're about to write output record(s) for
* the read with the given id.
*/
-void OutputQueue::beginRead(TReadId rdid, size_t threadId) {
- ThreadSafe t(&mutex_m, threadSafe_);
+void OutputQueue::beginReadImpl(TReadId rdid, size_t threadId) {
+#ifdef WITH_TBB
+ nstarted_.fetch_and_add(1);
+#else
nstarted_++;
+#endif
if(reorder_) {
assert_geq(rdid, cur_);
assert_eq(lines_.size(), finished_.size());
@@ -45,11 +48,19 @@ void OutputQueue::beginRead(TReadId rdid, size_t threadId) {
}
}
+void OutputQueue::beginRead(TReadId rdid, size_t threadId) {
+ if(reorder_ && threadSafe_) {
+ ThreadSafe ts(mutex_m);
+ beginReadImpl(rdid, threadId);
+ } else {
+ beginReadImpl(rdid, threadId);
+ }
+}
+
/**
* Writer is finished writing to
*/
-void OutputQueue::finishRead(const BTString& rec, TReadId rdid, size_t threadId) {
- ThreadSafe t(&mutex_m, threadSafe_);
+void OutputQueue::finishReadImpl(const BTString& rec, TReadId rdid, size_t threadId) {
if(reorder_) {
assert_geq(rdid, cur_);
assert_eq(lines_.size(), finished_.size());
@@ -63,20 +74,51 @@ void OutputQueue::finishRead(const BTString& rec, TReadId rdid, size_t threadId)
flush(false, false); // don't force; already have lock
} else {
// obuf_ is the OutFileBuf for the output file
- obuf_.writeString(rec);
- nfinished_++;
- nflushed_++;
+ int i = 0;
+ for(i=0; i < perThreadBufSize_; i++)
+ {
+ obuf_.writeString(perThreadBuf[threadId][i]);
+ //TODO: turn these into atomics
+ nfinished_++;
+ nflushed_++;
+ }
+ perThreadCounter[threadId] = 0;
+ }
+}
+
+void OutputQueue::finishRead(const BTString& rec, TReadId rdid, size_t threadId) {
+ if(reorder_ || perThreadCounter[threadId] >= perThreadBufSize_) {
+ if(threadSafe_) {
+ ThreadSafe ts(mutex_m);
+ finishReadImpl(rec, rdid, threadId);
+ } else {
+ finishReadImpl(rec, rdid, threadId);
+ }
+ }
+ if(!reorder_) {
+ perThreadBuf[threadId][perThreadCounter[threadId]++] = rec;
}
}
/**
* Write already-finished lines starting from cur_.
*/
-void OutputQueue::flush(bool force, bool getLock) {
+void OutputQueue::flushImpl(bool force) {
if(!reorder_) {
+ size_t i = 0;
+ int j = 0;
+ for(i=0;i<nthreads_;i++)
+ {
+ for(j=0;j<perThreadCounter[i];j++)
+ {
+ obuf_.writeString(perThreadBuf[i][j]);
+ nfinished_++;
+ nflushed_++;
+ }
+ perThreadCounter[i]=0;
+ }
return;
}
- ThreadSafe t(&mutex_m, getLock && threadSafe_);
size_t nflush = 0;
while(nflush < finished_.size() && finished_[nflush]) {
assert(started_[nflush]);
@@ -98,6 +140,18 @@ void OutputQueue::flush(bool force, bool getLock) {
}
}
+/**
+ * Write already-finished lines starting from cur_.
+ */
+void OutputQueue::flush(bool force, bool getLock) {
+ if(getLock && threadSafe_) {
+ ThreadSafe ts(mutex_m);
+ flushImpl(force);
+ } else {
+ flushImpl(force);
+ }
+}
+
#ifdef OUTQ_MAIN
#include <iostream>
diff --git a/outq.h b/outq.h
index 00408cf..f8a30cf 100644
--- a/outq.h
+++ b/outq.h
@@ -45,10 +45,10 @@ public:
bool reorder,
size_t nthreads,
bool threadSafe,
+ int perThreadBufSize,
TReadId rdid = 0) :
obuf_(obuf),
cur_(rdid),
- nstarted_(0),
nfinished_(0),
nflushed_(0),
lines_(RES_CAT),
@@ -56,11 +56,27 @@ public:
finished_(RES_CAT),
reorder_(reorder),
threadSafe_(threadSafe),
- mutex_m()
+ mutex_m(),
+ nthreads_(nthreads),
+ perThreadBufSize_(perThreadBufSize)
{
- assert(nthreads <= 1 || threadSafe);
+ nstarted_=0;
+ assert(nthreads_ <= 2 || threadSafe);
+ if(!reorder)
+ {
+ perThreadBuf = new BTString*[nthreads_];
+ perThreadCounter = new int[nthreads_];
+ size_t i = 0;
+ for(i=0;i<nthreads_;i++)
+ {
+ perThreadBuf[i] = new BTString[perThreadBufSize_];
+ perThreadCounter[i] = 0;
+ }
+ }
}
+ ~OutputQueue() { }
+
/**
* Caller is telling us that they're about to write output record(s) for
* the read with the given id.
@@ -109,7 +125,11 @@ protected:
OutFileBuf& obuf_;
TReadId cur_;
+#ifdef WITH_TBB
+ tbb::atomic<TReadId> nstarted_;
+#else
TReadId nstarted_;
+#endif
TReadId nfinished_;
TReadId nflushed_;
EList<BTString> lines_;
@@ -118,6 +138,18 @@ protected:
bool reorder_;
bool threadSafe_;
MUTEX_T mutex_m;
+
+ // used for output read buffer
+ size_t nthreads_;
+ BTString** perThreadBuf;
+ int* perThreadCounter;
+ int perThreadBufSize_;
+
+private:
+
+ void flushImpl(bool force);
+ void beginReadImpl(TReadId rdid, size_t threadId);
+ void finishReadImpl(const BTString& rec, TReadId rdid, size_t threadId);
};
class OutputQueueMark {
diff --git a/pat.cpp b/pat.cpp
index 6240eb7..16d6754 100644
--- a/pat.cpp
+++ b/pat.cpp
@@ -31,6 +31,15 @@
using namespace std;
+/*void set_bufsz(unsigned bufsz)
+{
+#if ZLIB_VERNUM < 0x1240
+ std::fprintf(stderr, "Warning: gzbuffer added in zlib1.2.4. Unable to change buffer size from default of 8192.\n");
+#else
+ gzbuffer(fp_, bufsz);
+#endif
+}*/
+
/**
* Calculate a per-read random seed based on a combination of
* the read data (incl. sequence, name, quals) and the global
@@ -188,7 +197,7 @@ pair<bool, int> SoloPatternComposer::nextBatch(PerThreadReadBuf& pt) {
true); // grab lock below
} while(!res.first && res.second == 0);
if(res.second == 0) {
- ThreadSafe ts(&mutex_m);
+ ThreadSafe ts(mutex_m);
if(cur + 1 > cur_) {
cur_++;
}
@@ -218,7 +227,7 @@ pair<bool, int> DualPatternComposer::nextBatch(PerThreadReadBuf& pt) {
true); // grab lock below
bool done = res.first;
if(!done && res.second == 0) {
- ThreadSafe ts(&mutex_m);
+ ThreadSafe ts(mutex_m);
if(cur + 1 > cur_) cur_++;
cur = cur_; // Move on to next PatternSource
continue; // on to next pair of PatternSources
@@ -229,7 +238,7 @@ pair<bool, int> DualPatternComposer::nextBatch(PerThreadReadBuf& pt) {
// Lock to ensure that this thread gets parallel reads
// in the two mate files
{
- ThreadSafe ts(&mutex_m);
+ ThreadSafe ts(mutex_m);
resa = (*srca_)[cur]->nextBatch(
pt,
true, // batch A
@@ -246,7 +255,7 @@ pair<bool, int> DualPatternComposer::nextBatch(PerThreadReadBuf& pt) {
<< "than in file specified with -2" << endl;
throw 1;
} else if(resa.second == 0 && resb.second == 0) {
- ThreadSafe ts(&mutex_m);
+ ThreadSafe ts(mutex_m);
if(cur + 1 > cur_) {
cur_++;
}
@@ -387,17 +396,12 @@ void PatternComposer::free_EList_pmembers( const EList<PatternSource*> &elist) {
* Returns pair<bool, int> where bool indicates whether we're
* completely done, and int indicates how many reads were read.
*/
-pair<bool, int> CFilePatternSource::nextBatch(
+pair<bool, int> CFilePatternSource::nextBatchImpl(
PerThreadReadBuf& pt,
- bool batch_a,
- bool lock)
+ bool batch_a)
{
bool done = false;
int nread = 0;
-
- // synchronization at this level because both reading and manipulation of
- // current file pointer have to be protected
- ThreadSafe ts(&mutex, lock);
pt.setReadId(readCnt_);
while(true) { // loop that moves on to next file when needed
do {
@@ -420,6 +424,21 @@ pair<bool, int> CFilePatternSource::nextBatch(
return make_pair(done, nread);
}
+pair<bool, int> CFilePatternSource::nextBatch(
+ PerThreadReadBuf& pt,
+ bool batch_a,
+ bool lock)
+{
+ if(lock) {
+ // synchronization at this level because both reading and manipulation of
+ // current file pointer have to be protected
+ ThreadSafe ts(mutex);
+ return nextBatchImpl(pt, batch_a);
+ } else {
+ return nextBatchImpl(pt, batch_a);
+ }
+}
+
/**
* Open the next file in the list of input files.
*/
@@ -432,8 +451,8 @@ void CFilePatternSource::open() {
}
else {
fclose(fp_);
- fp_ = NULL;
- }
+ fp_ = NULL;
+ }
}
while(filecur_ < infiles_.size()) {
if(infiles_[filecur_] == "-") {
@@ -454,12 +473,12 @@ void CFilePatternSource::open() {
if((compressed_ && zfp_ == NULL) || (!compressed_ && fp_ == NULL)) {
if(!errs_[filecur_]) {
cerr << "Warning: Could not open read file \""
- << infiles_[filecur_].c_str()
- << "\" for reading; skipping..." << endl;
+ << infiles_[filecur_].c_str()
+ << "\" for reading; skipping..." << endl;
errs_[filecur_] = true;
- }
- filecur_++;
- continue;
+ }
+ filecur_++;
+ continue;
}
}
is_open_ = true;
@@ -533,12 +552,10 @@ VectorPatternSource::VectorPatternSource(
* in the contsructor. This essentially modifies the pt as though we read
* in some number of patterns.
*/
-pair<bool, int> VectorPatternSource::nextBatch(
+pair<bool, int> VectorPatternSource::nextBatchImpl(
PerThreadReadBuf& pt,
- bool batch_a,
- bool lock)
+ bool batch_a)
{
- ThreadSafe ts(&mutex, lock);
pt.setReadId(cur_);
EList<Read>& readbuf = batch_a ? pt.bufa_ : pt.bufb_;
size_t readi = 0;
@@ -549,6 +566,19 @@ pair<bool, int> VectorPatternSource::nextBatch(
return make_pair(cur_ == bufs_.size(), readi);
}
+pair<bool, int> VectorPatternSource::nextBatch(
+ PerThreadReadBuf& pt,
+ bool batch_a,
+ bool lock)
+{
+ if(lock) {
+ ThreadSafe ts(mutex);
+ return nextBatchImpl(pt, batch_a);
+ } else {
+ return nextBatchImpl(pt, batch_a);
+ }
+}
+
/**
* Finishes parsing outside the critical section.
*/
@@ -593,7 +623,7 @@ bool VectorPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
while(c != '\t' && cur < buflen) {
if(isalpha(c)) {
assert_in(toupper(c), "ACGTN");
- if(nchar++ >= gTrim5) {
+ if(nchar++ >= pp_.trim5) {
assert_neq(0, asc2dnacat[c]);
r.patFw.append(asc2dna[c]); // ascii to int
}
@@ -607,7 +637,7 @@ bool VectorPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
// record amt trimmed from 5' end due to --trim5
r.trimmed5 = (int)(nchar - r.patFw.length());
// record amt trimmed from 3' end due to --trim3
- r.trimmed3 = (int)(r.patFw.trimEnd(gTrim3));
+ r.trimmed3 = (int)(r.patFw.trimEnd(pp_.trim3));
// Parse qualities
assert(r.qual.empty());
@@ -619,7 +649,7 @@ bool VectorPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
return false;
}
char cadd = charToPhred33(c, false, false);
- if(++nqual > gTrim5) {
+ if(++nqual > pp_.trim5) {
r.qual.append(cadd);
}
if(cur >= buflen) break;
@@ -632,7 +662,7 @@ bool VectorPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
tooManyQualities(r.name);
return false;
}
- r.qual.trimEnd(gTrim3);
+ r.qual.trimEnd(pp_.trim3);
assert(c == '\t' || c == '\n' || c == '\r' || cur >= buflen);
assert_eq(r.patFw.length(), r.qual.length());
}
@@ -723,21 +753,26 @@ bool FastaPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
assert(r.patFw.empty());
assert(c != '\n' && c != '\r');
assert_lt(cur, buflen);
- while(c != '\n' && cur < buflen) {
+ while(cur < buflen) {
if(c == '.') {
c = 'N';
}
if(isalpha(c)) {
// If it's past the 5'-end trim point
- if(nchar++ >= gTrim5) {
+ if(nchar++ >= pp_.trim5) {
r.patFw.append(asc2dna[c]);
}
}
assert_lt(cur, buflen);
c = r.readOrigBuf[cur++];
+ if ((c == '\n' || c == '\r')
+ && cur < buflen
+ && r.readOrigBuf[cur] != '>') {
+ c = r.readOrigBuf[cur++];
+ }
}
r.trimmed5 = (int)(nchar - r.patFw.length());
- r.trimmed3 = (int)(r.patFw.trimEnd(gTrim3));
+ r.trimmed3 = (int)(r.patFw.trimEnd(pp_.trim3));
for(size_t i = 0; i < r.patFw.length(); i++) {
r.qual.append('I');
@@ -881,7 +916,7 @@ bool FastaContinuousPatternSource::parse(
while(cur < buflen) {
if(isalpha(c)) {
assert_in(toupper(c), "ACGTN");
- if(nchar++ >= gTrim5) {
+ if(nchar++ >= pp_.trim5) {
assert_neq(0, asc2dnacat[c]);
ra.patFw.append(asc2dna[c]); // ascii to int
}
@@ -891,7 +926,7 @@ bool FastaContinuousPatternSource::parse(
// record amt trimmed from 5' end due to --trim5
ra.trimmed5 = (int)(nchar - ra.patFw.length());
// record amt trimmed from 3' end due to --trim3
- ra.trimmed3 = (int)(ra.patFw.trimEnd(gTrim3));
+ ra.trimmed3 = (int)(ra.patFw.trimEnd(pp_.trim3));
// Make fake qualities
assert(ra.qual.empty());
@@ -914,7 +949,7 @@ pair<bool, int> FastqPatternSource::nextBatchFromFile(
PerThreadReadBuf& pt,
bool batch_a)
{
- int c;
+ int c = -1;
EList<Read>* readbuf = batch_a ? &pt.bufa_ : &pt.bufb_;
if(first_) {
c = getc_wrapper();
@@ -1014,7 +1049,7 @@ bool FastqPatternSource::parse(Read &r, Read& rb, TReadId rdid) const {
}
if(isalpha(c)) {
// If it's past the 5'-end trim point
- if(nchar++ >= gTrim5) {
+ if(nchar++ >= pp_.trim5) {
r.patFw.append(asc2dna[c]);
}
}
@@ -1022,7 +1057,7 @@ bool FastqPatternSource::parse(Read &r, Read& rb, TReadId rdid) const {
c = r.readOrigBuf[cur++];
}
r.trimmed5 = (int)(nchar - r.patFw.length());
- r.trimmed3 = (int)(r.patFw.trimEnd(gTrim3));
+ r.trimmed3 = (int)(r.patFw.trimEnd(pp_.trim3));
assert_eq('+', c);
do {
@@ -1036,23 +1071,23 @@ bool FastqPatternSource::parse(Read &r, Read& rb, TReadId rdid) const {
assert(r.qual.empty());
if(nchar > 0) {
int nqual = 0;
- if (intQuals_) {
+ if (pp_.intQuals) {
int cur_int = 0;
while(c != '\t' && c != '\n' && c != '\r') {
cur_int *= 10;
cur_int += (int)(c - '0');
c = r.readOrigBuf[cur++];
if(c == ' ' || c == '\t' || c == '\n' || c == '\r') {
- char cadd = intToPhred33(cur_int, solQuals_);
+ char cadd = intToPhred33(cur_int, pp_.solexa64);
cur_int = 0;
assert_geq(cadd, 33);
- if(++nqual > gTrim5) {
+ if(++nqual > pp_.trim5) {
r.qual.append(cadd);
}
}
}
} else {
- c = charToPhred33(c, solQuals_, phred64Quals_);
+ c = charToPhred33(c, pp_.solexa64, pp_.phred64);
if(nqual++ >= r.trimmed5) {
r.qual.append(c);
}
@@ -1065,7 +1100,7 @@ bool FastqPatternSource::parse(Read &r, Read& rb, TReadId rdid) const {
if(c == '\r' || c == '\n') {
break;
}
- c = charToPhred33(c, solQuals_, phred64Quals_);
+ c = charToPhred33(c, pp_.solexa64, pp_.phred64);
if(nqual++ >= r.trimmed5) {
r.qual.append(c);
}
@@ -1164,7 +1199,7 @@ bool TabbedPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
while(c != '\t' && cur < buflen) {
if(isalpha(c)) {
assert_in(toupper(c), "ACGTN");
- if(nchar++ >= gTrim5) {
+ if(nchar++ >= pp_.trim5) {
assert_neq(0, asc2dnacat[c]);
r.patFw.append(asc2dna[c]); // ascii to int
}
@@ -1178,23 +1213,23 @@ bool TabbedPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
// record amt trimmed from 5' end due to --trim5
r.trimmed5 = (int)(nchar - r.patFw.length());
// record amt trimmed from 3' end due to --trim3
- r.trimmed3 = (int)(r.patFw.trimEnd(gTrim3));
+ r.trimmed3 = (int)(r.patFw.trimEnd(pp_.trim3));
// Parse qualities
assert(r.qual.empty());
c = ra.readOrigBuf[cur++];
int nqual = 0;
- if (intQuals_) {
+ if (pp_.intQuals) {
int cur_int = 0;
while(c != '\t' && c != '\n' && c != '\r' && cur < buflen) {
cur_int *= 10;
cur_int += (int)(c - '0');
c = ra.readOrigBuf[cur++];
if(c == ' ' || c == '\t' || c == '\n' || c == '\r') {
- char cadd = intToPhred33(cur_int, solQuals_);
+ char cadd = intToPhred33(cur_int, pp_.solexa64);
cur_int = 0;
assert_geq(cadd, 33);
- if(++nqual > gTrim5) {
+ if(++nqual > pp_.trim5) {
r.qual.append(cadd);
}
}
@@ -1205,8 +1240,8 @@ bool TabbedPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
wrongQualityFormat(r.name);
return false;
}
- char cadd = charToPhred33(c, solQuals_, phred64Quals_);
- if(++nqual > gTrim5) {
+ char cadd = charToPhred33(c, pp_.solexa64, pp_.phred64);
+ if(++nqual > pp_.trim5) {
r.qual.append(cadd);
}
if(cur >= buflen) break;
@@ -1220,7 +1255,7 @@ bool TabbedPatternSource::parse(Read& ra, Read& rb, TReadId rdid) const {
tooManyQualities(r.name);
return false;
}
- r.qual.trimEnd(gTrim3);
+ r.qual.trimEnd(pp_.trim3);
assert(c == '\t' || c == '\n' || c == '\r' || cur >= buflen);
assert_eq(r.patFw.length(), r.qual.length());
}
@@ -1276,7 +1311,7 @@ bool RawPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
assert(c != '\r' && c != '\n');
if(isalpha(c)) {
assert_in(toupper(c), "ACGTN");
- if(nchar++ >= gTrim5) {
+ if(nchar++ >= pp_.trim5) {
assert_neq(0, asc2dnacat[c]);
r.patFw.append(asc2dna[c]); // ascii to int
}
@@ -1286,7 +1321,7 @@ bool RawPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
// record amt trimmed from 5' end due to --trim5
r.trimmed5 = (int)(nchar - r.patFw.length());
// record amt trimmed from 3' end due to --trim3
- r.trimmed3 = (int)(r.patFw.trimEnd(gTrim3));
+ r.trimmed3 = (int)(r.patFw.trimEnd(pp_.trim3));
// Give the name field a dummy value
char cbuf[20];
diff --git a/pat.h b/pat.h
index a2556c7..57726a3 100644
--- a/pat.h
+++ b/pat.h
@@ -60,6 +60,8 @@ struct PatternParams {
bool solexa64_,
bool phred64_,
bool intQuals_,
+ int trim5_,
+ int trim3_,
int sampleLen_,
int sampleFreq_,
size_t skip_,
@@ -72,6 +74,8 @@ struct PatternParams {
solexa64(solexa64_),
phred64(phred64_),
intQuals(intQuals_),
+ trim5(trim5_),
+ trim3(trim3_),
sampleLen(sampleLen_),
sampleFreq(sampleFreq_),
skip(skip_),
@@ -85,6 +89,8 @@ struct PatternParams {
bool solexa64; // true -> qualities are on solexa64 scale
bool phred64; // true -> qualities are on phred64 scale
bool intQuals; // true -> qualities are space-separated numbers
+ int trim5; // amt to hard clip from 5' end
+ int trim3; // amt to hard clip from 3' end
int sampleLen; // length of sampled reads for FastaContinuous...
int sampleFreq; // frequency of sampled reads for FastaContinuous...
size_t skip; // skip the first 'skip' patterns
@@ -188,6 +194,7 @@ class PatternSource {
public:
PatternSource(const PatternParams& p) :
+ pp_(p),
readCnt_(0),
mutex() { }
@@ -231,11 +238,15 @@ public:
protected:
- /// The number of reads read by this PatternSource
+
+ // Reference to global input-parsing parameters
+ const PatternParams& pp_;
+
+ // The number of reads read by this PatternSource
volatile TReadId readCnt_;
- /// Lock enforcing mutual exclusion for (a) file I/O, (b) writing fields
- /// of this or another other shared object.
+ // Lock enforcing mutual exclusion for (a) file I/O, (b) writing fields
+ // of this or another other shared object.
MUTEX_T mutex;
};
@@ -283,6 +294,10 @@ public:
private:
+ pair<bool, int> nextBatchImpl(
+ PerThreadReadBuf& pt,
+ bool batch_a);
+
size_t cur_; // index for first read of next batch
size_t skip_; // # reads to skip
bool paired_; // whether reads are paired
@@ -404,16 +419,22 @@ protected:
return false;
}
- EList<std::string> infiles_; // filenames for read files
+ EList<std::string> infiles_; // filenames for read files
EList<bool> errs_; // whether we've already printed an error for each file
size_t filecur_; // index into infiles_ of next file to read
- FILE *fp_; // read file currently being read from
- gzFile zfp_;
+ FILE *fp_; // read file currently being read from
+ gzFile zfp_; // compressed version of fp_
bool is_open_; // whether fp_ is currently open
TReadId skip_; // number of reads to skip
bool first_; // parsing first record in first file?
char buf_[64*1024]; // file buffer
bool compressed_;
+
+private:
+
+ pair<bool, int> nextBatchImpl(
+ PerThreadReadBuf& pt,
+ bool batch_a);
};
/**
@@ -488,9 +509,6 @@ public:
const PatternParams& p,
bool secondName) : // whether it's --12/--tab5 or --tab6
CFilePatternSource(infiles, p),
- solQuals_(p.solexa64),
- phred64Quals_(p.phred64),
- intQuals_(p.intQuals),
secondName_(secondName) { }
/**
@@ -507,9 +525,6 @@ protected:
PerThreadReadBuf& pt,
bool batch_a);
- bool solQuals_; // base qualities are log odds
- bool phred64Quals_; // base qualities are on -64 scale
- bool intQuals_; // base qualities are space-separated strings
bool secondName_; // true if --tab6, false if --tab5
};
@@ -539,10 +554,7 @@ public:
QseqPatternSource(
const EList<std::string>& infiles,
const PatternParams& p) :
- CFilePatternSource(infiles, p),
- solQuals_(p.solexa64),
- phred64Quals_(p.phred64),
- intQuals_(p.intQuals) { }
+ CFilePatternSource(infiles, p) { }
/**
* Finalize qseq parsing outside critical section.
@@ -558,9 +570,6 @@ protected:
PerThreadReadBuf& pt,
bool batch_a);
- bool solQuals_; // base qualities are log odds
- bool phred64Quals_; // base qualities are on -64 scale
- bool intQuals_; // base qualities are space-separated strings
EList<std::string> qualToks_;
};
@@ -647,9 +656,6 @@ public:
const PatternParams& p, bool interleaved = false) :
CFilePatternSource(infiles, p),
first_(true),
- solQuals_(p.solexa64),
- phred64Quals_(p.phred64),
- intQuals_(p.intQuals),
interleaved_(interleaved) { }
virtual void reset() {
@@ -679,9 +685,6 @@ protected:
}
bool first_; // parsing first read in file
- bool solQuals_; // base qualities are log odds
- bool phred64Quals_; // base qualities are on -64 scale
- bool intQuals_; // base qualities are space-separated strings
bool interleaved_; // fastq reads are interleaved
};
diff --git a/presets.cpp b/presets.cpp
index 9fef89c..88066b6 100644
--- a/presets.cpp
+++ b/presets.cpp
@@ -35,22 +35,26 @@ void PresetsV0::apply(
// --sensitive -M 15 -R 2 -N 0 -L 22 -i S,1,1.15
// --very-sensitive -M 25 -R 3 -N 0 -L 19 -i S,1,0.50
if(preset == "very-fast") {
- policy += ";SEED=0,22";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=22";
policy += ";DPS=5";
policy += ";ROUNDS=1";
policy += ";IVAL=S,0,2.50";
} else if(preset == "fast") {
- policy += ";SEED=0,22";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=22";
policy += ";DPS=10";
policy += ";ROUNDS=2";
policy += ";IVAL=S,0,2.50";
} else if(preset == "sensitive") {
- policy += ";SEED=0,22";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=22";
policy += ";DPS=15";
policy += ";ROUNDS=2";
policy += ";IVAL=S,1,1.15";
} else if(preset == "very-sensitive") {
- policy += ";SEED=0,20";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=20";
policy += ";DPS=20";
policy += ";ROUNDS=3";
policy += ";IVAL=S,1,0.50";
@@ -61,7 +65,8 @@ void PresetsV0::apply(
// --sensitive-local -M 2 -N 0 -L 20 -i S,1,0.75 (default)
// --very-sensitive-local -M 3 -N 0 -L 20 -i S,1,0.50
else if(preset == "very-fast-local") {
- policy += ";SEED=0,25";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=25";
policy += ";DPS=5";
policy += ";ROUNDS=1";
policy += ";IVAL=S,1,2.00";
@@ -71,12 +76,14 @@ void PresetsV0::apply(
policy += ";ROUNDS=2";
policy += ";IVAL=S,1,1.75";
} else if(preset == "sensitive-local") {
- policy += ";SEED=0,20";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=20";
policy += ";DPS=15";
policy += ";ROUNDS=2";
policy += ";IVAL=S,1,0.75";
} else if(preset == "very-sensitive-local") {
- policy += ";SEED=0,20";
+ policy += ";SEED=0";
+ policy += ";SEEDLEN=20";
policy += ";DPS=20";
policy += ";ROUNDS=3";
policy += ";IVAL=S,1,0.50";
diff --git a/read_qseq.cpp b/read_qseq.cpp
index 286e28d..f428d65 100644
--- a/read_qseq.cpp
+++ b/read_qseq.cpp
@@ -148,7 +148,7 @@ bool QseqPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
}
if(isalpha(c)) {
assert_in(toupper(c), "ACGTN");
- if(++nchar > gTrim5) {
+ if(++nchar > pp_.trim5) {
assert_neq(0, asc2dnacat[c]);
r.patFw.append(asc2dna[c]);
}
@@ -164,12 +164,12 @@ bool QseqPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
// record amt trimmed from 5' end due to --trim5
r.trimmed5 = (int)(nchar - r.patFw.length());
// record amt trimmed from 3' end due to --trim3
- r.trimmed3 = (int)(r.patFw.trimEnd(gTrim3));
+ r.trimmed3 = (int)(r.patFw.trimEnd(pp_.trim3));
// 10. Qualities
assert(r.qual.empty());
int nqual = 0;
- if (intQuals_) {
+ if (pp_.intQuals) {
int cur_int = 0;
while(c != '\t') {
cur_int *= 10;
@@ -177,10 +177,10 @@ bool QseqPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
c = r.readOrigBuf[cur++];
assert(c != '\r' && c != '\n');
if(c == ' ' || c == '\t') {
- char cadd = intToPhred33(cur_int, solQuals_);
+ char cadd = intToPhred33(cur_int, pp_.solexa64);
cur_int = 0;
assert_geq(cadd, 33);
- if(++nqual > gTrim5) {
+ if(++nqual > pp_.trim5) {
r.qual.append(cadd);
}
}
@@ -195,7 +195,7 @@ bool QseqPatternSource::parse(Read& r, Read& rb, TReadId rdid) const {
} else if(c == '\t') {
break;
}
- c = charToPhred33(c, solQuals_, phred64Quals_);
+ c = charToPhred33(c, pp_.solexa64, pp_.phred64);
if(++nqual > r.trimmed5) {
r.qual.append(c);
}
diff --git a/scripts/sim/Sim.pm b/scripts/sim/Sim.pm
index 8c27b05..7874231 100644
--- a/scripts/sim/Sim.pm
+++ b/scripts/sim/Sim.pm
@@ -744,7 +744,7 @@ sub genPolicySEED() {
my $sd = substr("012", int(rand(2)), 1);
if(rand() < 0.9) {
# Length
- $sd .= ",".int(Math::Random::random_uniform(1, 12, 32));
+ $sd .= " -L".int(Math::Random::random_uniform(1, 12, 32));
}
return $sd;
}
diff --git a/scripts/test/simple_tests.pl b/scripts/test/simple_tests.pl
index b98edab..f1fdde7 100644
--- a/scripts/test/simple_tests.pl
+++ b/scripts/test/simple_tests.pl
@@ -452,6 +452,27 @@ my @cases = (
"\n\n\r\n>r1\nATCGATCAGTATCTG\r\n",
hits => [{ 2 => 1 }, { 3 => 1 }] },
+ { name => "Fasta multiline 1",
+ ref => [ "AGCATCGATCAGTATCTGA" ],
+ args => "-u 2",
+ fasta => "\n\n\r\n>r0\nCATCGAT\nCAGTATCTG\r\n".
+ "\n\n\r\n>r1\nATCGATCAGTATCTG\r\n",
+ hits => [{ 2 => 1 }, { 3 => 1 }] },
+
+ { name => "Fasta multiline 2",
+ ref => [ "AGCATCGATCAGTATCTGA" ],
+ args => "-u 2",
+ fasta => "\n\n\r\n>r0\nCATCGAT\nCAGTATCTG\r\n".
+ "\n\n\r\n>r1\nATCGATC\nAGTATCTG\r\n",
+ hits => [{ 2 => 1 }, { 3 => 1 }] },
+
+ { name => "Fasta multiline 3",
+ ref => [ "AGCATCGATCAGTATCTGA" ],
+ args => "-u 2",
+ fasta => "\n\n\r\n>r0\nCATCGAT\nCAGTATCTG\r\n".
+ "\n\n\r\n>\nATCGATC\nAGTATCTG\r\n",
+ hits => [{ 2 => 1 }, { 3 => 1 }] },
+
{ name => "Fasta paired 1",
ref => [ "AGCATCGATCAAAAACTGA" ],
# AGCATCGATC
@@ -1230,7 +1251,7 @@ my @cases = (
{ name => "Local alignment 2",
ref => [ "TTGA" ],
reads => [ "TTGT" ],
- args => "--local --policy \"MIN=L,1.0,0.75\\;SEED=0,3\\;IVAL=C,1,0\"",
+ args => "--local --policy \"MIN=L,1.0,0.75\\;SEED=0\\;SEEDLEN=3\\;IVAL=C,1,0\"",
report => "-a",
hits => [ { 0 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:3=1S" ],
@@ -1331,7 +1352,7 @@ my @cases = (
{ name => "Local alignment 4",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTCGTTTGT" ],
- args => "--local --policy \"SEED=0,3\\;IVAL=C,1,0\" --score-min=C,12",
+ args => "--local --policy \"SEED=0\\;SEEDLEN=3\\;IVAL=C,1,0\" --score-min=C,12",
report => "-a",
hits => [ { 0 => 1, 8 => 1 } ],
flags_map => [{
@@ -2178,7 +2199,7 @@ my @cases = (
# NNNNNGA------A-------------------G-NTTT
# ||||||||||||||||||||||||||||||||||
# CCAAT-ATTTTTAATTTCCTCTATTTTTCTCTCGTCTTG
- args => "--policy \"NP=Q\\;RDG=46.3220993654702\\;RFG=41.3796024365659\\;MIN=L,5.57015383125426,-3.28597145122829\\;NCEIL=L,0.263054599454459,0.130843661549367\\;SEED=1,29\\;IVAL=L,0.0169183264663712,3.75762168662522\" --overhang --trim5 6",
+ args => "--policy \"NP=Q\\;RDG=46.3220993654702\\;RFG=41.3796024365659\\;MIN=L,5.57015383125426,-3.28597145122829\\;NCEIL=L,0.263054599454459,0.130843661549367\\;SEED=1\\;SEEDLEN=29\\;IVAL=L,0.0169183264663712,3.75762168662522\" --overhang --trim5 6",
reads => [ "CTTTGCACCCCTCCCTTGTCGGCTCCCACCCATCCCCATCCGTTGTCCCCGCCCCCGCCCGCCGGTCGTCACTCCCCGTTTGCGTCATGCCCCTCACCCTCCCTTTGTCGGCTCGCACCCCTCCCCATCCGTTGTCCCCGCCCCCGCTCTCGGGGTCTTCACGCCCCGCTTGCTTCATGCCCCTCACTCGCACCCCG" ],
},
@@ -2380,7 +2401,7 @@ my @cases = (
{ ref => [ "TGC" ],
reads => [ "ATGC" ],
- args => "--overhang --policy \"SEED=0,3\\;IVAL=C,1,0\\;NCEIL=L,1,0\"",
+ args => "--overhang --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,1,0\"",
hits => [ { 0 => 1 } ],
cigar => [ "1S3M" ],
samoptflags => [
@@ -2389,7 +2410,7 @@ my @cases = (
{ ref => [ "TTGTTCGT" ],
reads => [ "TTGTTCG" ],
- args => "--policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
hits => [ { 0 => 1 } ],
cigar => [ "7M" ],
samoptflags => [ { "AS:i:0" => 1, "YT:Z:UU" => 1, "MD:Z:7" => 1 } ]
@@ -2434,7 +2455,7 @@ my @cases = (
# GTTCGTA
# ATTGTTC
reads => [ "TGTTCGT", "GTTCGTA", "ATTGTTC" ],
- args => "--overhang --policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
+ args => "--overhang --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
hits => [ { 1 => 1 }, { 2 => 1 }, { 0 => 1 } ],
cigar => [ "7M", "6M1S", "1S6M" ],
samoptflags => [
@@ -2446,7 +2467,7 @@ my @cases = (
# Same as previous case but --overhang not specified
{ ref => [ "TTGTTCGT" ],
reads => [ "TGTTCGT", "TTGTTCG", "GTTCGTA", "ATTGTTC" ],
- args => "--policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
hits => [ { 1 => 1 }, { 0 => 1 } ], # only the internal hits
cigar => [ "7M", "7M", "*", "*" ],
samoptflags => [
@@ -2461,7 +2482,7 @@ my @cases = (
# a special NCEIL setting.
{ ref => [ "TTGTTCGT" ],
reads => [ "TTGTTCG" ],
- args => "--overhang --policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
+ args => "--overhang --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
hits => [ { 0 => 1 } ]},
{ ref => [ "TTGTTCGT" ],
@@ -3601,7 +3622,7 @@ my @cases = (
{ ref => [ "AAAAAAAAAAAAAAAAAAAAAAAAACCCCCCCCCCCCCCCCCCCCCCCC" ],
reads => [ "AA", "AA", "AA", "AA", "CC", "CC", "CC", "CC", "AA", "AA", "AA", "AA", "CC", "CC", "CC", "CC" ],
names => [ "r1", "r1", "r1", "r1", "r2", "r2", "r2", "r2", "r3", "r3", "r3", "r3", "r4", "r4", "r4", "r4" ],
- args => "--policy \"SEED=1,1\"",
+ args => "--policy \"SEED=1\\;SEEDLEN=2\"",
check_random => 1,
report => "-k 1" },
@@ -3613,7 +3634,7 @@ my @cases = (
{ name => "Gap penalties 1",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTCTTTGTT" ], # budget = 3 + 12 * 3 = 39
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=29,10\\;RFG=25,15\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=29,10\\;RFG=25,15\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { 0 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:6=1D6=" ],
@@ -3627,7 +3648,7 @@ my @cases = (
{ name => "Gap penalties 2",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTCTTTGTT" ], # budget = 3 + 12 * 3 = 39
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=30,10\\;RFG=25,15\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=30,10\\;RFG=25,15\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { "*" => 1 } ],
flags => [ "XM:0,XP:0,XT:UU" ],
@@ -3639,7 +3660,7 @@ my @cases = (
{ name => "Gap penalties 3",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTCGATTTGTT" ], # budget = 3 + 14 * 3 = 45
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=25,15\\;RFG=30,15\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=25,15\\;RFG=30,15\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { 0 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:7=1I6=" ],
@@ -3684,7 +3705,7 @@ my @cases = (
{ name => "Gap penalties 4",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTCGATTTGTT" ], # budget = 3 + 14 * 3 = 45
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=25,15\\;RFG=30,16\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=25,15\\;RFG=30,16\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { "*" => 1 } ],
flags => [ "XM:0,XP:0,XT:UU" ],
@@ -3695,7 +3716,7 @@ my @cases = (
{ name => "Gap penalties 5",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTCGATTTGTT" ], # budget = 3 + 14 * 3 = 45
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=25,15\\;RFG=31,15\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=25,15\\;RFG=31,15\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { "*" => 1 } ],
flags => [ "XM:0,XP:0,XT:UU" ],
@@ -3706,7 +3727,7 @@ my @cases = (
{ name => "Gap penalties 6",
ref => [ "ATTGTTCGTTTGTTCGTA" ],
reads => [ "ATTGTTGTTTGATTCGTA" ], # budget = 3 + 18 * 3 = 57
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=19,10\\;RFG=18,10\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=19,10\\;RFG=18,10\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { 0 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:6=1D5=1I6=" ],
@@ -3716,7 +3737,7 @@ my @cases = (
{ name => "Gap penalties 7",
ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTGTTGTTTGATTCGT" ], # budget = 3 + 16 * 3 = 51
- args => "--policy \"MMP=C30\\;SEED=0,3\\;IVAL=C,1,0\\;RDG=16,10\\;RFG=16,10\\;MIN=L,-3,-3\"",
+ args => "--policy \"MMP=C30\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RDG=16,10\\;RFG=16,10\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { "*" => 1 } ],
flags => [ "XM:0,XP:0,XT:UU" ],
@@ -3766,7 +3787,7 @@ my @cases = (
reads => [ "ATACGCATCGAAC" ],
# 0123456789012345678901234567890
# 1 2 3
- args => "--policy \"NCEIL=L,0,0.1\\;SEED=0,10\\;IVAL=C,1,0\"",
+ args => "--policy \"NCEIL=L,0,0.1\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\"",
report => "-a",
hits => [ { 8 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:2=1X10=" ] },
@@ -3776,7 +3797,7 @@ my @cases = (
reads => [ "ATACGCATCGAAC" ],
# 0123456789012345678901234567890
# 1 2 3
- args => "--policy \"NCEIL=L,0,0.1\\;SEED=0,10\\;IVAL=C,1,0\"",
+ args => "--policy \"NCEIL=L,0,0.1\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\"",
report => "-a",
hits => [ { "*" => 1 } ] },
@@ -3785,7 +3806,7 @@ my @cases = (
# Alignment with 1 reference gap
{ ref => [ "TTTTGTTCGTTTG" ],
reads => [ "TTTTGTTCGATTTG" ], # budget = 3 + 14 * 3 = 45
- args => "--policy \"SEED=0,8\\;IVAL=C,1,0\\;MMP=C30\\;RDG=25,15\\;RFG=25,20\\;MIN=L,-3,-3\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;MMP=C30\\;RDG=25,15\\;RFG=25,20\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { 0 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:9=1I4=" ],
@@ -3815,7 +3836,7 @@ my @cases = (
# Alignment with 1 reference gap
{ ref => [ "TTGTTCGTTTGTT" ],
reads => [ "TTGTTCGATTTGTT" ], # budget = 3 + 14 * 3 = 45
- args => "--policy \"SEED=0,3\\;IVAL=C,1,0\\;MMP=C30\\;RDG=25,15\\;RFG=25,20\\;MIN=L,-3,-3\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;MMP=C30\\;RDG=25,15\\;RFG=25,20\\;MIN=L,-3,-3\"",
report => "-a",
hits => [ { 0 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:7=1I6=" ],
@@ -3829,7 +3850,7 @@ my @cases = (
{ ref => [ "ACNCA" ],
reads => [ "CA" ],
args => "",
- report => "-a --policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
+ report => "-a --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
hits => [ { 3 => 1 } ],
edits => [ ],
flags => [ "XM:0,XP:0,XT:UU,XC:2=" ],
@@ -3843,7 +3864,7 @@ my @cases = (
ref => [ "ACNCA" ],
reads => [ "AC" ],
args => "",
- report => "-a --policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
+ report => "-a --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
hits => [ { 0 => 1 } ],
edits => [ ],
flags => [ ] },
@@ -3854,7 +3875,7 @@ my @cases = (
# 0 1 2
reads => [ "CG" ],
args => "",
- report => "-a --policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
+ report => "-a --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
hits => [ { 13 => 2, 23 => 2 } ],
edits => [ ],
cigar => [ "2M", "2M" ],
@@ -3869,7 +3890,7 @@ my @cases = (
# 0 1 2 3
reads => [ "CG" ],
args => "",
- report => "-a --policy \"SEED=0,2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
+ report => "-a --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
hits => [ { 13 => 2, 23 => 2, 31 => 2 } ],
edits => [ ],
flags => [ "XM:0,XP:0,XT:UU,XC:2=",
@@ -3888,7 +3909,7 @@ my @cases = (
# 0 1 2 3
reads => [ "CG" ],
args => "",
- report => "-a --policy \"SEED=0,1\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
+ report => "-a --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,0,0\"",
hits => [ { 13 => 2, 23 => 2, 31 => 2 } ],
edits => [ ],
flags => [ "XM:0,XP:0,XT:UU,XC:2=",
@@ -3911,7 +3932,7 @@ my @cases = (
{ ref => [ "TTGTTYGT" ],
reads => [ "TTGTTGGT", "TTGTTCGT" ],
args => "",
- report => "-a --policy \"SEED=0,5\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
+ report => "-a --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
hits => [ { 0 => 1 }, { 0 => 1 } ],
norc => 1,
edits => [ "5:N>G", "5:N>C" ],
@@ -3935,7 +3956,7 @@ my @cases = (
# ^
# 4:CC>-
args => "",
- report => "-a --overhang --gbar 3 --policy \"MMP=C30\\;RDG=5,5\\;SEED=0,4\\;IVAL=C,1,0\\;RFG=25,20\\;MIN=L,-3,-3\"",
+ report => "-a --overhang --gbar 3 --policy \"MMP=C30\\;RDG=5,5\\;SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;RFG=25,20\\;MIN=L,-3,-3\"",
hits => [ { 0 => 1 } ],
edits => [ "4:CC>-" ],
flags => [ "XM:0,XP:0,XT:UU,XC:4=2D4=" ],
@@ -3951,7 +3972,7 @@ my @cases = (
reads => [ "ATATGCCCCCCCCCCTCCG" ], # 3 * 19 + 3 = 60
# ^
# 9:ATG>-
- args => "--policy \"SEED=0,8\\;IVAL=C,1,0\\;MMP=C30\\;RDG=5,5\\;RFG=25,15\\;MIN=L,-3,-3\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;MMP=C30\\;RDG=5,5\\;RFG=25,15\\;MIN=L,-3,-3\"",
hits => [ { 0 => 1 } ],
edits => [ "9:ATG>-" ],
norc => 1,
@@ -3970,7 +3991,7 @@ my @cases = (
# ^
# 10:GTA>-
args => "",
- report => "-a --overhang --policy \"SEED=0,8\\;IVAL=C,1,0\\;MMP=C30\\;RDG=5,5\\;RFG=25,20\\;MIN=L,-3,-3\"",
+ report => "-a --overhang --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;MMP=C30\\;RDG=5,5\\;RFG=25,20\\;MIN=L,-3,-3\"",
hits => [ { 0 => 1 } ],
edits => [ "10:GTA>-" ],
norc => 1,
@@ -4195,7 +4216,7 @@ my @cases = (
# Read 3 overhangs right end
{ ref => [ "TTGTTCGT" ],
reads => [ "GTTCGTA" ],
- args => "--overhang --policy \"SEED=0,3\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
+ args => "--overhang --policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;NCEIL=L,2,0\"",
hits => [ { 2 => 1 } ],
flags => [ "XM:0,XP:0,XT:UU,XC:6=1X" ] },
@@ -4226,7 +4247,7 @@ my @cases = (
# wedge a length-3 seed in (there's no room)
{ ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTATTAGT" ],
- args => "--policy \"SEED=0,3\\;IVAL=C,1,0\\;MMP=C1\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=3\\;IVAL=C,1,0\\;MMP=C1\"",
hits => [ { "*" => 1 } ],
flags => [ "XM:0,XP:0,XT:UU" ],
cigar => [ "*" ],
@@ -4238,7 +4259,7 @@ my @cases = (
# length-2 seed in
{ ref => [ "TTGTTCGTTTGTTCGT" ],
reads => [ "TTATTAGT" ],
- args => "--policy \"SEED=0,2\\;IVAL=C,1,0\\;MMP=C1\"",
+ args => "--policy \"SEED=0\\;SEEDLEN=2\\;IVAL=C,1,0\\;MMP=C1\"",
#
# TTGTTCGTTTGTTCGT TTGTTCGTTTGTTCGT TTGTTCGTTTGTTCGT
# || || || || | | || ||
diff --git a/search_globals.h b/search_globals.h
index 448c9c7..9b6a9f4 100644
--- a/search_globals.h
+++ b/search_globals.h
@@ -35,8 +35,6 @@ extern bool gMate1fw;
extern bool gMate2fw;
extern int gMinInsert;
extern int gMaxInsert;
-extern int gTrim5;
-extern int gTrim3;
extern int gGapBarrier;
extern int gAllowRedundant;
diff --git a/threading.h b/threading.h
index 98cdec3..67e4bca 100644
--- a/threading.h
+++ b/threading.h
@@ -29,11 +29,11 @@
# include <tbb/mutex.h>
# include <tbb/spin_mutex.h>
# include <tbb/queuing_mutex.h>
+# include <tbb/atomic.h>
# ifdef WITH_AFFINITY
# include <sched.h>
# include <tbb/task_group.h>
# include <tbb/task_scheduler_observer.h>
-# include <tbb/atomic.h>
# include <tbb/task_scheduler_init.h>
# endif
#else
@@ -59,6 +59,13 @@
# endif
#endif /* NO_SPINLOCK */
+#ifdef WITH_TBB
+struct thread_tracking_pair {
+ int tid;
+ tbb::atomic<int>* done;
+};
+#endif
+
/**
* Wrap a lock; obtain lock upon construction, release upon destruction.
@@ -66,38 +73,37 @@
class ThreadSafe {
public:
- ThreadSafe() : ptr_mutex(NULL) { }
-
- ThreadSafe(MUTEX_T* ptr_mutex, bool locked = true) : ptr_mutex(NULL) {
- if(locked) {
+ ThreadSafe(MUTEX_T& ptr_mutex) : mutex_(ptr_mutex) {
#if WITH_TBB && NO_SPINLOCK && WITH_QUEUELOCK
- //have to use the heap as we can't copy
- //the scoped lock
- this->ptr_mutex = new MUTEX_T::scoped_lock(*ptr_mutex);
#else
- this->ptr_mutex = ptr_mutex;
- ptr_mutex->lock();
+ mutex_.lock();
#endif
- }
}
~ThreadSafe() {
- if (ptr_mutex != NULL)
#if WITH_TBB && NO_SPINLOCK && WITH_QUEUELOCK
- delete ptr_mutex;
#else
- ptr_mutex->unlock();
+ mutex_.unlock();
#endif
}
private:
#if WITH_TBB && NO_SPINLOCK && WITH_QUEUELOCK
- MUTEX_T::scoped_lock* ptr_mutex;
+ MUTEX_T::scoped_lock mutex_;
#else
- MUTEX_T *ptr_mutex;
+ MUTEX_T& mutex_;
#endif
};
+#if defined(_TTHREAD_WIN32_)
+#define SLEEP(x) Sleep(x)
+#else
+#define SLEEP(x) do { \
+ const static timespec ts_tmp_ = {0, 1000000 * x}; \
+ nanosleep(&ts_tmp_, NULL); \
+} while(false)
+#endif
+
#ifdef WITH_TBB
#ifdef WITH_AFFINITY
//ripped entirely from;
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/debian-med/bowtie2.git
More information about the debian-med-commit
mailing list