[med-svn] [vsearch] 01/04: New upstream version 2.4.3
Andreas Tille
tille at debian.org
Mon Jun 19 10:05:47 UTC 2017
This is an automated email from the git hooks/post-receive script.
tille pushed a commit to branch master
in repository vsearch.
commit 38304ccb137098dc7781b4842d52e88ea7938c52
Author: Andreas Tille <tille at debian.org>
Date: Mon Jun 19 11:55:23 2017 +0200
New upstream version 2.4.3
---
.gitignore | 2 +
LICENSE.txt | 2 +-
README.md | 62 ++--
configure.ac | 12 +-
man/vsearch.1 | 64 ++--
src/Makefile.am | 36 +-
src/abundance.cc | 20 +-
src/abundance.h | 6 +-
src/align.cc | 100 +++---
src/align.h | 42 +--
src/align_simd.cc | 963 ++++++++++++++++++++++++++++++++++++++-------------
src/align_simd.h | 2 +-
src/allpairs.cc | 91 ++---
src/allpairs.h | 2 +-
src/arch.cc | 158 ++++++++-
src/arch.h | 13 +-
src/bitmap.cc | 6 +-
src/bitmap.h | 2 +-
src/chimera.cc | 110 +++---
src/chimera.h | 2 +-
src/cluster.cc | 96 ++---
src/cluster.h | 2 +-
src/cpu.cc | 49 ++-
src/cpu.h | 8 +-
src/db.cc | 60 ++--
src/db.h | 24 +-
src/dbhash.cc | 42 +--
src/dbhash.h | 24 +-
src/dbindex.cc | 24 +-
src/dbindex.h | 6 +-
src/derep.cc | 136 ++++----
src/derep.h | 2 +-
src/dynlibs.cc | 73 +++-
src/dynlibs.h | 10 +-
src/eestats.cc | 82 ++---
src/eestats.h | 2 +-
src/fasta.cc | 54 +--
src/fasta.h | 34 +-
src/fastq.cc | 66 ++--
src/fastq.h | 20 +-
src/fastqops.cc | 216 ++++++------
src/fastqops.h | 2 +-
src/fastx.cc | 40 ++-
src/fastx.h | 43 +--
src/linmemalign.cc | 156 ++++-----
src/linmemalign.h | 100 +++---
src/maps.cc | 2 +-
src/maps.h | 2 +-
src/mask.cc | 10 +-
src/mask.h | 2 +-
src/mergepairs.cc | 177 +++++-----
src/mergepairs.h | 2 +-
src/minheap.cc | 8 +-
src/minheap.h | 3 +-
src/msa.cc | 26 +-
src/msa.h | 4 +-
src/otutable.cc | 48 ++-
src/otutable.h | 4 +-
src/rerep.cc | 14 +-
src/rerep.h | 2 +-
src/results.cc | 83 ++---
src/results.h | 14 +-
src/search.cc | 71 ++--
src/search.h | 2 +-
src/searchcore.cc | 42 ++-
src/searchcore.h | 2 +-
src/searchexact.cc | 79 +++--
src/searchexact.h | 2 +-
src/showalign.cc | 64 ++--
src/showalign.h | 12 +-
src/shuffle.cc | 6 +-
src/shuffle.h | 2 +-
src/sortbylength.cc | 4 +-
src/sortbylength.h | 2 +-
src/sortbysize.cc | 6 +-
src/sortbysize.h | 2 +-
src/subsample.cc | 26 +-
src/subsample.h | 2 +-
src/unique.cc | 40 +--
src/unique.h | 2 +-
src/userfields.cc | 6 +-
src/userfields.h | 2 +-
src/util.cc | 134 +++----
src/util.h | 20 +-
src/vsearch.cc | 203 ++++++-----
src/vsearch.h | 218 +++++++-----
src/xstring.h | 2 +-
test/unclassified.sh | 70 ++++
88 files changed, 2688 insertions(+), 1800 deletions(-)
diff --git a/.gitignore b/.gitignore
index f4875a3..fba0eba 100644
--- a/.gitignore
+++ b/.gitignore
@@ -9,6 +9,8 @@
/config.h
/config.log
/config.status
+/config.guess
+/config.sub
/stamp-h1
Makefile
aclocal.m4
diff --git a/LICENSE.txt b/LICENSE.txt
index 96432e8..a8d3aa1 100644
--- a/LICENSE.txt
+++ b/LICENSE.txt
@@ -1,6 +1,6 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/README.md b/README.md
index c393373..a60d3d0 100644
--- a/README.md
+++ b/README.md
@@ -14,7 +14,7 @@ We have implemented a tool called VSEARCH which supports *de novo* and reference
VSEARCH stands for vectorized search, as the tool takes advantage of parallelism in the form of SIMD vectorization as well as multiple threads to perform accurate alignments at high speed. VSEARCH uses an optimal global aligner (full dynamic programming Needleman-Wunsch), in contrast to USEARCH which by default uses a heuristic seed and extend aligner. This usually results in more accurate alignments and overall improved sensitivity (recall) with VSEARCH, especially for alignments with gaps.
-VSEARCH binaries are provided for x86-64 systems running GNU/Linux or OS X (10.7 or higher). A beta version for 64-bit Windows (version 7 or higher) is also available.
+VSEARCH binaries are provided for x86-64 systems running GNU/Linux, macOS (version 10.7 or higher) and Windows (64-bit, version 7 or higher), as well as ppc64le systems running GNU/Linux.
VSEARCH can directly read input query and database files that are compressed using gzip and bzip2 (.gz and .bz2) if the zlib and bzip2 libraries are available.
@@ -35,9 +35,9 @@ In the example below, VSEARCH will identify sequences in the file database.fsa t
**Source distribution** To download the source distribution from a [release](https://github.com/torognes/vsearch/releases) and build the executable and the documentation, use the following commands:
```
-wget https://github.com/torognes/vsearch/archive/v2.3.4.tar.gz
-tar xzf v2.3.4.tar.gz
-cd vsearch-2.3.4
+wget https://github.com/torognes/vsearch/archive/v2.4.3.tar.gz
+tar xzf v2.4.3.tar.gz
+cd vsearch-2.4.3
./autogen.sh
./configure
make
@@ -46,6 +46,10 @@ make install # as root or sudo make install
You may customize the installation directory using the `--prefix=DIR` option to `configure`. If the compression libraries [zlib](http://www.zlib.net) and/or [bzip2](http://www.bzip.org) are installed on the system, they will be detected automatically and support for compressed files will be included in vsearch. Support for compressed files may be disabled using the `--disable-zlib` and `--disable-bzip2` options to `configure`. A PDF version of the manual will be created from the `vsearch [...]
+The IBM XL C++ compiler is recommended on ppc64le systems.
+
+The Windows binary was compiled using the [Mingw-w64](https://mingw-w64.org/) C++ cross-compiler.
+
**Cloning the repo** Instead of downloading the source distribution as a compressed archive, you could clone the repo and build it as shown below. The options to `configure` as described above are still valid.
```
@@ -57,33 +61,40 @@ make
make install # as root or sudo make install
```
-**Binary distribution** Starting with version 1.4.0, binary distribution files (.tar.gz) for GNU/Linux on x86-64 and Apple Mac OS X on x86-64 containing pre-compiled binaries as well as the documentation (man and pdf files) will be made available as part of each [release](https://github.com/torognes/vsearch/releases). The included executables include support for input files compressed by zlib and bzip2 (with files usually ending in `.gz` or `.bz2`). Download the appropriate executable fo [...]
+**Binary distribution** Starting with version 1.4.0, binary distribution files containing pre-compiled binaries as well as the documentation will be made available as part of each [release](https://github.com/torognes/vsearch/releases). The included executables include support for input files compressed by zlib and bzip2 (with files usually ending in `.gz` or `.bz2`).
+
+Binary distributions are provided for x86-64 systems running GNU/Linux, macOS (version 10.7 or higher) and Windows (64-bit, version 7 or higher), as well as ppc64le systems running GNU/Linux.
+
+Download the appropriate executable for your system using the following commands if you are using a Linux x86_64 system:
```sh
-wget https://github.com/torognes/vsearch/releases/download/v2.3.4/vsearch-2.3.4-linux-x86_64.tar.gz
-tar xzf vsearch-2.3.4-linux-x86_64.tar.gz
+wget https://github.com/torognes/vsearch/releases/download/v2.4.3/vsearch-2.4.3-linux-x86_64.tar.gz
+tar xzf vsearch-2.4.3-linux-x86_64.tar.gz
```
-Or these commands if you are using a Mac:
+Or these commands if you are using a Linux ppc64le system:
```sh
-wget https://github.com/torognes/vsearch/releases/download/v2.3.4/vsearch-2.3.4-osx-x86_64.tar.gz
-tar xzf vsearch-2.3.4-osx-x86_64.tar.gz
+wget https://github.com/torognes/vsearch/releases/download/v2.4.3/vsearch-2.4.3-linux-ppc64le.tar.gz
+tar xzf vsearch-2.4.3-linux-ppc64le.tar.gz
```
-You will now have the binary distribution in a folder called something like `vsearch-2.3.4-linux-x86_64` in which you will find three subfolders `bin`, `man` and `doc`. We recommend making a copy or a symbolic link to the vsearch binary `bin/vsearch` in a folder included in your `$PATH`, and a copy or a symbolic link to the vsearch man page `man/vsearch.1` in a folder included in your `$MANPATH`. The PDF version of the manual is available in `doc/vsearch_manual.pdf`.
+Or these commands if you are using a Mac:
+```sh
+wget https://github.com/torognes/vsearch/releases/download/v2.4.3/vsearch-2.4.3-macos-x86_64.tar.gz
+tar xzf vsearch-2.4.3-macos-x86_64.tar.gz
+```
-**Old binaries** Older VSEARCH binaries (until version 1.1.3) are available [here](https://github.com/torognes/vsearch/releases) for [GNU/Linux on x86-64 systems](https://github.com/torognes/vsearch/blob/master/bin/vsearch-1.1.3-linux-x86_64) and [Apple Mac OS X on x86-64 systems](https://github.com/torognes/vsearch/blob/master/bin/vsearch-1.1.3-osx-x86_64). These executables include support for input files compressed by zlib and bzip2 (with files usually ending in `.gz` or `.bz2`). Down [...]
+Or if you are using Windows, download and extract (unzip) the contents of this file:
-```sh
-cd ~
-mkdir -p bin
-cd bin
-wget https://github.com/torognes/vsearch/releases/download/v1.1.3/vsearch-1.1.3-linux-x86_64
-chmod a+x vsearch-1.1.3-linux-x86_64
-ln -s vsearch-1.1.3-linux-x86_64 vsearch
```
+https://github.com/torognes/vsearch/releases/download/v2.4.3/vsearch-2.4.3-win-x86_64.zip
+```
+
+Linux and Mac: You will now have the binary distribution in a folder called `vsearch-2.4.3-linux-x86_64` or `vsearch-2.4.3-macos-x86_64` in which you will find three subfolders `bin`, `man` and `doc`. We recommend making a copy or a symbolic link to the vsearch binary `bin/vsearch` in a folder included in your `$PATH`, and a copy or a symbolic link to the vsearch man page `man/vsearch.1` in a folder included in your `$MANPATH`. The PDF version of the manual is available in `doc/vsearch_m [...]
+
+Windows: You will now have the binary distribution in a folder called `vsearch-2.4.3-win-x86_64`. The vsearch executable is called `vsearch.exe`. The manual in PDF format is called `vsearch_manual.pdf`.
**Documentation** The VSEARCH user's manual is available in the `man` folder in the form of a [man page](https://github.com/torognes/vsearch/blob/master/doc/vsearch.1). A pdf version (vsearch_manual.pdf) will be generated by `make`. To install the manpage manually, copy the `vsearch.1` file or a create a symbolic link to `vsearch.1` in a folder included in your `$MANPATH`. The manual in both formats is also available with the binary distribution. The manual in PDF form (vsearch_manual.pd [...]
@@ -116,6 +127,8 @@ When compiling VSEARCH the header files for the following two optional libraries
* libz (zlib library) (zlib.h header file) (optional)
* libbz2 (bzip2lib library) (bzlib.h header file) (optional)
+On Windows these libraries are called zlib1.dll and bz2.dll.
+
VSEARCH will automatically check whether these libraries are available and load them dynamically.
@@ -212,7 +225,6 @@ VSEARCH is designed for rather short sequences, and will be slow when sequences
Some issues to work on:
* testing and debugging
-* performance evaluation
* heuristics for alignment of long sequences (e.g. banded alignment around selected diagonals)?
@@ -229,11 +241,13 @@ The main contributors to VSEARCH:
## Acknowledgements
-Thanks to the following people for patches and other suggestions for improvements:
-
-* Jeff Epler <jepler at unpythonic.net>
-* Andreas Tille <tille at debian.org>
+Special thanks to the following people for patches, suggestions, computer access etc:
+* Colin Brislawn
+* Jeff Epler
+* Christopher M. Sullivan
+* Andreas Tille
+* Sarah Westcott
## Citing VSEARCH
diff --git a/configure.ac b/configure.ac
index 97da8b5..6c83acd 100644
--- a/configure.ac
+++ b/configure.ac
@@ -2,7 +2,8 @@
# Process this file with autoconf to produce a configure script.
AC_PREREQ([2.63])
-AC_INIT([vsearch], [2.3.4], [torognes at ifi.uio.no])
+AC_INIT([vsearch], [2.4.3], [torognes at ifi.uio.no])
+AC_CANONICAL_TARGET
AM_INIT_AUTOMAKE([subdir-objects])
AC_LANG([C++])
AC_CONFIG_SRCDIR([src/vsearch.cc])
@@ -24,6 +25,8 @@ AC_PROG_INSTALL
# Checks for libraries.
AC_CHECK_LIB([pthread], [pthread_create])
AC_CHECK_LIB([dl], [dlopen])
+#AC_CHECK_LIB([regex], [regexec])
+AC_CHECK_LIB([psapi], [GetProcessMemoryInfo])
# Checks for header files.
AC_CHECK_HEADERS([getopt.h fcntl.h float.h regex.h ctype.h locale.h limits.h string.h sys/time.h dlfcn.h])
@@ -82,10 +85,17 @@ if test "x${have_zlib}" = "xyes"; then
AC_CHECK_HEADERS([zlib.h], [], [have_zlib=no])
fi
+case $target in
+ powerpc64*) target_ppc="yes" ;;
+esac
+
+AC_CHECK_HEADERS([windows.h], [AM_CONDITIONAL(TARGET_WIN, true)], [AM_CONDITIONAL(TARGET_WIN, false)])
+
AM_CONDITIONAL(HAVE_BZLIB, test "x${have_bzip2}" = "xyes")
AM_CONDITIONAL(HAVE_ZLIB, test "x${have_zlib}" = "xyes")
AM_CONDITIONAL(HAVE_PTHREADS, test "x${have_pthreads}" = "xyes")
AM_CONDITIONAL(HAVE_PS2PDF, test "x${have_ps2pdf}" = "xyes")
+AM_CONDITIONAL(TARGET_PPC, test "x${target_ppc}" = "xyes")
AM_PROG_CC_C_O
AC_CONFIG_FILES([Makefile
diff --git a/man/vsearch.1 b/man/vsearch.1
index ae702df..5be0517 100644
--- a/man/vsearch.1
+++ b/man/vsearch.1
@@ -1,5 +1,5 @@
.\" ============================================================================
-.TH vsearch 1 "December 9, 2016" "version 2.3.4" "USER COMMANDS"
+.TH vsearch 1 "April 6, 2017" "version 2.4.3" "USER COMMANDS"
.\" ============================================================================
.SH NAME
vsearch \(em chimera detection, clustering, dereplication and
@@ -125,16 +125,16 @@ processing.
Comparing nucleotide sequences is at the core of \fBvsearch\fR. To
speed up comparisons, \fBvsearch\fR implements an extremely fast
Needleman-Wunsch algorithm, making use of the Streaming SIMD
-Extensions (SSE2) of post-2003 x86-64 CPUs. If SSE2 instructions are
-not available, \fBvsearch\fR exits with an error message. Memory usage
-increases rapidly with sequence length: for example comparing two
-sequences of length 1 kb requires 8 MB of memory per thread, and
-comparing two 10 kb sequences requires 800 MB of memory per
-thread. For comparisons involving sequences with a length product
-greater than 25 million (for example two sequences of length 5 kb),
-\fBvsearch\fR uses a slower alignment method described by Hirschberg
-(1975) and Myers and Miller (1988), with much smaller memory
-requirements.
+Extensions (SSE2) of post-2003 x86-64 CPUs. If SSE2 instructions are
+not available, \fBvsearch\fR exits with an error message. On Power8
+CPUs it will use AltiVec/VSX/VMX instructions. Memory usage increases
+rapidly with sequence length: for example comparing two sequences of
+length 1 kb requires 8 MB of memory per thread, and comparing two 10
+kb sequences requires 800 MB of memory per thread. For comparisons
+involving sequences with a length product greater than 25 million (for
+example two sequences of length 5 kb), \fBvsearch\fR uses a slower
+alignment method described by Hirschberg (1975) and Myers and Miller
+(1988), with much smaller memory requirements.
.\" ----------------------------------------------------------------------------
.SS Input
\fBvsearch\fR accept as input fasta or fastq files containing one or
@@ -265,11 +265,10 @@ fatal error messages.
Number of computation threads to use (1 to 256). The number of threads
should be lesser or equal to the number of available CPU cores. The
default is to use all available resources and to launch one thread per
-logical core. The following commands are multi-threaded: uchime_ref,
-cluster_fast, cluster_size, cluster_smallmem, fastq_mergepairs,
-maskfasta, allpairs_global, usearch_global. Only one thread is used
-for the other commands.
-.\" fastq_mergepairs is not multithreaded in v2.1.0
+logical core. The following commands are multi-threaded:
+allpairs_global, cluster_fast, cluster_size, cluster_smallmem,
+fastq_mergepairs, maskfasta, search_exact, uchime_ref, and
+usearch_global. Only one thread is used for the other commands.
.TP
.B \-\-version | \-v
Output version information and exit.
@@ -1055,7 +1054,7 @@ with the \-\-fastq_maxns are also discarded (no limit by
default). Staggered reads are not merged unless the
\-\-fastq_allowmergestagger option is specified. The minimum length of
the overlap region between the reads may be specified with the
-\-\-minovlen option (default 10), and the overlap region may not
+\-\-minovlen option (default 16), and the overlap region may not
include more mismatches than specified with the \-\-maxdiffs option (5
by default), otherwise the read pair is discarded. The mimimum and
maximum length of the merged sequence may be specified with the
@@ -2581,8 +2580,8 @@ Frédéric Mahé.
.PP
Rognes T, Flouri T, Nichols B, Quince C, Mahé F. (2016)
VSEARCH: a versatile open source tool for metagenomics.
-\fIPeerJ Preprints\fR 4:e2409v1
-<https://doi.org/10.7287/peerj.preprints.2409v1>
+\fIPeerJ\fR 4:e2584 doi: 10.7717/peerj.2584
+<https://doi.org/10.7717/peerj.2584>
.PP
.\" ============================================================================
.SH REPORTING BUGS
@@ -2598,7 +2597,7 @@ Source code and binaries are available at
.PP
.\" ============================================================================
.SH COPYRIGHT
-Copyright (C) 2014-2016, Torbjørn Rognes, Frédéric Mahé and Tomás Flouri
+Copyright (C) 2014-2017, Torbjørn Rognes, Frédéric Mahé and Tomás Flouri
.PP
All rights reserved.
.PP
@@ -2669,10 +2668,6 @@ for the MD5 message digest algorithm.
\fBvsearch\fR includes public domain code written by Steve Reid and
others for the SHA1 message digest algorithm.
.IP -
-\fBvsearch\fR includes statistical data from PEAR
-<https://github.com/xflouris/PEAR> by Zhang, Kobert, Flouri and
-Stamatakis. Used with permission.
-.IP -
\fBvsearch\fR binaries may include code from the zlib library,
copyright Jean-Loup Gailly and Mark Adler.
.IP -
@@ -3048,6 +3043,27 @@ A minor speed improvement.
.TP
.BR v2.3.4\~ "released December 9th, 2016"
Fixed bug in output of sequence profiles and updated documentation.
+.TP
+.BR v2.4.0\~ "released February 8th, 2017"
+Added support for Linux on Power8 systems (ppc64le) and Windows on
+x86_64. Improved detection of pipes when reading FASTA and FASTQ
+files. Corrected option for specifiying output from fastq_eestats
+command in help text.
+.TP
+.BR v2.4.1\~ "released March 1st, 2017"
+Fixed an overflow bug in fastq_stats and fastq_eestats affecting
+analysis of very large FASTQ files. Fixed maximum memory usage
+reporting on Windows.
+.TP
+.BR v2.4.2\~ "released March 10th, 2017"
+Default value for fastq_minovlen increased to 16 in accordance with
+help text and for compatibility with usearch. Minor changes for
+improved accuracy of paired-end read merging.
+.TP
+.BR v2.4.3\~ "released April 6th, 2017"
+Fixed bug with progress bar for shuffling. Fixed missing N-lines in UC
+files with usearch_global, search_exact and allpairs_global when the
+output_no_hits option was not specified.
.RE
.LP
.\" ============================================================================
diff --git a/src/Makefile.am b/src/Makefile.am
index 02417b1..e56a8a2 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -1,6 +1,11 @@
bin_PROGRAMS = $(top_builddir)/bin/vsearch
-AM_CXXFLAGS=-Wall -Wsign-compare -O2 -g
+if TARGET_PPC
+AM_CXXFLAGS=-Wall -Wsign-compare -O3 -g -mcpu=power8
+else
+AM_CXXFLAGS=-Wall -Wsign-compare -O3 -g
+endif
+
AM_CFLAGS=$(AM_CXXFLAGS)
export MACOSX_DEPLOYMENT_TARGET
@@ -52,18 +57,36 @@ util.h \
vsearch.h \
xstring.h
+if TARGET_PPC
+libcpu_a_SOURCES = cpu.cc $(VSEARCHHEADERS)
+noinst_LIBRARIES = libcpu.a libcityhash.a
+else
libcpu_sse2_a_SOURCES = cpu.cc $(VSEARCHHEADERS)
libcpu_sse2_a_CXXFLAGS = $(AM_CXXFLAGS) -msse2
-
libcpu_ssse3_a_SOURCES = cpu.cc $(VSEARCHHEADERS)
libcpu_ssse3_a_CXXFLAGS = $(AM_CXXFLAGS) -mssse3 -DSSSE3
+noinst_LIBRARIES = libcpu_sse2.a libcpu_ssse3.a libcityhash.a
+endif
-libcityhash_a_SOURCES = city.cc city.h citycrc.h
-libcityhash_a_CXXFLAGS = $(AM_CXXFLAGS) -Wno-sign-compare
+libcityhash_a_SOURCES = city.cc city.h
-noinst_LIBRARIES = libcpu_sse2.a libcpu_ssse3.a libcityhash.a
+if TARGET_WIN
+
+libcityhash_a_CXXFLAGS = -Wall -Wno-sign-compare -O3 -g -D_MSC_VER
+__top_builddir__bin_vsearch_LDFLAGS = -static
+__top_builddir__bin_vsearch_LDADD = libregex.a libcityhash.a libcpu_ssse3.a libcpu_sse2.a
-__top_builddir__bin_vsearch_LDADD = libcpu_ssse3.a libcpu_sse2.a libcityhash.a
+else
+
+libcityhash_a_CXXFLAGS = -Wall -Wno-sign-compare -O3 -g
+
+if TARGET_PPC
+__top_builddir__bin_vsearch_LDADD = libcityhash.a libcpu.a
+else
+__top_builddir__bin_vsearch_LDADD = libcityhash.a libcpu_ssse3.a libcpu_sse2.a
+endif
+
+endif
__top_builddir__bin_vsearch_SOURCES = $(VSEARCHHEADERS) \
abundance.cc \
@@ -107,4 +130,3 @@ unique.cc \
userfields.cc \
util.cc \
vsearch.cc
-
diff --git a/src/abundance.cc b/src/abundance.cc
index 3a7704b..ea42295 100644
--- a/src/abundance.cc
+++ b/src/abundance.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -71,19 +71,19 @@ abundance_t * abundance_init(void)
void abundance_exit(abundance_t * a)
{
regfree(&a->regex);
- free(a);
+ xfree(a);
}
-long abundance_get(abundance_t * a, char * header)
+int64_t abundance_get(abundance_t * a, char * header)
{
/* read size/abundance annotation */
- long abundance = 1;
+ int64_t abundance = 1;
regmatch_t pmatch[4];
if (!regexec(&a->regex, header, 4, pmatch, 0))
{
- long number = atol(header + pmatch[2].rm_so);
+ int64_t number = atol(header + pmatch[2].rm_so);
if (number > 0)
abundance = number;
else
@@ -96,7 +96,7 @@ void abundance_fprint_header_with_size(abundance_t * a,
FILE * fp,
char * header,
int header_length,
- unsigned long size)
+ uint64_t size)
{
/* remove any previous size annotation */
/* regexp search for "(^|;)(\d+)(;|$)" */
@@ -110,7 +110,7 @@ void abundance_fprint_header_with_size(abundance_t * a,
int pat_end = pmatch[0].rm_eo;
fprintf(fp,
- "%.*s%s%.*s%ssize=%lu;",
+ "%.*s%s%.*s%ssize=%" PRIu64 ";",
pat_start, header,
(pat_start > 0 ? ";" : ""),
header_length - pat_end, header + pat_end,
@@ -121,7 +121,7 @@ void abundance_fprint_header_with_size(abundance_t * a,
else
{
fprintf(fp,
- "%s%ssize=%lu;",
+ "%s%ssize=%" PRIu64 ";",
header,
(((header_length == 0) ||
(header[header_length - 1] != ';')) ? ";" : ""),
@@ -165,14 +165,14 @@ char * abundance_strip_size(abundance_t * a,
int pat_start = pmatch[0].rm_so;
int pat_end = pmatch[0].rm_eo;
- ret = asprintf(&temp,
+ ret = xsprintf(&temp,
"%.*s%s%.*s",
pat_start, header,
((pat_start > 0) && (pat_end < header_length)) ? ";" : "",
header_length - pat_end, header + pat_end);
}
else
- ret = asprintf(&temp, "%s", header);
+ ret = xsprintf(&temp, "%s", header);
if (ret == -1)
fatal("Out of memory");
diff --git a/src/abundance.h b/src/abundance.h
index 001c1eb..fd81815 100644
--- a/src/abundance.h
+++ b/src/abundance.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -67,13 +67,13 @@ abundance_t * abundance_init(void);
void abundance_exit(abundance_t * a);
-long abundance_get(abundance_t * a, char * header);
+int64_t abundance_get(abundance_t * a, char * header);
void abundance_fprint_header_with_size(abundance_t * a,
FILE * fp,
char * header,
int header_length,
- unsigned long size);
+ uint64_t size);
void abundance_fprint_header_strip_size(abundance_t * a,
FILE * fp,
diff --git a/src/align.cc b/src/align.cc
index f4a7ca6..e62be70 100644
--- a/src/align.cc
+++ b/src/align.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -62,10 +62,10 @@
struct nwinfo_s
{
- long dir_alloc;
- long hearray_alloc;
+ int64_t dir_alloc;
+ int64_t hearray_alloc;
char * dir;
- long * hearray;
+ int64_t * hearray;
};
static const char maskup = 1;
@@ -163,10 +163,10 @@ struct nwinfo_s * nw_init()
void nw_exit(struct nwinfo_s * nw)
{
if (nw->dir)
- free(nw->dir);
+ xfree(nw->dir);
if (nw->hearray)
- free(nw->hearray);
- free(nw);
+ xfree(nw->hearray);
+ xfree(nw);
}
inline int nt_identical(char a, char b)
@@ -177,7 +177,7 @@ inline int nt_identical(char a, char b)
return 0;
}
-inline long getscore(long * score_matrix, char a, char b)
+inline int64_t getscore(int64_t * score_matrix, char a, char b)
{
return score_matrix[(chrmap_4bit[(int)a]<<4) + chrmap_4bit[(int)b]];
}
@@ -186,35 +186,35 @@ void nw_align(char * dseq,
char * dend,
char * qseq,
char * qend,
- long * score_matrix,
- long gapopen_q_left,
- long gapopen_q_interior,
- long gapopen_q_right,
- long gapopen_t_left,
- long gapopen_t_interior,
- long gapopen_t_right,
- long gapextend_q_left,
- long gapextend_q_interior,
- long gapextend_q_right,
- long gapextend_t_left,
- long gapextend_t_interior,
- long gapextend_t_right,
- long * nwscore,
- long * nwdiff,
- long * nwgaps,
- long * nwindels,
- long * nwalignmentlength,
+ int64_t * score_matrix,
+ int64_t gapopen_q_left,
+ int64_t gapopen_q_interior,
+ int64_t gapopen_q_right,
+ int64_t gapopen_t_left,
+ int64_t gapopen_t_interior,
+ int64_t gapopen_t_right,
+ int64_t gapextend_q_left,
+ int64_t gapextend_q_interior,
+ int64_t gapextend_q_right,
+ int64_t gapextend_t_left,
+ int64_t gapextend_t_interior,
+ int64_t gapextend_t_right,
+ int64_t * nwscore,
+ int64_t * nwdiff,
+ int64_t * nwgaps,
+ int64_t * nwindels,
+ int64_t * nwalignmentlength,
char ** nwalignment,
- long queryno,
- long dbseqno,
+ int64_t queryno,
+ int64_t dbseqno,
struct nwinfo_s * nw)
{
- long h, n, e, f, h_e, h_f;
- long *hep;
+ int64_t h, n, e, f, h_e, h_f;
+ int64_t *hep;
- long qlen = qend - qseq;
- long dlen = dend - dseq;
+ int64_t qlen = qend - qseq;
+ int64_t dlen = dend - dseq;
if (qlen * dlen > nw->dir_alloc)
{
@@ -222,16 +222,16 @@ void nw_align(char * dseq,
nw->dir = (char *) xrealloc(nw->dir, (size_t)nw->dir_alloc);
}
- long need = 2 * qlen * (long) sizeof(long);
+ int64_t need = 2 * qlen * (int64_t) sizeof(int64_t);
if (need > nw->hearray_alloc)
{
nw->hearray_alloc = need;
- nw->hearray = (long *) xrealloc(nw->hearray, (size_t)nw->hearray_alloc);
+ nw->hearray = (int64_t *) xrealloc(nw->hearray, (size_t)nw->hearray_alloc);
}
memset(nw->dir, 0, (size_t)(qlen*dlen));
- long i, j;
+ int64_t i, j;
for(i=0; i<qlen; i++)
{
@@ -322,15 +322,15 @@ void nw_align(char * dseq,
}
}
- long dist = nw->hearray[2*qlen-2];
+ int64_t dist = nw->hearray[2*qlen-2];
/* backtrack: count differences and save alignment in cigar string */
- long score = 0;
- long alength = 0;
- long matches = 0;
- long gaps = 0;
- long indels = 0;
+ int64_t score = 0;
+ int64_t alength = 0;
+ int64_t matches = 0;
+ int64_t gaps = 0;
+ int64_t indels = 0;
char * cigar = (char *) xmalloc((size_t)(qlen + dlen + 1));
char * cigarend = cigar+qlen+dlen+1;
@@ -344,10 +344,10 @@ void nw_align(char * dseq,
while ((i>0) && (j>0))
{
- long gapopen_q = (i < qlen) ? gapopen_q_interior : gapopen_q_right;
- long gapextend_q = (i < qlen) ? gapextend_q_interior : gapextend_q_right;
- long gapopen_t = (j < dlen) ? gapopen_t_interior : gapopen_t_right;
- long gapextend_t = (j < dlen) ? gapextend_t_interior : gapextend_t_right;
+ int64_t gapopen_q = (i < qlen) ? gapopen_q_interior : gapopen_q_right;
+ int64_t gapextend_q = (i < qlen) ? gapextend_q_interior : gapextend_q_right;
+ int64_t gapopen_t = (j < dlen) ? gapopen_t_interior : gapopen_t_right;
+ int64_t gapextend_t = (j < dlen) ? gapextend_t_interior : gapextend_t_right;
int d = nw->dir[qlen*(j-1)+(i-1)];
@@ -434,7 +434,7 @@ void nw_align(char * dseq,
/* move and reallocate cigar */
- long cigarlength = cigar+qlen+dlen-cigarend;
+ int64_t cigarlength = cigar+qlen+dlen-cigarend;
memmove(cigar, cigarend, (size_t)(cigarlength+1));
cigar = (char*) xrealloc(cigar, (size_t)(cigarlength+1));
@@ -448,14 +448,14 @@ void nw_align(char * dseq,
#if 1
if (score != dist)
{
- fprintf(stderr, "WARNING: Error with query no %ld and db sequence no %ld:\n", queryno, dbseqno);
- fprintf(stderr, "Initial and recomputed alignment score disagreement: %ld %ld\n", dist, score);
+ fprintf(stderr, "WARNING: Error with query no %" PRId64 " and db sequence no %" PRId64 ":\n", queryno, dbseqno);
+ fprintf(stderr, "Initial and recomputed alignment score disagreement: %" PRId64 " %" PRId64 "\n", dist, score);
fprintf(stderr, "Alignment: %s\n", cigar);
if (opt_log)
{
- fprintf(fp_log, "WARNING: Error with query no %ld and db sequence no %ld:\n", queryno, dbseqno);
- fprintf(fp_log, "Initial and recomputed alignment score disagreement: %ld %ld\n", dist, score);
+ fprintf(fp_log, "WARNING: Error with query no %" PRId64 " and db sequence no %" PRId64 ":\n", queryno, dbseqno);
+ fprintf(fp_log, "Initial and recomputed alignment score disagreement: %" PRId64 " %" PRId64 "\n", dist, score);
fprintf(fp_log, "Alignment: %s\n", cigar);
fprintf(fp_log, "\n");
}
diff --git a/src/align.h b/src/align.h
index 23d7ffa..1bd578f 100644
--- a/src/align.h
+++ b/src/align.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -68,25 +68,25 @@ void nw_align(char * dseq,
char * dend,
char * qseq,
char * qend,
- long * score_matrix,
- long gapopen_q_left,
- long gapopen_q_interior,
- long gapopen_q_right,
- long gapopen_t_left,
- long gapopen_t_interior,
- long gapopen_t_right,
- long gapextend_q_left,
- long gapextend_q_interior,
- long gapextend_q_right,
- long gapextend_t_left,
- long gapextend_t_interior,
- long gapextend_t_right,
- long * nwscore,
- long * nwdiff,
- long * nwgaps,
- long * nwindels,
- long * nwalignmentlength,
+ int64_t * score_matrix,
+ int64_t gapopen_q_left,
+ int64_t gapopen_q_interior,
+ int64_t gapopen_q_right,
+ int64_t gapopen_t_left,
+ int64_t gapopen_t_interior,
+ int64_t gapopen_t_right,
+ int64_t gapextend_q_left,
+ int64_t gapextend_q_interior,
+ int64_t gapextend_q_right,
+ int64_t gapextend_t_left,
+ int64_t gapextend_t_interior,
+ int64_t gapextend_t_right,
+ int64_t * nwscore,
+ int64_t * nwdiff,
+ int64_t * nwgaps,
+ int64_t * nwindels,
+ int64_t * nwalignmentlength,
char ** nwalignment,
- long queryno,
- long dbseqno,
+ int64_t queryno,
+ int64_t dbseqno,
struct nwinfo_s * nw);
diff --git a/src/align_simd.cc b/src/align_simd.cc
index 6d71004..06f142a 100644
--- a/src/align_simd.cc
+++ b/src/align_simd.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -91,21 +91,27 @@
#define MAXSEQLENPRODUCT 25000000
//#define MAXSEQLENPRODUCT 160000
-static long scorematrix[16][16];
+static int64_t scorematrix[16][16];
+
+#ifdef __PPC__
+typedef vector signed short VECTOR_SHORT;
+#else
+typedef __m128i VECTOR_SHORT;
+#endif
struct s16info_s
{
- __m128i matrix[32];
- __m128i * hearray;
- __m128i * dprofile;
- __m128i ** qtable;
+ VECTOR_SHORT matrix[32];
+ VECTOR_SHORT * hearray;
+ VECTOR_SHORT * dprofile;
+ VECTOR_SHORT ** qtable;
unsigned short * dir;
char * qseq;
- unsigned long diralloc;
+ uint64_t diralloc;
char * cigar;
char * cigarend;
- long cigaralloc;
+ int64_t cigaralloc;
int opcount;
char op;
@@ -125,14 +131,14 @@ struct s16info_s
CELL penalty_gap_extension_target_right;
};
-void _mm_print(__m128i x)
+void _mm_print(VECTOR_SHORT x)
{
unsigned short * y = (unsigned short*)&x;
for (int i=0; i<8; i++)
printf("%s%6d", (i>0?" ":""), y[7-i]);
}
-void _mm_print2(__m128i x)
+void _mm_print2(VECTOR_SHORT x)
{
signed short * y = (signed short*)&x;
for (int i=0; i<8; i++)
@@ -168,15 +174,31 @@ void dumpscorematrix(CELL * m)
}
}
+#ifdef __PPC__
+ const vector unsigned char perm_merge_long_low =
+ { 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
+ 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17 };
+
+ const vector unsigned char perm_merge_long_high =
+ { 0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
+ 0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f };
+#endif
void dprofile_fill16(CELL * dprofile_word,
CELL * score_matrix_word,
BYTE * dseq)
{
- __m128i xmm0, xmm1, xmm2, xmm3, xmm4, xmm5, xmm6, xmm7;
- __m128i xmm8, xmm9, xmm10, xmm11, xmm12, xmm13, xmm14, xmm15;
- __m128i xmm16, xmm17, xmm18, xmm19, xmm20, xmm21, xmm22, xmm23;
- __m128i xmm24, xmm25, xmm26, xmm27, xmm28, xmm29, xmm30, xmm31;
+#ifdef __PPC__
+ vector signed short reg0, reg1, reg2, reg3, reg4, reg5, reg6, reg7;
+ vector signed int reg8, reg9, reg10,reg11,reg12,reg13,reg14,reg15;
+ vector signed long long reg16,reg17,reg18,reg19,reg20,reg21,reg22,reg23;
+ vector signed long long reg24,reg25,reg26,reg27,reg28,reg29,reg30,reg31;
+#else
+ VECTOR_SHORT reg0, reg1, reg2, reg3, reg4, reg5, reg6, reg7;
+ VECTOR_SHORT reg8, reg9, reg10, reg11, reg12, reg13, reg14, reg15;
+ VECTOR_SHORT reg16, reg17, reg18, reg19, reg20, reg21, reg22, reg23;
+ VECTOR_SHORT reg24, reg25, reg26, reg27, reg28, reg29, reg30, reg31;
+#endif
/* does not require ssse3 */
/* approx 4*(5*8+2*40)=480 instructions */
@@ -200,58 +222,123 @@ void dprofile_fill16(CELL * dprofile_word,
for(int i=0; i<16; i += 8)
{
- xmm0 = _mm_load_si128((__m128i*)(score_matrix_word + d[0] + i));
- xmm1 = _mm_load_si128((__m128i*)(score_matrix_word + d[1] + i));
- xmm2 = _mm_load_si128((__m128i*)(score_matrix_word + d[2] + i));
- xmm3 = _mm_load_si128((__m128i*)(score_matrix_word + d[3] + i));
- xmm4 = _mm_load_si128((__m128i*)(score_matrix_word + d[4] + i));
- xmm5 = _mm_load_si128((__m128i*)(score_matrix_word + d[5] + i));
- xmm6 = _mm_load_si128((__m128i*)(score_matrix_word + d[6] + i));
- xmm7 = _mm_load_si128((__m128i*)(score_matrix_word + d[7] + i));
+#ifdef __PPC__
+ reg0 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[0] + i));
+ reg1 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[1] + i));
+ reg2 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[2] + i));
+ reg3 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[3] + i));
+ reg4 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[4] + i));
+ reg5 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[5] + i));
+ reg6 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[6] + i));
+ reg7 = vec_ld(0, (VECTOR_SHORT*)(score_matrix_word + d[7] + i));
+
+ reg8 = (vector signed int) vec_mergeh(reg0, reg1);
+ reg9 = (vector signed int) vec_mergel(reg0, reg1);
+ reg10 = (vector signed int) vec_mergeh(reg2, reg3);
+ reg11 = (vector signed int) vec_mergel(reg2, reg3);
+ reg12 = (vector signed int) vec_mergeh(reg4, reg5);
+ reg13 = (vector signed int) vec_mergel(reg4, reg5);
+ reg14 = (vector signed int) vec_mergeh(reg6, reg7);
+ reg15 = (vector signed int) vec_mergel(reg6, reg7);
+
+ reg16 = (vector signed long long) vec_mergeh(reg8, reg10);
+ reg17 = (vector signed long long) vec_mergel(reg8, reg10);
+ reg18 = (vector signed long long) vec_mergeh(reg12, reg14);
+ reg19 = (vector signed long long) vec_mergel(reg12, reg14);
+ reg20 = (vector signed long long) vec_mergeh(reg9, reg11);
+ reg21 = (vector signed long long) vec_mergel(reg9, reg11);
+ reg22 = (vector signed long long) vec_mergeh(reg13, reg15);
+ reg23 = (vector signed long long) vec_mergel(reg13, reg15);
+
+ reg24 = (vector signed long long) vec_perm
+ (reg16, reg18, perm_merge_long_low);
+ reg25 = (vector signed long long) vec_perm
+ (reg16, reg18, perm_merge_long_high);
+ reg26 = (vector signed long long) vec_perm
+ (reg17, reg19, perm_merge_long_low);
+ reg27 = (vector signed long long) vec_perm
+ (reg17, reg19, perm_merge_long_high);
+ reg28 = (vector signed long long) vec_perm
+ (reg20, reg22, perm_merge_long_low);
+ reg29 = (vector signed long long) vec_perm
+ (reg20, reg22, perm_merge_long_high);
+ reg30 = (vector signed long long) vec_perm
+ (reg21, reg23, perm_merge_long_low);
+ reg31 = (vector signed long long) vec_perm
+ (reg21, reg23, perm_merge_long_high);
+
+ vec_st((vector unsigned char)reg24, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+0) + CHANNELS*j));
+ vec_st((vector unsigned char)reg25, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+1) + CHANNELS*j));
+ vec_st((vector unsigned char)reg26, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+2) + CHANNELS*j));
+ vec_st((vector unsigned char)reg27, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+3) + CHANNELS*j));
+ vec_st((vector unsigned char)reg28, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+4) + CHANNELS*j));
+ vec_st((vector unsigned char)reg29, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+5) + CHANNELS*j));
+ vec_st((vector unsigned char)reg30, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+6) + CHANNELS*j));
+ vec_st((vector unsigned char)reg31, 0, (vector unsigned char *)
+ (dprofile_word + CDEPTH*CHANNELS*(i+7) + CHANNELS*j));
+
+#else
+
+ reg0 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[0] + i));
+ reg1 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[1] + i));
+ reg2 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[2] + i));
+ reg3 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[3] + i));
+ reg4 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[4] + i));
+ reg5 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[5] + i));
+ reg6 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[6] + i));
+ reg7 = _mm_load_si128((VECTOR_SHORT*)(score_matrix_word + d[7] + i));
- xmm8 = _mm_unpacklo_epi16(xmm0, xmm1);
- xmm9 = _mm_unpackhi_epi16(xmm0, xmm1);
- xmm10 = _mm_unpacklo_epi16(xmm2, xmm3);
- xmm11 = _mm_unpackhi_epi16(xmm2, xmm3);
- xmm12 = _mm_unpacklo_epi16(xmm4, xmm5);
- xmm13 = _mm_unpackhi_epi16(xmm4, xmm5);
- xmm14 = _mm_unpacklo_epi16(xmm6, xmm7);
- xmm15 = _mm_unpackhi_epi16(xmm6, xmm7);
+ reg8 = _mm_unpacklo_epi16(reg0, reg1);
+ reg9 = _mm_unpackhi_epi16(reg0, reg1);
+ reg10 = _mm_unpacklo_epi16(reg2, reg3);
+ reg11 = _mm_unpackhi_epi16(reg2, reg3);
+ reg12 = _mm_unpacklo_epi16(reg4, reg5);
+ reg13 = _mm_unpackhi_epi16(reg4, reg5);
+ reg14 = _mm_unpacklo_epi16(reg6, reg7);
+ reg15 = _mm_unpackhi_epi16(reg6, reg7);
- xmm16 = _mm_unpacklo_epi32(xmm8, xmm10);
- xmm17 = _mm_unpackhi_epi32(xmm8, xmm10);
- xmm18 = _mm_unpacklo_epi32(xmm12, xmm14);
- xmm19 = _mm_unpackhi_epi32(xmm12, xmm14);
- xmm20 = _mm_unpacklo_epi32(xmm9, xmm11);
- xmm21 = _mm_unpackhi_epi32(xmm9, xmm11);
- xmm22 = _mm_unpacklo_epi32(xmm13, xmm15);
- xmm23 = _mm_unpackhi_epi32(xmm13, xmm15);
+ reg16 = _mm_unpacklo_epi32(reg8, reg10);
+ reg17 = _mm_unpackhi_epi32(reg8, reg10);
+ reg18 = _mm_unpacklo_epi32(reg12, reg14);
+ reg19 = _mm_unpackhi_epi32(reg12, reg14);
+ reg20 = _mm_unpacklo_epi32(reg9, reg11);
+ reg21 = _mm_unpackhi_epi32(reg9, reg11);
+ reg22 = _mm_unpacklo_epi32(reg13, reg15);
+ reg23 = _mm_unpackhi_epi32(reg13, reg15);
- xmm24 = _mm_unpacklo_epi64(xmm16, xmm18);
- xmm25 = _mm_unpackhi_epi64(xmm16, xmm18);
- xmm26 = _mm_unpacklo_epi64(xmm17, xmm19);
- xmm27 = _mm_unpackhi_epi64(xmm17, xmm19);
- xmm28 = _mm_unpacklo_epi64(xmm20, xmm22);
- xmm29 = _mm_unpackhi_epi64(xmm20, xmm22);
- xmm30 = _mm_unpacklo_epi64(xmm21, xmm23);
- xmm31 = _mm_unpackhi_epi64(xmm21, xmm23);
+ reg24 = _mm_unpacklo_epi64(reg16, reg18);
+ reg25 = _mm_unpackhi_epi64(reg16, reg18);
+ reg26 = _mm_unpacklo_epi64(reg17, reg19);
+ reg27 = _mm_unpackhi_epi64(reg17, reg19);
+ reg28 = _mm_unpacklo_epi64(reg20, reg22);
+ reg29 = _mm_unpackhi_epi64(reg20, reg22);
+ reg30 = _mm_unpacklo_epi64(reg21, reg23);
+ reg31 = _mm_unpackhi_epi64(reg21, reg23);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+0) + CHANNELS*j), xmm24);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+1) + CHANNELS*j), xmm25);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+2) + CHANNELS*j), xmm26);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+3) + CHANNELS*j), xmm27);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+4) + CHANNELS*j), xmm28);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+5) + CHANNELS*j), xmm29);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+6) + CHANNELS*j), xmm30);
- _mm_store_si128((__m128i*)(dprofile_word +
- CDEPTH*CHANNELS*(i+7) + CHANNELS*j), xmm31);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+0) + CHANNELS*j), reg24);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+1) + CHANNELS*j), reg25);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+2) + CHANNELS*j), reg26);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+3) + CHANNELS*j), reg27);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+4) + CHANNELS*j), reg28);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+5) + CHANNELS*j), reg29);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+6) + CHANNELS*j), reg30);
+ _mm_store_si128((VECTOR_SHORT*)(dprofile_word +
+ CDEPTH*CHANNELS*(i+7) + CHANNELS*j), reg31);
+#endif
}
}
#if 0
@@ -268,6 +355,57 @@ void dprofile_fill16(CELL * dprofile_word,
no bits set: go diagonally
*/
+/*
+ On PPC the fifth parameter is a vector for the result in the lower 64 bits.
+ On x86_64 the fifth parameter is the address to write the result to.
+*/
+
+#ifdef __PPC__
+
+/* Handle differences between GNU and IBM compilers */
+
+#ifdef __IBMCPP__
+#define VECTORBYTEPERMUTE vec_bperm
+#else
+#define VECTORBYTEPERMUTE vec_vbpermq
+#endif
+
+const vector unsigned char perm = { 120, 112, 104, 96, 88, 80, 72, 64,
+ 56, 48, 40, 32, 24, 16, 8, 0 };
+
+#define ALIGNCORE(H, N, F, V, RES, QR_q, R_q, QR_t, R_t, H_MIN, H_MAX) \
+ { \
+ vector unsigned short W, X, Y, Z; \
+ vector unsigned int WX, YZ; \
+ vector short VV; \
+ VV = vec_ld(0, &(V)); \
+ H = vec_adds(H, VV); \
+ W = (vector unsigned short) VECTORBYTEPERMUTE \
+ ((vector unsigned char) vec_cmpgt(F, H), perm); \
+ H = vec_max(H, F); \
+ X = (vector unsigned short) VECTORBYTEPERMUTE \
+ ((vector unsigned char) vec_cmpgt(E, H), perm); \
+ H = vec_max(H, E); \
+ H_MIN = vec_min(H_MIN, H); \
+ H_MAX = vec_max(H_MAX, H); \
+ N = H; \
+ HF = vec_subs(H, QR_t); \
+ F = vec_subs(F, R_t); \
+ Y = (vector unsigned short) VECTORBYTEPERMUTE \
+ ((vector unsigned char) vec_cmpgt(F, HF), perm); \
+ F = vec_max(F, HF); \
+ HE = vec_subs(H, QR_q); \
+ E = vec_subs(E, R_q); \
+ Z = (vector unsigned short) VECTORBYTEPERMUTE \
+ ((vector unsigned char) vec_cmpgt(E, HE), perm); \
+ E = vec_max(E, HE); \
+ WX = (vector unsigned int) vec_mergel(W, X); \
+ YZ = (vector unsigned int) vec_mergel(Y, Z); \
+ RES = (vector unsigned long long) vec_mergeh(WX, YZ); \
+ }
+
+#else
+
#define ALIGNCORE(H, N, F, V, PATH, QR_q, R_q, QR_t, R_t, H_MIN, H_MAX) \
H = _mm_adds_epi16(H, V); \
*(PATH+0) = _mm_movemask_epi8(_mm_cmpgt_epi16(F, H)); \
@@ -286,45 +424,157 @@ void dprofile_fill16(CELL * dprofile_word,
*(PATH+3) = _mm_movemask_epi8(_mm_cmpgt_epi16(E, HE)); \
E = _mm_max_epi16(E, HE);
+#endif
-void aligncolumns_first(__m128i * Sm,
- __m128i * hep,
- __m128i ** qp,
- __m128i QR_q_i,
- __m128i R_q_i,
- __m128i QR_q_r,
- __m128i R_q_r,
- __m128i QR_t_0,
- __m128i R_t_0,
- __m128i QR_t_1,
- __m128i R_t_1,
- __m128i QR_t_2,
- __m128i R_t_2,
- __m128i QR_t_3,
- __m128i R_t_3,
- __m128i h0,
- __m128i h1,
- __m128i h2,
- __m128i h3,
- __m128i f0,
- __m128i f1,
- __m128i f2,
- __m128i f3,
- __m128i * _h_min,
- __m128i * _h_max,
- __m128i Mm,
- __m128i M_QR_t_left,
- __m128i M_R_t_left,
- __m128i M_QR_q_interior,
- __m128i M_QR_q_right,
- long ql,
+void aligncolumns_first(VECTOR_SHORT * Sm,
+ VECTOR_SHORT * hep,
+ VECTOR_SHORT ** qp,
+ VECTOR_SHORT QR_q_i,
+ VECTOR_SHORT R_q_i,
+ VECTOR_SHORT QR_q_r,
+ VECTOR_SHORT R_q_r,
+ VECTOR_SHORT QR_t_0,
+ VECTOR_SHORT R_t_0,
+ VECTOR_SHORT QR_t_1,
+ VECTOR_SHORT R_t_1,
+ VECTOR_SHORT QR_t_2,
+ VECTOR_SHORT R_t_2,
+ VECTOR_SHORT QR_t_3,
+ VECTOR_SHORT R_t_3,
+ VECTOR_SHORT h0,
+ VECTOR_SHORT h1,
+ VECTOR_SHORT h2,
+ VECTOR_SHORT h3,
+ VECTOR_SHORT f0,
+ VECTOR_SHORT f1,
+ VECTOR_SHORT f2,
+ VECTOR_SHORT f3,
+ VECTOR_SHORT * _h_min,
+ VECTOR_SHORT * _h_max,
+ VECTOR_SHORT Mm,
+ VECTOR_SHORT M_QR_t_left,
+ VECTOR_SHORT M_R_t_left,
+ VECTOR_SHORT M_QR_q_interior,
+ VECTOR_SHORT M_QR_q_right,
+ int64_t ql,
unsigned short * dir)
{
- __m128i h4, h5, h6, h7, h8, E, HE, HF;
- __m128i * vp;
- __m128i h_min = _mm_setzero_si128();
- __m128i h_max = _mm_setzero_si128();
- long i;
+#ifdef __PPC__
+
+ VECTOR_SHORT h4, h5, h6, h7, h8, E, HE, HF;
+ VECTOR_SHORT * vp;
+
+ VECTOR_SHORT h_min = vec_splat_s16(0);
+ VECTOR_SHORT h_max = vec_splat_s16(0);
+
+ vector unsigned long long RES1, RES2, RES;
+
+ int64_t i;
+
+ f0 = vec_subs(f0, QR_t_0);
+ f1 = vec_subs(f1, QR_t_1);
+ f2 = vec_subs(f2, QR_t_2);
+ f3 = vec_subs(f3, QR_t_3);
+
+ for(i=0; i < ql - 1; i++)
+ {
+ vp = qp[i+0];
+
+ h4 = hep[2*i+0];
+
+ E = hep[2*i+1];
+
+ /*
+ Initialize selected h and e values for next/this round.
+ First zero those cells where a new sequence starts
+ by using an unsigned saturated subtraction of a huge value to
+ set it to zero.
+ Then use signed subtraction to obtain the correct value.
+ */
+
+ h4 = (vector short) vec_subs((vector unsigned short) h4,
+ (vector unsigned short) Mm);
+ h4 = vec_subs(h4, M_QR_t_left);
+
+ E = (vector short) vec_subs((vector unsigned short) E,
+ (vector unsigned short) Mm);
+ E = vec_subs(E, M_QR_t_left);
+ E = vec_subs(E, M_QR_q_interior);
+
+ M_QR_t_left = vec_adds(M_QR_t_left, M_R_t_left);
+
+ ALIGNCORE(h0, h5, f0, vp[0], RES1,
+ QR_q_i, R_q_i, QR_t_0, R_t_0, h_min, h_max);
+ ALIGNCORE(h1, h6, f1, vp[1], RES2,
+ QR_q_i, R_q_i, QR_t_1, R_t_1, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+0));
+
+ ALIGNCORE(h2, h7, f2, vp[2], RES1,
+ QR_q_i, R_q_i, QR_t_2, R_t_2, h_min, h_max);
+ ALIGNCORE(h3, h8, f3, vp[3], RES2,
+ QR_q_i, R_q_i, QR_t_3, R_t_3, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+8));
+
+ hep[2*i+0] = h8;
+ hep[2*i+1] = E;
+
+ h0 = h4;
+ h1 = h5;
+ h2 = h6;
+ h3 = h7;
+ }
+
+ /* the final round - using query gap penalties for right end */
+
+ vp = qp[i+0];
+
+ E = hep[2*i+1];
+
+ E = (vector short) vec_subs((vector unsigned short) E,
+ (vector unsigned short) Mm);
+ E = vec_subs(E, M_QR_t_left);
+ E = vec_subs(E, M_QR_q_right);
+
+ ALIGNCORE(h0, h5, f0, vp[0], RES1,
+ QR_q_r, R_q_r, QR_t_0, R_t_0, h_min, h_max);
+ ALIGNCORE(h1, h6, f1, vp[1], RES2,
+ QR_q_r, R_q_r, QR_t_1, R_t_1, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+0));
+
+ ALIGNCORE(h2, h7, f2, vp[2], RES1,
+ QR_q_r, R_q_r, QR_t_2, R_t_2, h_min, h_max);
+ ALIGNCORE(h3, h8, f3, vp[3], RES2,
+ QR_q_r, R_q_r, QR_t_3, R_t_3, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+8));
+
+ hep[2*i+0] = h8;
+ hep[2*i+1] = E;
+
+ Sm[0] = h5;
+ Sm[1] = h6;
+ Sm[2] = h7;
+ Sm[3] = h8;
+
+ *_h_min = h_min;
+ *_h_max = h_max;
+
+#else
+
+ VECTOR_SHORT h4, h5, h6, h7, h8, E, HE, HF;
+ VECTOR_SHORT * vp;
+
+ VECTOR_SHORT h_min = _mm_setzero_si128();
+ VECTOR_SHORT h_max = _mm_setzero_si128();
+
+ int64_t i;
f0 = _mm_subs_epi16(f0, QR_t_0);
f1 = _mm_subs_epi16(f1, QR_t_1);
@@ -356,11 +606,11 @@ void aligncolumns_first(__m128i * Sm,
M_QR_t_left = _mm_adds_epi16(M_QR_t_left, M_R_t_left);
- ALIGNCORE(h0, h5, f0, vp[0], dir+16*i+ 0,
+ ALIGNCORE(h0, h5, f0, vp[0], dir+16*i+0,
QR_q_i, R_q_i, QR_t_0, R_t_0, h_min, h_max);
- ALIGNCORE(h1, h6, f1, vp[1], dir+16*i+ 4,
+ ALIGNCORE(h1, h6, f1, vp[1], dir+16*i+4,
QR_q_i, R_q_i, QR_t_1, R_t_1, h_min, h_max);
- ALIGNCORE(h2, h7, f2, vp[2], dir+16*i+ 8,
+ ALIGNCORE(h2, h7, f2, vp[2], dir+16*i+8,
QR_q_i, R_q_i, QR_t_2, R_t_2, h_min, h_max);
ALIGNCORE(h3, h8, f3, vp[3], dir+16*i+12,
QR_q_i, R_q_i, QR_t_3, R_t_3, h_min, h_max);
@@ -380,10 +630,11 @@ void aligncolumns_first(__m128i * Sm,
E = hep[2*i+1];
+
E = _mm_subs_epu16(E, Mm);
E = _mm_subs_epi16(E, M_QR_t_left);
E = _mm_subs_epi16(E, M_QR_q_right);
-
+
ALIGNCORE(h0, h5, f0, vp[0], dir+16*i+ 0,
QR_q_r, R_q_r, QR_t_0, R_t_0, h_min, h_max);
ALIGNCORE(h1, h6, f1, vp[1], dir+16*i+ 4,
@@ -403,47 +654,137 @@ void aligncolumns_first(__m128i * Sm,
*_h_min = h_min;
*_h_max = h_max;
+
+#endif
}
-void aligncolumns_rest(__m128i * Sm,
- __m128i * hep,
- __m128i ** qp,
- __m128i QR_q_i,
- __m128i R_q_i,
- __m128i QR_q_r,
- __m128i R_q_r,
- __m128i QR_t_0,
- __m128i R_t_0,
- __m128i QR_t_1,
- __m128i R_t_1,
- __m128i QR_t_2,
- __m128i R_t_2,
- __m128i QR_t_3,
- __m128i R_t_3,
- __m128i h0,
- __m128i h1,
- __m128i h2,
- __m128i h3,
- __m128i f0,
- __m128i f1,
- __m128i f2,
- __m128i f3,
- __m128i * _h_min,
- __m128i * _h_max,
- long ql,
+void aligncolumns_rest(VECTOR_SHORT * Sm,
+ VECTOR_SHORT * hep,
+ VECTOR_SHORT ** qp,
+ VECTOR_SHORT QR_q_i,
+ VECTOR_SHORT R_q_i,
+ VECTOR_SHORT QR_q_r,
+ VECTOR_SHORT R_q_r,
+ VECTOR_SHORT QR_t_0,
+ VECTOR_SHORT R_t_0,
+ VECTOR_SHORT QR_t_1,
+ VECTOR_SHORT R_t_1,
+ VECTOR_SHORT QR_t_2,
+ VECTOR_SHORT R_t_2,
+ VECTOR_SHORT QR_t_3,
+ VECTOR_SHORT R_t_3,
+ VECTOR_SHORT h0,
+ VECTOR_SHORT h1,
+ VECTOR_SHORT h2,
+ VECTOR_SHORT h3,
+ VECTOR_SHORT f0,
+ VECTOR_SHORT f1,
+ VECTOR_SHORT f2,
+ VECTOR_SHORT f3,
+ VECTOR_SHORT * _h_min,
+ VECTOR_SHORT * _h_max,
+ int64_t ql,
unsigned short * dir)
{
- __m128i h4, h5, h6, h7, h8, E, HE, HF;
- __m128i * vp;
- __m128i h_min = _mm_setzero_si128();
- __m128i h_max = _mm_setzero_si128();
- long i;
+#ifdef __PPC__
+
+ VECTOR_SHORT h4, h5, h6, h7, h8, E, HE, HF;
+ VECTOR_SHORT * vp;
+
+ VECTOR_SHORT h_min = vec_splat_s16(0);
+ VECTOR_SHORT h_max = vec_splat_s16(0);
+
+ vector unsigned long long RES1, RES2, RES;
+
+ int64_t i;
+
+ f0 = vec_subs(f0, QR_t_0);
+ f1 = vec_subs(f1, QR_t_1);
+ f2 = vec_subs(f2, QR_t_2);
+ f3 = vec_subs(f3, QR_t_3);
+
+ for(i=0; i < ql - 1; i++)
+ {
+ vp = qp[i+0];
+
+ h4 = hep[2*i+0];
+
+ E = hep[2*i+1];
+
+ ALIGNCORE(h0, h5, f0, vp[0], RES1,
+ QR_q_i, R_q_i, QR_t_0, R_t_0, h_min, h_max);
+ ALIGNCORE(h1, h6, f1, vp[1], RES2,
+ QR_q_i, R_q_i, QR_t_1, R_t_1, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+0));
+
+ ALIGNCORE(h2, h7, f2, vp[2], RES1,
+ QR_q_i, R_q_i, QR_t_2, R_t_2, h_min, h_max);
+ ALIGNCORE(h3, h8, f3, vp[3], RES2,
+ QR_q_i, R_q_i, QR_t_3, R_t_3, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+8));
+
+ hep[2*i+0] = h8;
+ hep[2*i+1] = E;
+
+ h0 = h4;
+ h1 = h5;
+ h2 = h6;
+ h3 = h7;
+ }
+
+ /* the final round - using query gap penalties for right end */
+
+ vp = qp[i+0];
+
+ E = hep[2*i+1];
+
+ ALIGNCORE(h0, h5, f0, vp[0], RES1,
+ QR_q_r, R_q_r, QR_t_0, R_t_0, h_min, h_max);
+ ALIGNCORE(h1, h6, f1, vp[1], RES2,
+ QR_q_r, R_q_r, QR_t_1, R_t_1, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+0));
+
+ ALIGNCORE(h2, h7, f2, vp[2], RES1,
+ QR_q_r, R_q_r, QR_t_2, R_t_2, h_min, h_max);
+ ALIGNCORE(h3, h8, f3, vp[3], RES2,
+ QR_q_r, R_q_r, QR_t_3, R_t_3, h_min, h_max);
+ RES = vec_perm(RES1, RES2, perm_merge_long_low);
+ vec_st((vector unsigned char) RES, 0,
+ (vector unsigned char *)(dir+16*i+8));
+
+ hep[2*i+0] = h8;
+ hep[2*i+1] = E;
+
+ Sm[0] = h5;
+ Sm[1] = h6;
+ Sm[2] = h7;
+ Sm[3] = h8;
+
+ *_h_min = h_min;
+ *_h_max = h_max;
+
+#else
+
+ VECTOR_SHORT h4, h5, h6, h7, h8, E, HE, HF;
+ VECTOR_SHORT * vp;
+
+ VECTOR_SHORT h_min = _mm_setzero_si128();
+ VECTOR_SHORT h_max = _mm_setzero_si128();
+
+ int64_t i;
f0 = _mm_subs_epi16(f0, QR_t_0);
f1 = _mm_subs_epi16(f1, QR_t_1);
f2 = _mm_subs_epi16(f2, QR_t_2);
f3 = _mm_subs_epi16(f3, QR_t_3);
+
for(i=0; i < ql - 1; i++)
{
vp = qp[i+0];
@@ -495,6 +836,8 @@ void aligncolumns_rest(__m128i * Sm,
*_h_min = h_min;
*_h_max = h_max;
+
+#endif
}
inline void pushop(s16info_s * s, char newop)
@@ -535,33 +878,33 @@ inline void finishop(s16info_s * s)
void backtrack16(s16info_s * s,
char * dseq,
- unsigned long dlen,
- unsigned long offset,
- unsigned long channel,
+ uint64_t dlen,
+ uint64_t offset,
+ uint64_t channel,
unsigned short * paligned,
unsigned short * pmatches,
unsigned short * pmismatches,
unsigned short * pgaps)
{
unsigned short * dirbuffer = s->dir;
- unsigned long dirbuffersize = s->qlen * s->maxdlen * 4;
- unsigned long qlen = s->qlen;
+ uint64_t dirbuffersize = s->qlen * s->maxdlen * 4;
+ uint64_t qlen = s->qlen;
char * qseq = s->qseq;
- unsigned long maskup = 3UL << (2*channel+ 0);
- unsigned long maskleft = 3UL << (2*channel+16);
- unsigned long maskextup = 3UL << (2*channel+32);
- unsigned long maskextleft = 3UL << (2*channel+48);
+ uint64_t maskup = 3UL << (2*channel+ 0);
+ uint64_t maskleft = 3UL << (2*channel+16);
+ uint64_t maskextup = 3UL << (2*channel+32);
+ uint64_t maskextleft = 3UL << (2*channel+48);
#if 0
printf("Dumping backtracking array\n");
- for(unsigned long i=0; i<qlen; i++)
+ for(uint64_t i=0; i<qlen; i++)
{
- for(unsigned long j=0; j<dlen; j++)
+ for(uint64_t j=0; j<dlen; j++)
{
- unsigned long d = *((unsigned long *) (dirbuffer +
+ uint64_t d = *((uint64_t *) (dirbuffer +
(offset + 16*s->qlen*(j/4) +
16*i + 4*(j&3)) % dirbuffersize));
if (d & maskup)
@@ -585,11 +928,11 @@ void backtrack16(s16info_s * s,
printf("Dumping gap extension array\n");
- for(unsigned long i=0; i<qlen; i++)
+ for(uint64_t i=0; i<qlen; i++)
{
- for(unsigned long j=0; j<dlen; j++)
+ for(uint64_t j=0; j<dlen; j++)
{
- unsigned long d = *((unsigned long *) (dirbuffer +
+ uint64_t d = *((uint64_t *) (dirbuffer +
(offset + 16*s->qlen*(j/4) +
16*i + 4*(j&3)) % dirbuffersize));
if (d & maskextup)
@@ -618,8 +961,8 @@ void backtrack16(s16info_s * s,
unsigned short mismatches = 0;
unsigned short gaps = 0;
- long i = qlen - 1;
- long j = dlen - 1;
+ int64_t i = qlen - 1;
+ int64_t j = dlen - 1;
s->cigarend = s->cigar + s->qlen + s->maxdlen + 1;
s->op = 0;
@@ -629,7 +972,7 @@ void backtrack16(s16info_s * s,
{
aligned++;
- unsigned long d = *((unsigned long *) (dirbuffer +
+ uint64_t d = *((uint64_t *) (dirbuffer +
(offset + 16*s->qlen*(j/4) +
16*i + 4*(j&3)) % dirbuffersize));
@@ -718,7 +1061,7 @@ struct s16info_s * search16_init(CELL score_match,
struct s16info_s * s = (struct s16info_s *)
xmalloc(sizeof(struct s16info_s));
- s->dprofile = (__m128i *) xmalloc(2*4*8*16);
+ s->dprofile = (VECTOR_SHORT *) xmalloc(2*4*8*16);
s->qlen = 0;
s->qseq = 0;
s->maxdlen = 0;
@@ -779,16 +1122,16 @@ void search16_exit(s16info_s * s)
{
/* free mem for dprofile, hearray, dir, qtable */
if (s->dir)
- free(s->dir);
+ xfree(s->dir);
if (s->hearray)
- free(s->hearray);
+ xfree(s->hearray);
if (s->dprofile)
- free(s->dprofile);
+ xfree(s->dprofile);
if (s->qtable)
- free(s->qtable);
+ xfree(s->qtable);
if (s->cigar)
- free(s->cigar);
- free(s);
+ xfree(s->cigar);
+ xfree(s);
}
void search16_qprep(s16info_s * s, char * qseq, int qlen)
@@ -797,13 +1140,13 @@ void search16_qprep(s16info_s * s, char * qseq, int qlen)
s->qseq = qseq;
if (s->hearray)
- free(s->hearray);
- s->hearray = (__m128i *) xmalloc(2 * s->qlen * sizeof(__m128i));
- memset(s->hearray, 0, 2 * s->qlen * sizeof(__m128i));
+ xfree(s->hearray);
+ s->hearray = (VECTOR_SHORT *) xmalloc(2 * s->qlen * sizeof(VECTOR_SHORT));
+ memset(s->hearray, 0, 2 * s->qlen * sizeof(VECTOR_SHORT));
if (s->qtable)
- free(s->qtable);
- s->qtable = (__m128i **) xmalloc(s->qlen * sizeof(__m128i*));
+ xfree(s->qtable);
+ s->qtable = (VECTOR_SHORT **) xmalloc(s->qlen * sizeof(VECTOR_SHORT*));
for(int i = 0; i < qlen; i++)
s->qtable[i] = s->dprofile + 4 * chrmap_4bit[(int)(qseq[i])];
@@ -822,14 +1165,14 @@ void search16(s16info_s * s,
CELL ** q_start = (CELL**) s->qtable;
CELL * dprofile = (CELL*) s->dprofile;
CELL * hearray = (CELL*) s->hearray;
- unsigned long qlen = s->qlen;
+ uint64_t qlen = s->qlen;
if (qlen == 0)
{
for (unsigned int cand_id = 0; cand_id < sequences; cand_id++)
{
unsigned int seqno = seqnos[cand_id];
- long length = db_getsequencelen(seqno);
+ int64_t length = db_getsequencelen(seqno);
paligned[cand_id] = length;
pmatches[cand_id] = 0;
@@ -848,7 +1191,7 @@ void search16(s16info_s * s,
char * cigar = 0;
if (length > 0)
{
- int ret = asprintf(&cigar, "%ldI", length);
+ int ret = xsprintf(&cigar, "%ldI", length);
if ((ret < 2) || !cigar)
fatal("Unable to allocate enough memory.");
}
@@ -863,12 +1206,12 @@ void search16(s16info_s * s,
}
/* find longest target sequence and reallocate direction buffer */
- unsigned long maxdlen = 0;
- for(long i = 0; i < sequences; i++)
+ uint64_t maxdlen = 0;
+ for(int64_t i = 0; i < sequences; i++)
{
- unsigned long dlen = db_getsequencelen(seqnos[i]);
+ uint64_t dlen = db_getsequencelen(seqnos[i]);
/* skip the very long sequences */
- if ((long)(s->qlen) * dlen <= MAXSEQLENPRODUCT)
+ if ((int64_t)(s->qlen) * dlen <= MAXSEQLENPRODUCT)
{
if (dlen > maxdlen)
maxdlen = dlen;
@@ -876,13 +1219,13 @@ void search16(s16info_s * s,
}
maxdlen = 4 * ((maxdlen + 3) / 4);
s->maxdlen = maxdlen;
- unsigned long dirbuffersize = s->qlen * s->maxdlen * 4;
+ uint64_t dirbuffersize = s->qlen * s->maxdlen * 4;
if (dirbuffersize > s->diralloc)
{
s->diralloc = dirbuffersize;
if (s->dir)
- free(s->dir);
+ xfree(s->dir);
s->dir = (unsigned short*) xmalloc(dirbuffersize *
sizeof(unsigned short));
}
@@ -893,47 +1236,72 @@ void search16(s16info_s * s,
{
s->cigaralloc = s->qlen + s->maxdlen + 1;
if (s->cigar)
- free(s->cigar);
+ xfree(s->cigar);
s->cigar = (char *) xmalloc(s->cigaralloc);
}
- __m128i T, M, T0;
-
- __m128i M_QR_target_left, M_R_target_left;
- __m128i M_QR_query_interior;
- __m128i M_QR_query_right;
-
- __m128i R_query_left;
- __m128i QR_query_interior, R_query_interior;
- __m128i QR_query_right, R_query_right;
- __m128i QR_target_left, R_target_left;
- __m128i QR_target_interior, R_target_interior;
- __m128i QR_target_right, R_target_right;
- __m128i QR_target[4], R_target[4];
+ VECTOR_SHORT T, M, T0;
+
+ VECTOR_SHORT M_QR_target_left, M_R_target_left;
+ VECTOR_SHORT M_QR_query_interior;
+ VECTOR_SHORT M_QR_query_right;
+
+ VECTOR_SHORT R_query_left;
+ VECTOR_SHORT QR_query_interior, R_query_interior;
+ VECTOR_SHORT QR_query_right, R_query_right;
+ VECTOR_SHORT QR_target_left, R_target_left;
+ VECTOR_SHORT QR_target_interior, R_target_interior;
+ VECTOR_SHORT QR_target_right, R_target_right;
+ VECTOR_SHORT QR_target[4], R_target[4];
- __m128i *hep, **qp;
+ VECTOR_SHORT *hep, **qp;
BYTE * d_begin[CHANNELS];
BYTE * d_end[CHANNELS];
- unsigned long d_offset[CHANNELS];
+ uint64_t d_offset[CHANNELS];
BYTE * d_address[CHANNELS];
- unsigned long d_length[CHANNELS];
- long seq_id[CHANNELS];
+ uint64_t d_length[CHANNELS];
+ int64_t seq_id[CHANNELS];
bool overflow[CHANNELS];
- __m128i dseqalloc[CDEPTH];
- __m128i S[4];
+ VECTOR_SHORT dseqalloc[CDEPTH];
+ VECTOR_SHORT S[4];
BYTE * dseq = (BYTE*) & dseqalloc;
BYTE zero = 0;
- unsigned long next_id = 0;
- unsigned long done = 0;
+ uint64_t next_id = 0;
+ uint64_t done = 0;
+
+#ifdef __PPC__
+
+ const vector short T0_init = { -1, 0, 0, 0, 0, 0, 0, 0 };
+ T0 = T0_init;
+
+ R_query_left = vec_splat((VECTOR_SHORT){s->penalty_gap_extension_query_left, 0, 0, 0, 0, 0, 0, 0}, 0);
+
+ QR_query_interior = vec_splat((VECTOR_SHORT){(short)(s->penalty_gap_open_query_interior + s->penalty_gap_extension_query_interior), 0, 0, 0, 0, 0, 0, 0}, 0);
+ R_query_interior = vec_splat((VECTOR_SHORT){s->penalty_gap_extension_query_interior, 0, 0, 0, 0, 0, 0, 0}, 0);
+
+ QR_query_right = vec_splat((VECTOR_SHORT){(short)(s->penalty_gap_open_query_right + s->penalty_gap_extension_query_right), 0, 0, 0, 0, 0, 0, 0}, 0);
+ R_query_right = vec_splat((VECTOR_SHORT){s->penalty_gap_extension_query_right, 0, 0, 0, 0, 0, 0, 0}, 0);
+
+ QR_target_left = vec_splat((VECTOR_SHORT){(short)(s->penalty_gap_open_target_left + s->penalty_gap_extension_target_left), 0, 0, 0, 0, 0, 0, 0}, 0);
+ R_target_left = vec_splat((VECTOR_SHORT){s->penalty_gap_extension_target_left, 0, 0, 0, 0, 0, 0, 0}, 0);
+
+ QR_target_interior = vec_splat((VECTOR_SHORT){(short)(s->penalty_gap_open_target_interior + s->penalty_gap_extension_target_interior), 0, 0, 0, 0, 0, 0, 0}, 0);
+ R_target_interior = vec_splat((VECTOR_SHORT){s->penalty_gap_extension_target_interior, 0, 0, 0, 0, 0, 0, 0}, 0);
+
+ QR_target_right = vec_splat((VECTOR_SHORT){(short)(s->penalty_gap_open_target_right + s->penalty_gap_extension_target_right), 0, 0, 0, 0, 0, 0, 0}, 0);
+ R_target_right = vec_splat((VECTOR_SHORT){s->penalty_gap_extension_target_right, 0, 0, 0, 0, 0, 0, 0}, 0);
+
+#else
+
T0 = _mm_set_epi16(0, 0, 0, 0, 0, 0, 0, 0xffff);
R_query_left = _mm_set1_epi16(s->penalty_gap_extension_query_left);
-
+
QR_query_interior = _mm_set1_epi16(s->penalty_gap_open_query_interior +
s->penalty_gap_extension_query_interior);
R_query_interior = _mm_set1_epi16(s->penalty_gap_extension_query_interior);
@@ -953,9 +1321,13 @@ void search16(s16info_s * s,
QR_target_right = _mm_set1_epi16(s->penalty_gap_open_target_right +
s->penalty_gap_extension_target_right);
R_target_right = _mm_set1_epi16(s->penalty_gap_extension_target_right);
+
+#endif
+
+
- hep = (__m128i*) hearray;
- qp = (__m128i**) q_start;
+ hep = (VECTOR_SHORT*) hearray;
+ qp = (VECTOR_SHORT**) q_start;
for (int c=0; c<CHANNELS; c++)
{
@@ -992,21 +1364,41 @@ void search16(s16info_s * s,
short score_min = SHRT_MIN + gap_penalty_max;
short score_max = SHRT_MAX;
+#ifdef __PPC__
+ const VECTOR_SHORT VZERO = vec_splat_s16(0);
+
+ for(int i=0; i<4; i++)
+ {
+ S[i] = vec_splat_s16(0);
+ dseqalloc[i] = vec_splat_s16(0);
+ }
+
+ VECTOR_SHORT H0 = vec_splat_s16(0);
+ VECTOR_SHORT H1 = vec_splat_s16(0);
+ VECTOR_SHORT H2 = vec_splat_s16(0);
+ VECTOR_SHORT H3 = vec_splat_s16(0);
+
+ VECTOR_SHORT F0 = vec_splat_s16(0);
+ VECTOR_SHORT F1 = vec_splat_s16(0);
+ VECTOR_SHORT F2 = vec_splat_s16(0);
+ VECTOR_SHORT F3 = vec_splat_s16(0);
+#else
for(int i=0; i<4; i++)
{
S[i] = _mm_setzero_si128();
dseqalloc[i] = _mm_setzero_si128();
}
- __m128i H0 = _mm_setzero_si128();
- __m128i H1 = _mm_setzero_si128();
- __m128i H2 = _mm_setzero_si128();
- __m128i H3 = _mm_setzero_si128();
-
- __m128i F0 = _mm_setzero_si128();
- __m128i F1 = _mm_setzero_si128();
- __m128i F2 = _mm_setzero_si128();
- __m128i F3 = _mm_setzero_si128();
+ VECTOR_SHORT H0 = _mm_setzero_si128();
+ VECTOR_SHORT H1 = _mm_setzero_si128();
+ VECTOR_SHORT H2 = _mm_setzero_si128();
+ VECTOR_SHORT H3 = _mm_setzero_si128();
+
+ VECTOR_SHORT F0 = _mm_setzero_si128();
+ VECTOR_SHORT F1 = _mm_setzero_si128();
+ VECTOR_SHORT F2 = _mm_setzero_si128();
+ VECTOR_SHORT F3 = _mm_setzero_si128();
+#endif
int easy = 0;
@@ -1048,14 +1440,39 @@ void search16(s16info_s * s,
else
{
/* one or more sequences ended */
- __m128i QR_diff = _mm_subs_epi16(QR_target_right,
+
+#ifdef __PPC__
+ VECTOR_SHORT QR_diff = vec_subs(QR_target_right,
QR_target_interior);
- __m128i R_diff = _mm_subs_epi16(R_target_right,
+ VECTOR_SHORT R_diff = vec_subs(R_target_right,
R_target_interior);
for(unsigned int j=0; j<CDEPTH; j++)
{
- __m128i M = _mm_setzero_si128();
- __m128i T = T0;
+ VECTOR_SHORT M = vec_splat_s16(0);
+ VECTOR_SHORT T = T0;
+ for(int c=0; c<CHANNELS; c++)
+ {
+ if ((d_begin[c] == d_end[c]) &&
+ (j >= ((d_length[c]+3) % 4)))
+ {
+ M = vec_xor(M, T);
+ }
+ T = vec_sld(T, VZERO, 2);
+ }
+ QR_target[j] = vec_adds(QR_target_interior,
+ vec_and(QR_diff, M));
+ R_target[j] = vec_adds(R_target_interior,
+ vec_and(R_diff, M));
+ }
+#else
+ VECTOR_SHORT QR_diff = _mm_subs_epi16(QR_target_right,
+ QR_target_interior);
+ VECTOR_SHORT R_diff = _mm_subs_epi16(R_target_right,
+ R_target_interior);
+ for(unsigned int j=0; j<CDEPTH; j++)
+ {
+ VECTOR_SHORT M = _mm_setzero_si128();
+ VECTOR_SHORT T = T0;
for(int c=0; c<CHANNELS; c++)
{
if ((d_begin[c] == d_end[c]) &&
@@ -1066,13 +1483,14 @@ void search16(s16info_s * s,
T = _mm_slli_si128(T, 2);
}
QR_target[j] = _mm_adds_epi16(QR_target_interior,
- _mm_and_si128(QR_diff, M));
+ _mm_and_si128(QR_diff, M));
R_target[j] = _mm_adds_epi16(R_target_interior,
- _mm_and_si128(R_diff, M));
+ _mm_and_si128(R_diff, M));
}
+#endif
}
- __m128i h_min, h_max;
+ VECTOR_SHORT h_min, h_max;
aligncolumns_rest(S, hep, qp,
QR_query_interior, R_query_interior,
@@ -1092,8 +1510,13 @@ void search16(s16info_s * s,
{
signed short h_min_array[8];
signed short h_max_array[8];
- _mm_storeu_si128((__m128i*)h_min_array, h_min);
- _mm_storeu_si128((__m128i*)h_max_array, h_max);
+#ifdef __PPC__
+ *(VECTOR_SHORT*)h_min_array = h_min;
+ *(VECTOR_SHORT*)h_max_array = h_max;
+#else
+ _mm_storeu_si128((VECTOR_SHORT*)h_min_array, h_min);
+ _mm_storeu_si128((VECTOR_SHORT*)h_max_array, h_max);
+#endif
signed short h_min_c = h_min_array[c];
signed short h_max_c = h_max_array[c];
if ((h_min_c <= score_min) || (h_max_c >= score_max))
@@ -1108,7 +1531,11 @@ void search16(s16info_s * s,
easy = 1;
+#ifdef __PPC__
+ M = vec_splat_s16(0);
+#else
M = _mm_setzero_si128();
+#endif
T = T0;
for (int c=0; c<CHANNELS; c++)
{
@@ -1130,18 +1557,22 @@ void search16(s16info_s * s,
{
/* sequence in channel c ended. change of sequence */
- M = _mm_xor_si128(M, T);
+#ifdef __PPC__
+ M = vec_xor(M, T);
+#else
+ M = _mm_xor_si128(M, T);
+#endif
- long cand_id = seq_id[c];
+ int64_t cand_id = seq_id[c];
if (cand_id >= 0)
{
/* save score */
char * dbseq = (char*) d_address[c];
- long dbseqlen = d_length[c];
- long z = (dbseqlen+3) % 4;
- long score = ((CELL*)S)[z*CHANNELS+c];
+ int64_t dbseqlen = d_length[c];
+ int64_t z = (dbseqlen+3) % 4;
+ int64_t score = ((CELL*)S)[z*CHANNELS+c];
if (overflow[c])
{
@@ -1169,7 +1600,7 @@ void search16(s16info_s * s,
/* get next sequence of reasonable length */
- long length = 0;
+ int64_t length = 0;
while ((length == 0) && (next_id < sequences))
{
@@ -1242,8 +1673,11 @@ void search16(s16info_s * s,
dseq[CHANNELS*j+c] = 0;
}
}
-
+#ifdef __PPC__
+ T = vec_sld(T, VZERO, 2);
+#else
T = _mm_slli_si128(T, 2);
+#endif
}
if (done == sequences)
@@ -1251,13 +1685,23 @@ void search16(s16info_s * s,
/* make masked versions of QR and R for gaps in target */
+#ifdef __PPC__
+ M_QR_target_left = vec_and(M, QR_target_left);
+ M_R_target_left = vec_and(M, R_target_left);
+#else
M_QR_target_left = _mm_and_si128(M, QR_target_left);
M_R_target_left = _mm_and_si128(M, R_target_left);
+#endif
/* make masked versions of QR for gaps in query at target left end */
+#ifdef __PPC__
+ M_QR_query_interior = vec_and(M, QR_query_interior);
+ M_QR_query_right = vec_and(M, QR_query_right);
+#else
M_QR_query_interior = _mm_and_si128(M, QR_query_interior);
M_QR_query_right = _mm_and_si128(M, QR_query_right);
+#endif
dprofile_fill16(dprofile, (CELL*) s->matrix, dseq);
@@ -1275,31 +1719,57 @@ void search16(s16info_s * s,
else
{
/* one or more sequences ended */
- __m128i QR_diff = _mm_subs_epi16(QR_target_right,
- QR_target_interior);
- __m128i R_diff = _mm_subs_epi16(R_target_right,
- R_target_interior);
+#ifdef __PPC__
+ VECTOR_SHORT QR_diff = vec_subs(QR_target_right,
+ QR_target_interior);
+ VECTOR_SHORT R_diff = vec_subs(R_target_right,
+ R_target_interior);
for(unsigned int j=0; j<CDEPTH; j++)
{
- __m128i M = _mm_setzero_si128();
- __m128i T = T0;
+ VECTOR_SHORT M = vec_splat_s16(0);
+ VECTOR_SHORT T = T0;
for(int c=0; c<CHANNELS; c++)
{
if ((d_begin[c] == d_end[c]) &&
(j >= ((d_length[c]+3) % 4)))
{
- M = _mm_xor_si128(M, T);
+ M = vec_xor(M, T);
}
- T = _mm_slli_si128(T, 2);
+ T = vec_sld(T, VZERO, 2);
}
- QR_target[j] = _mm_adds_epi16(QR_target_interior,
- _mm_and_si128(QR_diff, M));
- R_target[j] = _mm_adds_epi16(R_target_interior,
- _mm_and_si128(R_diff, M));
- }
+ QR_target[j] = vec_adds(QR_target_interior,
+ vec_and(QR_diff, M));
+ R_target[j] = vec_adds(R_target_interior,
+ vec_and(R_diff, M));
+ }
+#else
+ VECTOR_SHORT QR_diff = _mm_subs_epi16(QR_target_right,
+ QR_target_interior);
+
+ VECTOR_SHORT R_diff = _mm_subs_epi16(R_target_right,
+ R_target_interior);
+ for(unsigned int j=0; j<CDEPTH; j++)
+ {
+ VECTOR_SHORT M = _mm_setzero_si128();
+ VECTOR_SHORT T = T0;
+ for(int c=0; c<CHANNELS; c++)
+ {
+ if ((d_begin[c] == d_end[c]) &&
+ (j >= ((d_length[c]+3) % 4)))
+ {
+ M = _mm_xor_si128(M, T);
+ }
+ T = _mm_slli_si128(T, 2);
+ }
+ QR_target[j] = _mm_adds_epi16(QR_target_interior,
+ _mm_and_si128(QR_diff, M));
+ R_target[j] = _mm_adds_epi16(R_target_interior,
+ _mm_and_si128(R_diff, M));
+ }
+#endif
}
- __m128i h_min, h_max;
+ VECTOR_SHORT h_min, h_max;
aligncolumns_first(S, hep, qp,
QR_query_interior, R_query_interior,
@@ -1323,8 +1793,13 @@ void search16(s16info_s * s,
{
signed short h_min_array[8];
signed short h_max_array[8];
- _mm_storeu_si128((__m128i*)h_min_array, h_min);
- _mm_storeu_si128((__m128i*)h_max_array, h_max);
+#ifdef __PPC__
+ *(VECTOR_SHORT*)h_min_array = h_min;
+ *(VECTOR_SHORT*)h_max_array = h_max;
+#else
+ _mm_storeu_si128((VECTOR_SHORT*)h_min_array, h_min);
+ _mm_storeu_si128((VECTOR_SHORT*)h_max_array, h_max);
+#endif
signed short h_min_c = h_min_array[c];
signed short h_max_c = h_max_array[c];
if ((h_min_c <= score_min) ||
@@ -1334,6 +1809,17 @@ void search16(s16info_s * s,
}
}
+#ifdef __PPC__
+ H0 = vec_subs(H3, R_query_left);
+ H1 = vec_subs(H0, R_query_left);
+ H2 = vec_subs(H1, R_query_left);
+ H3 = vec_subs(H2, R_query_left);
+
+ F0 = vec_subs(F3, R_query_left);
+ F1 = vec_subs(F0, R_query_left);
+ F2 = vec_subs(F1, R_query_left);
+ F3 = vec_subs(F2, R_query_left);
+#else
H0 = _mm_subs_epi16(H3, R_query_left);
H1 = _mm_subs_epi16(H0, R_query_left);
H2 = _mm_subs_epi16(H1, R_query_left);
@@ -1343,6 +1829,7 @@ void search16(s16info_s * s,
F1 = _mm_subs_epi16(F0, R_query_left);
F2 = _mm_subs_epi16(F1, R_query_left);
F3 = _mm_subs_epi16(F2, R_query_left);
+#endif
dir += 4 * 4 * s->qlen;
diff --git a/src/align_simd.h b/src/align_simd.h
index 5e90a7c..77ed487 100644
--- a/src/align_simd.h
+++ b/src/align_simd.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/allpairs.cc b/src/allpairs.cc
index 808dc4b..3ff16e9 100644
--- a/src/allpairs.cc
+++ b/src/allpairs.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -71,7 +71,7 @@ static pthread_mutex_t mutex_input;
static pthread_mutex_t mutex_output;
static int qmatches;
static int queries;
-static long progress = 0;
+static int64_t progress = 0;
static FILE * fp_alnout = 0;
static FILE * fp_samout = 0;
static FILE * fp_userout = 0;
@@ -114,7 +114,7 @@ void allpairs_output_results(int hit_count,
char * qsequence_rc)
{
/* show results */
- long toreport = MIN(opt_maxhits, hit_count);
+ int64_t toreport = MIN(opt_maxhits, hit_count);
if (fp_alnout)
results_show_alnout(fp_alnout,
@@ -180,7 +180,7 @@ void allpairs_output_results(int hit_count,
qsequence_rc);
}
}
- else if (opt_output_no_hits)
+ else
{
if (fp_uc)
results_show_uc_one(fp_uc,
@@ -191,21 +191,24 @@ void allpairs_output_results(int hit_count,
qsequence_rc,
0);
- if (fp_userout)
- results_show_userout_one(fp_userout,
- 0,
- query_head,
- qsequence,
- qseqlen,
- qsequence_rc);
-
- if (fp_blast6out)
- results_show_blast6out_one(fp_blast6out,
- 0,
- query_head,
- qsequence,
- qseqlen,
- qsequence_rc);
+ if (opt_output_no_hits)
+ {
+ if (fp_userout)
+ results_show_userout_one(fp_userout,
+ 0,
+ query_head,
+ qsequence,
+ qseqlen,
+ qsequence_rc);
+
+ if (fp_blast6out)
+ results_show_blast6out_one(fp_blast6out,
+ 0,
+ query_head,
+ qsequence,
+ qseqlen,
+ qsequence_rc);
+ }
}
if (hit_count)
@@ -230,7 +233,7 @@ void allpairs_output_results(int hit_count,
}
}
-void allpairs_thread_run(long t)
+void allpairs_thread_run(int64_t t)
{
struct searchinfo_s sia;
@@ -266,7 +269,7 @@ void allpairs_thread_run(long t)
LinearMemoryAligner lma;
- long * scorematrix = lma.scorematrix_create(opt_match, opt_mismatch);
+ int64_t * scorematrix = lma.scorematrix_create(opt_match, opt_mismatch);
lma.set_parameters(scorematrix,
opt_gap_open_query_left,
@@ -356,13 +359,13 @@ void allpairs_thread_run(long t)
struct hit * hit = si->hits + h;
unsigned int target = pseqnos[h];
- long nwscore = pscores[h];
+ int64_t nwscore = pscores[h];
char * nwcigar;
- long nwalignmentlength;
- long nwmatches;
- long nwmismatches;
- long nwgaps;
+ int64_t nwalignmentlength;
+ int64_t nwmatches;
+ int64_t nwmismatches;
+ int64_t nwgaps;
if (nwscore == SHRT_MAX)
{
@@ -371,10 +374,10 @@ void allpairs_thread_run(long t)
linear memory aligner */
char * tseq = db_getsequence(target);
- long tseqlen = db_getsequencelen(target);
+ int64_t tseqlen = db_getsequencelen(target);
if (pcigar[h])
- free(pcigar[h]);
+ xfree(pcigar[h]);
nwcigar = xstrdup(lma.align(si->qsequence,
tseq,
@@ -418,7 +421,7 @@ void allpairs_thread_run(long t)
hit->matches = nwalignmentlength - hit->nwdiff;
hit->mismatches = hit->nwdiff - hit->nwindels;
- long dseqlen = db_getsequencelen(target);
+ int64_t dseqlen = db_getsequencelen(target);
hit->shortest = MIN(si->qseqlen, dseqlen);
hit->longest = MAX(si->qseqlen, dseqlen);
@@ -459,7 +462,7 @@ void allpairs_thread_run(long t)
/* free memory for alignment strings */
for(int i=0; i < si->hit_count; i++)
if (si->hits[i].aligned)
- free(si->hits[i].nwalignment);
+ xfree(si->hits[i].nwalignment);
}
else
{
@@ -470,28 +473,28 @@ void allpairs_thread_run(long t)
}
}
- free(finalhits);
+ xfree(finalhits);
- free(pcigar);
- free(pgaps);
- free(pmismatches);
- free(pmatches);
- free(paligned);
- free(pscores);
- free(pseqnos);
+ xfree(pcigar);
+ xfree(pgaps);
+ xfree(pmismatches);
+ xfree(pmatches);
+ xfree(paligned);
+ xfree(pscores);
+ xfree(pseqnos);
search16_exit(si->s);
nw_exit(nw);
- free(scorematrix);
+ xfree(scorematrix);
- free(si->hits);
+ xfree(si->hits);
}
void * allpairs_thread_worker(void * vp)
{
- long t = (long) vp;
+ int64_t t = (int64_t) vp;
allpairs_thread_run(t);
return 0;
}
@@ -507,7 +510,7 @@ void allpairs_thread_worker_run()
for(int t=0; t<opt_threads; t++)
{
if (pthread_create(pthread+t, &attr,
- allpairs_thread_worker, (void*)(long)t))
+ allpairs_thread_worker, (void*)(int64_t)t))
fatal("Cannot create thread");
}
@@ -612,7 +615,7 @@ void allpairs_global(char * cmdline, char * progheader)
pthread_mutex_init(&mutex_output, NULL);
progress = 0;
- progress_init("Aligning", MAX(0,((long)seqcount)*((long)seqcount-1))/2);
+ progress_init("Aligning", MAX(0,((int64_t)seqcount)*((int64_t)seqcount-1))/2);
allpairs_thread_worker_run();
progress_done();
@@ -629,7 +632,7 @@ void allpairs_global(char * cmdline, char * progheader)
pthread_mutex_destroy(&mutex_output);
pthread_mutex_destroy(&mutex_input);
- free(pthread);
+ xfree(pthread);
/* clean up, global */
db_free();
diff --git a/src/allpairs.h b/src/allpairs.h
index 0e08d96..b9b0aaa 100644
--- a/src/allpairs.h
+++ b/src/allpairs.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/arch.cc b/src/arch.cc
index 50671a2..2a0ff11 100644
--- a/src/arch.cc
+++ b/src/arch.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,23 +60,44 @@
#include "vsearch.h"
-unsigned long arch_get_memused()
+const int memalignment = 16;
+
+uint64_t arch_get_memused()
{
+#ifdef _WIN32
+
+ PROCESS_MEMORY_COUNTERS pmc;
+ GetProcessMemoryInfo(GetCurrentProcess(),
+ &pmc,
+ sizeof(PROCESS_MEMORY_COUNTERS));
+ return pmc.PeakWorkingSetSize;
+
+#else
+
struct rusage r_usage;
getrusage(RUSAGE_SELF, & r_usage);
-
-#ifdef __APPLE__
+
+# ifdef __APPLE__
/* Mac: ru_maxrss gives the size in bytes */
return r_usage.ru_maxrss;
-#else
+# else
/* Linux: ru_maxrss gives the size in kilobytes */
return r_usage.ru_maxrss * 1024;
+# endif
+
#endif
}
-unsigned long arch_get_memtotal()
+uint64_t arch_get_memtotal()
{
-#if defined(__APPLE__)
+#ifdef _WIN32
+
+ MEMORYSTATUSEX ms;
+ ms.dwLength = sizeof(MEMORYSTATUSEX);
+ GlobalMemoryStatusEx(&ms);
+ return ms.ullTotalPhys;
+
+#elif defined(__APPLE__)
int mib [] = { CTL_HW, HW_MEMSIZE };
int64_t ram = 0;
@@ -87,8 +108,8 @@ unsigned long arch_get_memtotal()
#elif defined(_SC_PHYS_PAGES) && defined(_SC_PAGESIZE)
- long phys_pages = sysconf(_SC_PHYS_PAGES);
- long pagesize = sysconf(_SC_PAGESIZE);
+ int64_t phys_pages = sysconf(_SC_PHYS_PAGES);
+ int64_t pagesize = sysconf(_SC_PAGESIZE);
if ((phys_pages == -1) || (pagesize == -1))
fatal("Cannot determine amount of RAM");
return pagesize * phys_pages;
@@ -102,3 +123,122 @@ unsigned long arch_get_memtotal()
#endif
}
+
+long arch_get_cores()
+{
+#ifdef _WIN32
+ SYSTEM_INFO si;
+ GetSystemInfo(&si);
+ return si.dwNumberOfProcessors;
+#else
+ return sysconf(_SC_NPROCESSORS_ONLN);
+#endif
+}
+
+void arch_get_user_system_time(double * user_time, double * system_time)
+{
+ *user_time = 0;
+ *system_time = 0;
+#ifdef _WIN32
+ HANDLE hProcess = GetCurrentProcess();
+ FILETIME ftCreation, ftExit, ftKernel, ftUser;
+ ULARGE_INTEGER ul;
+ GetProcessTimes(hProcess, &ftCreation, &ftExit, &ftKernel, &ftUser);
+ ul.u.HighPart = ftUser.dwHighDateTime;
+ ul.u.LowPart = ftUser.dwLowDateTime;
+ *user_time = ul.QuadPart * 100.0e-9;
+ ul.u.HighPart = ftKernel.dwHighDateTime;
+ ul.u.LowPart = ftKernel.dwLowDateTime;
+ *system_time = ul.QuadPart * 100.0e-9;
+#else
+ struct rusage r_usage;
+ getrusage(RUSAGE_SELF, & r_usage);
+ * user_time = r_usage.ru_utime.tv_sec * 1.0
+ + r_usage.ru_utime.tv_usec * 1.0e-6;
+ * system_time = r_usage.ru_stime.tv_sec * 1.0
+ + r_usage.ru_stime.tv_usec * 1.0e-6;
+#endif
+}
+
+void arch_srandom()
+{
+ /* initialize pseudo-random number generator */
+ unsigned int seed = opt_randseed;
+ if (seed == 0)
+ {
+#ifdef _WIN32
+ srand(GetTickCount());
+#else
+ int fd = open("/dev/urandom", O_RDONLY);
+ if (fd < 0)
+ fatal("Unable to open /dev/urandom");
+ if (read(fd, & seed, sizeof(seed)) < 0)
+ fatal("Unable to read from /dev/urandom");
+ close(fd);
+ srandom(seed);
+#endif
+ }
+ else
+ {
+#ifdef _WIN32
+ srand(seed);
+#else
+ srandom(seed);
+#endif
+ }
+}
+
+uint64_t arch_random()
+{
+#ifdef _WIN32
+ return rand();
+#else
+ return random();
+#endif
+}
+
+void * xmalloc(size_t size)
+{
+ if (size == 0)
+ size = 1;
+ void * t = 0;
+#ifdef _WIN32
+ t = _aligned_malloc(size, memalignment);
+#else
+ if (posix_memalign(& t, memalignment, size))
+ t = 0;
+#endif
+ if (!t)
+ fatal("Unable to allocate enough memory.");
+ return t;
+}
+
+void * xrealloc(void *ptr, size_t size)
+{
+ if (size == 0)
+ size = 1;
+#ifdef _WIN32
+ void * t = _aligned_realloc(ptr, size, memalignment);
+#else
+ void * t = realloc(ptr, size);
+#endif
+ if (!t)
+ fatal("Unable to reallocate enough memory.");
+ return t;
+}
+
+void xfree(void * ptr)
+{
+ if (ptr)
+ {
+#ifdef _WIN32
+ _aligned_free(ptr);
+#else
+ free(ptr);
+#endif
+ }
+ else
+ fatal("Trying to free a null pointer");
+}
+
+
diff --git a/src/arch.h b/src/arch.h
index 235349e..f5f659f 100644
--- a/src/arch.h
+++ b/src/arch.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -58,5 +58,12 @@
*/
-unsigned long arch_get_memused();
-unsigned long arch_get_memtotal();
+uint64_t arch_get_memused();
+uint64_t arch_get_memtotal();
+long arch_get_cores();
+void arch_get_user_system_time(double * user_time, double * system_time);
+void arch_srandom();
+uint64_t arch_random();
+void * xmalloc(size_t size);
+void * xrealloc(void * ptr, size_t size);
+void xfree(void * ptr);
diff --git a/src/bitmap.cc b/src/bitmap.cc
index b8e837c..901b90f 100644
--- a/src/bitmap.cc
+++ b/src/bitmap.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -71,6 +71,6 @@ bitmap_t * bitmap_init(unsigned int size)
void bitmap_free(bitmap_t* b)
{
if (b->bitmap)
- free(b->bitmap);
- free(b);
+ xfree(b->bitmap);
+ xfree(b);
}
diff --git a/src/bitmap.h b/src/bitmap.h
index d3275b7..b8e0d2f 100644
--- a/src/bitmap.h
+++ b/src/bitmap.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/chimera.cc b/src/chimera.cc
index 4e76fc3..93b3189 100644
--- a/src/chimera.cc
+++ b/src/chimera.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -85,15 +85,15 @@ static fastx_handle query_fasta_h;
static pthread_mutex_t mutex_input;
static pthread_mutex_t mutex_output;
static unsigned int seqno = 0;
-static unsigned long progress = 0;
+static uint64_t progress = 0;
static int chimera_count = 0;
static int nonchimera_count = 0;
static int borderline_count = 0;
static int total_count = 0;
-static long chimera_abundance = 0;
-static long nonchimera_abundance = 0;
-static long borderline_abundance = 0;
-static long total_abundance = 0;
+static int64_t chimera_abundance = 0;
+static int64_t nonchimera_abundance = 0;
+static int64_t borderline_abundance = 0;
+static int64_t total_abundance = 0;
static FILE * fp_chimeras = 0;
static FILE * fp_nonchimeras = 0;
static FILE * fp_uchimealns = 0;
@@ -124,11 +124,11 @@ struct chimera_info_s
unsigned short snwmatches[maxcandidates];
unsigned short snwmismatches[maxcandidates];
unsigned short snwgaps[maxcandidates];
- long nwscore[maxcandidates];
- long nwalignmentlength[maxcandidates];
- long nwmatches[maxcandidates];
- long nwmismatches[maxcandidates];
- long nwgaps[maxcandidates];
+ int64_t nwscore[maxcandidates];
+ int64_t nwalignmentlength[maxcandidates];
+ int64_t nwmatches[maxcandidates];
+ int64_t nwmismatches[maxcandidates];
+ int64_t nwgaps[maxcandidates];
char * nwcigar[maxcandidates];
int match_size;
@@ -755,7 +755,7 @@ int eval_parents(struct chimera_info_s * ci)
char qsym = chrmap_4bit[(int)(ci->qaln[i])];
char asym = chrmap_4bit[(int)(ci->paln[index_a][i])];
char bsym = chrmap_4bit[(int)(ci->paln[index_b][i])];
- char msym = i <= best_i ? asym : bsym;
+ char msym = (i <= best_i) ? asym : bsym;
if (qsym == asym)
match_QA++;
@@ -805,10 +805,10 @@ int eval_parents(struct chimera_info_s * ci)
"--------------------------------\n");
fprintf(fp_uchimealns, "Query (%5d nt) %s\n",
ci->query_len, ci->query_head);
- fprintf(fp_uchimealns, "ParentA (%5lu nt) %s\n",
+ fprintf(fp_uchimealns, "ParentA (%5" PRIu64 " nt) %s\n",
db_getsequencelen(seqno_a),
db_getheader(seqno_a));
- fprintf(fp_uchimealns, "ParentB (%5lu nt) %s\n",
+ fprintf(fp_uchimealns, "ParentB (%5" PRIu64 " nt) %s\n",
db_getsequencelen(seqno_b),
db_getheader(seqno_b));
fprintf(fp_uchimealns, "\n");
@@ -981,11 +981,11 @@ void query_exit(struct searchinfo_s * si)
nw_exit(si->nw);
if (si->qsequence)
- free(si->qsequence);
+ xfree(si->qsequence);
if (si->hits)
- free(si->hits);
+ xfree(si->hits);
if (si->kmers)
- free(si->kmers);
+ xfree(si->kmers);
}
void partition_query(struct chimera_info_s * ci)
@@ -1057,32 +1057,32 @@ void chimera_thread_exit(struct chimera_info_s * ci)
query_exit(ci->si + i);
if (ci->maxsmooth)
- free(ci->maxsmooth);
+ xfree(ci->maxsmooth);
if (ci->match)
- free(ci->match);
+ xfree(ci->match);
if (ci->smooth)
- free(ci->smooth);
+ xfree(ci->smooth);
if (ci->diffs)
- free(ci->diffs);
+ xfree(ci->diffs);
if (ci->votes)
- free(ci->votes);
+ xfree(ci->votes);
if (ci->model)
- free(ci->model);
+ xfree(ci->model);
if (ci->ignore)
- free(ci->ignore);
+ xfree(ci->ignore);
if (ci->maxi)
- free(ci->maxi);
+ xfree(ci->maxi);
if (ci->qaln)
- free(ci->qaln);
+ xfree(ci->qaln);
if (ci->paln[0])
- free(ci->paln[0]);
+ xfree(ci->paln[0]);
if (ci->paln[1])
- free(ci->paln[1]);
+ xfree(ci->paln[1]);
if (ci->query_seq)
- free(ci->query_seq);
+ xfree(ci->query_seq);
if (ci->query_head)
- free(ci->query_head);
+ xfree(ci->query_head);
}
void fasta_print_with_score(FILE * fp,
@@ -1100,7 +1100,7 @@ void fasta_print_with_score(FILE * fp,
else
snprintf(newheader, alloc, "%s%s=%.4lf;", head, score_name, score);
fasta_print(fp, newheader, seq, len);
- free(newheader);
+ xfree(newheader);
}
void fasta_print_relabel_with_score(FILE * fp,
@@ -1121,10 +1121,10 @@ void fasta_print_relabel_with_score(FILE * fp,
else
snprintf(newheader, alloc, "%s%s=%.4lf;", header, score_name, score);
fasta_print_relabel(fp, seq, len, newheader, strlen(newheader), abundance, ordinal);
- free(newheader);
+ xfree(newheader);
}
-unsigned long chimera_thread_core(struct chimera_info_s * ci)
+uint64_t chimera_thread_core(struct chimera_info_s * ci)
{
chimera_thread_init(ci);
@@ -1133,7 +1133,7 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
LinearMemoryAligner lma;
- long * scorematrix = lma.scorematrix_create(opt_match, opt_mismatch);
+ int64_t * scorematrix = lma.scorematrix_create(opt_match, opt_mismatch);
lma.set_parameters(scorematrix,
opt_gap_open_query_left,
@@ -1223,7 +1223,7 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
for(int j=0; j<hit_count; j++)
if (hits[j].accepted)
allhits_list[allhits_count++] = hits[j];
- free(hits);
+ xfree(hits);
}
for(int i=0; i < allhits_count; i++)
@@ -1241,7 +1241,7 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
/* deallocate cigar */
if (allhits_list[i].nwalignment)
- free(allhits_list[i].nwalignment);
+ xfree(allhits_list[i].nwalignment);
}
@@ -1261,13 +1261,13 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
for(int i=0; i < ci->cand_count; i++)
{
- long target = ci->cand_list[i];
- long nwscore = ci->snwscore[i];
+ int64_t target = ci->cand_list[i];
+ int64_t nwscore = ci->snwscore[i];
char * nwcigar;
- long nwalignmentlength;
- long nwmatches;
- long nwmismatches;
- long nwgaps;
+ int64_t nwalignmentlength;
+ int64_t nwmatches;
+ int64_t nwmismatches;
+ int64_t nwgaps;
if (nwscore == SHRT_MAX)
{
@@ -1276,10 +1276,10 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
linear memory aligner */
char * tseq = db_getsequence(target);
- long tseqlen = db_getsequencelen(target);
+ int64_t tseqlen = db_getsequencelen(target);
if (ci->nwcigar[i])
- free(ci->nwcigar[i]);
+ xfree(ci->nwcigar[i]);
nwcigar = xstrdup(lma.align(ci->query_seq,
tseq,
@@ -1419,7 +1419,7 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
for (int i=0; i < ci->cand_count; i++)
if (ci->nwcigar[i])
- free(ci->nwcigar[i]);
+ xfree(ci->nwcigar[i]);
if (opt_uchime_ref)
progress = fasta_get_position(query_fasta_h);
@@ -1434,18 +1434,18 @@ unsigned long chimera_thread_core(struct chimera_info_s * ci)
}
if (allhits_list)
- free(allhits_list);
+ xfree(allhits_list);
chimera_thread_exit(ci);
- free(scorematrix);
+ xfree(scorematrix);
return 0;
}
void * chimera_thread_worker(void * vp)
{
- return (void *) chimera_thread_core(cia + (long) vp);
+ return (void *) chimera_thread_core(cia + (int64_t) vp);
}
void chimera_threads_run()
@@ -1454,7 +1454,7 @@ void chimera_threads_run()
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
/* create worker threads */
- for(long t=0; t<opt_threads; t++)
+ for(int64_t t=0; t<opt_threads; t++)
{
if (pthread_create(pthread+t, & attr,
chimera_thread_worker, (void*)t))
@@ -1515,7 +1515,7 @@ void chimera()
tophits = opt_maxaccepts + opt_maxrejects;
- unsigned long progress_total;
+ uint64_t progress_total;
chimera_count = 0;
nonchimera_count = 0;
progress = 0;
@@ -1583,8 +1583,8 @@ void chimera()
"Found %d (%.1f%%) chimeras, %d (%.1f%%) non-chimeras,\n"
"and %d (%.1f%%) borderline sequences in %u unique sequences.\n"
"Taking abundance information into account, this corresponds to\n"
- "%ld (%.1f%%) chimeras, %ld (%.1f%%) non-chimeras,\n"
- "and %ld (%.1f%%) borderline sequences in %ld total sequences.\n",
+ "%" PRId64 " (%.1f%%) chimeras, %" PRId64 " (%.1f%%) non-chimeras,\n"
+ "and %" PRId64 " (%.1f%%) borderline sequences in %" PRId64 " total sequences.\n",
chimera_count,
100.0 * chimera_count / total_count,
nonchimera_count,
@@ -1618,12 +1618,12 @@ void chimera()
dbindex_free();
db_free();
-
+
pthread_mutex_destroy(&mutex_output);
pthread_mutex_destroy(&mutex_input);
- free(cia);
- free(pthread);
+ xfree(cia);
+ xfree(pthread);
close_chimera_file(fp_borderline);
close_chimera_file(fp_uchimeout);
diff --git a/src/chimera.h b/src/chimera.h
index 229dd55..ba077d6 100644
--- a/src/chimera.h
+++ b/src/chimera.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/cluster.cc b/src/cluster.cc
index 4ce9694..9ea345a 100644
--- a/src/cluster.cc
+++ b/src/cluster.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -74,7 +74,7 @@ typedef struct clusterinfo_s
static clusterinfo_t * clusterinfo = 0;
static int clusters = 0;
-static long * cluster_abundance;
+static int64_t * cluster_abundance;
static FILE * fp_centroids = 0;
static FILE * fp_uc = 0;
@@ -162,7 +162,7 @@ inline void cluster_query_core(struct searchinfo_s * si)
search_onequery(si, opt_qmask);
}
-inline void cluster_worker(long t)
+inline void cluster_worker(int64_t t)
{
/* wrapper for the main threaded core function for clustering */
for (int q = 0; q < ti[t].query_count; q++)
@@ -175,7 +175,7 @@ inline void cluster_worker(long t)
void * threads_worker(void * vp)
{
- long t = (long) vp;
+ int64_t t = (int64_t) vp;
thread_info_s * tip = ti + t;
pthread_mutex_lock(&tip->mutex);
/* loop until signalled to quit */
@@ -245,7 +245,7 @@ void threads_init()
tip->work = 0;
pthread_mutex_init(&tip->mutex, 0);
pthread_cond_init(&tip->cond, 0);
- if (pthread_create(&tip->thread, &attr, threads_worker, (void*)(long)t))
+ if (pthread_create(&tip->thread, &attr, threads_worker, (void*)(int64_t)t))
fatal("Cannot create thread");
}
}
@@ -270,7 +270,7 @@ void threads_exit()
pthread_cond_destroy(&tip->cond);
pthread_mutex_destroy(&tip->mutex);
}
- free(ti);
+ xfree(ti);
pthread_attr_destroy(&attr);
}
@@ -320,11 +320,11 @@ void cluster_query_exit(struct searchinfo_s * si)
nw_exit(si->nw);
if (si->qsequence)
- free(si->qsequence);
+ xfree(si->qsequence);
if (si->hits)
- free(si->hits);
+ xfree(si->hits);
if (si->kmers)
- free(si->kmers);
+ xfree(si->kmers);
}
char * relabel_otu(int clusterno, char * sequence, int seqlen)
@@ -365,7 +365,7 @@ void cluster_core_results_hit(struct hit * best,
db_getsequence(best->target),
db_getsequencelen(best->target));
otutable_add(query_head, label, qsize);
- free(label);
+ xfree(label);
}
else
otutable_add(query_head,
@@ -423,7 +423,7 @@ void cluster_core_results_nohit(int clusterno,
{
char * label = relabel_otu(clusterno, qsequence, qseqlen);
otutable_add(query_head, label, qsize);
- free(label);
+ xfree(label);
}
else
otutable_add(query_head, query_head, qsize);
@@ -492,7 +492,7 @@ void cluster_core_parallel()
int * extra_list = (int*) xmalloc(max_queries*sizeof(int));
LinearMemoryAligner lma;
- long * scorematrix = lma.scorematrix_create(opt_match, opt_mismatch);
+ int64_t * scorematrix = lma.scorematrix_create(opt_match, opt_mismatch);
lma.set_parameters(scorematrix,
opt_gap_open_query_left,
opt_gap_open_target_left,
@@ -513,7 +513,7 @@ void cluster_core_parallel()
int seqno = 0;
- long sum_nucleotides = 0;
+ int64_t sum_nucleotides = 0;
progress_init("Clustering", db_getnucleotidecount());
@@ -610,7 +610,7 @@ void cluster_core_parallel()
if (si->hit_count >= opt_maxaccepts + opt_maxrejects - 1)
{
if (si->hits[si->hit_count-1].aligned)
- free(si->hits[si->hit_count-1].nwalignment);
+ xfree(si->hits[si->hit_count-1].nwalignment);
si->hit_count--;
}
@@ -673,11 +673,11 @@ void cluster_core_parallel()
unsigned int nwtarget = target;
- long nwscore;
- long nwalignmentlength;
- long nwmatches;
- long nwmismatches;
- long nwgaps;
+ int64_t nwscore;
+ int64_t nwalignmentlength;
+ int64_t nwmatches;
+ int64_t nwmismatches;
+ int64_t nwgaps;
char * nwcigar = 0;
/* short variants for simd aligner */
@@ -697,7 +697,7 @@ void cluster_core_parallel()
& snwgaps,
& nwcigar);
- long tseqlen = db_getsequencelen(target);
+ int64_t tseqlen = db_getsequencelen(target);
if (snwscore == SHRT_MAX)
{
@@ -708,7 +708,7 @@ void cluster_core_parallel()
char * tseq = db_getsequence(target);
if (nwcigar)
- free(nwcigar);
+ xfree(nwcigar);
nwcigar = xstrdup(lma.align(si->qsequence,
tseq,
@@ -734,8 +734,8 @@ void cluster_core_parallel()
}
- long nwdiff = nwalignmentlength - nwmatches;
- long nwindels = nwdiff - nwmismatches;
+ int64_t nwdiff = nwalignmentlength - nwmatches;
+ int64_t nwindels = nwdiff - nwmismatches;
hit->aligned = 1;
hit->nwalignment = nwcigar;
@@ -786,7 +786,7 @@ void cluster_core_parallel()
{
new_hit_count = t;
if (hit->aligned)
- free(hit->nwalignment);
+ xfree(hit->nwalignment);
}
}
si->hit_count = new_hit_count;
@@ -856,7 +856,7 @@ void cluster_core_parallel()
for(int j=0; j<si->hit_count; j++)
if (si->hits[j].aligned)
if (si->hits[j].nwalignment)
- free(si->hits[j].nwalignment);
+ xfree(si->hits[j].nwalignment);
}
sum_nucleotides += si_p->qseqlen;
@@ -879,16 +879,16 @@ void cluster_core_parallel()
cluster_query_exit(si_minus+i);
}
- free(extra_list);
+ xfree(extra_list);
- free(si_plus);
+ xfree(si_plus);
if (opt_strand>1)
- free(si_minus);
+ xfree(si_minus);
/* terminate threads and clean up */
threads_exit();
- free(scorematrix);
+ xfree(scorematrix);
}
void cluster_core_serial()
@@ -970,7 +970,7 @@ void cluster_core_serial()
for(int i=0; i<si->hit_count; i++)
if (si->hits[i].aligned)
if (si->hits[i].nwalignment)
- free(si->hits[i].nwalignment);
+ xfree(si->hits[i].nwalignment);
}
progress_update(seqno);
@@ -1113,16 +1113,16 @@ void cluster(char * dbname,
if (opt_log)
{
- unsigned long slots = 1UL << (opt_wordlength << 1UL);
+ uint64_t slots = 1UL << (opt_wordlength << 1UL);
fprintf(fp_log, "\n");
fprintf(fp_log, " Alphabet nt\n");
- fprintf(fp_log, " Word width %ld\n", opt_wordlength);
- fprintf(fp_log, " Word ones %ld\n", opt_wordlength);
+ fprintf(fp_log, " Word width %" PRId64 "\n", opt_wordlength);
+ fprintf(fp_log, " Word ones %" PRId64 "\n", opt_wordlength);
fprintf(fp_log, " Spaced No\n");
fprintf(fp_log, " Hashed No\n");
fprintf(fp_log, " Coded No\n");
fprintf(fp_log, " Stepped No\n");
- fprintf(fp_log, " Slots %lu (%.1fk)\n", slots, slots/1000.0);
+ fprintf(fp_log, " Slots %" PRIu64 " (%.1fk)\n", slots, slots/1000.0);
fprintf(fp_log, " DBAccel 100%%\n");
fprintf(fp_log, "\n");
}
@@ -1135,10 +1135,10 @@ void cluster(char * dbname,
/* find size and abundance of each cluster and save stats */
- cluster_abundance = (long *) xmalloc(clusters * sizeof(long));
+ cluster_abundance = (int64_t *) xmalloc(clusters * sizeof(int64_t));
int * cluster_size = (int *) xmalloc(clusters * sizeof(int));
- memset(cluster_abundance, 0, clusters * sizeof(long));
+ memset(cluster_abundance, 0, clusters * sizeof(int64_t));
memset(cluster_size, 0, clusters * sizeof(int));
for(int i=0; i<seqcount; i++)
@@ -1149,14 +1149,14 @@ void cluster(char * dbname,
cluster_size[clusterno]++;
}
- long abundance_min = LONG_MAX;
- long abundance_max = 0;
+ int64_t abundance_min = LONG_MAX;
+ int64_t abundance_max = 0;
int size_max = 0;
int singletons = 0;
for(int z=0; z<clusters; z++)
{
- long abundance = cluster_abundance[z];
+ int64_t abundance = cluster_abundance[z];
if (abundance < abundance_min)
abundance_min = abundance;
if (abundance > abundance_max)
@@ -1213,7 +1213,7 @@ void cluster(char * dbname,
if (opt_uc)
{
- fprintf(fp_uc, "C\t%d\t%ld\t*\t*\t*\t*\t*\t%s\t*\n",
+ fprintf(fp_uc, "C\t%d\t%" PRId64 "\t*\t*\t*\t*\t*\t%s\t*\n",
clusterno,
cluster_abundance[clusterno],
db_getheader(seqno));
@@ -1249,7 +1249,7 @@ void cluster(char * dbname,
{
fclose(fp_clusters);
if (fn_clusters)
- free(fn_clusters);
+ xfree(fn_clusters);
}
}
@@ -1258,7 +1258,7 @@ void cluster(char * dbname,
if (!opt_quiet)
{
fprintf(stderr,
- "Clusters: %d Size min %ld, max %ld, avg %.1f\n",
+ "Clusters: %d Size min %" PRId64 ", max %" PRId64 ", avg %.1f\n",
clusters,
abundance_min,
abundance_max,
@@ -1273,7 +1273,7 @@ void cluster(char * dbname,
if (opt_log)
{
fprintf(fp_log,
- "Clusters: %d Size min %ld, max %ld, avg %.1f\n",
+ "Clusters: %d Size min %" PRId64 ", max %" PRId64 ", avg %.1f\n",
clusters,
abundance_min,
abundance_max,
@@ -1372,19 +1372,19 @@ void cluster(char * dbname,
if (fp_consout)
fclose(fp_consout);
- free(msa_target_list);
+ xfree(msa_target_list);
}
- free(cluster_abundance);
- free(cluster_size);
+ xfree(cluster_abundance);
+ xfree(cluster_size);
/* free cigar strings for all aligned sequences */
for(int i=0; i<seqcount; i++)
if (clusterinfo[i].cigar)
- free(clusterinfo[i].cigar);
+ xfree(clusterinfo[i].cigar);
- free(clusterinfo);
+ xfree(clusterinfo);
if (fp_biomout)
{
diff --git a/src/cluster.h b/src/cluster.h
index cd0546a..d00df6c 100644
--- a/src/cluster.h
+++ b/src/cluster.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/cpu.cc b/src/cpu.cc
index 1f6dd9a..76d91a0 100644
--- a/src/cpu.cc
+++ b/src/cpu.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,8 +60,49 @@
#include "vsearch.h"
-/* This file contains code dependent on special cpu features (e.g. ssse3) */
-/* The file will be compiled several times with different cpu options */
+/* This file contains code dependent on special cpu features. */
+/* The file may be compiled several times with different cpu options. */
+
+#ifdef __PPC__
+
+void increment_counters_from_bitmap(unsigned short * counters,
+ unsigned char * bitmap,
+ unsigned int totalbits)
+{
+ const vector unsigned char c1 =
+ { 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0 };
+ const vector unsigned char c2 =
+ { 0xfe, 0xfd, 0xfb, 0xf7, 0xef, 0xdf, 0xbf, 0x7f,
+ 0xfe, 0xfd, 0xfb, 0xf7, 0xef, 0xdf, 0xbf, 0x7f };
+ const vector unsigned char c3 =
+ { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
+ 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff };
+
+ unsigned short * p = (unsigned short *)(bitmap);
+ vector signed short * q = (vector signed short *) (counters);
+ int r = (totalbits + 15) / 16;
+
+ for(int j=0; j<r; j++)
+ {
+ vector unsigned char r0, r1, r2;
+ vector bool char r3;
+ vector signed short r4, r5;
+
+ r0 = * (vector unsigned char *) p;
+ p++;
+ r1 = vec_perm(r0, r0, c1);
+ r2 = vec_or(r1, c2);
+ r3 = vec_cmpeq(r2, c3);
+ r4 = (vector signed short) vec_unpackl(r3);
+ r5 = (vector signed short) vec_unpackh(r3);
+ *q = vec_subs(*q, r4);
+ q++;
+ *q = vec_subs(*q, r5);
+ q++;
+ }
+}
+
+#else
#ifdef SSSE3
void increment_counters_from_bitmap_ssse3(unsigned short * counters,
@@ -128,3 +169,5 @@ void increment_counters_from_bitmap_sse2(unsigned short * counters,
q++;
}
}
+
+#endif
diff --git a/src/cpu.h b/src/cpu.h
index 60fe132..961cb2a 100644
--- a/src/cpu.h
+++ b/src/cpu.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -58,6 +58,11 @@
*/
+#ifdef __PPC__
+void increment_counters_from_bitmap(unsigned short * counters,
+ unsigned char * bitmap,
+ unsigned int totalbits);
+#else
void increment_counters_from_bitmap_sse2(unsigned short * counters,
unsigned char * bitmap,
unsigned int totalbits);
@@ -65,3 +70,4 @@ void increment_counters_from_bitmap_sse2(unsigned short * counters,
void increment_counters_from_bitmap_ssse3(unsigned short * counters,
unsigned char * bitmap,
unsigned int totalbits);
+#endif
diff --git a/src/db.cc b/src/db.cc
index 5c3b8dd..1447676 100644
--- a/src/db.cc
+++ b/src/db.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -64,11 +64,11 @@
static fastx_handle h;
static bool is_fastq = 0;
-static unsigned long sequences = 0;
-static unsigned long nucleotides = 0;
-static unsigned long longest;
-static unsigned long shortest;
-static unsigned long longestheader;
+static uint64_t sequences = 0;
+static uint64_t nucleotides = 0;
+static uint64_t longest;
+static uint64_t shortest;
+static uint64_t longestheader;
seqinfo_t * seqindex;
char * datap;
@@ -78,7 +78,7 @@ bool db_is_fastq()
return is_fastq;
}
-char * db_getquality(unsigned long seqno)
+char * db_getquality(uint64_t seqno)
{
if (is_fastq)
return datap + seqindex[seqno].qual_p;
@@ -97,10 +97,10 @@ void db_read(const char * filename, int upcase)
is_fastq = fastx_is_fastq(h);
- long filesize = fastx_get_size(h);
+ int64_t filesize = fastx_get_size(h);
- char * prompt;
- if (asprintf(& prompt, "Reading file %s", filename) == -1)
+ char * prompt = 0;
+ if (xsprintf(& prompt, "Reading file %s", filename) == -1)
fatal("Out of memory");
progress_init(prompt, filesize);
@@ -111,13 +111,13 @@ void db_read(const char * filename, int upcase)
sequences = 0;
nucleotides = 0;
- long discarded_short = 0;
- long discarded_long = 0;
+ int64_t discarded_short = 0;
+ int64_t discarded_long = 0;
/* allocate space for data */
- unsigned long dataalloc = 0;
+ uint64_t dataalloc = 0;
datap = 0;
- unsigned long datalen = 0;
+ uint64_t datalen = 0;
/* allocate space for index */
size_t seqindex_alloc = 0;
@@ -207,14 +207,14 @@ void db_read(const char * filename, int upcase)
}
progress_done();
- free(prompt);
+ xfree(prompt);
fastx_close(h);
if (!opt_quiet)
{
if (sequences > 0)
fprintf(stderr,
- "%'lu nt in %'lu seqs, min %'lu, max %'lu, avg %'.0f\n",
+ "%'" PRIu64 " nt in %'" PRIu64 " seqs, min %'" PRIu64 ", max %'" PRIu64 ", avg %'.0f\n",
db_getnucleotidecount(),
db_getsequencecount(),
db_getshortestsequence(),
@@ -222,7 +222,7 @@ void db_read(const char * filename, int upcase)
db_getnucleotidecount() * 1.0 / db_getsequencecount());
else
fprintf(stderr,
- "%'lu nt in %'lu seqs\n",
+ "%'" PRIu64 " nt in %'" PRIu64 " seqs\n",
db_getnucleotidecount(),
db_getsequencecount());
}
@@ -231,7 +231,7 @@ void db_read(const char * filename, int upcase)
{
if (sequences > 0)
fprintf(fp_log,
- "%'lu nt in %'lu seqs, min %'lu, max %'lu, avg %'.0f\n\n",
+ "%'" PRIu64 " nt in %'" PRIu64 " seqs, min %'" PRIu64 ", max %'" PRIu64 ", avg %'.0f\n\n",
db_getnucleotidecount(),
db_getsequencecount(),
db_getshortestsequence(),
@@ -239,7 +239,7 @@ void db_read(const char * filename, int upcase)
db_getnucleotidecount() * 1.0 / db_getsequencecount());
else
fprintf(fp_log,
- "%'lu nt in %'lu seqs\n\n",
+ "%'" PRIu64 " nt in %'" PRIu64 " seqs\n\n",
db_getnucleotidecount(),
db_getsequencecount());
}
@@ -249,51 +249,51 @@ void db_read(const char * filename, int upcase)
if (discarded_short)
{
fprintf(stderr,
- "WARNING: %ld sequences shorter than %ld nucleotides discarded.\n",
+ "WARNING: %" PRId64 " sequences shorter than %" PRId64 " nucleotides discarded.\n",
discarded_short, opt_minseqlength);
if (opt_log)
fprintf(fp_log,
- "WARNING: %ld sequences shorter than %ld nucleotides discarded.\n\n",
+ "WARNING: %" PRId64 " sequences shorter than %" PRId64 " nucleotides discarded.\n\n",
discarded_short, opt_minseqlength);
}
if (discarded_long)
{
fprintf(stderr,
- "WARNING: %ld sequences longer than %ld nucleotides discarded.\n",
+ "WARNING: %" PRId64 " sequences longer than %" PRId64 " nucleotides discarded.\n",
discarded_long, opt_maxseqlength);
if (opt_log)
fprintf(fp_log,
- "WARNING: %ld sequences longer than %ld nucleotides discarded.\n\n",
+ "WARNING: %" PRId64 " sequences longer than %" PRId64 " nucleotides discarded.\n\n",
discarded_long, opt_maxseqlength);
}
show_rusage();
}
-unsigned long db_getsequencecount()
+uint64_t db_getsequencecount()
{
return sequences;
}
-unsigned long db_getnucleotidecount()
+uint64_t db_getnucleotidecount()
{
return nucleotides;
}
-unsigned long db_getlongestheader()
+uint64_t db_getlongestheader()
{
return longestheader;
}
-unsigned long db_getlongestsequence()
+uint64_t db_getlongestsequence()
{
return longest;
}
-unsigned long db_getshortestsequence()
+uint64_t db_getshortestsequence()
{
return shortest;
}
@@ -301,9 +301,9 @@ unsigned long db_getshortestsequence()
void db_free()
{
if (datap)
- free(datap);
+ xfree(datap);
if (seqindex)
- free(seqindex);
+ xfree(seqindex);
}
int compare_bylength(const void * a, const void * b)
diff --git a/src/db.h b/src/db.h
index 7c7415b..f95e458 100644
--- a/src/db.h
+++ b/src/db.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -73,27 +73,27 @@ typedef struct seqinfo_s seqinfo_t;
extern char * datap;
extern seqinfo_t * seqindex;
-inline char * db_getheader(unsigned long seqno)
+inline char * db_getheader(uint64_t seqno)
{
return datap + seqindex[seqno].header_p;
}
-inline char * db_getsequence(unsigned long seqno)
+inline char * db_getsequence(uint64_t seqno)
{
return datap + seqindex[seqno].seq_p;
}
-inline unsigned long db_getabundance(unsigned long seqno)
+inline uint64_t db_getabundance(uint64_t seqno)
{
return seqindex[seqno].size;
}
-inline unsigned long db_getsequencelen(unsigned long seqno)
+inline uint64_t db_getsequencelen(uint64_t seqno)
{
return seqindex[seqno].seqlen;
}
-inline unsigned long db_getheaderlen(unsigned long seqno)
+inline uint64_t db_getheaderlen(uint64_t seqno)
{
return seqindex[seqno].headerlen;
}
@@ -101,11 +101,11 @@ inline unsigned long db_getheaderlen(unsigned long seqno)
void db_read(const char * filename, int upcase);
void db_free();
-unsigned long db_getsequencecount();
-unsigned long db_getnucleotidecount();
-unsigned long db_getlongestheader();
-unsigned long db_getlongestsequence();
-unsigned long db_getshortestsequence();
+uint64_t db_getsequencecount();
+uint64_t db_getnucleotidecount();
+uint64_t db_getlongestheader();
+uint64_t db_getlongestsequence();
+uint64_t db_getshortestsequence();
/* Note: the sorting functions below must be called after db_read,
but before dbindex_prepare */
@@ -116,4 +116,4 @@ void db_sortbylength_shortest_first();
void db_sortbyabundance();
bool db_is_fastq();
-char * db_getquality(unsigned long seqno);
+char * db_getquality(uint64_t seqno);
diff --git a/src/dbhash.cc b/src/dbhash.cc
index aacbadc..8e28aa1 100644
--- a/src/dbhash.cc
+++ b/src/dbhash.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,12 +61,12 @@
#include "vsearch.h"
static bitmap_t * dbhash_bitmap;
-static unsigned long dbhash_size;
+static uint64_t dbhash_size;
static unsigned int dbhash_shift;
-static unsigned long dbhash_mask;
+static uint64_t dbhash_mask;
static struct dbhash_bucket_s * dbhash_table;
-int dbhash_seqcmp(char * a, char * b, unsigned long n)
+int dbhash_seqcmp(char * a, char * b, uint64_t n)
{
char * p = a;
char * q = b;
@@ -85,7 +85,7 @@ int dbhash_seqcmp(char * a, char * b, unsigned long n)
return chrmap_4bit[(int)(*p)] - chrmap_4bit[(int)(*q)];
}
-void dbhash_open(unsigned long maxelements)
+void dbhash_open(uint64_t maxelements)
{
/* adjust size of hash table for 2/3 fill rate */
/* and use a multiple of 2 */
@@ -111,20 +111,20 @@ void dbhash_close()
{
bitmap_free(dbhash_bitmap);
dbhash_bitmap = 0;
- free(dbhash_table);
+ xfree(dbhash_table);
dbhash_table = 0;
}
-long dbhash_search_first(char * seq,
- unsigned long seqlen,
+int64_t dbhash_search_first(char * seq,
+ uint64_t seqlen,
struct dbhash_search_info_s * info)
{
- unsigned long hash = hash_cityhash64(seq, seqlen);
+ uint64_t hash = hash_cityhash64(seq, seqlen);
info->hash = hash;
info->seq = seq;
info->seqlen = seqlen;
- unsigned long index = hash & dbhash_mask;
+ uint64_t index = hash & dbhash_mask;
struct dbhash_bucket_s * bp = dbhash_table + index;
while (bitmap_get(dbhash_bitmap, index)
@@ -145,12 +145,12 @@ long dbhash_search_first(char * seq,
return -1;
}
-long dbhash_search_next(struct dbhash_search_info_s * info)
+int64_t dbhash_search_next(struct dbhash_search_info_s * info)
{
- unsigned long hash = info->hash;
+ uint64_t hash = info->hash;
char * seq = info->seq;
- unsigned long seqlen = info->seqlen;
- unsigned long index = (info->index + 1) & dbhash_mask;
+ uint64_t seqlen = info->seqlen;
+ uint64_t index = (info->index + 1) & dbhash_mask;
struct dbhash_bucket_s * bp = dbhash_table + index;
while (bitmap_get(dbhash_bitmap, index)
@@ -171,11 +171,11 @@ long dbhash_search_next(struct dbhash_search_info_s * info)
return -1;
}
-void dbhash_add(char * seq, unsigned long seqlen, unsigned long seqno)
+void dbhash_add(char * seq, uint64_t seqlen, uint64_t seqno)
{
struct dbhash_search_info_s info;
- long ret = dbhash_search_first(seq, seqlen, & info);
+ int64_t ret = dbhash_search_first(seq, seqlen, & info);
while (ret >= 0)
ret = dbhash_search_next(&info);
@@ -185,10 +185,10 @@ void dbhash_add(char * seq, unsigned long seqlen, unsigned long seqno)
bp->seqno = seqno;
}
-void dbhash_add_one(unsigned long seqno)
+void dbhash_add_one(uint64_t seqno)
{
char * seq = db_getsequence(seqno);
- unsigned long seqlen = db_getsequencelen(seqno);
+ uint64_t seqlen = db_getsequencelen(seqno);
char * normalized = (char*) xmalloc(seqlen+1);
string_normalize(normalized, seq, seqlen);
dbhash_add(normalized, seqlen, seqno);
@@ -198,14 +198,14 @@ void dbhash_add_all()
{
progress_init("Hashing database sequences", db_getsequencecount());
char * normalized = (char*) xmalloc(db_getlongestsequence()+1);
- for(unsigned long seqno=0; seqno < db_getsequencecount(); seqno++)
+ for(uint64_t seqno=0; seqno < db_getsequencecount(); seqno++)
{
char * seq = db_getsequence(seqno);
- unsigned long seqlen = db_getsequencelen(seqno);
+ uint64_t seqlen = db_getsequencelen(seqno);
string_normalize(normalized, seq, seqlen);
dbhash_add(normalized, seqlen, seqno);
progress_update(seqno+1);
}
- free(normalized);
+ xfree(normalized);
progress_done();
}
diff --git a/src/dbhash.h b/src/dbhash.h
index a7b50b5..b1fbfc2 100644
--- a/src/dbhash.h
+++ b/src/dbhash.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,27 +60,27 @@
struct dbhash_bucket_s
{
- unsigned long hash;
- unsigned long seqno;
+ uint64_t hash;
+ uint64_t seqno;
};
struct dbhash_search_info_s
{
char * seq;
- unsigned long seqlen;
- unsigned long hash;
- unsigned long index;
+ uint64_t seqlen;
+ uint64_t hash;
+ uint64_t index;
};
-void dbhash_open(unsigned long maxelements);
+void dbhash_open(uint64_t maxelements);
void dbhash_close();
-void dbhash_add(char * seq, unsigned long seqlen, unsigned long seqno);
-void dbhash_add_one(unsigned long seqno);
+void dbhash_add(char * seq, uint64_t seqlen, uint64_t seqno);
+void dbhash_add_one(uint64_t seqno);
void dbhash_add_all();
-long dbhash_search_first(char * seq,
- unsigned long seqlen,
+int64_t dbhash_search_first(char * seq,
+ uint64_t seqlen,
struct dbhash_search_info_s * info);
-long dbhash_search_next(struct dbhash_search_info_s * info);
+int64_t dbhash_search_next(struct dbhash_search_info_s * info);
void dbhash_search_finish(struct dbhash_search_info_s * info);
diff --git a/src/dbindex.cc b/src/dbindex.cc
index 0222ded..6f02c93 100644
--- a/src/dbindex.cc
+++ b/src/dbindex.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,13 +61,13 @@
#include "vsearch.h"
unsigned int * kmercount;
-unsigned long * kmerhash;
+uint64_t * kmerhash;
unsigned int * kmerindex;
bitmap_t * * kmerbitmap;
unsigned int * dbindex_map;
static unsigned int kmerhashsize;
-static unsigned long kmerindexsize;
+static uint64_t kmerindexsize;
unsigned int dbindex_count;
static uhandle_s * dbindex_uh;
@@ -76,9 +76,9 @@ static uhandle_s * dbindex_uh;
static unsigned int bitmap_mincount;
-void fprint_kmer(FILE * f, unsigned int kk, unsigned long kmer)
+void fprint_kmer(FILE * f, unsigned int kk, uint64_t kmer)
{
- unsigned long x = kmer;
+ uint64_t x = kmer;
for(unsigned int i=0; i<kk; i++)
fprintf(f, "%c", sym_nt_2bit[(x >> (2*(kk-i-1))) & 3]);
}
@@ -167,8 +167,8 @@ void dbindex_prepare(int use_bitmap, int seqmask)
/* hash / bitmap setup */
/* convert hash counts to position in index */
- kmerhash = (unsigned long *) xmalloc((kmerhashsize+1) * sizeof(unsigned long));
- unsigned long sum = 0;
+ kmerhash = (uint64_t *) xmalloc((kmerhashsize+1) * sizeof(uint64_t));
+ uint64_t sum = 0;
for(unsigned int i = 0; i < kmerhashsize; i++)
{
kmerhash[i] = sum;
@@ -204,14 +204,14 @@ void dbindex_prepare(int use_bitmap, int seqmask)
void dbindex_free()
{
- free(kmerhash);
- free(kmerindex);
- free(kmercount);
- free(dbindex_map);
+ xfree(kmerhash);
+ xfree(kmerindex);
+ xfree(kmercount);
+ xfree(dbindex_map);
for(unsigned int kmer=0; kmer<kmerhashsize; kmer++)
if (kmerbitmap[kmer])
bitmap_free(kmerbitmap[kmer]);
- free(kmerbitmap);
+ xfree(kmerbitmap);
unique_exit(dbindex_uh);
}
diff --git a/src/dbindex.h b/src/dbindex.h
index a6d3c28..c5e1669 100644
--- a/src/dbindex.h
+++ b/src/dbindex.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -59,13 +59,13 @@
*/
extern unsigned int * kmercount; /* number of matching seqnos for each kmer */
-extern unsigned long * kmerhash; /* index into the list below for each kmer */
+extern uint64_t * kmerhash; /* index into the list below for each kmer */
extern unsigned int * kmerindex; /* the list of matching seqnos for kmers */
extern bitmap_t * * kmerbitmap;
extern unsigned int * dbindex_map;
extern unsigned int dbindex_count;
-void fprint_kmer(FILE * f, unsigned int k, unsigned long kmer);
+void fprint_kmer(FILE * f, unsigned int k, uint64_t kmer);
void dbindex_prepare(int use_bitmap, int seqmask);
void dbindex_addallsequences(int seqmask);
diff --git a/src/derep.cc b/src/derep.cc
index 793631a..0e17944 100644
--- a/src/derep.cc
+++ b/src/derep.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -66,7 +66,7 @@
struct bucket
{
- unsigned long hash;
+ uint64_t hash;
unsigned int seqno_first;
unsigned int seqno_last;
unsigned int size;
@@ -76,12 +76,12 @@ struct bucket
#ifdef BITMAP
unsigned char * hash_occupied = 0;
-void hash_set_occupied(unsigned long hashindex)
+void hash_set_occupied(uint64_t hashindex)
{
hash_occupied[(hashindex) >> 3] |= 1 << (hashindex & 7);
}
-int hash_is_occupied(unsigned long hashindex)
+int hash_is_occupied(uint64_t hashindex)
{
return hash_occupied[(hashindex) >> 3] & (1 << (hashindex & 7));
}
@@ -165,11 +165,11 @@ void derep_fulllength()
show_rusage();
- long dbsequencecount = db_getsequencecount();
+ int64_t dbsequencecount = db_getsequencecount();
/* adjust size of hash table for 2/3 fill rate */
- long hashtablesize = 1;
+ int64_t hashtablesize = 1;
int hash_shift = 0;
while (3 * dbsequencecount > 2 * hashtablesize)
{
@@ -189,9 +189,9 @@ void derep_fulllength()
memset(hash_occupied, 0, hashtablesize / 8);
#endif
- long clusters = 0;
- long sumsize = 0;
- unsigned long maxsize = 0;
+ int64_t clusters = 0;
+ int64_t sumsize = 0;
+ uint64_t maxsize = 0;
double median = 0.0;
double average = 0.0;
@@ -208,7 +208,7 @@ void derep_fulllength()
char * rc_seq_up = (char*) xmalloc(db_getlongestsequence() + 1);
progress_init("Dereplicating", dbsequencecount);
- for(long i=0; i<dbsequencecount; i++)
+ for(int64_t i=0; i<dbsequencecount; i++)
{
unsigned int seqlen = db_getsequencelen(i);
char * seq = db_getsequence(i);
@@ -228,8 +228,8 @@ void derep_fulllength()
collision when the number of sequences is about 5e9.
*/
- unsigned long hash = HASH(seq_up, seqlen);
- unsigned long j = hash & hash_mask;
+ uint64_t hash = HASH(seq_up, seqlen);
+ uint64_t j = hash & hash_mask;
struct bucket * bp = hashtable + j;
while (
@@ -257,9 +257,9 @@ void derep_fulllength()
/* no match on plus strand */
/* check minus strand as well */
- unsigned long rc_hash = HASH(rc_seq_up, seqlen);
+ uint64_t rc_hash = HASH(rc_seq_up, seqlen);
struct bucket * rc_bp = hashtable + rc_hash % hashtablesize;
- unsigned long k = rc_hash & hash_mask;
+ uint64_t k = rc_hash & hash_mask;
while (
#ifdef BITMAP
@@ -291,7 +291,7 @@ void derep_fulllength()
}
}
- long ab = opt_sizein ? db_getabundance(i) : 1;
+ int64_t ab = opt_sizein ? db_getabundance(i) : 1;
sumsize += ab;
if (bp->size)
@@ -323,8 +323,8 @@ void derep_fulllength()
}
progress_done();
- free(seq_up);
- free(rc_seq_up);
+ xfree(seq_up);
+ xfree(rc_seq_up);
show_rusage();
@@ -347,12 +347,12 @@ void derep_fulllength()
if (!opt_quiet)
fprintf(stderr,
- "%ld unique sequences, avg cluster %.1lf, median %.0f, max %lu\n",
+ "%" PRId64 " unique sequences, avg cluster %.1lf, median %.0f, max %" PRIu64 "\n",
clusters, average, median, maxsize);
if (opt_log)
fprintf(fp_log,
- "%ld unique sequences, avg cluster %.1lf, median %.0f, max %lu\n\n",
+ "%" PRId64 " unique sequences, avg cluster %.1lf, median %.0f, max %" PRIu64 "\n\n",
clusters, average, median, maxsize);
show_rusage();
@@ -360,11 +360,11 @@ void derep_fulllength()
/* count selected */
- long selected = 0;
- for (long i=0; i<clusters; i++)
+ int64_t selected = 0;
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
- long size = bp->size;
+ int64_t size = bp->size;
if ((size >= opt_minuniquesize) && (size <= opt_maxuniquesize))
{
selected++;
@@ -380,11 +380,11 @@ void derep_fulllength()
{
progress_init("Writing output file", clusters);
- long relabel_count = 0;
- for (long i=0; i<clusters; i++)
+ int64_t relabel_count = 0;
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
- long size = bp->size;
+ int64_t size = bp->size;
if ((size >= opt_minuniquesize) && (size <= opt_maxuniquesize))
{
relabel_count++;
@@ -410,20 +410,20 @@ void derep_fulllength()
if (opt_uc)
{
progress_init("Writing uc file, first part", clusters);
- for (long i=0; i<clusters; i++)
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
char * h = db_getheader(bp->seqno_first);
- long len = db_getsequencelen(bp->seqno_first);
+ int64_t len = db_getsequencelen(bp->seqno_first);
- fprintf(fp_uc, "S\t%ld\t%ld\t*\t*\t*\t*\t*\t%s\t*\n",
+ fprintf(fp_uc, "S\t%" PRId64 "\t%" PRId64 "\t*\t*\t*\t*\t*\t%s\t*\n",
i, len, h);
for (unsigned int next = nextseqtab[bp->seqno_first];
next != terminal;
next = nextseqtab[next])
fprintf(fp_uc,
- "H\t%ld\t%ld\t%.1f\t%s\t0\t0\t*\t%s\t%s\n",
+ "H\t%" PRId64 "\t%" PRId64 "\t%.1f\t%s\t0\t0\t*\t%s\t%s\n",
i, len, 100.0,
(match_strand[next] ? "-" : "+"),
db_getheader(next), h);
@@ -434,10 +434,10 @@ void derep_fulllength()
show_rusage();
progress_init("Writing uc file, second part", clusters);
- for (long i=0; i<clusters; i++)
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
- fprintf(fp_uc, "C\t%ld\t%u\t*\t*\t*\t*\t*\t%s\t*\n",
+ fprintf(fp_uc, "C\t%" PRId64 "\t%u\t*\t*\t*\t*\t*\t%s\t*\n",
i, bp->size, db_getheader(bp->seqno_first));
progress_update(i);
}
@@ -450,20 +450,20 @@ void derep_fulllength()
{
if (!opt_quiet)
fprintf(stderr,
- "%ld uniques written, %ld clusters discarded (%.1f%%)\n",
+ "%" PRId64 " uniques written, %" PRId64 " clusters discarded (%.1f%%)\n",
selected, clusters - selected,
100.0 * (clusters - selected) / clusters);
if (opt_log)
fprintf(fp_log,
- "%ld uniques written, %ld clusters discarded (%.1f%%)\n\n",
+ "%" PRId64 " uniques written, %" PRId64 " clusters discarded (%.1f%%)\n\n",
selected, clusters - selected,
100.0 * (clusters - selected) / clusters);
}
- free(match_strand);
- free(nextseqtab);
- free(hashtable);
+ xfree(match_strand);
+ xfree(nextseqtab);
+ xfree(hashtable);
db_free();
}
@@ -493,11 +493,11 @@ void derep_prefix()
show_rusage();
- long dbsequencecount = db_getsequencecount();
+ int64_t dbsequencecount = db_getsequencecount();
/* adjust size of hash table for 2/3 fill rate */
- long hashtablesize = 1;
+ int64_t hashtablesize = 1;
int hash_shift = 0;
while (3 * dbsequencecount > 2 * hashtablesize)
{
@@ -511,9 +511,9 @@ void derep_prefix()
memset(hashtable, 0, sizeof(bucket) * hashtablesize);
- long clusters = 0;
- long sumsize = 0;
- unsigned long maxsize = 0;
+ int64_t clusters = 0;
+ int64_t sumsize = 0;
+ uint64_t maxsize = 0;
double median = 0.0;
double average = 0.0;
@@ -529,11 +529,11 @@ void derep_prefix()
unsigned int len_longest = db_getlongestsequence();
unsigned int len_shortest = db_getshortestsequence();
- unsigned long * prefix_hashes = (unsigned long *)
- xmalloc(sizeof(unsigned long) * (len_longest+1));
+ uint64_t * prefix_hashes = (uint64_t *)
+ xmalloc(sizeof(uint64_t) * (len_longest+1));
progress_init("Dereplicating", dbsequencecount);
- for(long i=0; i<dbsequencecount; i++)
+ for(int64_t i=0; i<dbsequencecount; i++)
{
unsigned int seqlen = db_getsequencelen(i);
char * seq = db_getsequence(i);
@@ -541,7 +541,7 @@ void derep_prefix()
/* normalize sequence: uppercase and replace U by T */
string_normalize(seq_up, seq, seqlen);
- unsigned long ab = opt_sizein ? db_getabundance(i) : 1;
+ uint64_t ab = opt_sizein ? db_getabundance(i) : 1;
sumsize += ab;
/*
@@ -566,7 +566,7 @@ void derep_prefix()
/* compute hashes of all prefixes */
- unsigned long fnv1a_hash = 14695981039346656037UL;
+ uint64_t fnv1a_hash = 14695981039346656037UL;
prefix_hashes[0] = fnv1a_hash;
for(unsigned int j = 0; j < seqlen; j++)
{
@@ -579,7 +579,7 @@ void derep_prefix()
unsigned int prefix_len = seqlen;
- unsigned long hash = prefix_hashes[prefix_len];
+ uint64_t hash = prefix_hashes[prefix_len];
struct bucket * bp = hashtable + (hash & hash_mask);
while ((bp->size) &&
@@ -596,7 +596,7 @@ void derep_prefix()
/* at this point, bp points either to (1) a free empty hash bucket, or
(2) a bucket with an exact match. */
- unsigned long orig_hash = hash;
+ uint64_t orig_hash = hash;
struct bucket * orig_bp = bp;
if (bp->size)
@@ -673,9 +673,9 @@ void derep_prefix()
}
progress_done();
- free(prefix_hashes);
+ xfree(prefix_hashes);
- free(seq_up);
+ xfree(seq_up);
show_rusage();
@@ -696,23 +696,23 @@ void derep_prefix()
if (!opt_quiet)
fprintf(stderr,
- "%ld unique sequences, avg cluster %.1lf, median %.0f, max %lu\n",
+ "%" PRId64 " unique sequences, avg cluster %.1lf, median %.0f, max %" PRIu64 "\n",
clusters, average, median, maxsize);
if (opt_log)
fprintf(fp_log,
- "%ld unique sequences, avg cluster %.1lf, median %.0f, max %lu\n\n",
+ "%" PRId64 " unique sequences, avg cluster %.1lf, median %.0f, max %" PRIu64 "\n\n",
clusters, average, median, maxsize);
show_rusage();
/* count selected */
- long selected = 0;
- for (long i=0; i<clusters; i++)
+ int64_t selected = 0;
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
- long size = bp->size;
+ int64_t size = bp->size;
if ((size >= opt_minuniquesize) && (size <= opt_maxuniquesize))
{
selected++;
@@ -728,11 +728,11 @@ void derep_prefix()
{
progress_init("Writing output file", clusters);
- long relabel_count = 0;
- for (long i=0; i<clusters; i++)
+ int64_t relabel_count = 0;
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
- long size = bp->size;
+ int64_t size = bp->size;
if ((size >= opt_minuniquesize) && (size <= opt_maxuniquesize))
{
relabel_count++;
@@ -758,20 +758,20 @@ void derep_prefix()
if (opt_uc)
{
progress_init("Writing uc file, first part", clusters);
- for (long i=0; i<clusters; i++)
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
char * h = db_getheader(bp->seqno_first);
- long len = db_getsequencelen(bp->seqno_first);
+ int64_t len = db_getsequencelen(bp->seqno_first);
- fprintf(fp_uc, "S\t%ld\t%ld\t*\t*\t*\t*\t*\t%s\t*\n",
+ fprintf(fp_uc, "S\t%" PRId64 "\t%" PRId64 "\t*\t*\t*\t*\t*\t%s\t*\n",
i, len, h);
for (unsigned int next = nextseqtab[bp->seqno_first];
next != terminal;
next = nextseqtab[next])
fprintf(fp_uc,
- "H\t%ld\t%lu\t%.1f\t+\t0\t0\t*\t%s\t%s\n",
+ "H\t%" PRId64 "\t%" PRIu64 "\t%.1f\t+\t0\t0\t*\t%s\t%s\n",
i, db_getsequencelen(next), 100.0, db_getheader(next), h);
progress_update(i);
@@ -780,10 +780,10 @@ void derep_prefix()
show_rusage();
progress_init("Writing uc file, second part", clusters);
- for (long i=0; i<clusters; i++)
+ for (int64_t i=0; i<clusters; i++)
{
struct bucket * bp = hashtable + i;
- fprintf(fp_uc, "C\t%ld\t%u\t*\t*\t*\t*\t*\t%s\t*\n",
+ fprintf(fp_uc, "C\t%" PRId64 "\t%u\t*\t*\t*\t*\t*\t%s\t*\n",
i, bp->size, db_getheader(bp->seqno_first));
progress_update(i);
}
@@ -796,18 +796,18 @@ void derep_prefix()
{
if (!opt_quiet)
fprintf(stderr,
- "%ld uniques written, %ld clusters discarded (%.1f%%)\n",
+ "%" PRId64 " uniques written, %" PRId64 " clusters discarded (%.1f%%)\n",
selected, clusters - selected,
100.0 * (clusters - selected) / clusters);
if (opt_log)
fprintf(fp_log,
- "%ld uniques written, %ld clusters discarded (%.1f%%)\n\n",
+ "%" PRId64 " uniques written, %" PRId64 " clusters discarded (%.1f%%)\n\n",
selected, clusters - selected,
100.0 * (clusters - selected) / clusters);
}
- free(nextseqtab);
- free(hashtable);
+ xfree(nextseqtab);
+ xfree(hashtable);
db_free();
}
diff --git a/src/derep.h b/src/derep.h
index 754b891..8c9c35f 100644
--- a/src/derep.h
+++ b/src/derep.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/dynlibs.cc b/src/dynlibs.cc
index 4f6cc2e..9c9ddb7 100644
--- a/src/dynlibs.cc
+++ b/src/dynlibs.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,12 +61,17 @@
#include "vsearch.h"
#ifdef HAVE_ZLIB_H
-#ifdef __APPLE__
+# ifdef _WIN32
+const char gz_libname[] = "zlib1.dll";
+HMODULE gz_lib;
+# else
+# ifdef __APPLE__
const char gz_libname[] = "libz.dylib";
-#else
+# else
const char gz_libname[] = "libz.so";
-#endif
+# endif
void * gz_lib;
+# endif
gzFile (*gzdopen_p)(int, const char *);
int (*gzclose_p)(gzFile);
int (*gzread_p)(gzFile, void *, unsigned);
@@ -77,43 +82,69 @@ const char * (*gzerror_p)(gzFile, int*);
#endif
#ifdef HAVE_BZLIB_H
-#ifdef __APPLE__
+# ifdef _WIN32
+const char bz2_libname[] = "libbz2.dll";
+HMODULE bz2_lib;
+# else
+# ifdef __APPLE__
const char bz2_libname[] = "libbz2.dylib";
-#else
+# else
const char bz2_libname[] = "libbz2.so";
-#endif
+# endif
void * bz2_lib;
+# endif
BZFILE* (*BZ2_bzReadOpen_p)(int*, FILE*, int, int, void*, int);
void (*BZ2_bzReadClose_p)(int*, BZFILE*);
int (*BZ2_bzRead_p)(int*, BZFILE*, void*, int);
#endif
+#ifdef _WIN32
+FARPROC arch_dlsym(HMODULE handle, const char * symbol)
+#else
+void * arch_dlsym(void * handle, const char * symbol)
+#endif
+{
+#ifdef _WIN32
+ return GetProcAddress(handle, symbol);
+#else
+ return dlsym(handle, symbol);
+#endif
+}
+
void dynlibs_open()
{
#ifdef HAVE_ZLIB_H
+#ifdef _WIN32
+ gz_lib = LoadLibraryA(gz_libname);
+#else
gz_lib = dlopen(gz_libname, RTLD_LAZY);
+#endif
if (gz_lib)
{
- gzdopen_p = (gzFile (*)(int, const char*)) dlsym(gz_lib, "gzdopen");
- gzclose_p = (int (*)(gzFile)) dlsym(gz_lib, "gzclose");
- gzread_p = (int (*)(gzFile, void*, unsigned)) dlsym(gz_lib, "gzread");
- gzgetc_p = (int (*)(gzFile)) dlsym(gz_lib, "gzgetc");
- gzrewind_p = (int (*)(gzFile)) dlsym(gz_lib, "gzrewind");
- gzerror_p = (const char * (*)(gzFile, int*)) dlsym(gz_lib, "gzerror");
- gzungetc_p = (int (*)(int, gzFile)) dlsym(gz_lib, "gzungetc");
+ gzdopen_p = (gzFile (*)(int, const char*)) arch_dlsym(gz_lib, "gzdopen");
+ gzclose_p = (int (*)(gzFile)) arch_dlsym(gz_lib, "gzclose");
+ gzread_p = (int (*)(gzFile, void*, unsigned)) arch_dlsym(gz_lib, "gzread");
+ gzgetc_p = (int (*)(gzFile)) arch_dlsym(gz_lib, "gzgetc");
+ gzrewind_p = (int (*)(gzFile)) arch_dlsym(gz_lib, "gzrewind");
+ gzerror_p = (const char * (*)(gzFile, int*)) arch_dlsym(gz_lib, "gzerror");
+ gzungetc_p = (int (*)(int, gzFile)) arch_dlsym(gz_lib, "gzungetc");
}
#endif
#ifdef HAVE_BZLIB_H
+#ifdef _WIN32
+ bz2_lib = LoadLibraryA(bz2_libname);
+#else
bz2_lib = dlopen(bz2_libname, RTLD_LAZY);
+#endif
if (bz2_lib)
{
BZ2_bzReadOpen_p = (BZFILE* (*)(int*, FILE*, int, int, void*, int))
- dlsym(bz2_lib, "BZ2_bzReadOpen");
+ arch_dlsym(bz2_lib, "BZ2_bzReadOpen");
BZ2_bzReadClose_p = (void (*)(int*, BZFILE*))
- dlsym(bz2_lib, "BZ2_bzReadClose");
+ arch_dlsym(bz2_lib, "BZ2_bzReadClose");
BZ2_bzRead_p = (int (*)(int*, BZFILE*, void*, int))
- dlsym(bz2_lib, "BZ2_bzRead");
+ arch_dlsym(bz2_lib, "BZ2_bzRead");
}
#endif
}
@@ -122,12 +153,20 @@ void dynlibs_close()
{
#ifdef HAVE_ZLIB_H
if (gz_lib)
+#ifdef _WIN32
+ FreeLibrary(gz_lib);
+#else
dlclose(gz_lib);
+#endif
gz_lib = 0;
#endif
#ifdef HAVE_BZLIB_H
if (bz2_lib)
+#ifdef _WIN32
+ FreeLibrary(bz2_lib);
+#else
dlclose(bz2_lib);
+#endif
bz2_lib = 0;
#endif
}
diff --git a/src/dynlibs.h b/src/dynlibs.h
index 0d13cbc..c31cd1c 100644
--- a/src/dynlibs.h
+++ b/src/dynlibs.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -59,7 +59,11 @@
*/
#ifdef HAVE_ZLIB_H
+#ifdef _WIN32
+extern HMODULE gz_lib;
+#else
extern void * gz_lib;
+#endif
extern gzFile (*gzdopen_p)(int, const char *);
extern int (*gzclose_p)(gzFile);
extern int (*gzread_p)(gzFile, void*, unsigned);
@@ -70,7 +74,11 @@ extern const char * (*gzerror_p)(gzFile, int*);
#endif
#ifdef HAVE_BZLIB_H
+#ifdef _WIN32
+extern HMODULE bz2_lib;
+#else
extern void * bz2_lib;
+#endif
extern BZFILE* (*BZ2_bzReadOpen_p)(int*, FILE*, int, int, void*, int);
extern void (*BZ2_bzReadClose_p)(int*, BZFILE*);
extern int (*BZ2_bzRead_p)(int*, BZFILE*, void*, int);
diff --git a/src/eestats.cc b/src/eestats.cc
index bb6ea6b..0d50dd7 100644
--- a/src/eestats.cc
+++ b/src/eestats.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -65,7 +65,7 @@ double q2p(int q)
return exp10(- q / 10.0);
}
-long ee_start(int pos, int resolution)
+int64_t ee_start(int pos, int resolution)
{
return pos * (resolution * (pos + 1) + 2) / 2;
}
@@ -74,7 +74,7 @@ void fastq_eestats()
{
fastx_handle h = fastq_open(opt_fastq_eestats);
- unsigned long filesize = fastq_get_size(h);
+ uint64_t filesize = fastq_get_size(h);
FILE * fp_output = 0;
@@ -87,25 +87,25 @@ void fastq_eestats()
progress_init("Reading fastq file", filesize);
- unsigned long seq_count = 0;
- unsigned long symbols = 0;
+ uint64_t seq_count = 0;
+ uint64_t symbols = 0;
- long len_alloc = 10;
+ int64_t len_alloc = 10;
const int resolution = 1000;
int max_quality = opt_fastq_qmax - opt_fastq_qmin + 1;
- long ee_size = ee_start(len_alloc, resolution);
+ int64_t ee_size = ee_start(len_alloc, resolution);
- int * read_length_table = (int*) xmalloc(sizeof(int) * len_alloc);
- memset(read_length_table, 0, sizeof(int) * len_alloc);
+ uint64_t * read_length_table = (uint64_t*) xmalloc(sizeof(uint64_t) * len_alloc);
+ memset(read_length_table, 0, sizeof(uint64_t) * len_alloc);
- int * qual_length_table = (int*) xmalloc(sizeof(int) * len_alloc *
+ uint64_t * qual_length_table = (uint64_t*) xmalloc(sizeof(uint64_t) * len_alloc *
(max_quality+1));
- memset(qual_length_table, 0, sizeof(int) * len_alloc * (max_quality+1));
+ memset(qual_length_table, 0, sizeof(uint64_t) * len_alloc * (max_quality+1));
- int * ee_length_table = (int*) xmalloc(sizeof(int) * ee_size);
- memset(ee_length_table, 0, sizeof(int) * ee_size);
+ uint64_t * ee_length_table = (uint64_t*) xmalloc(sizeof(uint64_t) * ee_size);
+ memset(ee_length_table, 0, sizeof(uint64_t) * ee_size);
double * sum_ee_length_table = (double*) xmalloc(sizeof(double) * len_alloc);
memset(sum_ee_length_table, 0, sizeof(double) * len_alloc);
@@ -113,38 +113,38 @@ void fastq_eestats()
double * sum_pe_length_table = (double*) xmalloc(sizeof(double) * len_alloc);
memset(sum_pe_length_table, 0, sizeof(double) * len_alloc);
- long len_min = LONG_MAX;
- long len_max = 0;
+ int64_t len_min = LONG_MAX;
+ int64_t len_max = 0;
while(fastq_next(h, 0, chrmap_upcase))
{
seq_count++;
- long len = fastq_get_sequence_length(h);
+ int64_t len = fastq_get_sequence_length(h);
char * q = fastq_get_quality(h);
/* update length statistics */
- long new_alloc = len + 1;
+ int64_t new_alloc = len + 1;
if (new_alloc > len_alloc)
{
- long new_ee_size = ee_start(new_alloc, resolution);
+ int64_t new_ee_size = ee_start(new_alloc, resolution);
- read_length_table = (int*) xrealloc(read_length_table,
- sizeof(int) * new_alloc);
+ read_length_table = (uint64_t*) xrealloc(read_length_table,
+ sizeof(uint64_t) * new_alloc);
memset(read_length_table + len_alloc, 0,
- sizeof(int) * (new_alloc - len_alloc));
+ sizeof(uint64_t) * (new_alloc - len_alloc));
- qual_length_table = (int*) xrealloc(qual_length_table, sizeof(int) *
+ qual_length_table = (uint64_t*) xrealloc(qual_length_table, sizeof(uint64_t) *
new_alloc * (max_quality+1));
memset(qual_length_table + (max_quality+1) * len_alloc, 0,
- sizeof(int) * (new_alloc - len_alloc) * (max_quality+1));
+ sizeof(uint64_t) * (new_alloc - len_alloc) * (max_quality+1));
- ee_length_table = (int*) xrealloc(ee_length_table, sizeof(int) *
+ ee_length_table = (uint64_t*) xrealloc(ee_length_table, sizeof(uint64_t) *
new_ee_size);
memset(ee_length_table + ee_size, 0,
- sizeof(int) * (new_ee_size - ee_size));
+ sizeof(uint64_t) * (new_ee_size - ee_size));
sum_ee_length_table = (double*) xrealloc(sum_ee_length_table,
sizeof(double) * new_alloc);
@@ -171,7 +171,7 @@ void fastq_eestats()
double ee = 0.0;
- for(long i=0; i < len; i++)
+ for(int64_t i=0; i < len; i++)
{
read_length_table[i]++;
@@ -183,13 +183,13 @@ void fastq_eestats()
if (qual < opt_fastq_qmin)
{
- snprintf(msg, 200, "FASTQ quality value (%d) below qmin (%ld)",
+ snprintf(msg, 200, "FASTQ quality value (%d) below qmin (%" PRId64 ")",
qual, opt_fastq_qmin);
fatal(msg);
}
else if (qual > opt_fastq_qmax)
{
- snprintf(msg, 200, "FASTQ quality value (%d) above qmax (%ld)",
+ snprintf(msg, 200, "FASTQ quality value (%d) above qmax (%" PRId64 ")",
qual, opt_fastq_qmax);
fatal(msg);
}
@@ -210,7 +210,7 @@ void fastq_eestats()
ee += pe;
- int e_int = MIN(resolution*(i+1), (int)(resolution * ee));
+ int64_t e_int = MIN(resolution*(i+1), (int)(resolution * ee));
ee_length_table[ee_start(i, resolution) + e_int]++;
sum_ee_length_table[i] += ee;
@@ -225,9 +225,9 @@ void fastq_eestats()
"Min_Pe\tLow_Pe\tMed_Pe\tMean_Pe\tHi_Pe\tMax_Pe\t"
"Min_EE\tLow_EE\tMed_EE\tMean_EE\tHi_EE\tMax_EE\n");
- for(long i=0; i<len_max; i++)
+ for(int64_t i=0; i<len_max; i++)
{
- long reads = read_length_table[i];
+ int64_t reads = read_length_table[i];
double pctrecs = 100.0 * reads / seq_count;
double min_q = -1.0;
@@ -281,13 +281,13 @@ void fastq_eestats()
double hi_ee = -1.0;
double max_ee = -1.0;
- long ee_offset = ee_start(i, resolution);
- long max_errors = resolution * (i+1);
+ int64_t ee_offset = ee_start(i, resolution);
+ int64_t max_errors = resolution * (i+1);
n = 0;
- for(int e=0; e<=max_errors; e++)
+ for(int64_t e=0; e<=max_errors; e++)
{
- int x = ee_length_table[ee_offset + e];
+ int64_t x = ee_length_table[ee_offset + e];
n += x;
if ((min_ee<0) && (x > 0))
@@ -315,7 +315,7 @@ void fastq_eestats()
max_ee = (max_ee + 0.5) / resolution;
fprintf(fp_output,
- "%ld\t%ld\t%.1lf"
+ "%" PRId64 "\t%" PRId64 "\t%.1lf"
"\t%.1lf\t%.1lf\t%.1lf\t%.1lf\t%.1lf\t%.1lf"
"\t%.2lg\t%.2lg\t%.2lg\t%.2lg\t%.2lg\t%.2lg"
"\t%.2lf\t%.2lf\t%.2lf\t%.2lf\t%.2lf\t%.2lf\n",
@@ -325,11 +325,11 @@ void fastq_eestats()
min_ee, low_ee, med_ee, mean_ee, hi_ee, max_ee);
}
- free(read_length_table);
- free(qual_length_table);
- free(ee_length_table);
- free(sum_ee_length_table);
- free(sum_pe_length_table);
+ xfree(read_length_table);
+ xfree(qual_length_table);
+ xfree(ee_length_table);
+ xfree(sum_ee_length_table);
+ xfree(sum_pe_length_table);
fclose(fp_output);
diff --git a/src/eestats.h b/src/eestats.h
index 105f7f8..87aaa30 100644
--- a/src/eestats.h
+++ b/src/eestats.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/fasta.cc b/src/fasta.cc
index 84b3a60..39f023b 100644
--- a/src/fasta.cc
+++ b/src/fasta.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -123,13 +123,13 @@ void fasta_filter_sequence(fastx_handle h,
if ((c>=32) && (c<127))
snprintf(msg,
200,
- "illegal character '%c' on line %lu in fasta file",
+ "illegal character '%c' on line %" PRIu64 " in fasta file",
c,
h->lineno);
else
snprintf(msg,
200,
- "illegal unprintable character %#.2x (hexadecimal) on line %lu in fasta file",
+ "illegal unprintable character %#.2x (hexadecimal) on line %" PRIu64 " in fasta file",
c,
h->lineno);
fatal(msg);
@@ -162,7 +162,7 @@ bool fasta_next(fastx_handle h,
h->sequence_buffer.length = 0;
h->sequence_buffer.data[0] = 0;
- unsigned long rest = fastx_file_fill_buffer(h);
+ uint64_t rest = fastx_file_fill_buffer(h);
if (rest == 0)
return 0;
@@ -193,7 +193,7 @@ bool fasta_next(fastx_handle h,
rest);
/* copy to header buffer */
- unsigned long len = rest;
+ uint64_t len = rest;
if (lf)
{
/* LF found, copy up to and including LF */
@@ -226,7 +226,7 @@ bool fasta_next(fastx_handle h,
lf = (char *) memchr(h->file_buffer.data + h->file_buffer.position,
'\n', rest);
- unsigned long len = rest;
+ uint64_t len = rest;
if (lf)
{
/* LF found, copy up to and including LF */
@@ -247,37 +247,37 @@ bool fasta_next(fastx_handle h,
return 1;
}
-long fasta_get_abundance(fastx_handle h)
+int64_t fasta_get_abundance(fastx_handle h)
{
return abundance_get(global_abundance, h->header_buffer.data);
}
-unsigned long fasta_get_position(fastx_handle h)
+uint64_t fasta_get_position(fastx_handle h)
{
return h->file_position;
}
-unsigned long fasta_get_size(fastx_handle h)
+uint64_t fasta_get_size(fastx_handle h)
{
return h->file_size;
}
-unsigned long fasta_get_lineno(fastx_handle h)
+uint64_t fasta_get_lineno(fastx_handle h)
{
return h->lineno_start;
}
-unsigned long fasta_get_seqno(fastx_handle h)
+uint64_t fasta_get_seqno(fastx_handle h)
{
return h->seqno;
}
-unsigned long fasta_get_header_length(fastx_handle h)
+uint64_t fasta_get_header_length(fastx_handle h)
{
return h->header_buffer.length;
}
-unsigned long fasta_get_sequence_length(fastx_handle h)
+uint64_t fasta_get_sequence_length(fastx_handle h)
{
return h->sequence_buffer.length;
}
@@ -300,7 +300,7 @@ void fasta_print_header(FILE * fp, const char * hdr)
fprintf(fp, ">%s\n", hdr);
}
-void fasta_print_sequence(FILE * fp, char * seq, unsigned long len, int width)
+void fasta_print_sequence(FILE * fp, char * seq, uint64_t len, int width)
{
/*
The actual length of the sequence may be longer than "len", but only
@@ -313,8 +313,8 @@ void fasta_print_sequence(FILE * fp, char * seq, unsigned long len, int width)
fprintf(fp, "%.*s\n", (int)(len), seq);
else
{
- long rest = len;
- for(unsigned long i=0; i<len; i += width)
+ int64_t rest = len;
+ for(uint64_t i=0; i<len; i += width)
{
fprintf(fp, "%.*s\n", (int)(MIN(rest,width)), seq+i);
rest -= width;
@@ -323,14 +323,14 @@ void fasta_print_sequence(FILE * fp, char * seq, unsigned long len, int width)
}
void fasta_print(FILE * fp, const char * hdr,
- char * seq, unsigned long len)
+ char * seq, uint64_t len)
{
fasta_print_header(fp, hdr);
fasta_print_sequence(fp, seq, len, opt_fasta_width);
}
void fasta_print_ee(FILE * fp, const char * hdr,
- char * seq, unsigned long len,
+ char * seq, uint64_t len,
double ee)
{
fprintf(fp, ">%s;%6.4lf;\n", hdr, ee);
@@ -447,7 +447,7 @@ void fasta_print_relabel(FILE * fp,
}
void fasta_print_db_relabel(FILE * fp,
- unsigned long seqno,
+ uint64_t seqno,
int ordinal)
{
fasta_print_relabel(fp,
@@ -459,24 +459,24 @@ void fasta_print_db_relabel(FILE * fp,
ordinal);
}
-void fasta_print_db_sequence(FILE * fp, unsigned long seqno)
+void fasta_print_db_sequence(FILE * fp, uint64_t seqno)
{
char * seq = db_getsequence(seqno);
- long seqlen = db_getsequencelen(seqno);
+ int64_t seqlen = db_getsequencelen(seqno);
fasta_print_sequence(fp, seq, seqlen, opt_fasta_width);
}
-void fasta_print_db(FILE * fp, unsigned long seqno)
+void fasta_print_db(FILE * fp, uint64_t seqno)
{
char * hdr = db_getheader(seqno);
char * seq = db_getsequence(seqno);
- long seqlen = db_getsequencelen(seqno);
+ int64_t seqlen = db_getsequencelen(seqno);
fasta_print_header(fp, hdr);
fasta_print_sequence(fp, seq, seqlen, opt_fasta_width);
}
-void fasta_print_db_size(FILE * fp, unsigned long seqno, unsigned long size)
+void fasta_print_db_size(FILE * fp, uint64_t seqno, uint64_t size)
{
char * hdr = db_getheader(seqno);
int hdrlen = db_getheaderlen(seqno);
@@ -490,12 +490,12 @@ void fasta_print_db_size(FILE * fp, unsigned long seqno, unsigned long size)
fprintf(fp, "\n");
char * seq = db_getsequence(seqno);
- long seqlen = db_getsequencelen(seqno);
+ int64_t seqlen = db_getsequencelen(seqno);
fasta_print_sequence(fp, seq, seqlen, opt_fasta_width);
}
-void fasta_print_db_strip_size(FILE * fp, unsigned long seqno)
+void fasta_print_db_strip_size(FILE * fp, uint64_t seqno)
{
/* write FASTA but remove abundance information, as with --xsize option */
@@ -510,7 +510,7 @@ void fasta_print_db_strip_size(FILE * fp, unsigned long seqno)
fprintf(fp, "\n");
char * seq = db_getsequence(seqno);
- long seqlen = db_getsequencelen(seqno);
+ int64_t seqlen = db_getsequencelen(seqno);
fasta_print_sequence(fp, seq, seqlen, opt_fasta_width);
}
diff --git a/src/fasta.h b/src/fasta.h
index 0369d0b..79d6d9e 100644
--- a/src/fasta.h
+++ b/src/fasta.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -67,28 +67,28 @@ void fasta_close(fastx_handle h);
bool fasta_next(fastx_handle h,
bool truncateatspace,
const unsigned char * char_mapping);
-unsigned long fasta_get_position(fastx_handle h);
-unsigned long fasta_get_size(fastx_handle h);
-unsigned long fasta_get_lineno(fastx_handle h);
-unsigned long fasta_get_seqno(fastx_handle h);
+uint64_t fasta_get_position(fastx_handle h);
+uint64_t fasta_get_size(fastx_handle h);
+uint64_t fasta_get_lineno(fastx_handle h);
+uint64_t fasta_get_seqno(fastx_handle h);
char * fasta_get_header(fastx_handle h);
char * fasta_get_sequence(fastx_handle h);
-unsigned long fasta_get_header_length(fastx_handle h);
-unsigned long fasta_get_sequence_length(fastx_handle h);
-long fasta_get_abundance(fastx_handle h);
+uint64_t fasta_get_header_length(fastx_handle h);
+uint64_t fasta_get_sequence_length(fastx_handle h);
+int64_t fasta_get_abundance(fastx_handle h);
/* fasta output */
void fasta_print_header(FILE * fp, const char * hdr);
void fasta_print_sequence(FILE * fp, char * seq,
- unsigned long len, int width);
+ uint64_t len, int width);
void fasta_print(FILE * fp, const char * hdr,
- char * seq, unsigned long len);
+ char * seq, uint64_t len);
void fasta_print_ee(FILE * fp, const char * hdr,
- char * seq, unsigned long len,
+ char * seq, uint64_t len,
double ee);
void fasta_print_relabel_cluster(FILE * fp,
@@ -119,13 +119,13 @@ void fasta_print_relabel(FILE * fp,
int abundance,
int ordinal);
-void fasta_print_db(FILE * fp, unsigned long seqno);
+void fasta_print_db(FILE * fp, uint64_t seqno);
-void fasta_print_db_sequence(FILE * fp, unsigned long seqno);
+void fasta_print_db_sequence(FILE * fp, uint64_t seqno);
-void fasta_print_db_size(FILE * fp, unsigned long seqno,
- unsigned long size);
+void fasta_print_db_size(FILE * fp, uint64_t seqno,
+ uint64_t size);
-void fasta_print_db_strip_size(FILE * fp, unsigned long seqno);
+void fasta_print_db_strip_size(FILE * fp, uint64_t seqno);
-void fasta_print_db_relabel(FILE * fp, unsigned long seqno, int ordinal);
+void fasta_print_db_relabel(FILE * fp, uint64_t seqno, int ordinal);
diff --git a/src/fastq.cc b/src/fastq.cc
index 69bb3d7..44b0d70 100644
--- a/src/fastq.cc
+++ b/src/fastq.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,10 +60,10 @@
#include "vsearch.h"
-void fastq_fatal(unsigned long lineno, const char * msg)
+void fastq_fatal(uint64_t lineno, const char * msg)
{
char * string;
- if (asprintf(& string,
+ if (xsprintf(& string,
"Invalid line %lu in FASTQ file: %s",
lineno,
msg) == -1)
@@ -72,7 +72,7 @@ void fastq_fatal(unsigned long lineno, const char * msg)
if (string)
{
fatal(string);
- free(string);
+ xfree(string);
}
else
fatal("Out of memory");
@@ -81,24 +81,24 @@ void fastq_fatal(unsigned long lineno, const char * msg)
void buffer_filter_extend(fastx_handle h,
struct fastx_buffer_s * dest_buffer,
char * source_buf,
- unsigned long len,
+ uint64_t len,
unsigned int * char_action,
const unsigned char * char_mapping,
- unsigned long lineno_start)
+ uint64_t lineno_start)
{
buffer_makespace(dest_buffer, len+1);
/* Strip unwanted characters from the string and raise warnings or
errors on certain characters. */
- unsigned long lineno = lineno_start;
+ uint64_t lineno = lineno_start;
char * p = source_buf;
char * d = dest_buffer->data + dest_buffer->length;
char * q = d;
char msg[200];
- for(unsigned long i = 0; i < len; i++)
+ for(uint64_t i = 0; i < len; i++)
{
char c = *p++;
char m = char_action[(int)c];
@@ -170,12 +170,14 @@ bool fastq_next(fastx_handle h,
h->header_buffer.data[0] = 0;
h->sequence_buffer.length = 0;
h->sequence_buffer.data[0] = 0;
+ h->plusline_buffer.length = 0;
+ h->plusline_buffer.data[0] = 0;
h->quality_buffer.length = 0;
h->quality_buffer.data[0] = 0;
h->lineno_start = h->lineno;
- unsigned long rest = fastx_file_fill_buffer(h);
+ uint64_t rest = fastx_file_fill_buffer(h);
/* check end of file */
@@ -205,7 +207,7 @@ bool fastq_next(fastx_handle h,
rest);
/* copy to header buffer */
- unsigned long len = rest;
+ uint64_t len = rest;
if (lf)
{
/* LF found, copy up to and including LF */
@@ -219,7 +221,7 @@ bool fastq_next(fastx_handle h,
rest -= len;
}
- unsigned long lineno_seq = h->lineno;
+ uint64_t lineno_seq = h->lineno;
/* read sequence line(s) */
lf = 0;
@@ -241,7 +243,7 @@ bool fastq_next(fastx_handle h,
'\n', rest);
/* copy to sequence buffer */
- unsigned long len = rest;
+ uint64_t len = rest;
if (lf)
{
/* LF found, copy up to and including LF */
@@ -263,11 +265,9 @@ bool fastq_next(fastx_handle h,
fastq_fatal(lineno_seq, "Empty sequence line");
#endif
- unsigned long lineno_plus = h->lineno;
+ uint64_t lineno_plus = h->lineno;
/* read + line */
- fastx_buffer_s plusline_buffer;
- buffer_init(&plusline_buffer);
/* skip + character */
h->file_buffer.position++;
@@ -288,14 +288,14 @@ bool fastq_next(fastx_handle h,
'\n',
rest);
/* copy to plusline buffer */
- unsigned long len = rest;
+ uint64_t len = rest;
if (lf)
{
/* LF found, copy up to and including LF */
len = lf - (h->file_buffer.data + h->file_buffer.position) + 1;
h->lineno++;
}
- buffer_extend(& plusline_buffer,
+ buffer_extend(& h->plusline_buffer,
h->file_buffer.data + h->file_buffer.position,
len);
h->file_buffer.position += len;
@@ -305,28 +305,26 @@ bool fastq_next(fastx_handle h,
/* check that the plus line is empty or identical to @ line */
bool plusline_invalid = 0;
- if (h->header_buffer.length == plusline_buffer.length)
+ if (h->header_buffer.length == h->plusline_buffer.length)
{
if (memcmp(h->header_buffer.data,
- plusline_buffer.data,
+ h->plusline_buffer.data,
h->header_buffer.length))
plusline_invalid = 1;
}
else
{
- if ((plusline_buffer.length > 2) ||
- ((plusline_buffer.length == 2) && (plusline_buffer.data[0] != '\r')))
+ if ((h->plusline_buffer.length > 2) ||
+ ((h->plusline_buffer.length == 2) && (h->plusline_buffer.data[0] != '\r')))
plusline_invalid = 1;
}
if (plusline_invalid)
fastq_fatal(lineno_plus,
"'+' line must be empty or identical to header");
- buffer_free(&plusline_buffer);
-
/* read quality line(s) */
- unsigned long lineno_qual = h->lineno;
+ uint64_t lineno_qual = h->lineno;
lf = 0;
while (1)
@@ -349,7 +347,7 @@ bool fastq_next(fastx_handle h,
'\n', rest);
/* copy to quality buffer */
- unsigned long len = rest;
+ uint64_t len = rest;
if (lf)
{
/* LF found, copy up to and including LF */
@@ -386,37 +384,37 @@ char * fastq_get_quality(fastx_handle h)
return h->quality_buffer.data;
}
-unsigned long fastq_get_quality_length(fastx_handle h)
+uint64_t fastq_get_quality_length(fastx_handle h)
{
return h->quality_buffer.length;
}
-unsigned long fastq_get_position(fastx_handle h)
+uint64_t fastq_get_position(fastx_handle h)
{
return h->file_position;
}
-unsigned long fastq_get_size(fastx_handle h)
+uint64_t fastq_get_size(fastx_handle h)
{
return h->file_size;
}
-unsigned long fastq_get_lineno(fastx_handle h)
+uint64_t fastq_get_lineno(fastx_handle h)
{
return h->lineno_start;
}
-unsigned long fastq_get_seqno(fastx_handle h)
+uint64_t fastq_get_seqno(fastx_handle h)
{
return h->seqno;
}
-unsigned long fastq_get_header_length(fastx_handle h)
+uint64_t fastq_get_header_length(fastx_handle h)
{
return h->header_buffer.length;
}
-unsigned long fastq_get_sequence_length(fastx_handle h)
+uint64_t fastq_get_sequence_length(fastx_handle h)
{
return h->sequence_buffer.length;
}
@@ -510,12 +508,12 @@ void fastq_print_relabel(FILE * fp,
fastq_print_quality(fp, quality);
}
-long fastq_get_abundance(fastx_handle h)
+int64_t fastq_get_abundance(fastx_handle h)
{
return abundance_get(global_abundance, h->header_buffer.data);
}
-void fastq_print_db(FILE * fp, unsigned long seqno)
+void fastq_print_db(FILE * fp, uint64_t seqno)
{
char * hdr = db_getheader(seqno);
char * seq = db_getsequence(seqno);
diff --git a/src/fastq.h b/src/fastq.h
index 9fcfb58..43595d7 100644
--- a/src/fastq.h
+++ b/src/fastq.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -64,17 +64,17 @@ void fastq_close(fastx_handle h);
bool fastq_next(fastx_handle h,
bool truncateatspace,
const unsigned char * char_mapping);
-unsigned long fastq_get_position(fastx_handle h);
-unsigned long fastq_get_size(fastx_handle h);
-unsigned long fastq_get_lineno(fastx_handle h);
-unsigned long fastq_get_seqno(fastx_handle h);
+uint64_t fastq_get_position(fastx_handle h);
+uint64_t fastq_get_size(fastx_handle h);
+uint64_t fastq_get_lineno(fastx_handle h);
+uint64_t fastq_get_seqno(fastx_handle h);
char * fastq_get_header(fastx_handle h);
char * fastq_get_sequence(fastx_handle h);
char * fastq_get_quality(fastx_handle h);
-long fastq_get_abundance(fastx_handle h);
-unsigned long fastq_get_header_length(fastx_handle h);
-unsigned long fastq_get_sequence_length(fastx_handle h);
-unsigned long fastq_get_quality_length(fastx_handle h);
+int64_t fastq_get_abundance(fastx_handle h);
+uint64_t fastq_get_header_length(fastx_handle h);
+uint64_t fastq_get_sequence_length(fastx_handle h);
+uint64_t fastq_get_quality_length(fastx_handle h);
void fastq_print(FILE * fp, char * header, char * sequence, char * quality);
@@ -91,4 +91,4 @@ void fastq_print_relabel(FILE * fp,
int ordinal);
-void fastq_print_db(FILE * fp, unsigned long seqno);
+void fastq_print_db(FILE * fp, uint64_t seqno);
diff --git a/src/fastqops.cc b/src/fastqops.cc
index 93a81ac..d56bbe9 100644
--- a/src/fastqops.cc
+++ b/src/fastqops.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -67,13 +67,13 @@ int fastq_get_qual(char q)
if (qual < opt_fastq_qmin)
{
- snprintf(msg, 200, "FASTQ quality value (%d) below qmin (%ld)",
+ snprintf(msg, 200, "FASTQ quality value (%d) below qmin (%" PRId64 ")",
qual, opt_fastq_qmin);
fatal(msg);
}
else if (qual > opt_fastq_qmax)
{
- snprintf(msg, 200, "FASTQ quality value (%d) above qmax (%ld)",
+ snprintf(msg, 200, "FASTQ quality value (%d) above qmax (%" PRId64 ")",
qual, opt_fastq_qmax);
fatal(msg);
}
@@ -93,7 +93,7 @@ void filter(bool fastq_only, char * filename)
if ((opt_fastqout || opt_fastqout_discarded) && ! h->is_fastq)
fatal("Cannot write FASTQ output with a FASTA input file, lacking quality scores");
- unsigned long filesize = fastx_get_size(h);
+ uint64_t filesize = fastx_get_size(h);
FILE * fp_fastaout = 0;
FILE * fp_fastqout = 0;
@@ -128,7 +128,7 @@ void filter(bool fastq_only, char * filename)
fatal("Unable to open fastq output file for writing");
}
- unsigned long header_alloc = 0;
+ uint64_t header_alloc = 0;
char * header = 0;
if (opt_relabel)
{
@@ -138,20 +138,20 @@ void filter(bool fastq_only, char * filename)
progress_init("Reading input file", filesize);
- long kept = 0;
- long discarded = 0;
- long truncated = 0;
+ int64_t kept = 0;
+ int64_t discarded = 0;
+ int64_t truncated = 0;
char hex_md5[LEN_HEX_DIG_MD5];
char hex_sha1[LEN_HEX_DIG_SHA1];
while(fastx_next(h, 0, chrmap_no_change))
{
- long length = fastx_get_sequence_length(h);
+ int64_t length = fastx_get_sequence_length(h);
char * d = fastx_get_header(h);
char * p = fastx_get_sequence(h);
char * q = fastx_get_quality(h);
- long abundance = fastx_get_abundance(h);
+ int64_t abundance = fastx_get_abundance(h);
/* strip initial part */
if (opt_fastq_stripleft > 0)
@@ -187,7 +187,7 @@ void filter(bool fastq_only, char * filename)
double ee = 0.0;
if (h->is_fastq)
{
- for (long i = 0; i < length; i++)
+ for (int64_t i = 0; i < length; i++)
{
int qual = fastq_get_qual(q[i]);
ee += exp10(- qual / 10.0);
@@ -203,8 +203,8 @@ void filter(bool fastq_only, char * filename)
}
/* count n's */
- long ncount = 0;
- for (long i = 0; i < length; i++)
+ int64_t ncount = 0;
+ for (int64_t i = 0; i < length; i++)
{
int pc = p[i];
if ((pc == 'N') || (pc == 'n'))
@@ -222,7 +222,7 @@ void filter(bool fastq_only, char * filename)
kept++;
- if ((unsigned long)(length) < fastx_get_sequence_length(h))
+ if ((uint64_t)(length) < fastx_get_sequence_length(h))
{
truncated++;
p[length] = 0;
@@ -249,7 +249,7 @@ void filter(bool fastq_only, char * filename)
if (opt_relabel)
{
(void) snprintf(header, header_alloc,
- "%s%ld", opt_relabel, kept);
+ "%s%" PRId64, opt_relabel, kept);
d = header;
}
else if (opt_relabel_md5)
@@ -297,7 +297,7 @@ void filter(bool fastq_only, char * filename)
{
if (opt_relabel)
{
- (void) snprintf(header, header_alloc, "%s%ld", opt_relabel, discarded);
+ (void) snprintf(header, header_alloc, "%s%" PRId64, opt_relabel, discarded);
d = header;
}
else if (opt_relabel_md5)
@@ -323,13 +323,13 @@ void filter(bool fastq_only, char * filename)
progress_done();
fprintf(stderr,
- "%ld sequences kept (of which %ld truncated), %ld sequences discarded.\n",
+ "%" PRId64 " sequences kept (of which %" PRId64 " truncated), %" PRId64 " sequences discarded.\n",
kept,
truncated,
discarded);
if (header)
- free(header);
+ xfree(header);
if (opt_fastaout)
fclose(fp_fastaout);
@@ -358,10 +358,10 @@ void fastx_filter()
void fastq_chars()
{
- unsigned long sequence_chars[256];
- unsigned long quality_chars[256];
- unsigned long tail_chars[256];
- unsigned long total_chars = 0;
+ uint64_t sequence_chars[256];
+ uint64_t quality_chars[256];
+ uint64_t tail_chars[256];
+ uint64_t total_chars = 0;
int maxrun[256];
for(int c=0; c<256; c++)
@@ -374,17 +374,17 @@ void fastq_chars()
fastx_handle h = fastq_open(opt_fastq_chars);
- unsigned long filesize = fastq_get_size(h);
+ uint64_t filesize = fastq_get_size(h);
progress_init("Reading fastq file", filesize);
- unsigned long seq_count = 0;
+ uint64_t seq_count = 0;
int qmin_n = 255, qmax_n = 0;
while(fastq_next(h, 0, chrmap_upcase))
{
- long len = fastq_get_sequence_length(h);
+ int64_t len = fastq_get_sequence_length(h);
char * p = fastq_get_sequence(h);
char * q = fastq_get_quality(h);
@@ -394,7 +394,7 @@ void fastq_chars()
int run_char = -1;
int run = 0;
- long i = 0;
+ int64_t i = 0;
while(i<len)
{
int pc = *p++;
@@ -446,7 +446,7 @@ void fastq_chars()
fastq_close(h);
- fprintf(stderr, "Read %lu sequences.\n", seq_count);
+ fprintf(stderr, "Read %" PRIu64 " sequences.\n", seq_count);
char qmin = 0;
char qmax = 0;
@@ -510,7 +510,7 @@ void fastq_chars()
{
if (sequence_chars[c] > 0)
{
- fprintf(stderr, " %c %10lu %5.1f%% %6d",
+ fprintf(stderr, " %c %10" PRIu64 " %5.1f%% %6d",
c,
sequence_chars[c],
100.0 * sequence_chars[c] / total_chars,
@@ -534,7 +534,7 @@ void fastq_chars()
{
if (quality_chars[c] > 0)
{
- fprintf(stderr, " '%c' %5d %5.1f%% %10lu\n",
+ fprintf(stderr, " '%c' %5d %5.1f%% %10" PRIu64 "\n",
c,
c,
100.0 * quality_chars[c] / total_chars,
@@ -552,37 +552,37 @@ void fastq_stats()
{
fastx_handle h = fastq_open(opt_fastq_stats);
- unsigned long filesize = fastq_get_size(h);
+ uint64_t filesize = fastq_get_size(h);
progress_init("Reading fastq file", filesize);
- unsigned long seq_count = 0;
- unsigned long symbols = 0;
+ uint64_t seq_count = 0;
+ uint64_t symbols = 0;
- long read_length_alloc = 512;
+ int64_t read_length_alloc = 512;
- int * read_length_table = (int*) xmalloc(sizeof(int) * read_length_alloc);
- memset(read_length_table, 0, sizeof(int) * read_length_alloc);
+ uint64_t * read_length_table = (uint64_t*) xmalloc(sizeof(uint64_t) * read_length_alloc);
+ memset(read_length_table, 0, sizeof(uint64_t) * read_length_alloc);
- int * qual_length_table = (int*) xmalloc(sizeof(int) * read_length_alloc * 256);
- memset(qual_length_table, 0, sizeof(int) * read_length_alloc * 256);
+ uint64_t * qual_length_table = (uint64_t*) xmalloc(sizeof(uint64_t) * read_length_alloc * 256);
+ memset(qual_length_table, 0, sizeof(uint64_t) * read_length_alloc * 256);
- int * ee_length_table = (int *) xmalloc(sizeof(int) * read_length_alloc * 4);
- memset(ee_length_table, 0, sizeof(int) * read_length_alloc * 4);
+ uint64_t * ee_length_table = (uint64_t *) xmalloc(sizeof(uint64_t) * read_length_alloc * 4);
+ memset(ee_length_table, 0, sizeof(uint64_t) * read_length_alloc * 4);
- int * q_length_table = (int *) xmalloc(sizeof(int) * read_length_alloc * 4);
- memset(q_length_table, 0, sizeof(int) * read_length_alloc * 4);
+ uint64_t * q_length_table = (uint64_t *) xmalloc(sizeof(uint64_t) * read_length_alloc * 4);
+ memset(q_length_table, 0, sizeof(uint64_t) * read_length_alloc * 4);
double * sumee_length_table = (double *) xmalloc(sizeof(double) * read_length_alloc);
memset(sumee_length_table, 0, sizeof(double) * read_length_alloc);
- long len_min = LONG_MAX;
- long len_max = 0;
+ int64_t len_min = LONG_MAX;
+ int64_t len_max = 0;
int qmin = +1000;
int qmax = -1000;
- unsigned long quality_chars[256];
+ uint64_t quality_chars[256];
for(int c=0; c<256; c++)
quality_chars[c] = 0;
@@ -590,32 +590,32 @@ void fastq_stats()
{
seq_count++;
- long len = fastq_get_sequence_length(h);
+ int64_t len = fastq_get_sequence_length(h);
char * q = fastq_get_quality(h);
/* update length statistics */
if (len+1 > read_length_alloc)
{
- read_length_table = (int*) xrealloc(read_length_table,
- sizeof(int) * (len+1));
+ read_length_table = (uint64_t*) xrealloc(read_length_table,
+ sizeof(uint64_t) * (len+1));
memset(read_length_table + read_length_alloc, 0,
- sizeof(int) * (len + 1 - read_length_alloc));
+ sizeof(uint64_t) * (len + 1 - read_length_alloc));
- qual_length_table = (int*) xrealloc(qual_length_table,
- sizeof(int) * (len+1) * 256);
+ qual_length_table = (uint64_t*) xrealloc(qual_length_table,
+ sizeof(uint64_t) * (len+1) * 256);
memset(qual_length_table + 256 * read_length_alloc, 0,
- sizeof(int) * (len + 1 - read_length_alloc) * 256);
+ sizeof(uint64_t) * (len + 1 - read_length_alloc) * 256);
- ee_length_table = (int*) xrealloc(ee_length_table,
- sizeof(int) * (len+1) * 4);
+ ee_length_table = (uint64_t*) xrealloc(ee_length_table,
+ sizeof(uint64_t) * (len+1) * 4);
memset(ee_length_table + 4 * read_length_alloc, 0,
- sizeof(int) * (len + 1 - read_length_alloc) * 4);
+ sizeof(uint64_t) * (len + 1 - read_length_alloc) * 4);
- q_length_table = (int*) xrealloc(q_length_table,
- sizeof(int) * (len+1) * 4);
+ q_length_table = (uint64_t*) xrealloc(q_length_table,
+ sizeof(uint64_t) * (len+1) * 4);
memset(q_length_table + 4 * read_length_alloc, 0,
- sizeof(int) * (len + 1 - read_length_alloc) * 4);
+ sizeof(uint64_t) * (len + 1 - read_length_alloc) * 4);
sumee_length_table = (double *) xrealloc(sumee_length_table,
sizeof(double) * (len+1));
@@ -640,7 +640,7 @@ void fastq_stats()
double ee = 0.0;
int qmin_this = 1000;
- for(long i=0; i < len; i++)
+ for(int64_t i=0; i < len; i++)
{
int qc = q[i];
@@ -648,8 +648,8 @@ void fastq_stats()
if ((qual < opt_fastq_qmin) || (qual > opt_fastq_qmax))
{
char * msg;
- if (asprintf(& msg,
-"FASTQ quality value (%d) out of range (%ld-%ld).\n"
+ if (xsprintf(& msg,
+"FASTQ quality value (%d) out of range (%" PRId64 "-%" PRId64 ").\n"
"Please adjust the FASTQ quality base character or range with the\n"
"--fastq_ascii, --fastq_qmin or --fastq_qmax options. For a complete\n"
"diagnosis with suggested values, please run vsearch --fastq_chars file.",
@@ -657,7 +657,7 @@ void fastq_stats()
fatal(msg);
else
fatal("Out of memory");
- free(msg);
+ xfree(msg);
}
quality_chars[qc]++;
@@ -698,18 +698,18 @@ void fastq_stats()
/* compute various distributions */
- int * length_dist = (int*) xmalloc(sizeof(int) * (len_max+1));
- long * symb_dist = (long*) xmalloc(sizeof(long) * (len_max+1));
+ uint64_t * length_dist = (uint64_t*) xmalloc(sizeof(uint64_t) * (len_max+1));
+ int64_t * symb_dist = (int64_t*) xmalloc(sizeof(int64_t) * (len_max+1));
double * rate_dist = (double*) xmalloc(sizeof(double) * (len_max+1));
double * avgq_dist = (double*) xmalloc(sizeof(double) * (len_max+1));
double * avgee_dist = (double*) xmalloc(sizeof(double) * (len_max+1));
double * avgp_dist = (double*) xmalloc(sizeof(double) * (len_max+1));
- long length_accum = 0;
- long symb_accum = 0;
+ int64_t length_accum = 0;
+ int64_t symb_accum = 0;
- for(long i = 0; i <= len_max; i++)
+ for(int64_t i = 0; i <= len_max; i++)
{
length_accum += read_length_table[i];
length_dist[i] = length_accum;
@@ -717,8 +717,8 @@ void fastq_stats()
symb_accum += seq_count - length_accum;
symb_dist[i] = symb_accum;
- long q = 0;
- long x = 0;
+ int64_t q = 0;
+ int64_t x = 0;
double e_sum = 0.0;
for(int c=qmin; c<=qmax; c++)
{
@@ -741,10 +741,10 @@ void fastq_stats()
fprintf(fp_log, " L N Pct AccPct\n");
fprintf(fp_log, "------- ---------- ------- -------\n");
- for(long i = len_max; i >= len_min; i--)
+ for(int64_t i = len_max; i >= len_min; i--)
{
if (read_length_table[i] > 0)
- fprintf(fp_log, "%2s%5ld %10d %5.1lf%% %5.1lf%%\n",
+ fprintf(fp_log, "%2s%5" PRId64 " %10" PRIu64 " %5.1lf%% %5.1lf%%\n",
(i == len_max ? ">=" : " "),
i,
read_length_table[i],
@@ -757,14 +757,14 @@ void fastq_stats()
fprintf(fp_log, "ASCII Q Pe N Pct AccPct\n");
fprintf(fp_log, "----- --- ------- ---------- ------- -------\n");
- long qual_accum = 0;
+ int64_t qual_accum = 0;
for(int c = qmax ; c >= qmin ; c--)
{
if (quality_chars[c] > 0)
{
qual_accum += quality_chars[c];
fprintf(fp_log,
- " %c %3ld %7.5lf %10lu %6.1lf%% %6.1lf%%\n",
+ " %c %3" PRId64 " %7.5lf %10" PRIu64 " %6.1lf%% %6.1lf%%\n",
c,
c - opt_fastq_ascii,
q2p(c - opt_fastq_ascii),
@@ -778,7 +778,7 @@ void fastq_stats()
fprintf(fp_log, " L PctRecs AvgQ P(AvgQ) AvgP AvgEE Rate RatePct\n");
fprintf(fp_log, "----- ------- ---- ------- -------- ----- --------- --------\n");
- for(long i = 2; i <= len_max; i++)
+ for(int64_t i = 2; i <= len_max; i++)
{
double PctRecs = 100.0 * (seq_count - length_dist[i-1]) / seq_count;
double AvgQ = avgq_dist[i-1];
@@ -787,7 +787,7 @@ void fastq_stats()
double Rate = rate_dist[i-1];
fprintf(fp_log,
- "%5ld %6.1lf%% %4.1lf %7.5lf %8.6lf %5.2lf %9.6lf %7.3lf%%\n",
+ "%5" PRId64 " %6.1lf%% %4.1lf %7.5lf %8.6lf %5.2lf %9.6lf %7.3lf%%\n",
i,
PctRecs,
AvgQ,
@@ -802,9 +802,9 @@ void fastq_stats()
fprintf(fp_log, " L 1.0000 0.5000 0.2500 0.1000 1.0000 0.5000 0.2500 0.1000\n");
fprintf(fp_log, "----- ------- ------- ------- ------- ------- ------- ------- -------\n");
- for(long i = len_max+1; i >= 1; i--)
+ for(int64_t i = len_max+1; i >= 1; i--)
{
- long read_count[4];
+ int64_t read_count[4];
double read_percentage[4];
for(int z=0; z<4; z++)
@@ -819,7 +819,7 @@ void fastq_stats()
if (read_count[0] > 0)
{
fprintf(fp_log,
- "%5ld %7ld %7ld %7ld %7ld "
+ "%5" PRId64 " %7" PRId64 " %7" PRId64 " %7" PRId64 " %7" PRId64 " "
"%6.2lf%% %6.2lf%% %6.2lf%% %6.2lf%%\n",
i,
read_count[0], read_count[1],
@@ -835,54 +835,54 @@ void fastq_stats()
fprintf(fp_log, " Len Q=5 Q=10 Q=15 Q=20\n");
fprintf(fp_log, "----- ------ ------ ------ ------\n");
- for(long i = len_max; i >= len_max/2; i--)
+ for(int64_t i = len_max; i >= len_max/2; i--)
{
double read_percentage[4];
for(int z=0; z<4; z++)
read_percentage[z] = 100.0 * q_length_table[4*(i-1)+z] / seq_count;
- fprintf(fp_log, "%5ld %5.1lf%% %5.1lf%% %5.1lf%% %5.1lf%%\n",
+ fprintf(fp_log, "%5" PRId64 " %5.1lf%% %5.1lf%% %5.1lf%% %5.1lf%%\n",
i,
read_percentage[0], read_percentage[1],
read_percentage[2], read_percentage[3]);
}
fprintf(fp_log, "\n");
- fprintf(fp_log, "%10lu Recs (%.1lfM), 0 too long\n",
+ fprintf(fp_log, "%10" PRIu64 " Recs (%.1lfM), 0 too long\n",
seq_count, seq_count / 1.0e6);
fprintf(fp_log, "%10.1lf Avg length\n", 1.0 * symbols / seq_count);
fprintf(fp_log, "%9.1lfM Bases\n", symbols / 1.0e6);
}
- free(read_length_table);
- free(qual_length_table);
- free(ee_length_table);
- free(q_length_table);
- free(sumee_length_table);
-
- free(length_dist);
- free(symb_dist);
- free(rate_dist);
- free(avgq_dist);
- free(avgee_dist);
- free(avgp_dist);
+ xfree(read_length_table);
+ xfree(qual_length_table);
+ xfree(ee_length_table);
+ xfree(q_length_table);
+ xfree(sumee_length_table);
+
+ xfree(length_dist);
+ xfree(symb_dist);
+ xfree(rate_dist);
+ xfree(avgq_dist);
+ xfree(avgee_dist);
+ xfree(avgp_dist);
fastq_close(h);
- fprintf(stderr, "Read %lu sequences.\n", seq_count);
+ fprintf(stderr, "Read %" PRIu64 " sequences.\n", seq_count);
}
void fastx_revcomp()
{
- unsigned long buffer_alloc = 512;
+ uint64_t buffer_alloc = 512;
char * seq_buffer = (char*) xmalloc(buffer_alloc);
char * qual_buffer = (char*) xmalloc(buffer_alloc);
- unsigned long header_alloc = 512;
+ uint64_t header_alloc = 512;
char * header = (char*) xmalloc(header_alloc);
- unsigned long suffix_length = opt_label_suffix ? strlen(opt_label_suffix) : 0;
+ uint64_t suffix_length = opt_label_suffix ? strlen(opt_label_suffix) : 0;
fastx_handle h = fastx_open(opt_fastx_revcomp);
@@ -892,7 +892,7 @@ void fastx_revcomp()
if (opt_fastqout && ! h->is_fastq)
fatal("Cannot write FASTQ output with a FASTA input file, lacking quality scores");
- unsigned long filesize = fastx_get_size(h);
+ uint64_t filesize = fastx_get_size(h);
FILE * fp_fastaout = 0;
FILE * fp_fastqout = 0;
@@ -920,7 +920,7 @@ void fastx_revcomp()
{
/* header */
- unsigned long hlen = fastx_get_header_length(h);
+ uint64_t hlen = fastx_get_header_length(h);
if (hlen + suffix_length + 1 > header_alloc)
{
@@ -938,7 +938,7 @@ void fastx_revcomp()
/* sequence */
- unsigned long length = fastx_get_sequence_length(h);
+ uint64_t length = fastx_get_sequence_length(h);
if (length + 1 > buffer_alloc)
{
@@ -958,7 +958,7 @@ void fastx_revcomp()
if (fastx_is_fastq(h))
{
/* reverse quality values */
- for(unsigned long i=0; i<length; i++)
+ for(uint64_t i=0; i<length; i++)
qual_buffer[i] = q[length-1-i];
qual_buffer[length] = 0;
}
@@ -981,9 +981,9 @@ void fastx_revcomp()
fastx_close(h);
- free(header);
- free(seq_buffer);
- free(qual_buffer);
+ xfree(header);
+ xfree(seq_buffer);
+ xfree(qual_buffer);
}
void fastq_convert()
@@ -993,7 +993,7 @@ void fastq_convert()
if (!h)
fatal("Unable to open FASTQ file");
- unsigned long filesize = fastq_get_size(h);
+ uint64_t filesize = fastq_get_size(h);
FILE * fp_fastqout = 0;
@@ -1011,19 +1011,19 @@ void fastq_convert()
/* sequence */
- unsigned long length = fastq_get_sequence_length(h);
+ uint64_t length = fastq_get_sequence_length(h);
char * sequence = fastq_get_sequence(h);
/* convert quality values */
char * quality = fastq_get_quality(h);
- for(unsigned long i=0; i<length; i++)
+ for(uint64_t i=0; i<length; i++)
{
int q = quality[i] - opt_fastq_ascii;
if (q < opt_fastq_qmin)
{
fprintf(stderr,
- "\nFASTQ quality score (%d) below minimum (%ld) in entry no %lu starting on line %lu\n",
+ "\nFASTQ quality score (%d) below minimum (%" PRId64 ") in entry no %" PRIu64 " starting on line %" PRIu64 "\n",
q,
opt_fastq_qmin,
fastq_get_seqno(h) + 1,
@@ -1033,7 +1033,7 @@ void fastq_convert()
if (q > opt_fastq_qmax)
{
fprintf(stderr,
- "\nFASTQ quality score (%d) above maximum (%ld) in entry no %lu starting on line %lu\n",
+ "\nFASTQ quality score (%d) above maximum (%" PRId64 ") in entry no %" PRIu64 " starting on line %" PRIu64 "\n",
q,
opt_fastq_qmax,
fastq_get_seqno(h) + 1,
diff --git a/src/fastqops.h b/src/fastqops.h
index 8936c0d..024209d 100644
--- a/src/fastqops.h
+++ b/src/fastqops.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/fastx.cc b/src/fastx.cc
index c64658b..eadb29c 100644
--- a/src/fastx.cc
+++ b/src/fastx.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -95,14 +95,14 @@ void buffer_init(struct fastx_buffer_s * buffer)
void buffer_free(struct fastx_buffer_s * buffer)
{
if (buffer->data)
- free(buffer->data);
+ xfree(buffer->data);
buffer->data = 0;
buffer->alloc = 0;
buffer->length = 0;
buffer->position = 0;
}
-void buffer_makespace(struct fastx_buffer_s * buffer, unsigned long x)
+void buffer_makespace(struct fastx_buffer_s * buffer, uint64_t x)
{
/* make sure there is space for x more chars in buffer */
@@ -133,7 +133,7 @@ void buffer_truncate(struct fastx_buffer_s * buffer, bool truncateatspace)
void buffer_extend(struct fastx_buffer_s * dest_buffer,
char * source_buf,
- unsigned long len)
+ uint64_t len)
{
buffer_makespace(dest_buffer, len+1);
memcpy(dest_buffer->data + dest_buffer->length,
@@ -170,7 +170,7 @@ fastx_handle fastx_open(const char * filename)
if (fstat(fileno(h->fp), & fs))
fatal("Unable to fstat on input file (%s)", filename);
- h->is_pipe = ! S_ISREG(fs.st_mode);
+ h->is_pipe = S_ISFIFO(fs.st_mode);
h->file_size = 0;
@@ -256,7 +256,7 @@ fastx_handle fastx_open(const char * filename)
/* start filling up file buffer */
- unsigned long rest = fastx_file_fill_buffer(h);
+ uint64_t rest = fastx_file_fill_buffer(h);
if (rest < 2)
fatal("File too small");
@@ -323,6 +323,7 @@ fastx_handle fastx_open(const char * filename)
buffer_init(& h->header_buffer);
buffer_init(& h->sequence_buffer);
+ buffer_init(& h->plusline_buffer);
buffer_init(& h->quality_buffer);
h->stripped_all = 0;
@@ -351,7 +352,7 @@ void fastx_close(fastx_handle h)
fprintf(stderr, "WARNING: invalid characters stripped from fastq file:");
for (int i=0; i<256;i++)
if (h->stripped[i])
- fprintf(stderr, " %c(%lu)", i, h->stripped[i]);
+ fprintf(stderr, " %c(%" PRIu64 ")", i, h->stripped[i]);
fprintf(stderr, "\n");
if (opt_log)
@@ -359,7 +360,7 @@ void fastx_close(fastx_handle h)
fprintf(fp_log, "WARNING: invalid characters stripped from fastq file:");
for (int i=0; i<256;i++)
if (h->stripped[i])
- fprintf(fp_log, " %c(%lu)", i, h->stripped[i]);
+ fprintf(fp_log, " %c(%" PRIu64 ")", i, h->stripped[i]);
fprintf(fp_log, "\n");
}
}
@@ -397,6 +398,7 @@ void fastx_close(fastx_handle h)
buffer_free(& h->file_buffer);
buffer_free(& h->header_buffer);
buffer_free(& h->sequence_buffer);
+ buffer_free(& h->plusline_buffer);
buffer_free(& h->quality_buffer);
h->file_size = 0;
@@ -405,20 +407,20 @@ void fastx_close(fastx_handle h)
h->lineno = 0;
h->seqno = -1;
- free(h);
+ xfree(h);
h=0;
}
-unsigned long fastx_file_fill_buffer(fastx_handle h)
+uint64_t fastx_file_fill_buffer(fastx_handle h)
{
/* read more data if necessary */
- unsigned long rest = h->file_buffer.length - h->file_buffer.position;
+ uint64_t rest = h->file_buffer.length - h->file_buffer.position;
if (rest > 0)
return rest;
else
{
- unsigned long space = h->file_buffer.alloc - h->file_buffer.length;
+ uint64_t space = h->file_buffer.alloc - h->file_buffer.length;
if (space == 0)
{
@@ -504,7 +506,7 @@ bool fastx_next(fastx_handle h,
return fasta_next(h, truncateatspace, char_mapping);
}
-unsigned long fastx_get_position(fastx_handle h)
+uint64_t fastx_get_position(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_position(h);
@@ -513,7 +515,7 @@ unsigned long fastx_get_position(fastx_handle h)
}
-unsigned long fastx_get_size(fastx_handle h)
+uint64_t fastx_get_size(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_size(h);
@@ -522,7 +524,7 @@ unsigned long fastx_get_size(fastx_handle h)
}
-unsigned long fastx_get_lineno(fastx_handle h)
+uint64_t fastx_get_lineno(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_lineno(h);
@@ -531,7 +533,7 @@ unsigned long fastx_get_lineno(fastx_handle h)
}
-unsigned long fastx_get_seqno(fastx_handle h)
+uint64_t fastx_get_seqno(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_seqno(h);
@@ -555,7 +557,7 @@ char * fastx_get_sequence(fastx_handle h)
return fasta_get_sequence(h);
}
-unsigned long fastx_get_header_length(fastx_handle h)
+uint64_t fastx_get_header_length(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_header_length(h);
@@ -563,7 +565,7 @@ unsigned long fastx_get_header_length(fastx_handle h)
return fasta_get_header_length(h);
}
-unsigned long fastx_get_sequence_length(fastx_handle h)
+uint64_t fastx_get_sequence_length(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_sequence_length(h);
@@ -580,7 +582,7 @@ char * fastx_get_quality(fastx_handle h)
return 0;
}
-long fastx_get_abundance(fastx_handle h)
+int64_t fastx_get_abundance(fastx_handle h)
{
if (h->is_fastq)
return fastq_get_abundance(h);
diff --git a/src/fastx.h b/src/fastx.h
index ff5815d..bda8cc8 100644
--- a/src/fastx.h
+++ b/src/fastx.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,17 +61,17 @@
struct fastx_buffer_s
{
char * data;
- unsigned long length;
- unsigned long alloc;
- unsigned long position;
+ uint64_t length;
+ uint64_t alloc;
+ uint64_t position;
};
void buffer_init(struct fastx_buffer_s * buffer);
void buffer_free(struct fastx_buffer_s * buffer);
void buffer_extend(struct fastx_buffer_s * dest_buffer,
char * source_buf,
- unsigned long len);
-void buffer_makespace(struct fastx_buffer_s * buffer, unsigned long x);
+ uint64_t len);
+void buffer_makespace(struct fastx_buffer_s * buffer, uint64_t x);
void buffer_truncate(struct fastx_buffer_s * buffer, bool truncateatspace);
struct fastx_s
@@ -93,17 +93,18 @@ struct fastx_s
struct fastx_buffer_s header_buffer;
struct fastx_buffer_s sequence_buffer;
+ struct fastx_buffer_s plusline_buffer;
struct fastx_buffer_s quality_buffer;
- unsigned long file_size;
- unsigned long file_position;
+ uint64_t file_size;
+ uint64_t file_position;
- unsigned long lineno;
- unsigned long lineno_start;
- long seqno;
+ uint64_t lineno;
+ uint64_t lineno_start;
+ int64_t seqno;
- unsigned long stripped_all;
- unsigned long stripped[256];
+ uint64_t stripped_all;
+ uint64_t stripped[256];
int format;
};
@@ -119,16 +120,16 @@ void fastx_close(fastx_handle h);
bool fastx_next(fastx_handle h,
bool truncateatspace,
const unsigned char * char_mapping);
-unsigned long fastx_get_position(fastx_handle h);
-unsigned long fastx_get_size(fastx_handle h);
-unsigned long fastx_get_lineno(fastx_handle h);
-unsigned long fastx_get_seqno(fastx_handle h);
+uint64_t fastx_get_position(fastx_handle h);
+uint64_t fastx_get_size(fastx_handle h);
+uint64_t fastx_get_lineno(fastx_handle h);
+uint64_t fastx_get_seqno(fastx_handle h);
char * fastx_get_header(fastx_handle h);
char * fastx_get_sequence(fastx_handle h);
-unsigned long fastx_get_header_length(fastx_handle h);
-unsigned long fastx_get_sequence_length(fastx_handle h);
+uint64_t fastx_get_header_length(fastx_handle h);
+uint64_t fastx_get_sequence_length(fastx_handle h);
char * fastx_get_quality(fastx_handle h);
-long fastx_get_abundance(fastx_handle h);
+int64_t fastx_get_abundance(fastx_handle h);
-unsigned long fastx_file_fill_buffer(fastx_handle h);
+uint64_t fastx_file_fill_buffer(fastx_handle h);
diff --git a/src/linmemalign.cc b/src/linmemalign.cc
index 146227c..e9d291e 100644
--- a/src/linmemalign.cc
+++ b/src/linmemalign.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -72,7 +72,7 @@
The method has been adapted for the use of different
gap penalties for query/target/left/interior/right gaps.
- scorematrix consists of 16x16 long integers
+ scorematrix consists of 16x16 int64_t integers
Sequences and alignment matrix:
A/a/i/query/q/downwards/vertical/top/bottom
@@ -100,25 +100,25 @@ LinearMemoryAligner::LinearMemoryAligner()
LinearMemoryAligner::~LinearMemoryAligner()
{
if (cigar_string)
- free(cigar_string);
+ xfree(cigar_string);
if (HH)
- free(HH);
+ xfree(HH);
if (EE)
- free(EE);
+ xfree(EE);
if (XX)
- free(XX);
+ xfree(XX);
if (YY)
- free(YY);
+ xfree(YY);
}
-long * LinearMemoryAligner::scorematrix_create(long match, long mismatch)
+int64_t * LinearMemoryAligner::scorematrix_create(int64_t match, int64_t mismatch)
{
- long * newscorematrix = (long*) xmalloc(16*16*sizeof(long));
+ int64_t * newscorematrix = (int64_t*) xmalloc(16*16*sizeof(int64_t));
for(int i=0; i<16; i++)
for(int j=0; j<16; j++)
{
- long value;
+ int64_t value;
if ((i==0) || (j==0) || (i>4) || (j>4))
value = 0;
else if (i==j)
@@ -137,18 +137,18 @@ void LinearMemoryAligner::alloc_vectors(size_t x)
vector_alloc = x;
if (HH)
- free(HH);
+ xfree(HH);
if (EE)
- free(EE);
+ xfree(EE);
if (XX)
- free(XX);
+ xfree(XX);
if (YY)
- free(YY);
+ xfree(YY);
- HH = (long*) xmalloc(vector_alloc * (sizeof(long)));
- EE = (long*) xmalloc(vector_alloc * (sizeof(long)));
- XX = (long*) xmalloc(vector_alloc * (sizeof(long)));
- YY = (long*) xmalloc(vector_alloc * (sizeof(long)));
+ HH = (int64_t*) xmalloc(vector_alloc * (sizeof(int64_t)));
+ EE = (int64_t*) xmalloc(vector_alloc * (sizeof(int64_t)));
+ XX = (int64_t*) xmalloc(vector_alloc * (sizeof(int64_t)));
+ YY = (int64_t*) xmalloc(vector_alloc * (sizeof(int64_t)));
}
}
@@ -173,12 +173,12 @@ void LinearMemoryAligner::cigar_flush()
{
/* try writing string until enough memory has been allocated */
- long rest = cigar_alloc - cigar_length;
+ int64_t rest = cigar_alloc - cigar_length;
int n;
if (op_run > 1)
n = snprintf(cigar_string + cigar_length,
rest,
- "%ld%c", op_run, op);
+ "%" PRId64 "%c", op_run, op);
else
n = snprintf(cigar_string + cigar_length,
rest,
@@ -201,7 +201,7 @@ void LinearMemoryAligner::cigar_flush()
}
}
-void LinearMemoryAligner::cigar_add(char _op, long run)
+void LinearMemoryAligner::cigar_add(char _op, int64_t run)
{
if (op == _op)
op_run += run;
@@ -219,15 +219,15 @@ void LinearMemoryAligner::show_matrix()
{
printf("%2d:", i);
for(int j=0; j<16; j++)
- printf(" %2ld", scorematrix[16*i+j]);
+ printf(" %2" PRId64, scorematrix[16*i+j]);
printf("\n");
}
}
-void LinearMemoryAligner::diff(long a_start,
- long b_start,
- long a_len,
- long b_len,
+void LinearMemoryAligner::diff(int64_t a_start,
+ int64_t b_start,
+ int64_t a_len,
+ int64_t b_len,
bool gap_b_left, /* gap open left of b */
bool gap_b_right, /* gap open right of b */
bool a_left, /* includes left end of a */
@@ -265,10 +265,10 @@ void LinearMemoryAligner::diff(long a_start,
*/
- long MaxScore;
- long best;
+ int64_t MaxScore;
+ int64_t best;
- long Score = 0;
+ int64_t Score = 0;
/* First possibility */
@@ -317,7 +317,7 @@ void LinearMemoryAligner::diff(long a_start,
/* Third possibility */
- for (long j = 0; j < b_len; j++)
+ for (int64_t j = 0; j < b_len; j++)
{
// Insert zero or more from B, replace 1, insert rest of B
// -A--
@@ -365,8 +365,8 @@ void LinearMemoryAligner::diff(long a_start,
{
/* a_len >= 2, b_len >= 1 */
- long I = a_len / 2;
- long i, j;
+ int64_t I = a_len / 2;
+ int64_t i, j;
// Compute HH & EE in forward phase
// Upper part
@@ -388,14 +388,14 @@ void LinearMemoryAligner::diff(long a_start,
for (i = 1; i <= I; i++)
{
- long p = HH[0];
+ int64_t p = HH[0];
- long h = - (b_left ?
+ int64_t h = - (b_left ?
(gap_b_left ? 0 : go_t_l) + i * ge_t_l :
(gap_b_left ? 0 : go_t_i) + i * ge_t_i);
HH[0] = h;
- long f = LONG_MIN;
+ int64_t f = LONG_MIN;
for (j = 1; j <= b_len; j++)
{
@@ -436,13 +436,13 @@ void LinearMemoryAligner::diff(long a_start,
for (i = 1; i <= a_len - I; i++)
{
- long p = XX[0];
+ int64_t p = XX[0];
- long h = - (b_right ?
+ int64_t h = - (b_right ?
(gap_b_right ? 0 : go_t_r) + i * ge_t_r :
(gap_b_right ? 0 : go_t_i) + i * ge_t_i);
XX[0] = h;
- long f = LONG_MIN;
+ int64_t f = LONG_MIN;
for (j = 1; j <= b_len; j++)
{
@@ -468,14 +468,14 @@ void LinearMemoryAligner::diff(long a_start,
/* find maximum score along division line */
- long MaxScore0 = LONG_MIN;
- long best0 = -1;
+ int64_t MaxScore0 = LONG_MIN;
+ int64_t best0 = -1;
/* solutions with diagonal at break */
for (j=0; j <= b_len; j++)
{
- long Score = HH[j] + XX[b_len - j];
+ int64_t Score = HH[j] + XX[b_len - j];
if (Score > MaxScore0)
{
@@ -484,14 +484,14 @@ void LinearMemoryAligner::diff(long a_start,
}
}
- long MaxScore1 = LONG_MIN;
- long best1 = -1;
+ int64_t MaxScore1 = LONG_MIN;
+ int64_t best1 = -1;
/* solutions that end with a gap in b from both ends at break */
for (j=0; j <= b_len; j++)
{
- long g;
+ int64_t g;
if (b_left && (j==0))
g = go_t_l;
else if (b_right && (j==b_len))
@@ -499,7 +499,7 @@ void LinearMemoryAligner::diff(long a_start,
else
g = go_t_i;
- long Score = EE[j] + YY[b_len - j] + g;
+ int64_t Score = EE[j] + YY[b_len - j] + g;
if (Score > MaxScore1)
{
@@ -508,8 +508,8 @@ void LinearMemoryAligner::diff(long a_start,
}
}
- long P;
- long best;
+ int64_t P;
+ int64_t best;
if (MaxScore0 > MaxScore1)
{
@@ -570,19 +570,19 @@ void LinearMemoryAligner::diff(long a_start,
}
}
-void LinearMemoryAligner::set_parameters(long * _scorematrix,
- long _gap_open_query_left,
- long _gap_open_target_left,
- long _gap_open_query_interior,
- long _gap_open_target_interior,
- long _gap_open_query_right,
- long _gap_open_target_right,
- long _gap_extension_query_left,
- long _gap_extension_target_left,
- long _gap_extension_query_interior,
- long _gap_extension_target_interior,
- long _gap_extension_query_right,
- long _gap_extension_target_right)
+void LinearMemoryAligner::set_parameters(int64_t * _scorematrix,
+ int64_t _gap_open_query_left,
+ int64_t _gap_open_target_left,
+ int64_t _gap_open_query_interior,
+ int64_t _gap_open_target_interior,
+ int64_t _gap_open_query_right,
+ int64_t _gap_open_target_right,
+ int64_t _gap_extension_query_left,
+ int64_t _gap_extension_target_left,
+ int64_t _gap_extension_query_interior,
+ int64_t _gap_extension_target_interior,
+ int64_t _gap_extension_query_right,
+ int64_t _gap_extension_target_right)
{
scorematrix = _scorematrix;
@@ -609,8 +609,8 @@ void LinearMemoryAligner::set_parameters(long * _scorematrix,
char * LinearMemoryAligner::align(char * _a_seq,
char * _b_seq,
- long a_len,
- long b_len)
+ int64_t a_len,
+ int64_t b_len)
{
/* copy parameters */
a_seq = _a_seq;
@@ -635,39 +635,39 @@ char * LinearMemoryAligner::align(char * _a_seq,
void LinearMemoryAligner::alignstats(char * cigar,
char * _a_seq,
char * _b_seq,
- long * _nwscore,
- long * _nwalignmentlength,
- long * _nwmatches,
- long * _nwmismatches,
- long * _nwgaps)
+ int64_t * _nwscore,
+ int64_t * _nwalignmentlength,
+ int64_t * _nwmatches,
+ int64_t * _nwmismatches,
+ int64_t * _nwgaps)
{
a_seq = _a_seq;
b_seq = _b_seq;
- long nwscore = 0;
- long nwalignmentlength = 0;
- long nwmatches = 0;
- long nwmismatches = 0;
- long nwgaps = 0;
+ int64_t nwscore = 0;
+ int64_t nwalignmentlength = 0;
+ int64_t nwmatches = 0;
+ int64_t nwmismatches = 0;
+ int64_t nwgaps = 0;
- long a_pos = 0;
- long b_pos = 0;
+ int64_t a_pos = 0;
+ int64_t b_pos = 0;
char * p = cigar;
- long g;
+ int64_t g;
while (*p)
{
- long run = 1;
+ int64_t run = 1;
int scanlength = 0;
- sscanf(p, "%ld%n", &run, &scanlength);
+ sscanf(p, "%" PRId64 "%n", &run, &scanlength);
p += scanlength;
switch (*p++)
{
case 'M':
nwalignmentlength += run;
- for(long k=0; k<run; k++)
+ for(int64_t k=0; k<run; k++)
{
nwscore += subst_score(a_pos, b_pos);
diff --git a/src/linmemalign.h b/src/linmemalign.h
index 438f118..400df42 100644
--- a/src/linmemalign.h
+++ b/src/linmemalign.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,47 +61,47 @@
class LinearMemoryAligner
{
char op;
- long op_run;
- long cigar_alloc;
- long cigar_length;
+ int64_t op_run;
+ int64_t cigar_alloc;
+ int64_t cigar_length;
char * cigar_string;
char * a_seq;
char * b_seq;
- long * scorematrix;
+ int64_t * scorematrix;
- long q;
- long r;
+ int64_t q;
+ int64_t r;
/* gap penalties for open/extension query/target left/interior/right */
- long go_q_l;
- long go_t_l;
- long go_q_i;
- long go_t_i;
- long go_q_r;
- long go_t_r;
- long ge_q_l;
- long ge_t_l;
- long ge_q_i;
- long ge_t_i;
- long ge_q_r;
- long ge_t_r;
+ int64_t go_q_l;
+ int64_t go_t_l;
+ int64_t go_q_i;
+ int64_t go_t_i;
+ int64_t go_q_r;
+ int64_t go_t_r;
+ int64_t ge_q_l;
+ int64_t ge_t_l;
+ int64_t ge_q_i;
+ int64_t ge_t_i;
+ int64_t ge_q_r;
+ int64_t ge_t_r;
size_t vector_alloc;
- long * HH;
- long * EE;
- long * XX;
- long * YY;
+ int64_t * HH;
+ int64_t * EE;
+ int64_t * XX;
+ int64_t * YY;
void cigar_reset();
void cigar_flush();
- void cigar_add(char _op, long run);
+ void cigar_add(char _op, int64_t run);
- inline long subst_score(long x, long y)
+ inline int64_t subst_score(int64_t x, int64_t y)
{
/* return substitution score for replacing symbol at position x in a
with symbol at position y in b */
@@ -109,10 +109,10 @@ class LinearMemoryAligner
chrmap_4bit[(int)(a_seq[x])]];
}
- void diff(long a_start,
- long b_start,
- long a_len,
- long b_len,
+ void diff(int64_t a_start,
+ int64_t b_start,
+ int64_t a_len,
+ int64_t b_len,
bool gap_b_left, /* gap open left of b */
bool gap_b_right, /* gap open right of b */
bool a_left, /* includes left end of a */
@@ -130,34 +130,34 @@ public:
~LinearMemoryAligner();
- long * scorematrix_create(long match, long mismatch);
+ int64_t * scorematrix_create(int64_t match, int64_t mismatch);
- void set_parameters(long * _scorematrix,
- long _gap_open_query_left,
- long _gap_open_target_left,
- long _gap_open_query_interior,
- long _gap_open_target_interior,
- long _gap_open_query_right,
- long _gap_open_target_right,
- long _gap_extension_query_left,
- long _gap_extension_target_left,
- long _gap_extension_query_interior,
- long _gap_extension_target_interior,
- long _gap_extension_query_right,
- long _gap_extension_target_right);
+ void set_parameters(int64_t * _scorematrix,
+ int64_t _gap_open_query_left,
+ int64_t _gap_open_target_left,
+ int64_t _gap_open_query_interior,
+ int64_t _gap_open_target_interior,
+ int64_t _gap_open_query_right,
+ int64_t _gap_open_target_right,
+ int64_t _gap_extension_query_left,
+ int64_t _gap_extension_target_left,
+ int64_t _gap_extension_query_interior,
+ int64_t _gap_extension_target_interior,
+ int64_t _gap_extension_query_right,
+ int64_t _gap_extension_target_right);
char * align(char * _a_seq,
char * _b_seq,
- long M,
- long N);
+ int64_t M,
+ int64_t N);
void alignstats(char * cigar,
char * a_seq,
char * b_seq,
- long * nwscore,
- long * nwalignmentlength,
- long * nwmatches,
- long * nwmismatches,
- long * nwgaps);
+ int64_t * nwscore,
+ int64_t * nwalignmentlength,
+ int64_t * nwmatches,
+ int64_t * nwmismatches,
+ int64_t * nwgaps);
};
diff --git a/src/maps.cc b/src/maps.cc
index 4b0ca89..55de4da 100644
--- a/src/maps.cc
+++ b/src/maps.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/maps.h b/src/maps.h
index d011626..f4fd99b 100644
--- a/src/maps.h
+++ b/src/maps.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/mask.cc b/src/mask.cc
index 4d38f81..7ae04f6 100644
--- a/src/mask.cc
+++ b/src/mask.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -154,7 +154,7 @@ void dust(char * m, int len)
}
}
- free(s);
+ xfree(s);
}
static pthread_t * pthread;
@@ -199,14 +199,14 @@ void dust_all()
pthread = (pthread_t *) xmalloc(opt_threads * sizeof(pthread_t));
for(int t=0; t<opt_threads; t++)
- if (pthread_create(pthread+t, &attr, dust_all_worker, (void*)(long)t))
+ if (pthread_create(pthread+t, &attr, dust_all_worker, (void*)(int64_t)t))
fatal("Cannot create thread");
for(int t=0; t<opt_threads; t++)
if (pthread_join(pthread[t], NULL))
fatal("Cannot join thread");
- free(pthread);
+ xfree(pthread);
pthread_attr_destroy(&attr);
@@ -226,7 +226,7 @@ void hardmask(char * seq, int len)
void hardmask_all()
{
- for(unsigned long i=0; i<db_getsequencecount(); i++)
+ for(uint64_t i=0; i<db_getsequencecount(); i++)
hardmask(db_getsequence(i), db_getsequencelen(i));
}
diff --git a/src/mask.h b/src/mask.h
index 474bfa6..0bc3f9f 100644
--- a/src/mask.h
+++ b/src/mask.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/mergepairs.cc b/src/mergepairs.cc
index 6fa1973..7172977 100644
--- a/src/mergepairs.cc
+++ b/src/mergepairs.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,12 +61,11 @@
#include "vsearch.h"
#define INPUTCHUNKSIZE 10000
-#define SCOREMETHOD 2
/* scores */
const double alpha = 4.0;
-const double beta = -5.0;
+const double beta = -22.0;
/* static variables */
@@ -79,9 +78,9 @@ static FILE * fp_fastaout_notmerged_rev = 0;
static FILE * fp_eetabbedout = 0;
static fastx_handle fastq_fwd;
static fastx_handle fastq_rev;
-static long merged = 0;
-static long notmerged = 0;
-static long total = 0;
+static int64_t merged = 0;
+static int64_t notmerged = 0;
+static int64_t total = 0;
static char * merged_sequence = 0;
static char * merged_quality = 0;
static char * merged_header = 0;
@@ -92,7 +91,7 @@ static pthread_mutex_t mutex_progress;
static pthread_mutex_t mutex_counters;
static pthread_mutex_t mutex_output;
static pthread_cond_t cond_output;
-static long output_next;
+static int64_t output_next;
static char merge_qual_same[128][128];
static char merge_qual_diff[128][128];
static double match_score[128][128];
@@ -109,24 +108,24 @@ typedef struct merge_data_s
char * rev_seq_comp;
char * fwd_quality;
char * rev_quality;
- long header_alloc;
- long seq_alloc;
- long fwd_length;
- long rev_length;
- long fwd_trunc;
- long rev_trunc;
- long pair_no;
+ int64_t header_alloc;
+ int64_t seq_alloc;
+ int64_t fwd_length;
+ int64_t rev_length;
+ int64_t fwd_trunc;
+ int64_t rev_trunc;
+ int64_t pair_no;
char * merged_header;
char * merged_sequence;
char * merged_quality;
- long merged_header_alloc;
- long merged_seq_alloc;
+ int64_t merged_header_alloc;
+ int64_t merged_seq_alloc;
double ee_merged;
double ee_fwd;
double ee_rev;
- long fwd_errors;
- long rev_errors;
- long offset;
+ int64_t fwd_errors;
+ int64_t rev_errors;
+ int64_t offset;
bool merged;
} merge_data_t;
@@ -146,13 +145,13 @@ int get_qual(char q)
if (qual < opt_fastq_qmin)
{
- snprintf(msg, 200, "FASTQ quality value (%d) below qmin (%ld)",
+ snprintf(msg, 200, "FASTQ quality value (%d) below qmin (%" PRId64 ")",
qual, opt_fastq_qmin);
fatal(msg);
}
else if (qual > opt_fastq_qmax)
{
- snprintf(msg, 200, "FASTQ quality value (%d) above qmax (%ld)",
+ snprintf(msg, 200, "FASTQ quality value (%d) above qmax (%" PRId64 ")",
qual, opt_fastq_qmax);
fatal(msg);
}
@@ -207,11 +206,7 @@ void precompute_qual()
p = 1.0 - px - py + px * py * 4.0 / 3.0;
-#if SCOREMETHOD == 2
- match_score[x][y] = alpha * p + beta * (1.0 - p);
-#else
match_score[x][y] = alpha * p;
-#endif
/* Mismatch */
@@ -221,11 +216,7 @@ void precompute_qual()
p = 1.0 - (px + py) / 3.0 + px * py * 4.0 / 9.0;
-#if SCOREMETHOD == 2
- mism_score[x][y] = alpha * (1.0 - p) + beta * p;
-#else
mism_score[x][y] = beta * p;
-#endif
}
}
@@ -302,7 +293,7 @@ void keep(merge_data_t * ip)
}
if (opt_eetabbedout)
- fprintf(fp_eetabbedout, "%.2lf\t%.2lf\t%ld\t%ld\n",
+ fprintf(fp_eetabbedout, "%.2lf\t%.2lf\t%" PRId64 "\t%" PRId64 "\n",
ip->ee_fwd, ip->ee_rev, ip->fwd_errors, ip->rev_errors);
}
@@ -340,24 +331,24 @@ void merge(merge_data_t * ip)
/* The offset is the distance between the (truncated) 3' ends of the two
sequences */
- long rev_3prime_overhang = ip->offset > ip->fwd_trunc ?
+ int64_t rev_3prime_overhang = ip->offset > ip->fwd_trunc ?
ip->offset - ip->fwd_trunc : 0;
- long fwd_5prime_overhang = ip->fwd_trunc > ip->offset ?
+ int64_t fwd_5prime_overhang = ip->fwd_trunc > ip->offset ?
ip->fwd_trunc - ip->offset : 0;
- long mergelen = ip->fwd_trunc + ip->rev_trunc - ip->offset;
+ int64_t mergelen = ip->fwd_trunc + ip->rev_trunc - ip->offset;
ip->ee_merged = 0.0;
ip->ee_fwd = 0.0;
ip->ee_rev = 0.0;
- long fwd_pos = 0;
- long rev_pos = ip->rev_trunc - 1 +
+ int64_t fwd_pos = 0;
+ int64_t rev_pos = ip->rev_trunc - 1 +
fwd_5prime_overhang - rev_3prime_overhang;
ip->fwd_errors = 0;
ip->rev_errors = 0;
- for(long i = 0; i < mergelen; i++)
+ for(int64_t i = 0; i < mergelen; i++)
{
bool has_fwd = 0;
if ((fwd_pos >= 0) && (fwd_pos < ip->fwd_trunc))
@@ -429,16 +420,16 @@ void merge(merge_data_t * ip)
}
double overlap_score(merge_data_t * ip,
- long fwd_pos_start,
- long rev_pos_start,
- long overlap)
+ int64_t fwd_pos_start,
+ int64_t rev_pos_start,
+ int64_t overlap)
{
- long fwd_pos = fwd_pos_start;
- long rev_pos = rev_pos_start;
+ int64_t fwd_pos = fwd_pos_start;
+ int64_t rev_pos = rev_pos_start;
double score = 0.0;
- long diffs = 0;
+ int64_t diffs = 0;
- for (long j=0; j < overlap; j++)
+ for (int64_t j=0; j < overlap; j++)
{
char fwd_sym = ip->fwd_sequence[fwd_pos];
char rev_sym = ip->rev_seq_comp[rev_pos];
@@ -460,29 +451,28 @@ double overlap_score(merge_data_t * ip,
return score;
}
-long optimize(merge_data_t * ip)
+int64_t optimize(merge_data_t * ip)
{
- // long i1 = opt_fastq_minovlen;
- long i1 = 1;
+ int64_t i1 = 1;
i1 = MAX(i1, ip->fwd_trunc + ip->rev_trunc - opt_fastq_maxmergelen);
- long i2 = opt_fastq_allowmergestagger ?
+ int64_t i2 = opt_fastq_allowmergestagger ?
(ip->fwd_trunc + ip->rev_trunc - opt_fastq_minovlen) :
ip->fwd_trunc;
i2 = MIN(i2, ip->fwd_trunc + ip->rev_trunc - opt_fastq_minmergelen);
- long best_i = 0;
+ int64_t best_i = 0;
double best_score = 0.0;
- for(long i = i1; i <= i2; i++)
+ for(int64_t i = i1; i <= i2; i++)
{
- long fwd_3prime_overhang = i > ip->rev_trunc ? i - ip->rev_trunc : 0;
- long rev_3prime_overhang = i > ip->fwd_trunc ? i - ip->fwd_trunc : 0;
- long overlap = i - fwd_3prime_overhang - rev_3prime_overhang;
- long fwd_pos_start = ip->fwd_trunc - fwd_3prime_overhang - 1;
- long rev_pos_start = ip->rev_trunc - rev_3prime_overhang - overlap;
+ int64_t fwd_3prime_overhang = i > ip->rev_trunc ? i - ip->rev_trunc : 0;
+ int64_t rev_3prime_overhang = i > ip->fwd_trunc ? i - ip->fwd_trunc : 0;
+ int64_t overlap = i - fwd_3prime_overhang - rev_3prime_overhang;
+ int64_t fwd_pos_start = ip->fwd_trunc - fwd_3prime_overhang - 1;
+ int64_t rev_pos_start = ip->rev_trunc - rev_3prime_overhang - overlap;
double score = overlap_score(ip,
fwd_pos_start,
@@ -515,11 +505,11 @@ void process(merge_data_t * ip)
/* truncate sequences by quality */
- long fwd_trunc = ip->fwd_length;
+ int64_t fwd_trunc = ip->fwd_length;
if (!skip)
{
- for (long i = 0; i < ip->fwd_length; i++)
+ for (int64_t i = 0; i < ip->fwd_length; i++)
if (get_qual(ip->fwd_quality[i]) <= opt_fastq_truncqual)
{
fwd_trunc = i;
@@ -531,11 +521,11 @@ void process(merge_data_t * ip)
ip->fwd_trunc = fwd_trunc;
- long rev_trunc = ip->rev_length;
+ int64_t rev_trunc = ip->rev_length;
if (!skip)
{
- for (long i = 0; i < ip->rev_length; i++)
+ for (int64_t i = 0; i < ip->rev_length; i++)
if (get_qual(ip->rev_quality[i]) <= opt_fastq_truncqual)
{
rev_trunc = i;
@@ -553,8 +543,8 @@ void process(merge_data_t * ip)
if (!skip)
{
- long fwd_ncount = 0;
- for (long i = 0; i < fwd_trunc; i++)
+ int64_t fwd_ncount = 0;
+ for (int64_t i = 0; i < fwd_trunc; i++)
if (ip->fwd_sequence[i] == 'N')
{
ip->fwd_quality[i] = opt_fastq_ascii;
@@ -566,8 +556,8 @@ void process(merge_data_t * ip)
if (!skip)
{
- long rev_ncount = 0;
- for (long i = 0; i < rev_trunc; i++)
+ int64_t rev_ncount = 0;
+ for (int64_t i = 0; i < rev_trunc; i++)
if (ip->rev_sequence[i] == 'N')
{
ip->rev_quality[i] = opt_fastq_ascii;
@@ -582,14 +572,13 @@ void process(merge_data_t * ip)
if (!skip)
ip->offset = optimize(ip);
- // if (ip->offset)
if (ip->offset >= opt_fastq_minovlen)
merge(ip);
}
bool read_pair(merge_data_t * ip)
{
- long suffix_len = opt_label_suffix ? strlen(opt_label_suffix) : 0;
+ int64_t suffix_len = opt_label_suffix ? strlen(opt_label_suffix) : 0;
if (fastq_next(fastq_fwd, 0, chrmap_upcase))
{
@@ -598,9 +587,9 @@ bool read_pair(merge_data_t * ip)
/* allocate more memory if necessary */
- long fwd_header_len = fastq_get_header_length(fastq_fwd);
- long rev_header_len = fastq_get_header_length(fastq_rev);
- long header_needed = MAX(fwd_header_len, rev_header_len) + 1;
+ int64_t fwd_header_len = fastq_get_header_length(fastq_fwd);
+ int64_t rev_header_len = fastq_get_header_length(fastq_rev);
+ int64_t header_needed = MAX(fwd_header_len, rev_header_len) + 1;
if (header_needed > ip->header_alloc)
{
@@ -611,7 +600,7 @@ bool read_pair(merge_data_t * ip)
ip->fwd_length = fastq_get_sequence_length(fastq_fwd);
ip->rev_length = fastq_get_sequence_length(fastq_rev);
- long seq_needed = MAX(ip->fwd_length, ip->rev_length) + 1;
+ int64_t seq_needed = MAX(ip->fwd_length, ip->rev_length) + 1;
if (seq_needed > ip->seq_alloc)
{
@@ -624,7 +613,7 @@ bool read_pair(merge_data_t * ip)
}
- long merged_seq_needed = ip->fwd_length + ip->rev_length + 1;
+ int64_t merged_seq_needed = ip->fwd_length + ip->rev_length + 1;
if (merged_seq_needed > ip->merged_seq_alloc)
{
@@ -635,7 +624,7 @@ bool read_pair(merge_data_t * ip)
merged_seq_needed);
}
- long merged_header_needed = fwd_header_len + suffix_len + 1;
+ int64_t merged_header_needed = fwd_header_len + suffix_len + 1;
if (merged_header_needed > ip->merged_header_alloc)
{
@@ -655,7 +644,7 @@ bool read_pair(merge_data_t * ip)
/* make a complementary sequence of the reverse sequence */
- for(long i=0; i<ip->rev_length; i++)
+ for(int64_t i=0; i<ip->rev_length; i++)
ip->rev_seq_comp[i] = chrmap_complement[(int)(ip->rev_sequence[i])];
ip->rev_seq_comp[ip->rev_length] = 0;
@@ -678,7 +667,7 @@ void pair()
merge_data_t * merge_buffer = (merge_data_t*)
xmalloc(INPUTCHUNKSIZE * sizeof(merge_data_s));
- for(long i=0; i<INPUTCHUNKSIZE; i++)
+ for(int64_t i=0; i<INPUTCHUNKSIZE; i++)
{
merge_data_s * ip = merge_buffer + i;
ip->fwd_header = 0;
@@ -700,7 +689,7 @@ void pair()
{
pthread_mutex_lock(&mutex_input);
progress_update(fastq_get_position(fastq_fwd));
- long pairs = 0;
+ int64_t pairs = 0;
while (more && (pairs < INPUTCHUNKSIZE))
{
more = read_pair(merge_buffer + pairs);
@@ -709,14 +698,14 @@ void pair()
}
pthread_mutex_unlock(&mutex_input);
- for(long i=0; i<pairs; i++)
+ for(int64_t i=0; i<pairs; i++)
process(merge_buffer + i);
pthread_mutex_lock(&mutex_output);
while(output_next < merge_buffer[0].pair_no)
pthread_cond_wait(&cond_output, &mutex_output);
- for(long i=0; i<pairs; i++)
+ for(int64_t i=0; i<pairs; i++)
{
merge_data_s * ip = merge_buffer + i;
if (ip->merged)
@@ -730,31 +719,31 @@ void pair()
pthread_mutex_unlock(&mutex_output);
}
- for(long i=0; i<INPUTCHUNKSIZE; i++)
+ for(int64_t i=0; i<INPUTCHUNKSIZE; i++)
{
merge_data_s * ip = merge_buffer + i;
if (ip->fwd_header)
- free(ip->fwd_header);
+ xfree(ip->fwd_header);
if (ip->rev_header)
- free(ip->rev_header);
+ xfree(ip->rev_header);
if (ip->fwd_sequence)
- free(ip->fwd_sequence);
+ xfree(ip->fwd_sequence);
if (ip->rev_sequence)
- free(ip->rev_sequence);
+ xfree(ip->rev_sequence);
if (ip->rev_seq_comp)
- free(ip->rev_seq_comp);
+ xfree(ip->rev_seq_comp);
if (ip->fwd_quality)
- free(ip->fwd_quality);
+ xfree(ip->fwd_quality);
if (ip->rev_quality)
- free(ip->rev_quality);
+ xfree(ip->rev_quality);
}
- free(merge_buffer);
+ xfree(merge_buffer);
}
void * pair_worker(void * vp)
{
- long t = (long) vp;
+ int64_t t = (int64_t) vp;
(void) t;
pair();
return 0;
@@ -776,14 +765,14 @@ void pair_all()
pthread = (pthread_t *) xmalloc(opt_threads * sizeof(pthread_t));
for(int t=0; t<opt_threads; t++)
- if (pthread_create(pthread+t, &attr, pair_worker, (void*)(long)t))
+ if (pthread_create(pthread+t, &attr, pair_worker, (void*)(int64_t)t))
fatal("Cannot create thread");
for(int t=0; t<opt_threads; t++)
if (pthread_join(pthread[t], NULL))
fatal("Cannot join thread");
- free(pthread);
+ xfree(pthread);
pthread_attr_destroy(&attr);
@@ -825,7 +814,7 @@ void fastq_mergepairs()
/* main */
- unsigned long filesize = fastq_get_size(fastq_fwd);
+ uint64_t filesize = fastq_get_size(fastq_fwd);
progress_init("Merging reads", filesize);
pair_all();
@@ -836,16 +825,16 @@ void fastq_mergepairs()
fatal("More reverse reads than forward reads");
fprintf(stderr,
- "%10lu Pairs\n",
+ "%10" PRIu64 " Pairs\n",
total);
fprintf(stderr,
- "%10lu Merged (%.1lf%%)\n",
+ "%10" PRIu64 " Merged (%.1lf%%)\n",
merged,
100.0 * merged / total);
fprintf(stderr,
- "%10lu Not merged (%.1lf%%)\n",
+ "%10" PRIu64 " Not merged (%.1lf%%)\n",
notmerged,
100.0 * notmerged / total);
@@ -872,9 +861,9 @@ void fastq_mergepairs()
fastq_fwd = 0;
if (merged_sequence)
- free(merged_sequence);
+ xfree(merged_sequence);
if (merged_quality)
- free(merged_quality);
+ xfree(merged_quality);
if (merged_header)
- free(merged_header);
+ xfree(merged_header);
}
diff --git a/src/mergepairs.h b/src/mergepairs.h
index ee0cf3a..223e077 100644
--- a/src/mergepairs.h
+++ b/src/mergepairs.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/minheap.cc b/src/minheap.cc
index fd7a18e..4c42801 100644
--- a/src/minheap.cc
+++ b/src/minheap.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -137,8 +137,8 @@ minheap_init(int size)
void
minheap_exit(minheap_t * m)
{
- free(m->array);
- free(m);
+ xfree(m->array);
+ xfree(m);
}
static int swaps = 0;
@@ -275,7 +275,7 @@ minheap_test()
for(int i=samples; i>=0; i--)
{
- elem_t x = {(unsigned int)(random()),0,1};
+ elem_t x = {(unsigned int)(rand()),0,1};
minheap_add(m, & x);
}
diff --git a/src/minheap.h b/src/minheap.h
index cd4debf..356c12e 100644
--- a/src/minheap.h
+++ b/src/minheap.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -90,3 +90,4 @@ void minheap_exit(minheap_t * m);
void minheap_add(minheap_t * m, elem_t * n);
elem_t minheap_pop(minheap_t * m);
+void minheap_dump(minheap_t * m);
diff --git a/src/msa.cc b/src/msa.cc
index 6814aa0..3aa6cc8 100644
--- a/src/msa.cc
+++ b/src/msa.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -111,7 +111,7 @@ void msa_add(char c)
void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
int cluster,
int target_count, struct msa_target_s * target_list,
- long totalabundance)
+ int64_t totalabundance)
{
int centroid_seqno = target_list[0].seqno;
int centroid_len = db_getsequencelen(centroid_seqno);
@@ -127,9 +127,9 @@ void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
int pos = 0;
while (p < e)
{
- long run = 1;
+ int64_t run = 1;
int scanlength = 0;
- sscanf(p, "%ld%n", &run, &scanlength);
+ sscanf(p, "%" PRId64 "%n", &run, &scanlength);
p += scanlength;
char op = *p++;
switch (op)
@@ -159,11 +159,11 @@ void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
char * cons = (char *) xmalloc(alnlen+1);
/* Find longest target sequence on reverse strand and allocate buffer */
- long longest_reversed = 0;
+ int64_t longest_reversed = 0;
for(int i=0; i < target_count; i++)
if (target_list[i].strand)
{
- long len = db_getsequencelen(target_list[i].seqno);
+ int64_t len = db_getsequencelen(target_list[i].seqno);
if (len > longest_reversed)
longest_reversed = len;
}
@@ -208,9 +208,9 @@ void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
char * e = p + strlen(p);
while (p < e)
{
- long run = 1;
+ int64_t run = 1;
int scanlength = 0;
- sscanf(p, "%ld%n", &run, &scanlength);
+ sscanf(p, "%" PRId64 "%n", &run, &scanlength);
p += scanlength;
char op = *p++;
@@ -262,7 +262,7 @@ void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
}
if (rc_buffer)
- free(rc_buffer);
+ xfree(rc_buffer);
/* consensus */
@@ -362,8 +362,8 @@ void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
fprintf(fp_profile, "\n");
}
- free(maxi);
- free(aln);
- free(cons);
- free(profile);
+ xfree(maxi);
+ xfree(aln);
+ xfree(cons);
+ xfree(profile);
}
diff --git a/src/msa.h b/src/msa.h
index be8377a..923d18c 100644
--- a/src/msa.h
+++ b/src/msa.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -68,4 +68,4 @@ struct msa_target_s
void msa(FILE * fp_msaout, FILE * fp_consout, FILE * fp_profile,
int cluster,
int target_count, struct msa_target_s * target_list,
- long totalabundance);
+ int64_t totalabundance);
diff --git a/src/otutable.cc b/src/otutable.cc
index 5075a0c..1e6c625 100644
--- a/src/otutable.cc
+++ b/src/otutable.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,10 +60,6 @@
#include "vsearch.h"
-#include <string>
-#include <set>
-#include <map>
-
/*
@@ -80,9 +76,9 @@
typedef std::set<std::string> string_set_t;
typedef std::pair<std::string, std::string> string_pair_t;
-typedef std::map<string_pair_t, unsigned long> string_pair_map_t;
+typedef std::map<string_pair_t, uint64_t> string_pair_map_t;
typedef std::map<std::string, std::string> otu_tax_map_t;
-typedef std::map<std::string, unsigned long> string_no_map_t;
+typedef std::map<std::string, uint64_t> string_no_map_t;
struct otutable_s
{
@@ -131,7 +127,7 @@ void otutable_done()
otutable.otu_sample_count.clear();
}
-void otutable_add(char * query_header, char * target_header, long abundance)
+void otutable_add(char * query_header, char * target_header, int64_t abundance)
{
/* read sample annotation in query */
@@ -195,7 +191,7 @@ void otutable_add(char * query_header, char * target_header, long abundance)
strncpy(tax_name, start_tax, len_tax);
tax_name[len_tax] = 0;
otutable.otu_tax_map[otu_name] = tax_name;
- free(tax_name);
+ xfree(tax_name);
}
/* store data */
@@ -207,13 +203,13 @@ void otutable_add(char * query_header, char * target_header, long abundance)
otutable.otu_sample_count[string_pair_t(otu_name,sample_name)]
+= abundance;
- free(otu_name);
- free(sample_name);
+ xfree(otu_name);
+ xfree(sample_name);
}
void otutable_print_otutabout(FILE * fp)
{
- long progress = 0;
+ int64_t progress = 0;
progress_init("Writing OTU table (classic)", otutable.otu_set.size());
fprintf(fp, "#OTU ID");
@@ -236,7 +232,7 @@ void otutable_print_otutabout(FILE * fp)
it_sample != otutable.sample_set.end();
it_sample++)
{
- unsigned long a = 0;
+ uint64_t a = 0;
if ((it_map != otutable.otu_sample_count.end()) &&
(it_map->first.first == *it_otu) &&
(it_map->first.second == *it_sample))
@@ -244,7 +240,7 @@ void otutable_print_otutabout(FILE * fp)
a = it_map->second;
it_map++;
}
- fprintf(fp, "\t%ld", a);
+ fprintf(fp, "\t%" PRIu64, a);
}
if (! otutable.otu_tax_map.empty())
{
@@ -262,11 +258,11 @@ void otutable_print_otutabout(FILE * fp)
void otutable_print_mothur_shared_out(FILE * fp)
{
- long progress = 0;
+ int64_t progress = 0;
progress_init("Writing OTU table (mothur)", otutable.sample_set.size());
fprintf(fp, "label\tGroup\tnumOtus");
- long numotus = 0;
+ int64_t numotus = 0;
for (string_set_t::iterator it_otu = otutable.otu_set.begin();
it_otu != otutable.otu_set.end();
it_otu++)
@@ -283,13 +279,13 @@ void otutable_print_mothur_shared_out(FILE * fp)
it_sample != otutable.sample_set.end();
it_sample++)
{
- fprintf(fp, "vsearch\t%s\t%ld", it_sample->c_str(), numotus);
+ fprintf(fp, "vsearch\t%s\t%" PRId64, it_sample->c_str(), numotus);
for (string_set_t::iterator it_otu = otutable.otu_set.begin();
it_otu != otutable.otu_set.end();
it_otu++)
{
- unsigned long a = 0;
+ uint64_t a = 0;
if ((it_map != otutable.sample_otu_count.end()) &&
(it_map->first.first == *it_sample) &&
(it_map->first.second == *it_otu))
@@ -297,7 +293,7 @@ void otutable_print_mothur_shared_out(FILE * fp)
a = it_map->second;
it_map++;
}
- fprintf(fp, "\t%ld", a);
+ fprintf(fp, "\t%" PRIu64, a);
}
fprintf(fp, "\n");
@@ -308,11 +304,11 @@ void otutable_print_mothur_shared_out(FILE * fp)
void otutable_print_biomout(FILE * fp)
{
- long progress = 0;
+ int64_t progress = 0;
progress_init("Writing OTU table (biom 1.0)", otutable.otu_sample_count.size());
- long rows = otutable.otu_set.size();
- long columns = otutable.sample_set.size();
+ int64_t rows = otutable.otu_set.size();
+ int64_t columns = otutable.sample_set.size();
static time_t time_now = time(0);
struct tm tm_now;
@@ -330,7 +326,7 @@ void otutable_print_biomout(FILE * fp)
"\t\"date\": \"%s\",\n"
"\t\"matrix_type\": \"sparse\",\n"
"\t\"matrix_element_type\": \"int\",\n"
- "\t\"shape\": [%ld,%ld],\n",
+ "\t\"shape\": [%" PRId64 ",%" PRId64 "],\n",
opt_biomout,
PROG_NAME, PROG_VERSION,
date,
@@ -338,7 +334,7 @@ void otutable_print_biomout(FILE * fp)
columns);
string_no_map_t otu_no_map;
- unsigned long otu_no = 0;
+ uint64_t otu_no = 0;
fprintf(fp, "\t\"rows\":[");
for (string_set_t::iterator it_otu = otutable.otu_set.begin();
@@ -367,7 +363,7 @@ void otutable_print_biomout(FILE * fp)
fprintf(fp, "\t],\n");
string_no_map_t sample_no_map;
- unsigned long sample_no = 0;
+ uint64_t sample_no = 0;
fprintf(fp, "\t\"columns\":[");
for (string_set_t::iterator it_sample = otutable.sample_set.begin();
@@ -394,7 +390,7 @@ void otutable_print_biomout(FILE * fp)
otu_no = otu_no_map[it_map->first.first];
sample_no = sample_no_map[it_map->first.second];
- fprintf(fp, "\n\t\t[%ld,%ld,%lu]", otu_no, sample_no, it_map->second);
+ fprintf(fp, "\n\t\t[%" PRIu64 ",%" PRIu64 ",%" PRIu64 "]", otu_no, sample_no, it_map->second);
first = false;
progress_update(++progress);
}
diff --git a/src/otutable.h b/src/otutable.h
index 4a8e866..d42390e 100644
--- a/src/otutable.h
+++ b/src/otutable.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2016, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,7 +60,7 @@
void otutable_init();
void otutable_done();
-void otutable_add(char * query_header, char * target_header, long abundance);
+void otutable_add(char * query_header, char * target_header, int64_t abundance);
void otutable_print_otutabout(FILE * fp);
void otutable_print_mothur_shared_out(FILE * fp);
void otutable_print_biomout(FILE * fp);
diff --git a/src/rerep.cc b/src/rerep.cc
index cf15b1b..f091459 100644
--- a/src/rerep.cc
+++ b/src/rerep.cc
@@ -3,7 +3,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -75,18 +75,18 @@ void rereplicate()
}
fastx_handle fh = fasta_open(opt_rereplicate);
- long filesize = fasta_get_size(fh);
+ int64_t filesize = fasta_get_size(fh);
progress_init("Rereplicating", filesize);
- long i = 0;
- long n = 0;
+ int64_t i = 0;
+ int64_t n = 0;
while (fasta_next(fh, ! opt_notrunclabels, chrmap_no_change))
{
n++;
- long abundance = fasta_get_abundance(fh);
+ int64_t abundance = fasta_get_abundance(fh);
- for(long j=0; j<abundance; j++)
+ for(int64_t j=0; j<abundance; j++)
{
i++;
if (opt_output)
@@ -103,7 +103,7 @@ void rereplicate()
}
progress_done();
- fprintf(stderr, "Rereplicated %ld reads from %ld amplicons\n", i, n);
+ fprintf(stderr, "Rereplicated %" PRId64 " reads from %" PRId64 " amplicons\n", i, n);
fasta_close(fh);
diff --git a/src/rerep.h b/src/rerep.h
index 823177f..bad4cb4 100644
--- a/src/rerep.h
+++ b/src/rerep.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/results.cc b/src/results.cc
index e35184a..70d236f 100644
--- a/src/results.cc
+++ b/src/results.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -64,7 +64,7 @@ void results_show_fastapairs_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc)
{
/* http://www.drive5.com/usearch/manual/fastapairs.html */
@@ -79,7 +79,7 @@ void results_show_fastapairs_one(FILE * fp,
fasta_print_sequence(fp,
qrow + hp->trim_q_left + hp->trim_t_left,
hp->internal_alignmentlength, 0);
- free(qrow);
+ xfree(qrow);
char * trow = align_getrow(db_getsequence(hp->target),
hp->nwalignment,
@@ -89,7 +89,7 @@ void results_show_fastapairs_one(FILE * fp,
fasta_print_sequence(fp,
trow + hp->trim_q_left + hp->trim_t_left,
hp->internal_alignmentlength, 0);
- free(trow);
+ xfree(trow);
fprintf(fp, "\n");
}
@@ -100,7 +100,7 @@ void results_show_blast6out_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc)
{
@@ -142,7 +142,7 @@ void results_show_blast6out_one(FILE * fp,
}
fprintf(fp,
- "%s\t%s\t%.1f\t%d\t%d\t%d\t%d\t%d\t%d\t%lu\t%d\t%d\n",
+ "%s\t%s\t%.1f\t%d\t%d\t%d\t%d\t%d\t%d\t%" PRIu64 "\t%d\t%d\n",
query_head,
db_getheader(hp->target),
hp->id,
@@ -166,7 +166,7 @@ void results_show_uc_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc,
int clusterno)
{
@@ -189,10 +189,10 @@ void results_show_uc_one(FILE * fp,
if (hp)
{
bool perfect = (hp->matches == qseqlen) &&
- ((unsigned long)(qseqlen) == db_getsequencelen(hp->target));
+ ((uint64_t)(qseqlen) == db_getsequencelen(hp->target));
fprintf(fp,
- "H\t%d\t%ld\t%.1f\t%c\t0\t0\t%s\t%s\t%s\n",
+ "H\t%d\t%" PRId64 "\t%.1f\t%c\t0\t0\t%s\t%s\t%s\n",
clusterno,
qseqlen,
hp->id,
@@ -207,7 +207,7 @@ void results_show_uc_one(FILE * fp,
void results_show_userout_one(FILE * fp, struct hit * hp,
char * query_head,
- char * qsequence, long qseqlen,
+ char * qsequence, int64_t qseqlen,
char * rc)
{
@@ -224,7 +224,7 @@ void results_show_userout_one(FILE * fp, struct hit * hp,
int field = userfields_requested[c];
char * tsequence = 0;
- long tseqlen = 0;
+ int64_t tseqlen = 0;
char * t_head = 0;
if (hp)
@@ -264,31 +264,31 @@ void results_show_userout_one(FILE * fp, struct hit * hp,
fprintf(fp, "%d", hp ? hp->internal_indels : 0);
break;
case 8: /* qlo */
- fprintf(fp, "%ld", hp ? (hp->strand ? qseqlen : 1) : 0);
+ fprintf(fp, "%" PRId64, hp ? (hp->strand ? qseqlen : 1) : 0);
break;
case 9: /* qhi */
- fprintf(fp, "%ld", hp ? (hp->strand ? 1 : qseqlen) : 0);
+ fprintf(fp, "%" PRId64, hp ? (hp->strand ? 1 : qseqlen) : 0);
break;
case 10: /* tlo */
fprintf(fp, "%d", hp ? 1 : 0);
break;
case 11: /* thi */
- fprintf(fp, "%ld", tseqlen);
+ fprintf(fp, "%" PRId64, tseqlen);
break;
case 12: /* pv */
fprintf(fp, "%d", hp ? hp->matches : 0);
break;
case 13: /* ql */
- fprintf(fp, "%ld", qseqlen);
+ fprintf(fp, "%" PRId64, qseqlen);
break;
case 14: /* tl */
- fprintf(fp, "%ld", hp ? tseqlen : 0);
+ fprintf(fp, "%" PRId64, hp ? tseqlen : 0);
break;
case 15: /* qs */
- fprintf(fp, "%ld", qseqlen);
+ fprintf(fp, "%" PRId64, qseqlen);
break;
case 16: /* ts */
- fprintf(fp, "%ld", hp ? tseqlen : 0);
+ fprintf(fp, "%" PRId64, hp ? tseqlen : 0);
break;
case 17: /* alnlen */
fprintf(fp, "%d", hp ? hp->internal_alignmentlength : 0);
@@ -331,7 +331,7 @@ void results_show_userout_one(FILE * fp, struct hit * hp,
fprintf(fp, "%.*s",
(int)(hp->internal_alignmentlength),
qrow + hp->trim_q_left + hp->trim_t_left);
- free(qrow);
+ xfree(qrow);
}
break;
case 27: /* trow */
@@ -344,7 +344,7 @@ void results_show_userout_one(FILE * fp, struct hit * hp,
fprintf(fp, "%.*s",
(int)(hp->internal_alignmentlength),
trow + hp->trim_q_left + hp->trim_t_left);
- free(trow);
+ xfree(trow);
}
break;
case 28: /* qframe */
@@ -389,13 +389,13 @@ void results_show_userout_one(FILE * fp, struct hit * hp,
fprintf(fp, "%d", hp ? hp->trim_q_left + 1 : 0);
break;
case 40: /* qihi */
- fprintf(fp, "%ld", hp ? qseqlen - hp->trim_q_right : 0);
+ fprintf(fp, "%" PRId64, hp ? qseqlen - hp->trim_q_right : 0);
break;
case 41: /* tilo */
fprintf(fp, "%d", hp ? hp->trim_t_left + 1 : 0);
break;
case 42: /* tihi */
- fprintf(fp, "%ld", hp ? tseqlen - hp->trim_t_right : 0);
+ fprintf(fp, "%" PRId64, hp ? tseqlen - hp->trim_t_right : 0);
break;
}
}
@@ -407,7 +407,7 @@ void results_show_alnout(FILE * fp,
int hitcount,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc)
{
/* http://drive5.com/usearch/manual/alnout.html */
@@ -428,7 +428,7 @@ void results_show_alnout(FILE * fp,
if (opt_top_hits_only && (hp->id < top_hit_id))
break;
- fprintf(fp,"%3.0f%% %6lu %s\n",
+ fprintf(fp,"%3.0f%% %6" PRIu64 " %s\n",
hp->id,
db_getsequencelen(hp->target),
db_getheader(hp->target));
@@ -445,16 +445,16 @@ void results_show_alnout(FILE * fp,
char * dseq = db_getsequence(hp->target);
- long dseqlen = db_getsequencelen(hp->target);
+ int64_t dseqlen = db_getsequencelen(hp->target);
char dummy;
- int qlenlen = snprintf(&dummy, 1, "%ld", qseqlen);
- int tlenlen = snprintf(&dummy, 1, "%ld", dseqlen);
+ int qlenlen = snprintf(&dummy, 1, "%" PRId64, qseqlen);
+ int tlenlen = snprintf(&dummy, 1, "%" PRId64, dseqlen);
int numwidth = MAX(qlenlen, tlenlen);
- fprintf(fp," Query %*ldnt >%s\n", numwidth,
+ fprintf(fp," Query %*" PRId64 "nt >%s\n", numwidth,
qseqlen, query_head);
- fprintf(fp,"Target %*ldnt >%s\n", numwidth,
+ fprintf(fp,"Target %*" PRId64 "nt >%s\n", numwidth,
dseqlen, db_getheader(hp->target));
int rowlen = opt_rowlen == 0 ? qseqlen+dseqlen : opt_rowlen;
@@ -612,14 +612,14 @@ void results_show_samheader(FILE * fp,
{
fprintf(fp, "@HD\tVN:1.0\tSO:unsorted\tGO:query\n");
- for(unsigned long i=0; i<db_getsequencecount(); i++)
+ for(uint64_t i=0; i<db_getsequencecount(); i++)
{
char md5hex[LEN_HEX_DIG_MD5];
get_hex_seq_digest_md5(md5hex,
db_getsequence(i),
db_getsequencelen(i));
fprintf(fp,
- "@SQ\tSN:%s\tLN:%lu\tM5:%s\tUR:file:%s\n",
+ "@SQ\tSN:%s\tLN:%" PRIu64 "\tM5:%s\tUR:file:%s\n",
db_getheader(i),
db_getsequencelen(i),
md5hex,
@@ -639,7 +639,7 @@ void results_show_samout(FILE * fp,
int hitcount,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc)
{
/*
@@ -704,18 +704,21 @@ void results_show_samout(FILE * fp,
& md);
fprintf(fp,
- "%s\t%u\t%s\t%lu\t%u\t%s\t%s\t%lu\t%lu\t%s\t%s\t"
+ "%s\t%u\t%s\t%" PRIu64
+ "\t%u\t%s\t%s\t%" PRIu64
+ "\t%" PRIu64
+ "\t%s\t%s\t"
"AS:i:%.0f\tXN:i:%d\tXM:i:%d\tXO:i:%d\t"
"XG:i:%d\tNM:i:%d\tMD:Z:%s\tYT:Z:%s\n",
query_head,
0x10 * hp->strand | (t>0 ? 0x100 : 0),
db_getheader(hp->target),
- 1UL,
+ (uint64_t) 1,
255,
cigar.get_string(),
"*",
- 0UL,
- 0UL,
+ (uint64_t) 0,
+ (uint64_t) 0,
hp->strand ? rc : qsequence,
"*",
hp->id,
@@ -731,16 +734,16 @@ void results_show_samout(FILE * fp,
else if (opt_output_no_hits)
{
fprintf(fp,
- "%s\t%u\t%s\t%lu\t%u\t%s\t%s\t%lu\t%lu\t%s\t%s\n",
+ "%s\t%u\t%s\t%" PRIu64 "\t%u\t%s\t%s\t%" PRIu64 "\t%" PRIu64 "\t%s\t%s\n",
query_head,
0x04,
"*",
- 0UL,
+ (uint64_t) 0,
255,
"*",
"*",
- 0UL,
- 0UL,
+ (uint64_t) 0,
+ (uint64_t) 0,
qsequence,
"*");
}
diff --git a/src/results.h b/src/results.h
index 48c1cfd..f9f0c43 100644
--- a/src/results.h
+++ b/src/results.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -63,21 +63,21 @@ void results_show_alnout(FILE * fp,
int hitcount,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc);
void results_show_blast6out_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc);
void results_show_uc_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc,
int clusterno);
@@ -85,14 +85,14 @@ void results_show_userout_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc);
void results_show_fastapairs_one(FILE * fp,
struct hit * hp,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc);
void results_show_samheader(FILE * fp,
@@ -104,5 +104,5 @@ void results_show_samout(FILE * fp,
int hitcount,
char * query_head,
char * qsequence,
- long qseqlen,
+ int64_t qseqlen,
char * rc);
diff --git a/src/search.cc b/src/search.cc
index fef3586..fde6700 100644
--- a/src/search.cc
+++ b/src/search.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -101,7 +101,7 @@ void search_output_results(int hit_count,
pthread_mutex_lock(&mutex_output);
/* show results */
- long toreport = MIN(opt_maxhits, hit_count);
+ int64_t toreport = MIN(opt_maxhits, hit_count);
if (fp_alnout)
results_show_alnout(fp_alnout,
@@ -172,7 +172,7 @@ void search_output_results(int hit_count,
qsequence_rc);
}
}
- else if (opt_output_no_hits)
+ else
{
if (fp_uc)
results_show_uc_one(fp_uc,
@@ -183,21 +183,24 @@ void search_output_results(int hit_count,
qsequence_rc,
0);
- if (fp_userout)
- results_show_userout_one(fp_userout,
- 0,
- query_head,
- qsequence,
- qseqlen,
- qsequence_rc);
-
- if (fp_blast6out)
- results_show_blast6out_one(fp_blast6out,
- 0,
- query_head,
- qsequence,
- qseqlen,
- qsequence_rc);
+ if (opt_output_no_hits)
+ {
+ if (fp_userout)
+ results_show_userout_one(fp_userout,
+ 0,
+ query_head,
+ qsequence,
+ qseqlen,
+ qsequence_rc);
+
+ if (fp_blast6out)
+ results_show_blast6out_one(fp_blast6out,
+ 0,
+ query_head,
+ qsequence,
+ qseqlen,
+ qsequence_rc);
+ }
}
if (hit_count)
@@ -225,7 +228,7 @@ void search_output_results(int hit_count,
pthread_mutex_unlock(&mutex_output);
}
-int search_query(long t)
+int search_query(int64_t t)
{
for (int s = 0; s < opt_strand; s++)
{
@@ -264,14 +267,14 @@ int search_query(long t)
/* free memory for alignment strings */
for(int i=0; i<hit_count; i++)
if (hits[i].aligned)
- free(hits[i].nwalignment);
+ xfree(hits[i].nwalignment);
- free(hits);
+ xfree(hits);
return hit_count;
}
-void search_thread_run(long t)
+void search_thread_run(int64_t t)
{
while (1)
{
@@ -320,7 +323,7 @@ void search_thread_run(long t)
strcpy(si_plus[t].qsequence, qseq);
/* get progress as amount of input file read */
- unsigned long progress = fasta_get_position(query_fasta_h);
+ uint64_t progress = fasta_get_position(query_fasta_h);
/* let other threads read input */
pthread_mutex_unlock(&mutex_input);
@@ -400,20 +403,20 @@ void search_thread_exit(struct searchinfo_s * si)
nw_exit(si->nw);
#endif
unique_exit(si->uh);
- free(si->hits);
+ xfree(si->hits);
minheap_exit(si->m);
- free(si->kmers);
+ xfree(si->kmers);
if (si->query_head)
- free(si->query_head);
+ xfree(si->query_head);
if (si->qsequence)
- free(si->qsequence);
+ xfree(si->qsequence);
}
void * search_thread_worker(void * vp)
{
- long t = (long) vp;
+ int64_t t = (int64_t) vp;
search_thread_run(t);
return 0;
}
@@ -432,7 +435,7 @@ void search_thread_worker_run()
if (si_minus)
search_thread_init(si_minus+t);
if (pthread_create(pthread+t, &attr,
- search_thread_worker, (void*)(long)t))
+ search_thread_worker, (void*)(int64_t)t))
fatal("Cannot create thread");
}
@@ -641,10 +644,10 @@ void usearch_global(char * cmdline, char * progheader)
pthread_mutex_destroy(&mutex_output);
pthread_mutex_destroy(&mutex_input);
- free(pthread);
- free(si_plus);
+ xfree(pthread);
+ xfree(si_plus);
if (si_minus)
- free(si_minus);
+ xfree(si_minus);
fasta_close(query_fasta_h);
@@ -678,7 +681,7 @@ void usearch_global(char * cmdline, char * progheader)
if (opt_dbmatched || opt_dbnotmatched)
{
- for(long i=0; i<seqcount; i++)
+ for(int64_t i=0; i<seqcount; i++)
if (dbmatched[i])
{
if (opt_dbmatched)
@@ -696,7 +699,7 @@ void usearch_global(char * cmdline, char * progheader)
}
}
- free(dbmatched);
+ xfree(dbmatched);
if (opt_dbmatched)
fclose(fp_dbmatched);
diff --git a/src/search.h b/src/search.h
index 8d2f962..d2da935 100644
--- a/src/search.h
+++ b/src/search.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/searchcore.cc b/src/searchcore.cc
index 17c9eba..7a97a44 100644
--- a/src/searchcore.cc
+++ b/src/searchcore.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -185,12 +185,16 @@ void search_topscores(struct searchinfo_s * si)
if (bitmap)
{
+#ifdef __PPC__
+ increment_counters_from_bitmap(si->kmers, bitmap, indexed_count);
+#else
if (ssse3_present)
increment_counters_from_bitmap_ssse3(si->kmers,
bitmap, indexed_count);
else
increment_counters_from_bitmap_sse2(si->kmers,
bitmap, indexed_count);
+#endif
}
else
{
@@ -223,7 +227,7 @@ void search_topscores(struct searchinfo_s * si)
minheap_sort(si->m);
}
-int seqncmp(char * a, char * b, unsigned long n)
+int seqncmp(char * a, char * b, uint64_t n)
{
for(unsigned int i = 0; i<n; i++)
{
@@ -256,12 +260,12 @@ void align_trim(struct hit * hit)
char * p = hit->nwalignment;
char op;
- long run;
+ int64_t run;
if (*p)
{
run = 1;
int scanlength = 0;
- sscanf(p, "%ld%n", &run, &scanlength);
+ sscanf(p, "%" PRId64 "%n", &run, &scanlength);
op = *(p+scanlength);
if (op != 'M')
{
@@ -285,7 +289,7 @@ void align_trim(struct hit * hit)
while ((p > hit->nwalignment) && (*(p-1) <= '9'))
p--;
run = 1;
- sscanf(p, "%ld", &run);
+ sscanf(p, "%" PRId64, &run);
hit->trim_aln_right = e - p;
if (op == 'D')
hit->trim_q_right = run;
@@ -355,8 +359,8 @@ int search_acceptable_unaligned(struct searchinfo_s * si,
char * qseq = si->qsequence;
char * dlabel = db_getheader(target);
char * dseq = db_getsequence(target);
- long dseqlen = db_getsequencelen(target);
- long tsize = db_getabundance(target);
+ int64_t dseqlen = db_getsequencelen(target);
+ int64_t tsize = db_getabundance(target);
if (
/* maxqsize */
@@ -518,16 +522,16 @@ void align_delayed(struct searchinfo_s * si)
}
else
{
- long target = hit->target;
- long nwscore = nwscore_list[i];
+ int64_t target = hit->target;
+ int64_t nwscore = nwscore_list[i];
char * nwcigar;
- long nwalignmentlength;
- long nwmatches;
- long nwmismatches;
- long nwgaps;
+ int64_t nwalignmentlength;
+ int64_t nwmatches;
+ int64_t nwmismatches;
+ int64_t nwgaps;
- long dseqlen = db_getsequencelen(target);
+ int64_t dseqlen = db_getsequencelen(target);
if (nwscore == SHRT_MAX)
{
@@ -538,7 +542,7 @@ void align_delayed(struct searchinfo_s * si)
char * dseq = db_getsequence(target);
if (nwcigar_list[i])
- free(nwcigar_list[i]);
+ xfree(nwcigar_list[i]);
nwcigar = xstrdup(si->lma->align(si->qsequence,
dseq,
@@ -593,7 +597,7 @@ void align_delayed(struct searchinfo_s * si)
/* free ignored alignments */
while (i < target_count)
- free(nwcigar_list[i++]);
+ xfree(nwcigar_list[i++]);
si->finalized = si->hit_count;
}
@@ -606,7 +610,7 @@ void search_onequery(struct searchinfo_s * si, int seqmask)
si->lma = new LinearMemoryAligner;
- long * scorematrix = si->lma->scorematrix_create(opt_match, opt_mismatch);
+ int64_t * scorematrix = si->lma->scorematrix_create(opt_match, opt_mismatch);
si->lma->set_parameters(scorematrix,
opt_gap_open_query_left,
@@ -679,7 +683,7 @@ void search_onequery(struct searchinfo_s * si, int seqmask)
align_delayed(si);
delete si->lma;
- free(scorematrix);
+ xfree(scorematrix);
}
struct hit * search_findbest2_byid(struct searchinfo_s * si_p,
@@ -752,7 +756,7 @@ void search_joinhits(struct searchinfo_s * si_p,
if (h->accepted)
hits[a++] = *h;
else if (h->aligned)
- free(h->nwalignment);
+ xfree(h->nwalignment);
}
}
diff --git a/src/searchcore.h b/src/searchcore.h
index 82a405a..d9a1aa5 100644
--- a/src/searchcore.h
+++ b/src/searchcore.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/searchexact.cc b/src/searchexact.cc
index 09f7a18..2205fc5 100644
--- a/src/searchexact.cc
+++ b/src/searchexact.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -90,7 +90,7 @@ static FILE * fp_otutabout = 0;
static FILE * fp_mothur_shared_out = 0;
static FILE * fp_biomout = 0;
-void add_hit(struct searchinfo_s * si, unsigned long seqno)
+void add_hit(struct searchinfo_s * si, uint64_t seqno)
{
if (search_acceptable_unaligned(si, seqno))
{
@@ -111,7 +111,7 @@ void add_hit(struct searchinfo_s * si, unsigned long seqno)
hp->matches = si->qseqlen;
hp->mismatches = 0;
- int ret = asprintf(&hp->nwalignment, "%dM", si->qseqlen);
+ int ret = xsprintf(&hp->nwalignment, "%dM", si->qseqlen);
if ((ret == -1) || (!hp->nwalignment))
fatal("Out of memory");
@@ -149,19 +149,19 @@ void search_exact_onequery(struct searchinfo_s * si)
dbhash_search_info_s info;
char * seq = si->qsequence;
- unsigned long seqlen = si->qseqlen;
+ uint64_t seqlen = si->qseqlen;
char * normalized = (char*) xmalloc(seqlen+1);
string_normalize(normalized, seq, seqlen);
si->hit_count = 0;
- long ret = dbhash_search_first(normalized, seqlen, & info);
+ int64_t ret = dbhash_search_first(normalized, seqlen, & info);
while (ret >= 0)
{
add_hit(si, ret);
ret = dbhash_search_next(&info);
}
- free(normalized);
+ xfree(normalized);
}
void search_exact_output_results(int hit_count,
@@ -175,7 +175,7 @@ void search_exact_output_results(int hit_count,
pthread_mutex_lock(&mutex_output);
/* show results */
- long toreport = MIN(opt_maxhits, hit_count);
+ int64_t toreport = MIN(opt_maxhits, hit_count);
if (fp_alnout)
results_show_alnout(fp_alnout,
@@ -246,7 +246,7 @@ void search_exact_output_results(int hit_count,
qsequence_rc);
}
}
- else if (opt_output_no_hits)
+ else
{
if (fp_uc)
results_show_uc_one(fp_uc,
@@ -257,21 +257,24 @@ void search_exact_output_results(int hit_count,
qsequence_rc,
0);
- if (fp_userout)
- results_show_userout_one(fp_userout,
- 0,
- query_head,
- qsequence,
- qseqlen,
- qsequence_rc);
-
- if (fp_blast6out)
- results_show_blast6out_one(fp_blast6out,
- 0,
- query_head,
- qsequence,
- qseqlen,
- qsequence_rc);
+ if (opt_output_no_hits)
+ {
+ if (fp_userout)
+ results_show_userout_one(fp_userout,
+ 0,
+ query_head,
+ qsequence,
+ qseqlen,
+ qsequence_rc);
+
+ if (fp_blast6out)
+ results_show_blast6out_one(fp_blast6out,
+ 0,
+ query_head,
+ qsequence,
+ qseqlen,
+ qsequence_rc);
+ }
}
if (hit_count)
@@ -299,7 +302,7 @@ void search_exact_output_results(int hit_count,
pthread_mutex_unlock(&mutex_output);
}
-int search_exact_query(long t)
+int search_exact_query(int64_t t)
{
for (int s = 0; s < opt_strand; s++)
{
@@ -338,14 +341,14 @@ int search_exact_query(long t)
/* free memory for alignment strings */
for(int i=0; i<hit_count; i++)
if (hits[i].aligned)
- free(hits[i].nwalignment);
+ xfree(hits[i].nwalignment);
- free(hits);
+ xfree(hits);
return hit_count;
}
-void search_exact_thread_run(long t)
+void search_exact_thread_run(int64_t t)
{
while (1)
{
@@ -392,7 +395,7 @@ void search_exact_thread_run(long t)
strcpy(si_plus[t].qsequence, qseq);
/* get progress as amount of input file read */
- unsigned long progress = fasta_get_position(query_fasta_h);
+ uint64_t progress = fasta_get_position(query_fasta_h);
/* let other threads read input */
pthread_mutex_unlock(&mutex_input);
@@ -450,16 +453,16 @@ void search_exact_thread_init(struct searchinfo_s * si)
void search_exact_thread_exit(struct searchinfo_s * si)
{
/* thread specific clean up */
- free(si->hits);
+ xfree(si->hits);
if (si->query_head)
- free(si->query_head);
+ xfree(si->query_head);
if (si->qsequence)
- free(si->qsequence);
+ xfree(si->qsequence);
}
void * search_exact_thread_worker(void * vp)
{
- long t = (long) vp;
+ int64_t t = (int64_t) vp;
search_exact_thread_run(t);
return 0;
}
@@ -478,7 +481,7 @@ void search_exact_thread_worker_run()
if (si_minus)
search_exact_thread_init(si_minus+t);
if (pthread_create(pthread+t, &attr,
- search_exact_thread_worker, (void*)(long)t))
+ search_exact_thread_worker, (void*)(int64_t)t))
fatal("Cannot create thread");
}
@@ -622,7 +625,7 @@ void search_exact_done()
dbhash_close();
db_free();
- free(dbmatched);
+ xfree(dbmatched);
if (opt_dbmatched)
fclose(fp_dbmatched);
@@ -683,10 +686,10 @@ void search_exact(char * cmdline, char * progheader)
pthread_mutex_destroy(&mutex_output);
pthread_mutex_destroy(&mutex_input);
- free(pthread);
- free(si_plus);
+ xfree(pthread);
+ xfree(si_plus);
if (si_minus)
- free(si_minus);
+ xfree(si_minus);
fasta_close(query_fasta_h);
@@ -720,7 +723,7 @@ void search_exact(char * cmdline, char * progheader)
if (opt_dbmatched || opt_dbnotmatched)
{
- for(long i=0; i<seqcount; i++)
+ for(int64_t i=0; i<seqcount; i++)
if (dbmatched[i])
{
if (opt_dbmatched)
diff --git a/src/searchexact.h b/src/searchexact.h
index 6aa150c..0deb04d 100644
--- a/src/searchexact.h
+++ b/src/searchexact.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/showalign.cc b/src/showalign.cc
index 2e97792..079eaf7 100644
--- a/src/showalign.cc
+++ b/src/showalign.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2016, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -60,20 +60,20 @@
#include "vsearch.h"
-static long line_pos;
+static int64_t line_pos;
static char * q_seq;
static char * d_seq;
-static long q_start;
-static long d_start;
+static int64_t q_start;
+static int64_t d_start;
-static long q_pos;
-static long d_pos;
+static int64_t q_pos;
+static int64_t d_pos;
-static long q_strand;
+static int64_t q_strand;
-static long alignlen;
+static int64_t alignlen;
static char * q_line;
static char * a_line;
@@ -87,8 +87,8 @@ static int headwidth = 5;
static const char * q_name;
static const char * d_name;
-static long q_len;
-static long d_len;
+static int64_t q_len;
+static int64_t d_len;
inline int nt_identical(char a, char b)
{
@@ -98,11 +98,11 @@ inline int nt_identical(char a, char b)
return 0;
}
-inline void putop(char c, long len)
+inline void putop(char c, int64_t len)
{
- long delta = q_strand ? -1 : +1;
+ int64_t delta = q_strand ? -1 : +1;
- long count = len;
+ int64_t count = len;
while(count)
{
if (line_pos == 0)
@@ -152,24 +152,24 @@ inline void putop(char c, long len)
a_line[line_pos] = 0;
d_line[line_pos] = 0;
- long q1 = q_start + 1;
+ int64_t q1 = q_start + 1;
if (q1 > q_len)
q1 = q_len;
- long q2 = q_strand ? q_pos +2 : q_pos;
+ int64_t q2 = q_strand ? q_pos +2 : q_pos;
- long d1 = d_start + 1;
+ int64_t d1 = d_start + 1;
if (d1 > d_len)
d1 = d_len;
- long d2 = d_pos;
+ int64_t d2 = d_pos;
fprintf(out, "\n");
- fprintf(out, "%*s %*ld %c %s %ld\n", headwidth, q_name, poswidth,
+ fprintf(out, "%*s %*" PRId64 " %c %s %" PRId64 "\n", headwidth, q_name, poswidth,
q1, q_strand ? '-' : '+', q_line, q2);
fprintf(out, "%*s %*s %s\n", headwidth, "", poswidth,
"", a_line);
- fprintf(out, "%*s %*ld %c %s %ld\n", headwidth, d_name, poswidth,
+ fprintf(out, "%*s %*" PRId64 " %c %s %" PRId64 "\n", headwidth, d_name, poswidth,
d1, '+', d_line, d2);
line_pos = 0;
@@ -180,15 +180,15 @@ inline void putop(char c, long len)
void align_show(FILE * f,
char * seq1,
- long seq1len,
- long seq1off,
+ int64_t seq1len,
+ int64_t seq1off,
const char * seq1name,
char * seq2,
- long seq2len,
- long seq2off,
+ int64_t seq2len,
+ int64_t seq2off,
const char * seq2name,
char * cigar,
- long cigarlen,
+ int64_t cigarlen,
int numwidth,
int namewidth,
int alignwidth,
@@ -223,9 +223,9 @@ void align_show(FILE * f,
while(p < e)
{
- long len;
+ int64_t len;
int n;
- if (!sscanf(p, "%ld%n", & len, & n))
+ if (!sscanf(p, "%" PRId64 "%n", & len, & n))
{
n = 0;
len = 1;
@@ -237,9 +237,9 @@ void align_show(FILE * f,
putop(0, 1);
- free(q_line);
- free(a_line);
- free(d_line);
+ xfree(q_line);
+ xfree(a_line);
+ xfree(d_line);
}
char * align_getrow(char * seq, char * cigar, int alen, int origin)
@@ -251,9 +251,9 @@ char * align_getrow(char * seq, char * cigar, int alen, int origin)
while(*p)
{
- long len;
+ int64_t len;
int n;
- if (!sscanf(p, "%ld%n", & len, & n))
+ if (!sscanf(p, "%" PRId64 "%n", & len, & n))
{
n = 0;
len = 1;
@@ -272,7 +272,7 @@ char * align_getrow(char * seq, char * cigar, int alen, int origin)
else
{
/* insert len gap symbols */
- for(long i = 0; i < len; i++)
+ for(int64_t i = 0; i < len; i++)
*r++ = '-';
}
}
diff --git a/src/showalign.h b/src/showalign.h
index d179648..8dd11d9 100644
--- a/src/showalign.h
+++ b/src/showalign.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -64,15 +64,15 @@ void align_fprint_uncompressed_alignment(FILE * f, char * cigar);
void align_show(FILE * f,
char * seq1,
- long seq1len,
- long seq1off,
+ int64_t seq1len,
+ int64_t seq1off,
const char * seq1name,
char * seq2,
- long seq2len,
- long seq2off,
+ int64_t seq2len,
+ int64_t seq2off,
const char * seq2name,
char * cigar,
- long cigarlen,
+ int64_t cigarlen,
int numwidth,
int namewidth,
int alignwidth,
diff --git a/src/shuffle.cc b/src/shuffle.cc
index 37ade2c..74e83a5 100644
--- a/src/shuffle.cc
+++ b/src/shuffle.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -90,7 +90,7 @@ void shuffle()
deck[j] = t;
passed++;
- progress_update(i);
+ progress_update(passed);
}
progress_done();
show_rusage();
@@ -107,7 +107,7 @@ void shuffle()
show_rusage();
- free(deck);
+ xfree(deck);
db_free();
fclose(fp_output);
}
diff --git a/src/shuffle.h b/src/shuffle.h
index 35fd124..3263b89 100644
--- a/src/shuffle.h
+++ b/src/shuffle.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/sortbylength.cc b/src/sortbylength.cc
index 3366457..efb5480 100644
--- a/src/sortbylength.cc
+++ b/src/sortbylength.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -159,7 +159,7 @@ void sortbylength()
progress_done();
show_rusage();
- free(sortinfo);
+ xfree(sortinfo);
db_free();
fclose(fp_output);
}
diff --git a/src/sortbylength.h b/src/sortbylength.h
index aeb991e..3bf5c3e 100644
--- a/src/sortbylength.h
+++ b/src/sortbylength.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/sortbysize.cc b/src/sortbysize.cc
index 279db2d..aca3d42 100644
--- a/src/sortbysize.cc
+++ b/src/sortbysize.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -114,7 +114,7 @@ void sortbysize()
for(int i=0; i<dbsequencecount; i++)
{
- long size = db_getabundance(i);
+ int64_t size = db_getabundance(i);
if((size >= opt_minsize) && (size <= opt_maxsize))
{
@@ -162,7 +162,7 @@ void sortbysize()
progress_done();
show_rusage();
- free(sortinfo);
+ xfree(sortinfo);
db_free();
fclose(fp_output);
}
diff --git a/src/sortbysize.h b/src/sortbysize.h
index 3718e08..3d91a93 100644
--- a/src/sortbysize.h
+++ b/src/sortbysize.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/subsample.cc b/src/subsample.cc
index 118bb7c..f7a1177 100644
--- a/src/subsample.cc
+++ b/src/subsample.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -103,7 +103,7 @@ void subsample()
int dbsequencecount = db_getsequencecount();
- unsigned long mass_total = 0;
+ uint64_t mass_total = 0;
if (!opt_sizein)
mass_total = dbsequencecount;
@@ -111,7 +111,7 @@ void subsample()
for(int i=0; i<dbsequencecount; i++)
mass_total += db_getabundance(i);
- fprintf(stderr, "Got %lu reads from %d amplicons\n",
+ fprintf(stderr, "Got %" PRIu64 " reads from %d amplicons\n",
mass_total, dbsequencecount);
@@ -122,7 +122,7 @@ void subsample()
random_init();
- unsigned long n; /* number of reads to sample */
+ uint64_t n; /* number of reads to sample */
if (opt_sample_size)
n = opt_sample_size;
else
@@ -131,18 +131,18 @@ void subsample()
if (n > mass_total)
fatal("Cannot subsample more reads than in the original sample");
- unsigned long x = n; /* number of reads left */
+ uint64_t x = n; /* number of reads left */
int a = 0; /* amplicon number */
- unsigned long r = 0; /* read being checked */
- unsigned long m = 0; /* accumulated mass */
+ uint64_t r = 0; /* read being checked */
+ uint64_t m = 0; /* accumulated mass */
- unsigned long mass = /* mass of current amplicon */
+ uint64_t mass = /* mass of current amplicon */
opt_sizein ? db_getabundance(0) : 1;
progress_init("Subsampling", mass_total);
while (x > 0)
{
- unsigned long random = random_ulong(mass_total - r);
+ uint64_t random = random_ulong(mass_total - r);
if (random < x)
{
@@ -169,8 +169,8 @@ void subsample()
progress_init("Writing output", dbsequencecount);
for(int i=0; i<dbsequencecount; i++)
{
- long ab_sub = abundance[i];
- long ab_discarded = (opt_sizein ? db_getabundance(i) : 1) - ab_sub;
+ int64_t ab_sub = abundance[i];
+ int64_t ab_discarded = (opt_sizein ? db_getabundance(i) : 1) - ab_sub;
if (ab_sub > 0)
{
@@ -223,9 +223,9 @@ void subsample()
}
progress_done();
- free(abundance);
+ xfree(abundance);
- fprintf(stderr, "Subsampled %lu reads from %d amplicons\n", n, samples);
+ fprintf(stderr, "Subsampled %" PRIu64 " reads from %d amplicons\n", n, samples);
db_free();
diff --git a/src/subsample.h b/src/subsample.h
index ae5e044..905d9af 100644
--- a/src/subsample.h
+++ b/src/subsample.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/unique.cc b/src/unique.cc
index 76fd45f..cb203bb 100644
--- a/src/unique.cc
+++ b/src/unique.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -83,8 +83,8 @@ struct uhandle_s
int size;
int alloc;
- unsigned long bitmap_size;
- unsigned long * bitmap;
+ uint64_t bitmap_size;
+ uint64_t * bitmap;
};
struct uhandle_s * unique_init()
@@ -106,12 +106,12 @@ struct uhandle_s * unique_init()
void unique_exit(struct uhandle_s * uh)
{
if (uh->bitmap)
- free(uh->bitmap);
+ xfree(uh->bitmap);
if (uh->hash)
- free(uh->hash);
+ xfree(uh->hash);
if (uh->list)
- free(uh->list);
- free(uh);
+ xfree(uh->list);
+ xfree(uh);
}
int unique_compare(const void * a, const void * b)
@@ -147,21 +147,21 @@ void unique_count_bitmap(struct uhandle_s * uh,
xrealloc(uh->list, sizeof(unsigned int) * uh->alloc);
}
- unsigned long size = 1UL << (k << 1UL);
+ uint64_t size = 1UL << (k << 1UL);
/* reallocate bitmap arrays if necessary */
if (uh->bitmap_size < size)
{
- uh->bitmap = (unsigned long *) xrealloc(uh->bitmap, size >> 3UL);
+ uh->bitmap = (uint64_t *) xrealloc(uh->bitmap, size >> 3UL);
uh->bitmap_size = size;
}
memset(uh->bitmap, 0, size >> 3UL);
- unsigned long bad = 0;
- unsigned long kmer = 0;
- unsigned long mask = size - 1UL;
+ uint64_t bad = 0;
+ uint64_t kmer = 0;
+ uint64_t mask = size - 1UL;
char * s = seq;
char * e1 = s + k-1;
char * e2 = s + seqlen;
@@ -194,8 +194,8 @@ void unique_count_bitmap(struct uhandle_s * uh,
if (!bad)
{
- unsigned long x = kmer >> 6UL;
- unsigned long y = 1UL << (kmer & 63UL);
+ uint64_t x = kmer >> 6UL;
+ uint64_t y = 1UL << (kmer & 63UL);
if (!(uh->bitmap[x] & y))
{
/* not seen before */
@@ -238,8 +238,8 @@ void unique_count_hash(struct uhandle_s * uh,
memset(uh->hash, 0, sizeof(struct bucket_s) * uh->size);
- unsigned long bad = 0;
- unsigned long j;
+ uint64_t bad = 0;
+ uint64_t j;
unsigned int kmer = 0;
unsigned int mask = (1<<(2*k)) - 1;
char * s = seq;
@@ -260,7 +260,7 @@ void unique_count_hash(struct uhandle_s * uh,
kmer |= chrmap_2bit[(int)(*s++)];
}
- unsigned long unique = 0;
+ uint64_t unique = 0;
while (s < e2)
{
@@ -321,8 +321,8 @@ int unique_count_shared(struct uhandle_s * uh,
for(int i = 0; i<listlen; i++)
{
unsigned int kmer = list[i];
- unsigned long x = kmer >> 6UL;
- unsigned long y = 1UL << (kmer & 63UL);
+ uint64_t x = kmer >> 6UL;
+ uint64_t y = 1UL << (kmer & 63UL);
if (uh->bitmap[x] & y)
count++;
}
@@ -332,7 +332,7 @@ int unique_count_shared(struct uhandle_s * uh,
for(int i = 0; i<listlen; i++)
{
unsigned int kmer = list[i];
- unsigned long j = HASH((char*)&kmer, (k+3)/4) & uh->hash_mask;
+ uint64_t j = HASH((char*)&kmer, (k+3)/4) & uh->hash_mask;
while((uh->hash[j].count) && (uh->hash[j].kmer != kmer))
j = (j + 1) & uh->hash_mask;
if (uh->hash[j].count)
diff --git a/src/unique.h b/src/unique.h
index f9c8ceb..9d4062c 100644
--- a/src/unique.h
+++ b/src/unique.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/userfields.cc b/src/userfields.cc
index 6d722ff..0c26505 100644
--- a/src/userfields.cc
+++ b/src/userfields.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -124,7 +124,7 @@ int parse_userfields_arg(char * arg)
if (*p++ == '+')
userfields_requested_count++;
- userfields_requested = (int*) xmalloc(sizeof(int) * (unsigned long)userfields_requested_count);
+ userfields_requested = (int*) xmalloc(sizeof(int) * (uint64_t)userfields_requested_count);
p = arg;
@@ -138,7 +138,7 @@ int parse_userfields_arg(char * arg)
if (!q)
q = e;
- unsigned long n = (unsigned long)(q - p);
+ uint64_t n = (uint64_t)(q - p);
char ** u = (char**) userfields_names;
diff --git a/src/userfields.h b/src/userfields.h
index 62e876c..88606f9 100644
--- a/src/userfields.h
+++ b/src/userfields.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/src/util.cc b/src/util.cc
index c28aac0..71a5281 100644
--- a/src/util.cc
+++ b/src/util.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -61,13 +61,13 @@
#include "vsearch.h"
static const char * progress_prompt;
-static unsigned long progress_next;
-static unsigned long progress_size;
-static unsigned long progress_chunk;
-static const unsigned long progress_granularity = 200;
+static uint64_t progress_next;
+static uint64_t progress_size;
+static uint64_t progress_chunk;
+static const uint64_t progress_granularity = 200;
static bool progress_terminal;
-void progress_init(const char * prompt, unsigned long size)
+void progress_init(const char * prompt, uint64_t size)
{
progress_terminal = isatty(fileno(stderr));
progress_prompt = prompt;
@@ -84,7 +84,7 @@ void progress_init(const char * prompt, unsigned long size)
}
}
-void progress_update(unsigned long progress)
+void progress_update(uint64_t progress)
{
if ((! opt_quiet) && progress_terminal)
if (progress >= progress_next)
@@ -108,18 +108,6 @@ void progress_done()
}
}
-long gcd(long a, long b)
-{
- if (b == 0)
- {
- return a;
- }
- else
- {
- return gcd(b, a % b);
- }
-}
-
void __attribute__((noreturn)) fatal(const char * msg)
{
fprintf(stderr, "\n\n");
@@ -151,27 +139,6 @@ void __attribute__((noreturn)) fatal(const char * format,
exit(EXIT_FAILURE);
}
-void * xmalloc(size_t size)
-{
- const size_t alignment = 16;
- void * t = 0;
- if (posix_memalign(& t, alignment, size))
- fatal("Unable to allocate enough memory.");
- if (!t)
- fatal("Unable to allocate enough memory.");
- return t;
-}
-
-void * xrealloc(void *ptr, size_t size)
-{
- if (size == 0)
- size = 1;
- void * t = realloc(ptr, size);
- if (!t)
- fatal("Unable to allocate enough memory.");
- return t;
-}
-
char * xstrdup(const char *s)
{
size_t len = strlen(s);
@@ -190,12 +157,28 @@ char * xstrchrnul(char *s, int c)
return (char *)s + strlen(s);
}
-unsigned long hash_cityhash64(char * s, unsigned long n)
+int xsprintf(char * * ret, const char * format, ...)
+{
+ va_list ap;
+ va_start(ap, format);
+ int len = vsnprintf(0, 0, format, ap);
+ va_end(ap);
+ if (len < 0)
+ fatal("Error with vsnprintf in xsprintf");
+ char * p = (char *) xmalloc(len + 1);
+ va_start(ap, format);
+ len = vsnprintf(p, len + 1, format, ap);
+ va_end(ap);
+ *ret = p;
+ return len;
+}
+
+uint64_t hash_cityhash64(char * s, uint64_t n)
{
return CityHash64((const char*)s, n);
}
-long getusec(void)
+int64_t getusec(void)
{
struct timeval tv;
if(gettimeofday(&tv,0) != 0) return 0;
@@ -205,52 +188,39 @@ long getusec(void)
void show_rusage()
{
#if 0
- struct rusage r_usage;
- getrusage(RUSAGE_SELF, & r_usage);
+ double user_time = 0.0;
+ double system_time = 0.0;
- fprintf(stderr, "Time: %.3fs (user)", r_usage.ru_utime.tv_sec * 1.0 + (double) r_usage.ru_utime.tv_usec * 1.0e-6);
- fprintf(stderr, " %.3fs (sys)", r_usage.ru_stime.tv_sec * 1.0 + r_usage.ru_stime.tv_usec * 1.0e-6);
-
- fprintf(stderr, " Memory: %luMB\n", arch_get_memused() / 1024 / 1024);
+ arch_get_user_system_time(&user_time, &system_time);
+ double megabytes = arch_get_memused() / 1024.0 / 1024.0;
+
+ fprintf(stderr, "Time: %.3fs (user) %.3fs (sys) Memory: %.0lfMB\n",
+ user_time, system_time, megabytes);
+
if (opt_log)
- {
- fprintf(fp_log, "Time: %.3fs (user)", r_usage.ru_utime.tv_sec * 1.0 + (double) r_usage.ru_utime.tv_usec * 1.0e-6);
- fprintf(fp_log, " %.3fs (sys)", r_usage.ru_stime.tv_sec * 1.0 + r_usage.ru_stime.tv_usec * 1.0e-6);
-
- fprintf(fp_log, " Memory: %luMB\n", arch_get_memused() / 1024 / 1024);
- }
+ fprintf(fp_log, "Time: %.3fs (user) %.3fs (sys) Memory: %.0lfMB\n",
+ user_time, system_time, megabytes);
#endif
}
-void reverse_complement(char * rc, char * seq, long len)
+void reverse_complement(char * rc, char * seq, int64_t len)
{
/* Write the reverse complementary sequence to rc.
The memory for rc must be long enough for the rc of the sequence
(identical to the length of seq + 1. */
- for(long i=0; i<len; i++)
+ for(int64_t i=0; i<len; i++)
rc[i] = chrmap_complement[(int)(seq[len-1-i])];
rc[len] = 0;
}
void random_init()
{
- /* initialize pseudo-random number generator */
- unsigned int seed = opt_randseed;
- if (seed == 0)
- {
- int fd = open("/dev/urandom", O_RDONLY);
- if (fd < 0)
- fatal("Unable to open /dev/urandom");
- if (read(fd, & seed, sizeof(seed)) < 0)
- fatal("Unable to read from /dev/urandom");
- close(fd);
- }
- srandom(seed);
+ arch_srandom();
}
-long random_int(long n)
+int64_t random_int(int64_t n)
{
/*
Generate a random integer in the range 0 to n-1, inclusive.
@@ -261,28 +231,28 @@ long random_int(long n)
avoid modulo bias.
*/
- long random_max = RAND_MAX;
- long limit = random_max - (random_max + 1) % n;
- long r = random();
+ int64_t random_max = RAND_MAX;
+ int64_t limit = random_max - (random_max + 1) % n;
+ int64_t r = arch_random();
while (r > limit)
- r = random();
+ r = arch_random();
return r % n;
}
-unsigned long random_ulong(unsigned long n)
+uint64_t random_ulong(uint64_t n)
{
/*
Generate a random integer in the range 0 to n-1, inclusive,
n must be > 0
*/
- unsigned long random_max = ULONG_MAX;
- unsigned long limit = random_max - (random_max - n + 1) % n;
- unsigned long r = ((random() << 48) ^ (random() << 32) ^
- (random() << 16) ^ (random()));
+ uint64_t random_max = ULONG_MAX;
+ uint64_t limit = random_max - (random_max - n + 1) % n;
+ uint64_t r = ((arch_random() << 48) ^ (arch_random() << 32) ^
+ (arch_random() << 16) ^ (arch_random()));
while (r > limit)
- r = ((random() << 48) ^ (random() << 32) ^
- (random() << 16) ^ (random()));
+ r = ((arch_random() << 48) ^ (arch_random() << 32) ^
+ (arch_random() << 16) ^ (arch_random()));
return r % n;
}
@@ -337,7 +307,7 @@ void get_hex_seq_digest_sha1(char * hex, char * seq, int seqlen)
SHA1((const unsigned char*) normalized, (size_t) seqlen, digest);
- free(normalized);
+ xfree(normalized);
for(int i=0; i<LEN_DIG_SHA1; i++)
{
@@ -360,7 +330,7 @@ void get_hex_seq_digest_md5(char * hex, char * seq, int seqlen)
MD5(normalized, (size_t) seqlen, digest);
- free(normalized);
+ xfree(normalized);
for(int i=0; i<LEN_DIG_MD5; i++)
{
diff --git a/src/util.h b/src/util.h
index 012d335..d52150c 100644
--- a/src/util.h
+++ b/src/util.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -79,28 +79,26 @@
#define LEN_HEX_DIG_MD5 (2*LEN_DIG_MD5+1)
#define LEN_HEX_DIG_SHA1 (2*LEN_DIG_SHA1+1)
-long gcd(long a, long b);
void fatal(const char * msg);
void fatal(const char * format, const char * message);
-void * xmalloc(size_t size);
-void * xrealloc(void * ptr, size_t size);
char * xstrdup(const char *s);
char * xstrchrnul(char *s, int c);
-unsigned long hash_cityhash64(char * s, unsigned long n);
-long getusec(void);
+int xsprintf(char * * ret, const char * format, ...);
+uint64_t hash_cityhash64(char * s, uint64_t n);
+int64_t getusec(void);
void show_rusage();
-void progress_init(const char * prompt, unsigned long size);
-void progress_update(unsigned long progress);
+void progress_init(const char * prompt, uint64_t size);
+void progress_update(uint64_t progress);
void progress_done();
void random_init();
-long random_int(long n);
-unsigned long random_ulong(unsigned long n);
+int64_t random_int(int64_t n);
+uint64_t random_ulong(uint64_t n);
void string_normalize(char * normalized, char * s, unsigned int len);
-void reverse_complement(char * rc, char * seq, long len);
+void reverse_complement(char * rc, char * seq, int64_t len);
void fprint_hex(FILE * fp, unsigned char * data, int len);
diff --git a/src/vsearch.cc b/src/vsearch.cc
index 291b9af..ac89394 100644
--- a/src/vsearch.cc
+++ b/src/vsearch.cc
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -186,82 +186,83 @@ int opt_slots;
int opt_uchimeout5;
int opt_usersort;
int opt_version;
-long opt_dbmask;
-long opt_fasta_width;
-long opt_fastq_ascii;
-long opt_fastq_asciiout;
-long opt_fastq_maxdiffs;
-long opt_fastq_maxlen;
-long opt_fastq_maxmergelen;
-long opt_fastq_maxns;
-long opt_fastq_minlen;
-long opt_fastq_minmergelen;
-long opt_fastq_minovlen;
-long opt_fastq_qmax;
-long opt_fastq_qmaxout;
-long opt_fastq_qmin;
-long opt_fastq_qminout;
-long opt_fastq_stripleft;
-long opt_fastq_tail;
-long opt_fastq_trunclen;
-long opt_fastq_trunclen_keep;
-long opt_fastq_truncqual;
-long opt_fulldp;
-long opt_hardmask;
-long opt_iddef;
-long opt_idprefix;
-long opt_idsuffix;
-long opt_leftjust;
-long opt_match;
-long opt_maxaccepts;
-long opt_maxdiffs;
-long opt_maxgaps;
-long opt_maxhits;
-long opt_maxqsize;
-long opt_maxrejects;
-long opt_maxseqlength;
-long opt_maxsize;
-long opt_maxsubs;
-long opt_maxuniquesize;
-long opt_mincols;
-long opt_minseqlength;
-long opt_minsize;
-long opt_mintsize;
-long opt_minuniquesize;
-long opt_minwordmatches;
-long opt_mismatch;
-long opt_notrunclabels;
-long opt_output_no_hits;
-long opt_qmask;
-long opt_randseed;
-long opt_rightjust;
-long opt_rowlen;
-long opt_sample_size;
-long opt_self;
-long opt_selfid;
-long opt_sizein;
-long opt_sizeout;
-long opt_strand;
-long opt_threads;
-long opt_top_hits_only;
-long opt_topn;
-long opt_uc_allhits;
-long opt_wordlength;
+int64_t opt_dbmask;
+int64_t opt_fasta_width;
+int64_t opt_fastq_ascii;
+int64_t opt_fastq_asciiout;
+int64_t opt_fastq_maxdiffs;
+int64_t opt_fastq_maxlen;
+int64_t opt_fastq_maxmergelen;
+int64_t opt_fastq_maxns;
+int64_t opt_fastq_minlen;
+int64_t opt_fastq_minmergelen;
+int64_t opt_fastq_minovlen;
+int64_t opt_fastq_qmax;
+int64_t opt_fastq_qmaxout;
+int64_t opt_fastq_qmin;
+int64_t opt_fastq_qminout;
+int64_t opt_fastq_stripleft;
+int64_t opt_fastq_tail;
+int64_t opt_fastq_trunclen;
+int64_t opt_fastq_trunclen_keep;
+int64_t opt_fastq_truncqual;
+int64_t opt_fulldp;
+int64_t opt_hardmask;
+int64_t opt_iddef;
+int64_t opt_idprefix;
+int64_t opt_idsuffix;
+int64_t opt_leftjust;
+int64_t opt_match;
+int64_t opt_maxaccepts;
+int64_t opt_maxdiffs;
+int64_t opt_maxgaps;
+int64_t opt_maxhits;
+int64_t opt_maxqsize;
+int64_t opt_maxrejects;
+int64_t opt_maxseqlength;
+int64_t opt_maxsize;
+int64_t opt_maxsubs;
+int64_t opt_maxuniquesize;
+int64_t opt_mincols;
+int64_t opt_minseqlength;
+int64_t opt_minsize;
+int64_t opt_mintsize;
+int64_t opt_minuniquesize;
+int64_t opt_minwordmatches;
+int64_t opt_mismatch;
+int64_t opt_notrunclabels;
+int64_t opt_output_no_hits;
+int64_t opt_qmask;
+int64_t opt_randseed;
+int64_t opt_rightjust;
+int64_t opt_rowlen;
+int64_t opt_sample_size;
+int64_t opt_self;
+int64_t opt_selfid;
+int64_t opt_sizein;
+int64_t opt_sizeout;
+int64_t opt_strand;
+int64_t opt_threads;
+int64_t opt_top_hits_only;
+int64_t opt_topn;
+int64_t opt_uc_allhits;
+int64_t opt_wordlength;
/* Other variables */
/* cpu features available */
-long mmx_present = 0;
-long sse_present = 0;
-long sse2_present = 0;
-long sse3_present = 0;
-long ssse3_present = 0;
-long sse41_present = 0;
-long sse42_present = 0;
-long popcnt_present = 0;
-long avx_present = 0;
-long avx2_present = 0;
+int64_t altivec_present = 0;
+int64_t mmx_present = 0;
+int64_t sse_present = 0;
+int64_t sse2_present = 0;
+int64_t sse3_present = 0;
+int64_t ssse3_present = 0;
+int64_t sse41_present = 0;
+int64_t sse42_present = 0;
+int64_t popcnt_present = 0;
+int64_t avx_present = 0;
+int64_t avx2_present = 0;
static char * progname;
static char progheader[80];
@@ -276,13 +277,18 @@ abundance_t * global_abundance;
char * STDIN_NAME = (char*) "/dev/stdin";
char * STDOUT_NAME = (char*) "/dev/stdout";
+#ifndef __PPC__
#define cpuid(f1, f2, a, b, c, d) \
__asm__ __volatile__ ("cpuid" \
: "=a" (a), "=b" (b), "=c" (c), "=d" (d) \
: "a" (f1), "c" (f2));
+#endif
void cpu_features_detect()
{
+#ifdef __PPC__
+ altivec_present = 1;
+#else
unsigned int a, b, c, d;
cpuid(0, 0, a, b, c, d);
@@ -307,11 +313,14 @@ void cpu_features_detect()
avx2_present = (b >> 5) & 1;
}
}
+#endif
}
void cpu_features_show()
{
fprintf(stderr, "CPU features:");
+ if (altivec_present)
+ fprintf(stderr, " altivec");
if (mmx_present)
fprintf(stderr, " mmx");
if (sse_present)
@@ -493,11 +502,11 @@ void args_get_gap_penalty_string(char * arg, int is_open)
}
-long args_getlong(char * arg)
+int64_t args_getlong(char * arg)
{
int len = 0;
- long temp = 0;
- int ret = sscanf(arg, "%ld%n", &temp, &len);
+ int64_t temp = 0;
+ int ret = sscanf(arg, "%" PRId64 "%n", &temp, &len);
if ((ret == 0) || (((unsigned int)(len)) < strlen(arg)))
fatal("Illegal option argument");
return temp;
@@ -571,7 +580,7 @@ void args_init(int argc, char **argv)
opt_fastq_mergepairs = 0;
opt_fastq_minlen = 1;
opt_fastq_minmergelen = 0;
- opt_fastq_minovlen = 10;
+ opt_fastq_minovlen = 16;
opt_fastq_nostagger = 1;
opt_fastqout_notmerged_fwd = 0;
opt_fastqout_notmerged_rev = 0;
@@ -969,7 +978,7 @@ void args_init(int argc, char **argv)
break;
case 13:
- opt_threads = (long) args_getdouble(optarg);
+ opt_threads = (int64_t) args_getdouble(optarg);
break;
case 14:
@@ -1769,12 +1778,17 @@ void args_init(int argc, char **argv)
if ((opt_iddef < 0) || (opt_iddef > 4))
fatal("The argument to --iddef must in the range 0 to 4");
+#if 0
+
if (opt_match <= 0)
fatal("The argument to --match must be positive");
if (opt_mismatch >= 0)
fatal("The argument to --mismatch must be negative");
+#endif
+
+
if (opt_alignwidth < 0)
fatal("The argument to --alignwidth must not be negative");
@@ -1853,7 +1867,7 @@ void args_init(int argc, char **argv)
opt_minwordmatches = minwordmatches_defaults[opt_wordlength];
if (opt_threads == 0)
- opt_threads = sysconf(_SC_NPROCESSORS_ONLN);
+ opt_threads = arch_get_cores();
/* set default opt_minseqlength depending on command */
@@ -2086,7 +2100,8 @@ void cmd_help()
" --fastq_qmax INT maximum base quality value for FASTQ input (41)\n"
" --fastq_qmin INT minimum base quality value for FASTQ input (0)\n"
" Output\n"
- " --log FILENAME output statistics\n"
+ " --log FILENAME output file for statistics with --fastq_stats\n"
+ " --output FILENAME output file for statistics with --fastq_eestats\n"
"\n"
"Filtering\n"
" --fastx_filter FILENAME filter and truncate sequences in FASTA/FASTQ file\n"
@@ -2384,7 +2399,7 @@ void cmd_rereplicate()
void cmd_derep()
{
if ((!opt_output) && (!opt_uc))
- fatal("Output file for derepl_fulllength must be specified with --output or --uc");
+ fatal("Output file for derep_fulllength must be specified with --output or --uc");
if (opt_derep_fulllength)
derep_fulllength();
@@ -2446,7 +2461,7 @@ void cmd_none()
fprintf(stderr,
"For help, please enter: %s --help\n"
"\n"
- "For further details, please see the manual by entering: man vsearch\n"
+ "For further details, please see the manual.\n"
"\n"
"Example commands:\n"
"\n"
@@ -2585,7 +2600,7 @@ void fillheader()
"%s v%s_%s, %.1fGB RAM, %ld cores",
PROG_NAME, PROG_VERSION, PROG_ARCH,
arch_get_memtotal() / 1024.0 / 1024.0 / 1024.0,
- sysconf(_SC_NPROCESSORS_ONLN));
+ arch_get_cores());
}
void getentirecommandline(int argc, char** argv)
@@ -2594,13 +2609,14 @@ void getentirecommandline(int argc, char** argv)
for (int i=0; i<argc; i++)
len += strlen(argv[i]);
- cmdline = (char*) xmalloc(len+argc+1);
+ cmdline = (char*) xmalloc(len+argc);
cmdline[0] = 0;
for (int i=0; i<argc; i++)
{
+ if (i>0)
+ strcat(cmdline, " ");
strcat(cmdline, argv[i]);
- strcat(cmdline, " ");
}
}
@@ -2617,14 +2633,13 @@ void show_header()
int main(int argc, char** argv)
{
fillheader();
+
getentirecommandline(argc, argv);
cpu_features_detect();
args_init(argc, argv);
- dynlibs_open();
-
if (opt_log)
{
fp_log = fopen(opt_log, "w");
@@ -2643,8 +2658,12 @@ int main(int argc, char** argv)
show_header();
+ dynlibs_open();
+
+#ifndef __PPC__
if (!sse2_present)
fatal("Sorry, this program requires a cpu with SSE2.");
+#endif
global_abundance = abundance_init();
@@ -2700,7 +2719,6 @@ int main(int argc, char** argv)
if (opt_log)
{
-
time_finish = time(0);
struct tm tm_finish;
localtime_r(& time_finish, & tm_finish);
@@ -2709,10 +2727,11 @@ int main(int argc, char** argv)
fprintf(fp_log, "\n");
fprintf(fp_log, "Finished %s", time_string);
- time_t time_diff = time_finish - time_start;
+ double time_diff = difftime(time_finish, time_start);
fprintf(fp_log, "\n");
- fprintf(fp_log, "Elapsed time %02lu:%02lu\n",
- time_diff / 60, time_diff % 60);
+ fprintf(fp_log, "Elapsed time %02.0lf:%02.0lf\n",
+ floor(time_diff / 60.0),
+ floor(time_diff - 60.0 * floor(time_diff / 60.0)));
double maxmem = arch_get_memused() / 1048576.0;
if (maxmem < 1024.0)
fprintf(fp_log, "Max memory %.1lfMB\n", maxmem);
@@ -2721,9 +2740,7 @@ int main(int argc, char** argv)
fclose(fp_log);
}
- free(cmdline);
-
+ xfree(cmdline);
abundance_exit(global_abundance);
-
dynlibs_close();
}
diff --git a/src/vsearch.h b/src/vsearch.h
index d13d4a6..f2c244b 100644
--- a/src/vsearch.h
+++ b/src/vsearch.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
@@ -58,10 +58,22 @@
*/
+#define __STDC_FORMAT_MACROS 1
+#define __restrict
+
+#ifdef HAVE_CONFIG_H
+#include "config.h"
+#endif
+
+#include <regex.h>
+
+#include <string>
+#include <set>
+#include <map>
+
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/time.h>
-#include <sys/resource.h>
#include <stdio.h>
#include <string.h>
#include <pthread.h>
@@ -72,16 +84,28 @@
#include <locale.h>
#include <math.h>
#include <ctype.h>
-#include <regex.h>
#include <fcntl.h>
#include <unistd.h>
#include <float.h>
-#include <dlfcn.h>
+#include <inttypes.h>
+#include <stdarg.h>
-#ifdef HAVE_CONFIG_H
-#include "config.h"
+#define PROG_NAME PACKAGE
+#define PROG_VERSION PACKAGE_VERSION
+
+#ifdef __PPC__
+
+#ifdef __LITTLE_ENDIAN__
+#define PROG_CPU "ppc64le"
+#include <altivec.h>
+#else
+#error Big endian ppc64 CPUs not supported
#endif
+#else
+
+#define PROG_CPU "x86_64"
+
#ifdef __SSE2__
#include <emmintrin.h>
#endif
@@ -90,12 +114,32 @@
#include <tmmintrin.h>
#endif
+#endif
+
#ifdef __APPLE__
+#define PROG_OS "osx"
+#include <sys/resource.h>
#include <sys/sysctl.h>
-#else
+#endif
+
+#ifdef __linux__
+#define PROG_OS "linux"
+#include <sys/resource.h>
#include <sys/sysinfo.h>
#endif
+#ifdef _WIN32
+#define PROG_OS "win"
+#include <windows.h>
+#include <psapi.h>
+#endif
+
+#define PROG_ARCH PROG_OS "_" PROG_CPU
+
+#ifdef HAVE_DLFCN_H
+#include <dlfcn.h>
+#endif
+
#ifdef HAVE_ZLIB_H
#include <zlib.h>
#endif
@@ -104,16 +148,16 @@
#include <bzlib.h>
#endif
-#include "dynlibs.h"
#include "city.h"
-#include "citycrc.h"
#include "md5.h"
#include "sha1.h"
+
+#include "arch.h"
+#include "dynlibs.h"
#include "util.h"
#include "xstring.h"
#include "align_simd.h"
#include "maps.h"
-#include "arch.h"
#include "abundance.h"
#include "db.h"
#include "align.h"
@@ -149,15 +193,6 @@
#include "rerep.h"
#include "otutable.h"
-#define PROG_NAME PACKAGE
-#define PROG_VERSION PACKAGE_VERSION
-
-#ifdef __APPLE__
-#define PROG_ARCH "osx_x86_64"
-#else
-#define PROG_ARCH "linux_x86_64"
-#endif
-
/* options */
extern bool opt_bzip2_decompress;
@@ -284,78 +319,79 @@ extern int opt_slots;
extern int opt_uchimeout5;
extern int opt_usersort;
extern int opt_version;
-extern long opt_dbmask;
-extern long opt_fasta_width;
-extern long opt_fastq_ascii;
-extern long opt_fastq_asciiout;
-extern long opt_fastq_maxdiffs;
-extern long opt_fastq_maxlen;
-extern long opt_fastq_maxmergelen;
-extern long opt_fastq_maxns;
-extern long opt_fastq_minlen;
-extern long opt_fastq_minmergelen;
-extern long opt_fastq_minovlen;
-extern long opt_fastq_qmax;
-extern long opt_fastq_qmaxout;
-extern long opt_fastq_qmin;
-extern long opt_fastq_qminout;
-extern long opt_fastq_stripleft;
-extern long opt_fastq_tail;
-extern long opt_fastq_trunclen;
-extern long opt_fastq_trunclen_keep;
-extern long opt_fastq_truncqual;
-extern long opt_fulldp;
-extern long opt_hardmask;
-extern long opt_iddef;
-extern long opt_idprefix;
-extern long opt_idsuffix;
-extern long opt_leftjust;
-extern long opt_match;
-extern long opt_maxaccepts;
-extern long opt_maxdiffs;
-extern long opt_maxgaps;
-extern long opt_maxhits;
-extern long opt_maxqsize;
-extern long opt_maxrejects;
-extern long opt_maxseqlength;
-extern long opt_maxsize;
-extern long opt_maxsubs;
-extern long opt_maxuniquesize;
-extern long opt_mincols;
-extern long opt_minseqlength;
-extern long opt_minsize;
-extern long opt_mintsize;
-extern long opt_minuniquesize;
-extern long opt_minwordmatches;
-extern long opt_mismatch;
-extern long opt_notrunclabels;
-extern long opt_output_no_hits;
-extern long opt_qmask;
-extern long opt_randseed;
-extern long opt_rightjust;
-extern long opt_rowlen;
-extern long opt_sample_size;
-extern long opt_self;
-extern long opt_selfid;
-extern long opt_sizein;
-extern long opt_sizeout;
-extern long opt_strand;
-extern long opt_threads;
-extern long opt_top_hits_only;
-extern long opt_topn;
-extern long opt_uc_allhits;
-extern long opt_wordlength;
-
-extern long mmx_present;
-extern long sse_present;
-extern long sse2_present;
-extern long sse3_present;
-extern long ssse3_present;
-extern long sse41_present;
-extern long sse42_present;
-extern long popcnt_present;
-extern long avx_present;
-extern long avx2_present;
+extern int64_t opt_dbmask;
+extern int64_t opt_fasta_width;
+extern int64_t opt_fastq_ascii;
+extern int64_t opt_fastq_asciiout;
+extern int64_t opt_fastq_maxdiffs;
+extern int64_t opt_fastq_maxlen;
+extern int64_t opt_fastq_maxmergelen;
+extern int64_t opt_fastq_maxns;
+extern int64_t opt_fastq_minlen;
+extern int64_t opt_fastq_minmergelen;
+extern int64_t opt_fastq_minovlen;
+extern int64_t opt_fastq_qmax;
+extern int64_t opt_fastq_qmaxout;
+extern int64_t opt_fastq_qmin;
+extern int64_t opt_fastq_qminout;
+extern int64_t opt_fastq_stripleft;
+extern int64_t opt_fastq_tail;
+extern int64_t opt_fastq_trunclen;
+extern int64_t opt_fastq_trunclen_keep;
+extern int64_t opt_fastq_truncqual;
+extern int64_t opt_fulldp;
+extern int64_t opt_hardmask;
+extern int64_t opt_iddef;
+extern int64_t opt_idprefix;
+extern int64_t opt_idsuffix;
+extern int64_t opt_leftjust;
+extern int64_t opt_match;
+extern int64_t opt_maxaccepts;
+extern int64_t opt_maxdiffs;
+extern int64_t opt_maxgaps;
+extern int64_t opt_maxhits;
+extern int64_t opt_maxqsize;
+extern int64_t opt_maxrejects;
+extern int64_t opt_maxseqlength;
+extern int64_t opt_maxsize;
+extern int64_t opt_maxsubs;
+extern int64_t opt_maxuniquesize;
+extern int64_t opt_mincols;
+extern int64_t opt_minseqlength;
+extern int64_t opt_minsize;
+extern int64_t opt_mintsize;
+extern int64_t opt_minuniquesize;
+extern int64_t opt_minwordmatches;
+extern int64_t opt_mismatch;
+extern int64_t opt_notrunclabels;
+extern int64_t opt_output_no_hits;
+extern int64_t opt_qmask;
+extern int64_t opt_randseed;
+extern int64_t opt_rightjust;
+extern int64_t opt_rowlen;
+extern int64_t opt_sample_size;
+extern int64_t opt_self;
+extern int64_t opt_selfid;
+extern int64_t opt_sizein;
+extern int64_t opt_sizeout;
+extern int64_t opt_strand;
+extern int64_t opt_threads;
+extern int64_t opt_top_hits_only;
+extern int64_t opt_topn;
+extern int64_t opt_uc_allhits;
+extern int64_t opt_wordlength;
+
+extern int64_t altivec_present;
+extern int64_t mmx_present;
+extern int64_t sse_present;
+extern int64_t sse2_present;
+extern int64_t sse3_present;
+extern int64_t ssse3_present;
+extern int64_t sse41_present;
+extern int64_t sse42_present;
+extern int64_t popcnt_present;
+extern int64_t avx_present;
+extern int64_t avx2_present;
extern FILE * fp_log;
diff --git a/src/xstring.h b/src/xstring.h
index 5b1d310..317ad0e 100644
--- a/src/xstring.h
+++ b/src/xstring.h
@@ -2,7 +2,7 @@
VSEARCH: a versatile open source tool for metagenomics
- Copyright (C) 2014-2015, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
+ Copyright (C) 2014-2017, Torbjorn Rognes, Frederic Mahe and Tomas Flouri
All rights reserved.
Contact: Torbjorn Rognes <torognes at ifi.uio.no>,
diff --git a/test/unclassified.sh b/test/unclassified.sh
index 3ed03d7..24f8f23 100644
--- a/test/unclassified.sh
+++ b/test/unclassified.sh
@@ -214,6 +214,76 @@ C=$(printf ">seq1\nAACC\n>seq2\nGGTT\n" | \
# clean
unset C
+
+#*****************************************************************************#
+# #
+# Avoid progress indicator if stderr is not a terminal (issue 156) #
+# #
+#*****************************************************************************#
+
+# https://github.com/torognes/vsearch/issues/156
+# Avoid updating the progress indicator when stderr is not a terminal
+
+# In practice, stderr is not a tty when --log is used and is a
+# file. Maybe the issue should be renamed "Avoid writing progress
+# indicator to log file"?
+
+DESCRIPTION="do not output progress when stderr is a tty and stdout is a tty"
+"${VSEARCH}" \
+ --fastx_mask <(printf ">seq1\nACGTattggatcccttataTTA\n") \
+ --fastaout - 2>&1 | \
+ grep -q "Writing output" && \
+ failure "${DESCRIPTION}" || \
+ success "${DESCRIPTION}" # should we avoid visually mixed output?
+
+DESCRIPTION="output progress when stderr is a redirection and stdout is a tty"
+"${VSEARCH}" \
+ --fastx_mask <(printf ">seq1\nACGTattggatcccttataTTA\n") \
+ --fastaout - 2>&1 | \
+ grep -q "Writing output" && \
+ success "${DESCRIPTION}" || \
+ failure "${DESCRIPTION}"
+
+DESCRIPTION="output progress when log, stderr and stdout are ttys"
+"${VSEARCH}" \
+ --fastx_mask <(printf ">seq1\nACGTattggatcccttataTTA\n") \
+ --log - \
+ --fastaout - 2>&1 | \
+ grep -q "Writing output" && \
+ success "${DESCRIPTION}" || \
+ failure "${DESCRIPTION}"
+
+DESCRIPTION="do not output progress when stderr is a redirection"
+"${VSEARCH}" \
+ --fastx_mask <(printf ">seq1\nACGTattggatcccttataTTA\n") \
+ --fastaout /dev/null 2>&1 | \
+ grep -q "Writing output" && \
+ failure "${DESCRIPTION}" || \
+ success "${DESCRIPTION}" # can vsearch know if stderr is
+ # attached to anything else than a
+ # tty?
+
+DESCRIPTION="do not output progress when log is a file and stderr is a redirection"
+PROGRESS=$(mktemp)
+"${VSEARCH}" \
+ --fastx_mask <(printf ">seq1\nACGTattggatcccttataTTA\n") \
+ --log ${PROGRESS} \
+ --fastaout - > /dev/null 2>> ${PROGRESS}
+grep -q "Writing output" ${PROGRESS} && \
+ failure "${DESCRIPTION}" || \
+ success "${DESCRIPTION}"
+rm ${PROGRESS}
+
+DESCRIPTION="do not output progress when log is a process substitution (named pipe)"
+"${VSEARCH}" \
+ --fastx_mask <(printf ">seq1\nACGTattggatcccttataTTA\n") \
+ --log >(grep -q "Writing output" && echo yes) \
+ --fastaout - 2>&1 | \
+ grep -q "Writing output" && \
+ failure "${DESCRIPTION}" || \
+ success "${DESCRIPTION}"
+
+
#*****************************************************************************#
# #
# fastq_trunclen and discarded short sequences (issue 203) #
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/debian-med/vsearch.git
More information about the debian-med-commit
mailing list