[Debian-med-packaging] Bug#970344: ncbi-blast+: tblastn segfaults intermittently with multiple threads

[Étienne Mollier]

Package: ncbi-blast+
Version: 2.10.0-3
Severity: normal
Tags: upstream

Greetings,

Andreas first noticed this issue when running autopkgtest on the
package kaptive.  The test crashes intermittently on amd64 and
i386, with segmentation fault.  At the same time, the following
message is appearing in `dmesg` output:

	traps: tblastn[3579372] general protection fault ip:7ff551327dde sp:7ffdb8072790 error:0 in libblast.so[7ff55130f000+63000]

What happens is that the kaptive.py script runs tblastn, which
in turn seemed to be the source of the problem.  To reproduce
the issue, one needs the following resources:

  * sample data from the kaptive package, which can be
    downloaded for instance at:

        https://salsa.debian.org/med-team/kaptive/-/raw/master/sample_data/exact_match.fasta

  * temp_gene_seqs.fasta, which I try to attach to the present
    report if the engine accepts attachments of 264 kiB; it is
    an artifact that remains from a attempt to run kaptive which
    failed with with the error above.

Prior to running the reproducer, data has to be prepared:

	unxz temp_gene_seqs.fasta.xz
	makeblastdb -dbtype nucl -in exact_match.fasta

Then the following command may or may not crash:

	tblastn -db_gencode 11 -seg no -db exact_match.fasta \
		-query temp_gene_seqs.fasta \
		-num_threads 4 \
		-outfmt '6 qseqid sseqid qstart qend sstart send evalue bitscore length pident qlen qseq sseq'

One has to run tblastn several times to see the test either
fail, or run successfully.  Feel free to adjust the number of
CPUs to see the frequency at which issues occur, as there seem
to be different behaviors depending on the thread count.

When the number of threads run (option -num_threads) is 1, I do
not see any issues when running the tblastn command.  Above 4
concurrent threads, the reproducer almost always crashes.

Apparently there are no such issue seen on arm64, or other non
x86 architectures, for the moment.

I hope this helps,
Kind Regards,
Étienne.


PS: as a side note, at high thread count, I saw some other
variants of kernel messages, but I'm not sure how they related
to the above issue, they also seemed to occur earlier in the
process:

        tblastn[882692]: segfault at 7efcf02adb7c ip 00007efce84bd90d sp 00007efce4fc1070 error 6 in libblast.so[7efce848a000+63000]
        Code: fc ff 48 63 4d 1c 48 89 45 28 48 89 c6 48 8b 45 08 48 01 c1 48 39 c8 72 1e e9 2e ff ff ff 0f 1f 80 00 00 00 00 49 63 54 95 00 <83> 04 96 01 48 39 c1 0f 84 15 ff ff ff 48 0f be 10 48 83 c0 01 41

        tblastn[882958]: segfault at 7f30ec42b000 ip 00007f3118e12acf sp 00007f31159160c0 error 4 in libblast.so[7f3118ddf000+63000]
        Code: 48 89 73 08 41 80 3c 03 00 75 50 49 63 04 82 49 8d 04 81 8b 10 8d 72 01 89 30 48 8b 43 30 0f 1f 44 00 00 48 89 c1 48 83 c0 04 <3b> 50 fc 75 f4 89 31 f2 0f 10 43 38 66 0f 2f 05 7d 0c 03 00 41 b8


-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.7.14 (SMP w/6 CPU threads; PREEMPT)
Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages ncbi-blast+ depends on:
ii  libbz2-1.0      1.0.8-4
ii  libc6           2.31-3
ii  libgcc-s1       10.2.0-7
ii  libgomp1        10.2.0-7
ii  liblmdb0        0.9.24-1
ii  libmbedcrypto3  2.16.5-1
ii  libmbedtls12    2.16.5-1
ii  libpcre3        2:8.39-13
ii  libstdc++6      10.2.0-7
ii  ncbi-data       6.1.20170106+dfsg1-8
ii  perl            5.30.3-4
ii  python3         3.8.2-3
ii  zlib1g          1:1.2.11.dfsg-2

ncbi-blast+ recommends no packages.

ncbi-blast+ suggests no packages.

-- no debconf information

-- 
Étienne Mollier <etienne.mollier at mailoo.org>
Old rsa/3072: 5ab1 4edf 63bb ccff 8b54  2fa9 59da 56fe fff3 882d
New rsa/4096: 8f91 b227 c7d6 f2b1 948c  8236 793c f67e 8f0d 11da
Sent from /dev/pts/3, please excuse my verbosity.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: temp_gene_seqs.fasta.xz
Type: application/x-xz
Size: 266412 bytes
Desc: not available
URL: <http://alioth-lists.debian.net/pipermail/debian-med-packaging/attachments/20200914/20dcff66/attachment-0001.xz>