<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Apr 17, 2020, at 13:18, Andreas Tille <<a href="mailto:tille@debian.org" class="">tille@debian.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi Matthew,<br class=""><br class="">On Fri, Apr 17, 2020 at 08:18:29AM -0700, Matthew Fernandez wrote:<br class=""><blockquote type="cite" class=""><blockquote type="cite" class="">Thanks for the patch which I applied to packaging Git. I assume you<br class="">want to express that while these fixes are definitely good coding<br class="">practice the bus error problem is not fixed by it, right?<br class=""></blockquote><br class="">Thanks, Andreas. It may fix the bus error, but I don’t have a MIPS machine<br class="">to test on. Some of those logging calls had the potential to leave you with<br class="">a misaligned stack pointer. IIUC unaligned loads on MIPS could cause such a<br class="">bus error.<br class=""></blockquote><br class="">I tried with hope ... but failed:<br class=""><br class="">(sid_mipsel-dchroot)tille@eller:~/clustalo$ gdb --args src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force<br class="">GNU gdb (Debian 9.1-3) 9.1<br class="">...<br class="">Reading symbols from src/clustalo...<br class="">(gdb) run<br class="">Starting program: /home/tille/clustalo/src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force<br class="">[Thread debugging using libthread_db enabled]<br class="">Using host libthread_db library "/lib/mipsel-linux-gnu/libthread_db.so.1".<br class=""><br class="">Program received signal SIGBUS, Bus error.<br class="">0x5556a1b8 in PairDistances (distmat=0x7fff278c, mseq=0x55692a30, pairdist_type=<optimized out>, bPercID=<optimized out>, istart=0, iend=3, jstart=0, jend=3, fdist_in=0x0, <br class=""> fdist_out=0x0) at pair_dist.c:346<br class="">346 NewProgress(&prProgress, LogGetFP(&rLog, LOG_INFO),<br class=""></div></div></blockquote><div><br class=""></div>OK, let me try a little harder :)</div><div><br class=""></div><div> $ # enable debugging symbols and Address Sanitizer</div><div> $ CFLAGS="-g -fsanitize=address" CXXFLAGS="-g -fsanitize=address" ./configure</div><div> …</div><div> $ make clean && make</div><div> …</div><div><div> $ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force</div><div> =================================================================</div><div> ==30264==ERROR: AddressSanitizer: dynamic-stack-buffer-overflow on address 0x7ffcfcbf5784 at pc 0x5620f0aa478c bp 0x7ffcfcbf56c0 sp 0x7ffcfcbf56b8</div><div> WRITE of size 4 at 0x7ffcfcbf5784 thread T0</div><div> #0 0x5620f0aa478b in PairDistances /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336</div><div> #1 0x5620f0a91d9f in AlignmentOrder /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:835</div><div> #2 0x5620f0a95c04 in Align /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:1221</div><div> #3 0x5620f0a90d76 in MyMain /home/matthew/clustal-omega-1.2.4/src/mymain.c:1192</div><div> #4 0x5620f0a88ca2 in main /home/matthew/clustal-omega-1.2.4/src/main.cpp:469</div><div> #5 0x7f3773d9009a in __libc_start_main ../csu/libc-start.c:308</div><div> #6 0x5620f0a89ad9 in _start (/home/matthew/clustal-omega-1.2.4/src/clustalo+0x2dad9)</div><div><br class=""></div><div> Address 0x7ffcfcbf5784 is located in stack of thread T0</div><div> SUMMARY: AddressSanitizer: dynamic-stack-buffer-overflow /home/matthew/clustal-omega-1.2.4/src/clustal/pair_dist.c:336 in PairDistances</div><div> Shadow bytes around the buggy address:</div><div> 0x10001f976aa0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div> 0x10001f976ab0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div> 0x10001f976ac0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div> 0x10001f976ad0: 00 00 00 00 00 00 00 00 00 00 00 00 ca ca ca ca</div><div> 0x10001f976ae0: 04 cb cb cb cb cb cb cb 00 00 00 00 ca ca ca ca</div><div> =>0x10001f976af0:[04]cb cb cb cb cb cb cb 00 00 00 00 00 00 00 00</div><div> 0x10001f976b00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div> 0x10001f976b10: f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2 00 f2 f2 f2</div><div> 0x10001f976b20: f2 f2 f2 f2 00 00 00 00 00 00 00 00 00 f2 f2 f2</div><div> 0x10001f976b30: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00</div><div> 0x10001f976b40: 00 00 00 00 00 00 f1 f1 f1 f1 00 f2 f2 f2 f2 f2</div><div> Shadow byte legend (one shadow byte represents 8 application bytes):</div><div> Addressable: 00</div><div> Partially addressable: 01 02 03 04 05 06 07</div><div> Heap left redzone: fa</div><div> Freed heap region: fd</div><div> Stack left redzone: f1</div><div> Stack mid redzone: f2</div><div> Stack right redzone: f3</div><div> Stack after return: f5</div><div> Stack use after scope: f8</div><div> Global redzone: f9</div><div> Global init order: f6</div><div> Poisoned by user: f7</div><div> Container overflow: fc</div><div> Array cookie: ac</div><div> Intra object redzone: bb</div><div> ASan internal: fe</div><div> Left alloca redzone: ca</div><div> Right alloca redzone: cb</div><div> ==30264==ABORTING</div><div><br class=""></div><div>Looking at line 336 of pair_dist.c, it looks like the bound on the containing loop is wrong. So let’s try adjusting that:</div><div><br class=""></div><div> $ vim src/clustal/pair_dist.c</div><div><div> $ git diff src/clustal/pair_dist.c</div><div> diff --git a/src/clustal/pair_dist.c b/src/clustal/pair_dist.c</div><div> index e6dbdc3..bb79e61 100644</div><div> --- a/src/clustal/pair_dist.c</div><div> +++ b/src/clustal/pair_dist.c</div><div> @@ -321,7 +321,7 @@ PairDistances(symmatrix_t **distmat, mseq_t *mseq, int pairdist_type, bool bPerc</div><div><br class=""></div><div> /* FIXME: can get rid of iChunkStart, iChunkEnd now that we're using the arrays */</div><div> iChunkStart = iend;</div><div> - for(iChunk = 0; iChunk <= iNumberOfThreads; iChunk++)</div><div> + for(iChunk = 0; iChunk < iNumberOfThreads; iChunk++)</div><div> {</div><div> iChunkEnd = iChunkStart;</div><div> if (iChunk == iNumberOfThreads - 1){</div><div class=""> $ make</div><div class=""> …</div><div class=""><div class=""> $ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal --force</div><div class=""> =================================================================</div><div class=""> ==30601==ERROR: AddressSanitizer: global-buffer-overflow on address 0x561188847864 at pc 0x5611886da6e7 bp 0x7fffe6d77ef0 sp 0x7fffe6d77ee8</div><div class=""> READ of size 4 at 0x561188847864 thread T0</div><div class=""> #0 0x5611886da6e6 in FullAlignment::Build(HMM&, Hit&, char*) /home/matthew/clustal-omega-1.2.4/src/hhalign/hhfullalignment-C.h:250</div><div class=""> #1 0x5611886df3eb in HitList::PrintAlignments(char**, char**, char*, char*, HMM&, char*, char) /home/matthew/clustal-omega-1.2.4/src/hhalign/hhhitlist-C.h:197</div><div class=""> #2 0x5611886f379b in hhalign /home/matthew/clustal-omega-1.2.4/src/hhalign/hhalign.cpp:1211</div><div class=""> #3 0x56118863f848 in HHalignWrapper /home/matthew/clustal-omega-1.2.4/src/clustal/hhalign_wrapper.c:1342</div><div class=""> #4 0x561188637db1 in Align /home/matthew/clustal-omega-1.2.4/src/clustal-omega.c:1250</div><div class=""> #5 0x561188632d76 in MyMain /home/matthew/clustal-omega-1.2.4/src/mymain.c:1192</div><div class=""> #6 0x56118862aca2 in main /home/matthew/clustal-omega-1.2.4/src/main.cpp:469</div><div class=""> #7 0x7f6d857f109a in __libc_start_main ../csu/libc-start.c:308</div><div class=""> #8 0x56118862bad9 in _start (/home/matthew/clustal-omega-1.2.4/src/clustalo+0x2dad9)</div><div class=""><br class=""></div><div class=""> 0x561188847864 is located 60 bytes to the left of global variable 'Sim' defined in 'hhdecl-C.h:234:7' (0x5611888478a0) of size 1764</div><div class=""> 0x561188847864 is located 0 bytes to the right of global variable 'S' defined in 'hhdecl-C.h:235:7' (0x561188847180) of size 1764</div><div class=""> SUMMARY: AddressSanitizer: global-buffer-overflow /home/matthew/clustal-omega-1.2.4/src/hhalign/hhfullalignment-C.h:250 in FullAlignment::Build(HMM&, Hit&, char*)</div><div class=""> Shadow bytes around the buggy address:</div><div class=""> 0x0ac2b1100eb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100ec0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100ed0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100ee0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> =>0x0ac2b1100f00: 00 00 00 00 00 00 00 00 00 00 00 00[04]f9 f9 f9</div><div class=""> 0x0ac2b1100f10: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100f40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> 0x0ac2b1100f50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00</div><div class=""> Shadow byte legend (one shadow byte represents 8 application bytes):</div><div class=""> Addressable: 00</div><div class=""> Partially addressable: 01 02 03 04 05 06 07</div><div class=""> Heap left redzone: fa</div><div class=""> Freed heap region: fd</div><div class=""> Stack left redzone: f1</div><div class=""> Stack mid redzone: f2</div><div class=""> Stack right redzone: f3</div><div class=""> Stack after return: f5</div><div class=""> Stack use after scope: f8</div><div class=""> Global redzone: f9</div><div class=""> Global init order: f6</div><div class=""> Poisoned by user: f7</div><div class=""> Container overflow: fc</div><div class=""> Array cookie: ac</div><div class=""> Intra object redzone: bb</div><div class=""> ASan internal: fe</div><div class=""> Left alloca redzone: ca</div><div class=""> Right alloca redzone: cb</div><div class=""> ==30601==ABORTING</div></div><div class=""><br class=""></div><div class="">Looking at line 250 of hhfullalignment-C.h, we can see it’s reading the array <span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">‘</span>S’ out of bounds here. Someone has helpfully left a debugging line below this, so let’s shuffle it ahead of the faulting access and remove the part where it is also performing the faulting access:</div><div class=""><br class=""></div><div class=""> $ vim src/hhalign/hhfullalignment-C.h</div><div class=""><div class=""> $ git diff src/hhalign/hhfullalignment-C.h</div><div class=""> diff --git a/src/hhalign/hhfullalignment-C.h b/src/hhalign/hhfullalignment-C.h</div><div class=""> index 8f40fd1..fd759f9 100644</div><div class=""> --- a/src/hhalign/hhfullalignment-C.h</div><div class=""> +++ b/src/hhalign/hhfullalignment-C.h</div><div class=""> @@ -247,8 +247,8 @@ FullAlignment::Build(HMM& q, Hit& hit, char zcError[])</div><div class=""> char qc=qa->seq[ q.nfirst][ qa->m[ q.nfirst][hit.i[step]] ];</div><div class=""> char tc=ta->seq[hit.nfirst][ ta->m[hit.nfirst][hit.j[step]] ];</div><div class=""> if (qc==tc) identities++; // count identical amino acids</div><div class=""> + fprintf(stderr,"%3i %3i %3i %3i %3i %1c %1c %6.2f %6.2f %6.2f \n",step,hit.nsteps,hit.i[step],hit.j[step],int(state),qc,tc,score_sim,hit.P_posterior[step],hit.sum_of_probs); //DEBUG</div><div class=""> score_sim += S[(int)aa2i(qc)][(int)aa2i(tc)];</div><div class=""> - // fprintf(stderr,"%3i %3i %3i %3i %3i %1c %1c %6.2f %6.2f %6.2f %6.2f \n",step,hit.nsteps,hit.i[step],hit.j[step],int(state),qc,tc,S[(int)aa2i(qc)][(int)aa2i(tc)],score_sim,hit.P_posterior[step],hit.sum_of_probs); //DEBUG</div><div class=""> }</div><div class=""> }</div><div class=""><br class=""></div><div class=""> $ make</div></div><div class=""> …</div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"> $ ./src/clustalo -i debian/tests/biopython_testdata/f002 --guidetree-out temp_test.dnd -o temp_test.aln --outfmt clustal —force</div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"> …</div><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"> 28 582 386 559 10 N - 127.25 0.01 2.91</div><div class=""> =================================================================</div><div class=""> ==30936==ERROR: AddressSanitizer: global-buffer-overflow on address 0x563d5b2258a4 at pc 0x563d5b0b79e8 bp 0x7ffd269e0e40 sp 0x7ffd269e0e38</div><div class=""> READ of size 4 at 0x563d5b2258a4 thread T0</div><div class=""> #0 0x563d5b0b79e7 in FullAlignment::Build(HMM&, Hit&, char*) /home/matthew/clustal-omega-1.2.4/src/hhalign/hhfullalignment-C.h:251</div><div class=""> …</div><div class=""><br class=""></div><div class="">So the values of qc and tc at this point are 'N' and '-', respectively. This results in an access to S[20][21], which is indeed out of range as S is a 21x21 array. To go further, I think I need some guidance from a domain expert. Is aa2i() ever expected to be called with a value that maps to GAP or ANY? Maybe S is actually meant to be a 22x22 array? Maybe the loop in <span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">hhfullalignment-C.h is meant to skip any iteration for which qc or tc map to GAP?</span></div><div class=""><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class=""></span></div><div class=""><font color="#000000" class=""><span style="caret-color: rgb(0, 0, 0);" class="">By the way, Andreas, I am doing this debugging on the upstream 1.2.4 release on an x86-64 machine so I still have no certainty that this is related to the root cause of your observed problem on MIPS.</span></font></div></div></div></body></html>