[Debian-med-packaging] Bug#845753: Possible workaround

[Christian Seiler]

Hi,

On 12/14/2016 04:16 PM, Dirk Eddelbuettel wrote:
> One quick thought: does it die in _compilation_ which we have seen with other
> (C++-heavy) packages?

No, g++ works fine here. (The C++ file itself is trivial if you
look at it.)

Current package in Debian:
http://sources.debian.net/src/r-cran-treescape/1.10.18-2/

> Otherwise if it fails _after_ compilation we may be able to get by turning
> some default aspects of R CMD INSTALL off:
> 
>       --no-byte-compile do not byte-compile R code

That doesn't help, still fails. :-(

>       --no-test-load    skip test of loading installed package

That doesn't help either. :-(

>From the build log when it fails (8 MiB stack limit):

* installing *source* package 'treescape' ...
** package 'treescape' successfully unpacked and MD5 sums checked
** libs
g++ -I/usr/share/R/include -DNDEBUG   -I"/usr/lib/R/site-library/Rcpp/include"   -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-PAdLwq/r-base-3.3.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c CPP_update_combinations.cpp -o CPP_update_combinations.o
g++ -I/usr/share/R/include -DNDEBUG   -I"/usr/lib/R/site-library/Rcpp/include"   -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-PAdLwq/r-base-3.3.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -shared -L/usr/lib/R/lib -Wl,-z,relro -o treescape.so CPP_update_combinations.o RcppExports.o -L/usr/lib/R/lib -lR
installing to /home/christian/r-cran-treescape-1.10.18/debian/r-cran-treescape/usr/lib/R/site-library/treescape/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
Creating a generic function for 'toJSON' from package 'jsonlite' in package 'googleVis'
Warning in rgl.init(initValue, onlyNULL) :
  RGL: unable to open X11 display
Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE
Error: segfault from C stack overflow
* removing '/home/christian/r-cran-treescape-1.10.18/debian/r-cran-treescape/usr/lib/R/site-library/treescape'

(Ignore the X warnings, they are irrelevant here, I'm too lazy to run
it in xvfb and it's in a VM without X.)

When it succeeds (195.3 MiB stack limit):

* installing *source* package 'treescape' ...
** package 'treescape' successfully unpacked and MD5 sums checked
** libs
g++ -I/usr/share/R/include -DNDEBUG   -I"/usr/lib/R/site-library/Rcpp/include"   -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-PAdLwq/r-base-3.3.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c CPP_update_combinations.cpp -o CPP_update_combinations.o
g++ -I/usr/share/R/include -DNDEBUG   -I"/usr/lib/R/site-library/Rcpp/include"   -fpic  -g -O2 -fdebug-prefix-map=/build/r-base-PAdLwq/r-base-3.3.2=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c RcppExports.cpp -o RcppExports.o
g++ -shared -L/usr/lib/R/lib -Wl,-z,relro -o treescape.so CPP_update_combinations.o RcppExports.o -L/usr/lib/R/lib -lR
installing to /home/christian/r-cran-treescape-1.10.18/debian/r-cran-treescape/usr/lib/R/site-library/treescape/libs
** R
** data
*** moving datasets to lazyload DB
** inst
** preparing package for lazy loading
Creating a generic function for 'toJSON' from package 'jsonlite' in package 'googleVis'
Warning in rgl.init(initValue, onlyNULL) :
  RGL: unable to open X11 display
Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
Creating a generic function for 'toJSON' from package 'jsonlite' in package 'googleVis'
Warning in rgl.init(initValue, onlyNULL) :
  RGL: unable to open X11 display
Warning: 'rgl_init' failed, running with rgl.useNULL = TRUE
* DONE (treescape)

So the problem occurs at the following step:

  ** preparing package for lazy loading

And, to recap the specific circumstances where this problem appears:

 - 32bit
 - Little Endian architecture
 - Linux 3.16
 - Standard stack size limit (8 MiB)
 - treescape module version >= 1.10.17

Change only one of these things and it will work:

 - 64bit Little Endian Linux 3.16, standard stack limit: works
      e.g. amd64, arm64
 - 32bit Big Endian Linux 3.16, standard stack limit: works
      e.g. powerpc
 - 32bit Little Endian Linux 4.7.x or higher, standard stack limit: works
      e.g. i386 on my own system with newer kernel, or the mipsel
      build server of Debian with a backported kernel
 - 32bit Little Endian Linux 3.16, huge stack limit: works
 - older version 1.9.18: works

Note that different kernel versions really mean just the kernel,
the libraries and tools are 100% identical. (I mean libc, R, gcc,
and so on.)

I'm at a complete loss why the kernel version is even relevant here
btw., since the program uses a huge stack, but there is no system
call related stuff going on (and the package doesn't appear to
interface directly with the kernel anyway, it's much too high level
for that), and the problem persists for the given kernel over a lot
of very different architectures.

gdb stack trace of the R process when the stack limit is hit can be
found in this email of mine:
https://lists.debian.org/debian-mentors/2016/12/msg00412.html
(Debug symbols were installed, R version 3.3.2.)

Regards,
Christian