[pymvpa] "SVD did not converge" - how can I make PyMVPA use lapack version?

[Michael Bannert]

Dear Yaroslav,

Thanks for your reply! I'm starting to wonder if it would really make a 
difference to use 'dgesvd' or 'numpy'/'scipy' implementations because 
numpy also uses lapack apparently at least since 1.11

http://devdocs.io/numpy~1.11/generated/numpy.linalg.svd

Do you still think dgesvd could be more robust as claimed in previous 
PyMVPA threads?

On 06.03.18 19:49, Yaroslav Halchenko wrote:
> 
> On Tue, 06 Mar 2018, Michael Bannert wrote:
> 
>> Dear all,
> 
>> I would like to use the lapack version of SVD since it is supposed to be
>> more robust than the numpy/scipy implementations (
>> http://lists.alioth.debian.org/pipermail/pkg-exppsy-pymvpa/2014q2/002754.html
>> ).
> 
> I guess you are talking about the message coming from the searchlight
> hyperalignment, which has an argument for hyperalignment, which has an
> argument for which transformation to use (ProcrusteanMapper by default),
> and ProcrusteanMapper has a parameter 'svd' which defaults to 'numpy'
> but could be set to "dgesvd" which would take that one from the
> lapack... see e.g.
> http://github.com/PyMVPA/PyMVPA/blob/HEAD/doc/examples/hyperalignment.py

Yes, it shows up in the context of hyperalignment (not the searchlight 
version, though - haven't checked it yet).
> 
> 
>> I have installed on my a macOS using Homebrew, which seemed to work but it
>> gave me this message:
> 
>> """
>> This formula is keg-only, which means it was not symlinked into /usr/local,
>> because macOS already provides this software and installing another version
>> in parallel can cause all kinds of trouble.
> 
>> For compilers to find this software you may need to set:
>>      LDFLAGS:  -L/usr/local/opt/lapack/lib
>>      CPPFLAGS: -I/usr/local/opt/lapack/include
>> For pkg-config to find this software you may need to set:
>>      PKG_CONFIG_PATH: /usr/local/opt/lapack/lib/pkgconfig
>> """
> 
>> I defined the environment variables as suggested but PyMVPA's
>> externals.exists('liblapack.so') still returns false.
> 
>> What can I do to make it work?
> 
>> Best & thanks,
>> Michael
> 
>> PS: Eventually, I will need to use it on a remote Linux system where lapack
>> is installed as /usr/lib64/liblapack.so.3 but PyMVPA fails to find it via
>> "externals.exists('liblapack.so')" too.
> 
> note that you need to have ctypes available for that functionality
> 
> print externals.exists('ctypes')

I have ctypes on both Mac and Linux systems.

> 
> this is what is happening underneath:
> 
>   python -c "from ctypes import cdll; print(cdll.LoadLibrary('liblapack.so'))"
> 
> so you could try smth like
> 
> LD_PRELOAD=/usr/lib64/liblapack.so.3 python -c "from ctypes import cdll; print(cdll.LoadLibrary('liblapack.so'))
> 
> may be... and/or check if you have /usr/lib64/liblapack.so symlink to
> /usr/lib64/liblapack.so.3
> 

Yes, the symbolic link did the trick. Thank you - I learned something today.

Best,
Michael