[Neurodebian-users] NeuroFedora packaging priorities

[Yaroslav Halchenko]

On Sun, 18 Nov 2018, Yury V. Zaytsev wrote:

> On Sun, 18 Nov 2018, Yaroslav Halchenko wrote:

> > > TLDR; Debian may be awesome, but RHEL is also awesome ;-)

> > Sorry, I might be missing something -- does RHEL now provide a rich set
> > of neuroscience related python (and not only) packages?

> It depends on how you define "provides"; RHEL certainly does not provide
> them in form of pre-built binaries for the OS-level package management
> system (rpm).

> My argument was, however, that managing pure Python dependencies is trivial
> for anyone who has a clue,

heh... yeah... besides that is the problem we are trying to avoid here
-- "managing ... dependencies".  pypi/pip (and conda to that extend)
provide no testing of correct or "coherent" operation of the
distribution as a whole at any given point in time.  At best testing is
done only for any given package at its build time.  So any next
upload of core library could break an unknown number of dependent on it
packages.  That is my point behind using a stable system which does
provide and cares not only about the core  packages but also all other
packages included within it, that they operate correctly together at the
versions provided in that release.

I run into compatibility issues between different packages ALL THE TIME!
Some times it might be represented by some failing test in a leaf
package you might be using which otherwise passes tests fine on the
other systems.

So, IMHO "managing pure Python (or any other to that extent)
dependencies is ACTUALLY NOT trivial" if you want to have some
assurance of correct (inter)operation.

But ok -- I should stop ranting ;)

> On top of that, specifically in the HPC context, I used to make my own NumPy
> / SciPy builds with icc against MKL anyways to squeeze every last 2%
> performance out of my CPUs, not even speaking of MPI stuff. If you are
> routinely running simulations that take 1 000 000 CPU hours, some 20 000
> hours do make a difference... so no way I would have used distro packages
> built to work on a wide range of CPUs, and thus not exploiting latest
> instruction set extensions.

FWIW, there is apt-build  and possibly other helpers to provide local
targeted rebuilds of debian packages with optimizations added by
your to your liking


> > don't you need some kind of a root "daemon" to make use of guix/nix ?

> Nope, not at all:

>   $ nix run --store ~/nix nixpkgs.nix nixpkgs.bashInteractive

> This will create an user-owned nix store, which does not require root
> privileges, and spawn a shell where this store will be mounted to /nix.

ah, cool to know.  might try it some time if I see it solving some
problem for me

-- 
Yaroslav O. Halchenko
Center for Open Neuroscience     http://centerforopenneuroscience.org
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834                       Fax: +1 (603) 646-1419
WWW:   http://www.linkedin.com/in/yarik