Atlas proposal

[Sylvestre Ledru]

Hello,

First, thanks to all people participating to this discussion. 

Le mercredi 18 août 2010 à 13:29 +0100, Ian Jackson a écrit :
> I think the key thing to remember here is that both writing fast code,
> and its subsequent deployment and performance tuning, are very hard
> problems which are fields of research in their own right.  Our job in
> Debian is not primarily to try to outdo others' work by producing
> programs which are better in all of performance, portability,
> convenience, and impact on other users of the system.  
> 
> Our job is to package up and present the existing work so that our
> users have access to, and their pick of, the best trade-offs currently
> available.  So:
> 
> Josselin Mouette writes ("Re: Atlas proposal"):
> > Just ship two packages: libatlas3gf-base and libatlas3gf-auto, with
> > appropriate Conflicts/Provides just as we have now. And have
> > libatlas3gf-auto depend on all build-dependencies, ship the source, and
> > build itself in its postinst. Gentoo-style.
> 
> I think this is exactly the right thing to do.  It also solves the
> problem correctly in cluster or hpc farm deployments: each node gets
> a version of the library optimised according to its local properties.
I agree that it is the best solution. It is pretty close of what I
suggested.


> Admins who have more complicated setups (eg, multiple users
> timesharing on a single node) can either just install the -base
> package, or they can use the debconf parameters to detune the library.
> The -base package should be optimised for a modest number of CPUs (4,
> say) and the smallest possible cache size etc., so that the
> performance isn't crippled on even the slowest machines.
I can set the number of threads but not the number of CPUs (the number
of CPU is not enough anyway). I could say 2 threads should be launched
(?). I don't really known which values would be the best for Debian...

> > In libatlas3gf-auto you could use debconf for overriding things like the
> > number of threads or some optimizations, but I wouldn?t use debconf for
> > selecting the implementation.
> 
> Right.  The debconf questions should be fairly low priority and should
> default to "make it fast on this very computer".  dpkg-reconfigure
> should be made to always rebuild the package (eg if the host hardware
> has changed).
> 
> The only remaining questions are: where in the filesystem should all
> of this take place and which files should be deleted or kept ?
Hmm, I already have all the mechanism to build the optimized package on
the machine of the user.

This command in the source directory of Atlas:
$ fakeroot debian/rules custom
will produce the optimized packages:
../libatlas3gf-base_*.deb

If it is possible, I really would like to keep this way to do things
since it is factorized with the -base configure/build/install system.

For now, the solution I imagine is:
* build the package libatlas3gf-base just like it is now
* the libatlas3gf-auto package contains the upstream tarball + the
content of the debian/ directory
* libatlas3gf-auto has dependencies on the build dependencies
* when -auto is install, it will unpack both the tarball and the debian
directory
* it will start the build (fakeroot debian/rules custom)
* at the end, it installs the .deb packages and removes the
libatlas3gf-auto

How does it sound ? Is it possible ?

Does anybody know an other packages in the archive which does that ? (I
just checked with fftw and optimization are done at runtime).

Sylvestre