Atlas proposal

[Goswin von Brederlow]

Sylvestre Ledru <sylvestre at debian.org> writes:

> For now, the solution I imagine is:
> * build the package libatlas3gf-base just like it is now
> * the libatlas3gf-auto package contains the upstream tarball + the
> content of the debian/ directory
> * libatlas3gf-auto has dependencies on the build dependencies
> * when -auto is install, it will unpack both the tarball and the debian
> directory
> * it will start the build (fakeroot debian/rules custom)
> * at the end, it installs the .deb packages and removes the
> libatlas3gf-auto
>
> How does it sound ? Is it possible ?
>
> Does anybody know an other packages in the archive which does that ? (I
> just checked with fftw and optimization are done at runtime).
>
> Sylvestre

As said in other mails the "Installs the .deb" part won't work for
locking reasons.

But I have some experience with this from automatic kernel deb building
I did a few years back. So here is how I did solve the issue there:

I have a package "linux-custom" that depends on the kernel source,
patches, module sources and anything needed to build a kernel. It also
contains the .config. This coresponds to your libatlas3gf-auto package.

I also have a package "local-archive" that depends on reprepro,
generates a local signing key on first install, adds that to the apt-get
keyring and adds a file:/// url in /etc/apt/sources.list.d/.


Now when a kernel compile was needed (kernel source, patch or module
source was updated) a build job gets started in the background. It is a
good idea to delay the build a bit so dpkg/apt can finish before the
compile hogs the cpu. In my case I touch
/var/lib/linux-custom/need-rebuild and a nightly cron job starts the
build process if that exists. You might want something else or some
combination of things so the package rebuild is restarted if the system
is rebooted during build and so on.

Success or failure a mail is send to notify the user of the build result
(success just says the new package is there, failure includes
logs). When the build succeeds the generated package is added to the
local reprepro archive. The next time apt-get update/upgrade is run the
custom package is installed.

Using a local archive instead of calling dpkg -i directly avoids the
locking issues. It also allows exporting the archive for a pool of
systems. No need to build libatlas3gf 200 times for a 200 node cluster.
On the down side the upgrade of the package, while being completly
transparent, is not completly automatic. It requires 2 update/upgrade
runs.



To support building atlas on one system and installing it on many you
may want to use Depends: libatlas3gf-base | libatlas3gf-auto |
libatlas3gf-remote, libatlas3gf-base | libatlas3gf-custom.

The -base is the prebuild package, the -auto sets up the automatic
building and libatlas3gf-remote adds a file in /etc/apt/sources.list.d/
with an entry configured via debconf.  Having any one of the three
ensures atlas package updates will be available. The second part then
ensures that eigther the base or custom flavour of the lib is installed.

The important part though is that you would allow installing the -custom
package without requiring the -auto package as well.

MfG
        Goswin