[Debian-med-packaging] Question about proper archive area for packages that require big data for operation

[Laszlo Kajan]

Dear Ben!

On 27/04/13 00:46, Ben Hutchings wrote:
> On Fri, Apr 26, 2013 at 03:21:43PM +0200, Laszlo Kajan wrote:
>> Dear FTP Masters!
>>
>> On 23/04/13 15:13, Benjamin Drung wrote:
>> [...]
>>> You can use xz for the source and binary package to reduce the size. The
>>> default compression level for xz reduces the size of the source tarball
>>> from 415 MB to 272 MB:
>>>
>>> $ ls -1s --si metastudent-data_1.0.0.tar*
>>> 823M metastudent-data_1.0.0.tar
>>> 381M metastudent-data_1.0.0.tar.bz2
>>> 415M metastudent-data_1.0.0.tar.gz
>>> 272M metastudent-data_1.0.0.tar.xz
>>> $ ls -1sh metastudent-data_1.0.0.tar*
>>> 784M metastudent-data_1.0.0.tar
>>> 363M metastudent-data_1.0.0.tar.bz2
>>> 396M metastudent-data_1.0.0.tar.gz
>>> 259M metastudent-data_1.0.0.tar.xz
>>
>> Following Benjamin's suggestion and the data.debian.org document [1], we have prepared a 'metastudent-data' arch:all package that is ~130MB (xz
>> compressed).
>> The package builds required architecture dependent databases in the postinst script. The purpose of this is to save space in the archive that
>> each architecture dependent version would take up.
> [...]
> 
> Does this mean that installing the package results in having two
> uncompressed copies of the data on disk?  If so, wouldn't it be
> better to do:

Indeed, the original arch:all version, and the native one. The arch:all version is not needed any more after conversion, and could be removed.
Thanks for drawing my attention to this.

> 1. Compress the database (with xz).
> 2. Build the package without compression (contents are already
>    compressed so re-compressing would be a waste of time).
> 3. In postinst, decompress and convert the database to native.
> 
> However, I would expect the vast majority of installations to be on
> amd64, so if you always generate a 64-bit little-endian database
> and avoid duplicating when installing on such a machine then it
> would be better for most users (not so nice for others).
> 
> (Incidentally, arch:all packages generating arch-specific data have
> interesting interactions with multi-arch.  I doubt many people with
> multi-arch systems would want this package to generate multiple
> versions of the database, but you never know...)

I see. According to [1], Arch:all with Multi-Arch:same is an error.
[1] https://wiki.ubuntu.com/MultiarchSpec

So at this point I see one way forward:

1: Move the postinst script into a new Arch:any package that depends on 'metastudent-data'. This Arch:any package would build the native
database in postinst (with no multiarch support for now).

What do you think?

Best regards,
Laszlo