[Debian-med-packaging] Bug#909716: Request for sponsoring changes in idba

[Andreas Tille]

Hi Pranav,

On Sun, Mar 08, 2020 at 11:49:38AM +0530, Pranav Ballaney wrote:
> Hi Andreas,
> My mid-sem exams are over now, and I can resume work on idba.

I hope you are happy with your result.
 
> On Mon, Feb 24, 2020 at 2:21 PM Andreas Tille <andreas at an3as.eu> wrote:
> 
> I'm now realising that idba build system builds a lot of other tools
> > inside $(CURDIR)/bin which might (or might not?) be sensible to install.
> 
> Yes, there are over 20 tools that come with this package. I've been trying
> to find documentation, but haven't had any luck so far. Did you find any
> while packaging?

I think I simply moved to the binary package by the tools requested by
some (random?) users here.

> Otherwise we could email the original author and ask for help, because
> without proper documentation it will be difficult to decide which tools to
> include in the binary.

I remember that upstream was quite responsive - so I guess asking is a
good idea.
 
> > I have not verified the test suite but it is not possible to download
> > anything while building the package since every package needs to be able
> > to build on a non-networked machine.  So every download has to be
> > patched from the test suite anyway.
> 
> Sorry, I used an incorrect term here. By downloads I was talking only about
> the test data. For now, I have included it within debian/tests but if it is
> possible to run sim_reads during testing, it will reduce the data required
> for testing, because a lot of it can be generated using sim_reads and
> sim_reads_tran.

It makes perfectly sense to run the simulation of test data while testing.
 
> Its way more sensible to provide all needed tools inside the package.
> > so please decide which of **all** the tools in $(CURDIR)/bin should end
> > up in the binary package.  You can install these easily by for instance
> > adding a line like
> >    bin/sim_reads        usr/bin
> > to the file debian/install.
> >
> 
> Thanks. Let's see if we can find documentation first, else I'll look into
> each program and try to figure out which ones to include. At first glance,
> it seems like sim_reads and sim_reads_tran were made only to generate data
> for testing, but I will look into this in depth (and other tools too) and
> get back to you about this.

May be adding some binary package idba-extra (or some better name you
might find) to ship these additional tools that might be not needed in
every day use but might be helpful anyway for some use case - for
instance testing the package.
 
> > In short: Please check prefere creation of the files in the test over
> > static
> > data.  Having a few small files provided statically is fine, thought.
> 
> While I get the general idea, I don't exactly understand what Architecture:
> all and Architecture: any packages are. Could you provide a link to a
> resource about these?

Architecture any packages are containing binary code that needs to be
compiled for specific architecture.  Examples are programs written in C
that are compiled into object code dedicated for amd64, arm64, misc,
etc. and than linked.  Architecture all packages could be either pure
data which can be shared on all those architectures, documentation
packages or scripting languages like Python or Perl code (if not using
binary code for performance reasons).

> Also, would it be possible to have some programs available only during
> testing, but not in the finall installation? Then we could run sim_reads
> that way and avoid all the static data.

Yes, for instance in an extra binary package.  While we could even run
autopkgtest inside a compiled source tree I personally prefer to ship
every tool inside some binary Debian package to enable users reproducing
the test results on their installation.
 
> > Provided that your time permits
> >
> >    1. Decide about what executables in $(CURDIR)/bin should be installed
> 
>    2. Generate as much data as possible
> 
>    3. Check whether test suite tries to download anything and ignore
> >       these tests or provide these data statically
> >    4. If we need large static data files create an extra binary package
> >
> > Feel free to ask me about any of these steps if you might need any help!
> 
> I need some help regarding binary packages. How can I create one? Does it
> require adding a file or folder inside the idba repo, or is it another repo
> entirely?

You just add another paragraph to debian/control.  Here is the paragraph
of the main idba binary package:

Package: idba
Architecture: any
Depends: ${shlibs:Depends},
         ${misc:Depends},
         python3
Description: iterative De Bruijn Graph short read assemblers
 IDBA stands for iterative de Bruijn graph assembler. In computational
 ...

You can simply copy this paragraph and find a new name, say

Package: idba-extra
Architecture: any
Depends: ${shlibs:Depends},
         ${misc:Depends},
         python3,    # remove this in case Python might not be needed
         idba        # in case this extra package needs idba   
Description: iterative De Bruijn Graph short read assemblers (extra tools)
 IDBA stands for iterative de Bruijn graph assembler. In computational
 ...
 This package ships extra tools for using idba in specific situations.


Than you do

    git mv debian/install  debian/idba.install
    git mv debian/manpages debian/idba.manpages

(which is strictly speaking redundant since by default debian/install
and debian/manpages are affecting the first binary package - but I
consider this better) and create a

    $EDITOR debian/idba-extra.install

package listing all tools you want to install into this package.

> Thanks for the hints about dch and installing test files and tools. I'll
> keep these in mind.

Hope this short intro is helpful.

Kind regards

       Andreas.


-- 
http://fam-tille.de