[Debian-med-packaging] Binary (rda) files in source tarball of BioC graph

[Andreas Tille]

Hi Paul,

On Thu, Aug 29, 2013 at 05:13:25PM -0400, Paul Tagliamonte wrote:
> On Thu, Aug 29, 2013 at 11:05:47PM +0200, Andreas Tille wrote:
> > > The serialized R objects can be input and manipulated in R by
> > > humans, I guess in the same way that png files are read by image
> > > viewers.
> 
> Erm, wat?
> 
> X is created by humans like Y is consumed by humans?
> 
> Also, this doesn't exactly hold, since Y may be created by Z (and often
> is, in the case of SVG files), and renders Y a binary result of the
> prefered form of modification.
> 
> This is Binaries 101, guys :)

Yes, as far as I understood.

> The issue I brought up was *ensuring* that we *can* reproduce *all*
> binaries. *I* would very much like to see such binaries rebuilt a
> build-time, unless it's samped data from some sort instrument, or
> actually created by hand lovingly.
> 
> > > In general, data objects in Bioconductor packages are complicated --
> > > not simple tables, but highly coordinated data structures. They have
> 
> As are ELF files ;)

No, as far as I understood.

> > > diverse origins, and the binary representation offers benefits to
> > > users and to our build and distribution channels; many are used to
> > > test or illustrate (in vignettes or man page examples) package
> > > functionality. It's not logistically feasible for us to provide
> > > ASCII representations of these objects.
> 
> I don't understand this. Not feasible? Is this to say we don't have the
> ability to rebuild this binary using the package data in main?
> 
> This is like saying "This PNG is from overlaying this 9 SVGs. Providing
> the SVGs is too much work so we didn't do it"

No.  From my perspective (and BioC maintainers in CC might correct me if
I'm wrong) it is like firing up Gimp to create a PNG and create a
drawing.  There is no (or at least not necessarily) such thing like some
original ASCII data.  You are just using R to manipulate the data as you
are using an image manipulation programm to work on your drawing.

> > > It's not escaped our notice that binary files are not a good engineering solution!
> 
> I don't mind this (personally), I do mind the fact that we can't rebuild
> these generated binaries.

If there is nothing to *re*build you can not rebuild anything.  What we
are requiring is that you can *change* the files - which you can do by
using R.

> > > I hope that provides some context,
> > 
> > Yes, it does.  Thanks for the hint.  I tried to document this issue on a
> > newly created Wiki page
> > 
> >    https://wiki.debian.org/GNU_R
> > 
> > which hopefully might be helpful for future R packagers.
> > 
> > Ftpmasters, if you need some further information / clarification I
> > should put on the Wiki page, please let me know.
> > 
> > Kind regards and thanks again for your insight
> 
> I would strongly encourage you to tell folks to treat them as we treat
> any binary - rebuild it at build time from source, if it's come from
> *any* sort of generated means.

How do we treat Office documents which are also clearly binaries?  We
also had cases where people were using a PDF editor to create
documentation.  I have no idea about your insight into R but I
personally do trust those experts if they are talking about their data.

And as I said previously:  We have currently 67 packages in Debian main
which did passed previous ftpmasters attention and as far as I know this
was due to some similar discussion.  I know that currently ftpmasters
have some time constraints which might be the reason why nobody else
from them was involved into the discussion on debian-devel[1].  Perhaps
we might find a shortcut to the discussion if you ask your fellows
explicitly how they might think about this and what might have lead to
their previous decision.

Kind regards

        Andreas.

[1] https://lists.debian.org/debian-devel/2013/08/msg00069.html

-- 
http://fam-tille.de