patches-applied historical imports (usd import)

[Ian Jackson]

Nish Aravamudan writes ("patches-applied historical imports (usd import)"):
> 1) Some source packages (bouncycastle, php7.0 are the ones I can think
> of off the top of my head) upstream tarballs contain .gitattributes
> files, which will change the behavior of git itself when checking out a
> branch.

I need to do some tests to see exactly how to work around this.
Can you point me at an affected source package version ?

> 2) How do we determine if a source package is 1.0 vs. 3.0? I am
> currently using `dpkg-source --print-format`, but have found one source
> package (util-linux 2.13~rc3-5), where dpkg-source emits:
> 
> syntax error in /tmp/tmp3y515osf/util-linux-2.13~rc3/debian/control at
> line 14: duplicate field Depends found
> 
> and thus we error out.

How exciting.  dgit looks at debian/source/format, which is what
dpkg-source reads when generating the package.

> 3) Imagine the following graph in the git repository:
> 
>    A           D
>  ->o.....f....>o->
>    ^ .       . ^
>    |  c     e  |
>    o   .   .   o
>    ^    . .    ^
>    |     B     |
>    o     o     o
>    ^     ^     ^
>    |     |     |
>  ->o---->o---->o-> 
>    a     b     d
...
> Let's assert that there is a problem with obtaining the patches-applied
> version of b. This can occur for (at least) the following reasons:
> 
> i) as in 2), we might not be able to determine the source package format
> (implementation detail, to some degree), so are unable to correctly
> derive if there is a patches-applied state that is distinct from b.

You can use dpkg-source -x to extract the patches-applied state
corresponding to any .dsc, I think.  Any case where you can't is a bug
in dpkg-source.

> ii) some patches may fail to apply with a trivial `quilt push`. This
> occurs with, at least, a historical publish of samba.

Do you have an example source package ?

> In theory, there are other reasons/cases where this might happen and the
> importer needs to never fail (so it is of some use to run automatically
> :)

I haven't run dgit's dsc importer on a whole historical archive but
Peter Green of Raspian has been running it and filing bugs.  I haven't
seen such a bug recently so I hope it has been working for all the
packages he's seen.

> The questions I have relate to what to do when we encounter this
> situation, which in turn is divided into two parts:

I think this situation must be excluded.

This kind of difficulty is one of the main reasons for the failure of
previous efforts to transition universally to git.  It is why dgit
does not currently work, in Debian, with a full history of each
package.  Instead there's a bit of a bodge, with a locally generated
history.  Joey Hess and I came up with this approach at the Vaumarcus
Debconf.

Eventually I intend to import the whole history but not yet.

If you intend to go forward without fully solving this problem, your
choices are not good.

> Let's presume that this failure is not persistent and that D is able to
> be imported successfully, we again have to make decisions about
> parenting. I think it only makes sense for one of c or f to exist, based
> upon what we decide is the right policy above.

This is particularly troublesome.  When you can generate the missing
package, you will not be able to make it an ancestor of your existing
history.

You could rewrite the existing history and then pseudomerge the old
and new histories together, but that means duplicating most of the
history metadata.


If you wait with making an official import until it is suitable for
use with dgit, then I might consider using the relevant parts of your
imports history as the official Debian dgit history.

That would save you from the problem which I foresee in our future,
where Debian and Ubuntu have disjoint imports of the same history.


Ian.

-- 
Ian Jackson <ijackson at chiark.greenend.org.uk>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.