What to do with .git directories in source package uploads?

[Nish Aravamudan]

On 24.05.2017 [22:56:52 +0100], Ian Jackson wrote:
> Nish Aravamudan writes ("What to do with .git directories in source package uploads?"):
> > I noticed today that our Ubuntu git importer failed to import
> > src:nplan (which is only in Ubuntu). This is because the upload
> > (incorrectly) includes a .git directory. I see that dgit's behavior
> > for the same DSC file with `dgit import-dsc` is to delete the .git
> > directory from the source package before import. Is that a reflection
> > of Debian policy?
> 
> Debian ftpmasters REJECT out of NEW packages that contain .git
> directories.  But there is a loophole, which I think is accidental,
> that lets later uplods containing .git directories slip through.

Ah ha! That's interesting. I'm guessing Launchpad has the same loophole.

> > 1) What dgit currently does, which is remove .git directories from
> > source packages, with a warning. This does mean that an imported
> > object does not match an uploaded exactly. Perhaps this is OK, but for
> > instance, in the case of nplan (purely as an example), if a developer
> > were to then use our git repository to develop a follow-on change, the
> > resulting debdiff would include (surprisingly to the end-user) a
> > removal of .git/)
> 
> I think that is actually correct.  If Debian ftpmasters fix the bug in
> their system, so that uploads containing .git are properly rejected,
> it would even be necessary.

Yep, that makes sense -- and that is implicit policy then, if not
necessarly official/documented.

> Furthermore many Debian package-handling tools will delete the .git
> anyway.

Yep :)

> > 2) In order from longest to shortest:
> >         Change any file in the source package matching ^\(\.+git\)$ to
> >              \.\1
> >         [essentially prefixing a '.' to any file consisting of only
> > leading '.' and the string 'git' ]
> ...
> > I'm wondering, are there are any explicit policy statements that say a
> > build system cannot rely on a git directory being present? Say a
> > debian/rules that does a `git describe`? I don't want to break such a
> > source package if it does exist.
> 
> I'm almost certain that all the .git directories found in actual
> package uploads are accidents caused my buggy or misoperated packaging
> tools.

Yep, that's my interpretation as well. I do see `man dpkg-source` refers
to "3.0 (git)" source packages, but it's still marked as experimental
and I'm not sure there is an example to play with.

> > I'm planning on implementing 2), as it leaves us the most wiggle-room
> > and we haven't asserted anything about hash stability yet in our
> > imports. But I'd love for some input on why 1) was chosen, and, as I
> > mentioned earlier, if it's considered 'policy' or not.
> 
> Your scheme is a possible alternative approach, indeed.  It might
> result in slightly confusing git trees.  It doesn't lose information,
> it's true, which is a slight advantage.

Yeah, it's certainly a trade-off.

> But if there are packages where the .git directory is somehow actually
> used in the build, your scheme won't help because a build from the
> imported git tree will use your history, not the history found in the
> .dsc's .git.

Good point! The theory I have (and it's not necessarily a good one) is
that if one is using our tooling, we can (with some finagling) not use
.git ourselves and then back up the renamed files in ^\.+git$ without
the leading . and then run dpkg-buildpackage.

Again, by no means ideal, but it would in theory work.

If deleting .git is the most agreeable option, I'm happy to do it in our
tooling as well :) It seems the simplest.

-Nish
-- 
Nishanth Aravamudan
Ubuntu Server
Canonical Ltd