v3 Debian source package formats

Sat Oct 4 05:35:28 UTC 2008

On Fri, Oct 03 2008, George Danchev wrote:

> On Thursday 02 October 2008 20:33:00 Manoj Srivastava wrote:
>> On Thu, Oct 02 2008, George Danchev wrote:
>> > Quoting Manoj Srivastava <srivasta at acm.org>:
>
> --cut--
>
>> > Nobody is comparing git with quilt, really. Obviously you are trying
>> > to compare different unit types as a result of muddle thinking. It is
>> > not git *or* quilt, but DVCS *and* some sort of patch series and I see
>> > LKML and git developers constantly performing that. So, forget about
>> > quilt and think of the least common multiple as an abstract change
>> > units every can inherit and extend.
>>
>>         Fine. Then I  posit that the best way to help people is to give
>>  them the same environment that I work with (preferred form of
>>  modification and all). This means putting in ./debian/topic a patchset
>>  for each pure featrue branch, that  is not serialized, so people can
>>  try out each fauture by itself, makes it easier for upstream or
>>  downstream to get a single  feature cherry picked.
>
> Now, that is something different, and if I understand correctly it is
> targetting 3.0 (git) format, that's fine.

        Actually, this is not targeting the 3.0 formats at all. I
 intend to start shipping my packages with version 1.0 format, and a
 ./debian/topics directory with patches corresponding to every pure
 feature branch I have.

> The best part is that the users will get your environement via the
> debian source package, i.e. what is officially released by Debian and
> what buildd's had been fed with, right ?

        Right. The diff.gz is what the buildd's are fed, each of the
 patchsets in ./debian/topic recreate exactly the feature branches used
 to develop the code, the orig.tar.gz is the upstream branch. So, just
 using the orig.ar.gz and each of these patches, one can exactly
 replicate the tip of all the branches in use.

> I can also see your effort to avoid serialization, since it somehow
> doesn't fit well enough in the distributed model.

        I don't think fit into the distributed model has anything to do
 with it.  I find serialized patches harder to read than pure feature
 patches. Also, I find it hard to try out just one feature if all the
 features are ordered and serialized; the features at the end of the
 chain are very hard to decipher.

        I am just making it easier for people to read the patches.

> However, I'm not exactly sure if users really need that since they
> will hardly intend to develop in parallel with you via the debian
> source package they just apt-get source'd, instead they would love to
> perform a mere audit test, i.e. how Debian patched a particular
> upstream version, when the simple patch series is enough.

        Well, given that it is not the distributed vs one-off mode that
 makes me dislike serialization, this bit is working off a an assumption
 that misconstrues my motives a bit. I am providing easier to read
 patches, allow people to concentrate on one feature at a time, and
 patch upstream with features independently.  Now, for non-audit
 purposes, this is just fine, since people can examine all changes made
 to upstream.

        For security audit, you still need to audit the source that was
 compiled; and that is represented by the diff.gz. Looking at a dozen or
 so patches (PAM had nearer two dozen at one point) can be hard, since
 an exploit can be broken into bits and innocuous bits be present in
 each patch, but combined, be a security leak. So you audit the
 integrated sources. The ./debian/topic stuff is already gives people a
 conceptual understanding of each features changes, and shoule be enough
 to explain the obscure bits of the diff.gz.

        I think I want the patch series to be more than that. I want my
 source packages to fully enable people in the free software community,
 wh like to see the code to try out variations, and try one or more
 feature on their own, I can then ship features that are not merged into
 the integration branch as too controversial, and people can try even
 these features if they wish.

>>         The diff.gz is then the integration branch, and produces the
>>  sources that the package was built from.
>>
>>         The one thing you lose is the serialization changes (the efort
>>  that is spent in serializing the features).
>
> OTOH, nothing stops the user-side to get their patch series back via
> `tg export --quilt' on the top of your 3.0 (git) package if needed, or
> am I missing something here ?

        No, they can, if ever we do get 3.0(git) accepted. I understand
 there is some resistance. I am also willing to use topgit, just so the
 branches may be serialized by anyone with access to the git repo, eve
 if I just end up using the persistent integration branch that tg
 creates; and not convert it into a quilt series in the source package.

        Mind you, 3.0 (git) does not support submodules, so I am
 unlikely to be able to use the 3.0 format as it exists.

> Given that, the main question is: do we really need to push all that
> distributed burden to the mortal user, when he merely needs to see
> (review/audit) divergencies from upstream -> a mere patch series will
> do, and will simplify the things.

        For me, it is simpler to generate the feature patches (no topgit
 needed) than it is to serialize them. And since I believe the
 unserialized patches are easier to read, and understand, and also allow
 people to try upstream + single feature more easily, allow upstream
 (or, for that matter, downstream) to grab a single feature directly, I
 think I'll stop at using topgit, so people who want to serialize can
 use topgit to do so, but I will prefer to ship the pure unserialized
 patches. 

> OTOH, if he really wants to develop in parallel with you or upstream
> developers he will jump thru all the hoops of the distributed model
> and will hardly depend on your DVCS-ready debian source
> package. Undoubtedly, from the debian developer point of view, users
> doing that via the debian source package (since it is a ready to go
> git repo) would be a pleasant surprise, but do all users-reviewers are
> ready to pay that price. I'm quite uncertain about that, and only time
> will tell, so I might be wrong.

        I am not providing the patches for audit purposes. I am
 providing the patches so people can see what features I have installed;
 even if they do not understand git. They can see each feature, and the
 integration of all (most?) of the features.

        Now, you seem to be coming at this from the code audit
 viewpoint, and not to trust the developer when they say these pure
 feature branches were put together into that integration branch. I
 think it is not unreasonable to ask people to audit the integration
 branch (diff.gz), using the feature branches (./debian/topic/featureX)
 to understand how the diff.gz is put together.

>
>>         Given the advantages (ability to checkout any feature
>>  independently, exact replication of the developers working
>>  environment), I am willing to fail on the linkage between feature
>>  branches and the integration branch being re-done on the fly during
>>  build.
>
> re-doing that build-time might be fragile, though I can't think of any 
> unexpected results right now, but seems to be doable.

        Hmm. I would be willing to use topgit just so people can
 serialize the branches, even if I am not shipping the serialized
 patches.

        manoj

-- 
Be careful!  UGLY strikes 9 out of 10!
Manoj Srivastava <srivasta at acm.org> <http://www.golden-gryphon.com/>  
1024D/BF24424C print 4966 F272 D093 B493 410B  924B 21BA DABB BF24 424C