rethinking patch management with GIT / topgit

[Thomas Koch]

martin f krafft:
> also sprach martin f krafft <madduck at debian.org> [2010.03.27.0801 +0100]:
<SNIP>
> Before I forget: if you have not yet done so, could you please
> create a vcs-pkg tag on your blog and aggregate it on
> http://vcs-pkg.org/planet/? Thanks!
done. thanks for the invitation.
> 
> Now to the solution you propose: I have multiple, mixed reactions to
> it:
> 
> 2. I have always disliked .topdeps, which feels like it's doing
>    something that should and could be better done by Git itself. As
>    a result, .topdeps feels brittle to me. Your proposal to use
>    merge commits may be usable to replace .topdeps, but I'll need to
>    explore it a bit more. A reference implementation would be really
>    useful.
I started a prototype in perl yesterday, but it still needs some hours. I'm 
also new to perl so I could need a hand of a perl expert. If you've time, I'd 
also be glad to meet for a hacking session.
Do you propose a more suitable language for this tool?
The merge commits do not intend to replace .topdeps. The intend of the merges 
is solely to not loose commits. .topdeps is replaced by the file "branches" 
shown in my blogpost.

> 2b. How would your implementation represent a tree with three
>     feature branches, where B and C both depend on A, e.g.
> 
>                  B
>                 /
>     upstream — A
>                 \
>                  C

First, the commits tree itself represents this structure. Additionally to 
this, we want to keep track of the branch names, the top commit of these 
branches and the names of the branches they depend on, which is recorded in 
the patchset's meta branch in a file like the following, suggested name 
"branches":

ROOT: upstream
BRANCH: B a8eSOMECOMMITSHA1xyzfaf46d87
DEPENDENCIES:
    A
BRANCH: C f4eSOMECOMMITSHA1xyzfaf46d87
DEPENDENCIES:
    A

I'm not sure yet about the serialization format (only that I don't like XML). 
Any suggestions?
The above informations are necessary to
- have a status of the patchsets: are there updates in upstream of in a patch 
branch?
- generate the patches for the export from their root to their top

> 3. It seems to me that you are proposing to store meta information
>    in plain files in meta branches (cf. .topdeps). If at all
>    possible, I'd say we should avoid touching the worktree
>    altogether, but to design a protocol by which all the information
>    we need is kept in the Git DAG and its objects.
The worktree won't be touched by my proposal. What makes you think that? All 
changes to the patchset's meta branches are done by the toolchain in the 
background without touching the worktree.
I've actually started my prototype by copying code from pristine-tar. All 
changes to the pristine-tar branch are made in a temporary directory outside 
the worktree.

> 4. Rather than a meta branch, if we had a means to store and update
>    information for each branch, we could solve the problem and store
>    .topmsg, as well as the top-bases/* pointer in there.
>    A rudimentary implementation might be using Git notes, e.g.: walk
>    up the ancestry until you find a specially-formatted Git note,
>    and use that as branch meta data store. If the data change, then
>    write a new note to the top-most commit.
I'm not up to date about GIT notes. Could you point me to a description? As 
far as I've seen, GIT notes is still experimental? Are GIT notes of existing 
commits immutable? That would be required to make sure history won't get lost 
or modified.

> 5. It occurs to me that what we want to implement somehow should get
>    rid of .top* files and top-bases/*, and instead allow us to track
>    "commits" at a much higher level. E.g. a branch creation is
>    a commit, updating feature branches is a commit (I see no reason
>    to allow updating of single feature branches independently, but
>    would say either all or none), integrating feature branches is
>    a commit. The way to represent that might be using a meta
>    hierarchy with the merge commits you suggest, but which also keep
>    ancestry.
Sorry, but I believe you think way to abstract. What I like about my proposal 
is, that it only stores a few bytes of meta informations additional to a 
plain, regular, unspectacular GIT repository.
It should even be possible (I guess) for build my tool VCS agnostic and 
migrate a patchset with all meta informations from GIT to Hg.

> 6. One of the main issues I have with TopGit is that it's not really
>    possible to consistently tag a release with all its branches.
>    E.g. you need to tag all branches, and all top-bases at the time
>    of the TopGit tag, which appears brittle itself. If indeed we can
>    find a method where a single commit represents the entire state
>    of a tree+feature branches at a time, and one could generate all
>    kinds of patch formats from it, and even fork the tree with all
>    feature branches from this point, then that would be splendid.
I guess so. This is one of the main design goals beside the ability to have 
multiple independent patchsets in the same repository.

Thomas Koch, http://www.koch.ro