bzr-loom plugin

[James Westby]

Hi all,

I wanted to bring something to your attention, that may
serve as a good discussion point as well.

There is a new plugin for Bazaar, called the loom plugin.

https://launchpad.net/bzr-loom

You may have seen the name on the following Ubuntu document

https://wiki.ubuntu.com/NoMoreSourcePackages

The aim of the loom plugin is to let you develop a set of related
changes to a branch when the changes aren't that easy to do.

Consider the following situation. You have an codebase you are working
on, and there is a trunk branch that you are aiming to get your changes
in. The changes that you want to make arThe reason e not trivial, they
touch a load
of files, involve deep changes and much refactoring, and as such you
need to submit it in smaller chunks to stand a chance of getting it 
reviewed.

To use the plugin to do this you first turn your local branch in to
a loom branch (the main reason for this is so that someone without
the plugin can't access the branch and so mess up the structure).
You then create a set of "threads" on top of the code that represent
each logical change you want to make, and you end up with something
that looks like the following:

   add-the-docs
   add-the-ui
   make-core-changes
   add-helper-functions
   cleanup
   trunk

This looks something like a git patch series that you submit, so each
of these would be a commit in a git branch. However the loom in fact
uses a branch for each.

Now you use the plugin to make the changes in each area, and any time
you make a change at a lower level and move up it adds a merge 
(fast-forward if possible), and you end up with something like

   add-the-docs                    *----*----*----*
                                  /    /    /    /
   add-the-ui                    *----*----*----*
                                /    /    /    /
   make-core-changes           *----*----*----*
                              /    /    /    /
   add-helper-functions      *----*----*----*
                            /    /    /    /
   cleanup                 *----*----*----*
                          /    /    /    / 
   trunk                 *----*----*----*

You can then submit these back upstream. If upstream is bzr then you
can do it with a merge directive, which is patch + history, if not
then you can generate plain patches.

This looks like a set of git branches where you just maintain a strict
ordering and merging policy, but it is more than that. It also looks
like stgit, mq or similar, but it is more than the usual use of those.

Not only do you have a set of branches, but the tips are themselves
versioned, so that you can move about in the history of the whole
set.

This isn't such a useful property on its own, but as soon as you have
history you can do useful merging. This means that I can collaborate 
with you on the whole set. I can merge your changes to the whole thing, 
and the threads will be left alone, fast-forwarded, or merged as
necessary. (I've not yet tried out something that creates a "conflict"
at this level, for instance deleting a thread, so I don't know
what that looks like).

Now, on to the reason that I mention it on this list. This is useful
for developing large changes that I mention above, but it can also
be used for maintaining packages. Consider the following set of
threads

  debian
  fix-123456
  upstream

Here we have upstream code that has had a patch applied to fix a bug,
then the debian/ directory added as a separate thread. This means that
when a new upstream comes along you can pull it in to the upstream 
thread, and then merge up through the threads and build the new version.

If you submit the fix upstream then that thread becomes "empty" and can
be dropped.

There will also be commands to help when you have

   fix-234567
   fix-123456
   upsteam

If you want to submit the top thread first then it will create a new
branch of upstream and cherrypick that thread in to it, and then allow
you to fix up any conflicts and send it.

The fact that it is versioned then makes it easy to collaborate on the
package as a simple "bzr pull" or "bzr merge" (or "bzr merge --pull" for
the git workflow's fans) allows you to grab all of someone else's
changes.

The plugin is pretty new, and still needs some work doing on it to get
it fully useful, but the core is there to work from now. Also we haven't
started to develop the packaging-specific bits of this, but I hope to
start on this soon.

Does anyone have any comments on this? I don't expect that everyone will
drop what they are doing and switch to bzr because of this, but it would
be interesting to know whether this matches the way that you would want
to work.

Also, I am under no illusions that what I have described above is 
impossible in other systems, I just want to use the concrete of one
solution implemented in bzr to discuss the abstract.

My apologies for any inaccuracies above, I'm still getting to grips with
all of the intricacies. I haven't used it in anger yet, but I have 
pulled in a package converting .diff.gz and debian/patches/ to threads,
and then just played about.

Thanks,

James