[Debichem-devel] Git migration plan

[Daniel Leidert]

Hi,

I'm currently working on the GIT migration for the whole archive. To my
knowledge, subversion services are determined to be shut down at May
1st. So we are just in time. However, the migration is not a simple
process, especially when we want to keep most of the history around
7500 commits from more than 11 years of subversion usage. So please
read this mail carefully, if you want to help the migration.

I suggest, that if you agree to the plan and nothing serious comes up,
we can do the migration on February 1st.

During my work I found, that git-svn is probably the wrong tool to do
the migration[*]. An alternative is svn-all-fast-export (from KDEs
svn2git), which works on the base of a ruleset. It needs to process our
archive just once, can deal with broken package history and our tag and
branch layout. I saw later, that Stuart used the same tool in his last
approach.

I spent the last days to write a ruleset [1], that deals with all the
glitches in our repository (at least those I found) and uploaded the
resulting GIT repositories to [2]. The script I use is attached to this
mail. It runs svn-all-fast-export again a local and synchronized
debichem subversion repository and needs less than 5 minutes to create
and upload everything.

Now many of our packages have a clean subversion history and the
resulting GIT repositories look fine to me. Others had a more
complicated life, especially when branched off or tagged several times.
For these I did my best to make sensible decision writing the ruleset.
I checked the results by looking at the graphical log version (git log
--all --graph) and the references (git show-ref). I also imported the
results several times to salsa.d.o.

These are the basics of the ruleset. Please respond if you disagree to
these conventions (in general or just for a special package):

- Packages ignored are those
  - converted to GIT a long time ago (e.g. jmol, pymol)
  - non-free and long-time unmaintained
  - without a relevant history in SVN (e.g. xdrawchem, chemical-structiures)
  - recently converted, complete history and tags (e.g. gelemental, gperiodic)
  - dead (e.g. gchempaint)
  Their files go into a fake repository called "junk" (based on Stuart
Prescotts ruleset).

- All Ubuntu history (/ubuntu/, /edgy/) is ignored due to the fact,
that most of this stuff was added by svn add and not by branching stuff
off. So the files in these locations usually don't have a (svn)
relation to the files in /unstable/. This stuff also is very very old.

- Packages, that have been recently converted to GIT, but missing all
previous tags there, are still converted to a GIT repository (e.g.
rdkit, avogadro). I'd like to nake a decision about them afterwards.

- Not yet uploaded packages (mostly /wnpp/<package>, but also e.g.
/unstable/plumed/) are usually put into a branch called wnpp, not
master.

- All stuff of the debichem team (PROSPECTIVE_PACKAGES, tools, icons,
etc.) is put into a repository called "debichem".

So if you would like to help, these are the tasks and open questions
left before the conversion:

- Check the created GIT repositories (maybe your favorites package(s))
for glitches, especially broken history. Tell me, if a branch is
missing or if you have any other issue.

- Is it ok to use a branch wnpp or shall it be named differently?

- Is the "junk" approach ok? Do you have any other suggestion (see my
comments at the top of debichem.rules)?

- Are there hooks, templates, ... etc. that should be created for every
repository? Do we need to setup a repository template at
salsa.debian.org for every repository? Is there anybody already
familiar with the latter?

After the conversion:

- The tags are mostly correct. A few might be not and can be fixed
after the conversion. A few might also be missing (due to not being
tagged in subversion) and can also be added later.

- Tags called backup/* are used by svn-all-fast-export to mark the last
commit, before a branch (sometimes tags, but these should all be fixed
already) was deleted. These can be removed if you feel no need to keep
them. Please check the graphical log and/or the subversion history if
necessary.

- openbabel: The development for Debian experimental is currently in
unstable/openbabel, thus put into the master branch. There is no way to
"fix" this in subversion. But it is possible to move the relevant
commits into a Git branch after the conversion.

That's it. Please don't hesitate to ask or comment.

Regards, Daniel

[1] https://anonscm.debian.org/viewvc/debichem/debichem.rules?view=co
[2] https://people.debian.org/~dleidert/debichem/

[*] git-svn needs to process the whole archive with every package
migration, wasting a lot of time and resources. It gets the tags
wrong due to #887881 and mixed revision tags in our repository. Further
we sometimes have some broken history due to human mistakes. It is also
not able to deal with our branch layout.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: svn2git.sh
Type: application/x-shellscript
Size: 546 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/debichem-devel/attachments/20180125/0661c8fc/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://lists.alioth.debian.org/pipermail/debichem-devel/attachments/20180125/0661c8fc/attachment.sig>