[bind10-dev] Subversion to Git conversion

[Shane Kerr]

All,

Here is a proposal for the Subversion to Git conversion. I apologize for
the length, but please have a look and provide feedback here.


Motivation
----------
We want to convert from Subversion to Git. 

This came from repeated annoyances at using Subversion to do revision
control. It made software review more difficult than necessary. There
are plenty of other reasons(1), but this was what led us down the path
to desire a distributed revision control system.

Research
--------
Evan looked at Mercurial, Michael looked at Git. They both were
convinced that their tool could do an excellent job. Ultimately we
decided between the two based on a coin toss, and Git won.

We wanted to make sure that we could use the same tool for BIND 9,
BIND 10, and ultimately all ISC software. BIND 9 is using CVS, and has
an incredibly old and crufty repository. Amar Takhar did some
investigation and showed that a conversion from CVS to Git for BIND 9
was possible.

Requirements
------------
We have some requirements for BIND repositories:

 * We must be able to make security fixes that do not go into public
   repositories. (We don't want the hackers to have access until the
   fix is done and the good guys have it.)
 * We must be able to publish user-specific changes, which do not
   affect the main tree. This is the case where users have specific
   hacks or features that are not generally useful.
 * Michael would like a public read/write repository so that
   non-vetted code has a place to live, other than scattered around
   the Internet.

Git Repositories
----------------
We will eventually have the following repositories and relationships:

            +------------+
+---------->| Developer  +--------------------------+
|           |    R/W     +---------+                |
|           +-----+------+         |                |
|                 |                |                |
|                 |push            |push            |push
|                 v                v                v
|           +------------+   +------------+   +------------+
|           |  Public    |   | Customer   |   |   Forum    |
|           |    R/O     |   |    R/O     |   |    R/O     |
|           +-----+------+   +------------+   +------------+
|                 |
|                 |push
|                 v
|   pull    +------------+
|(reviewed) |  Public    |
+-----------+    R/W     |
            +------------+

The "push" in the diagram can be post-commit hooks, which keep the
various repositories updated on the fly.

* "Developer R/W" is basically the same as the current BIND 10
  repository, except that it will not be world-readable. This is where
  the BIND 10 team commits changes after we go through our review
  process.
* "Public R/O" is a copy of the Developer repository, but is
  read-only. When we have a security issue, we turn off the
  replication mechanism until we have made an announcement of the
  issue which can include a fix.
* "Customer R/O" is for read-only access to private branches. We push
  changes to these from our Developer repository.
* "Forum R/O" is basically a copy of the Developer repository, but is
  read-only. It is accessible to BIND Forum members.
* "Public R/W" is where developers not on the BIND 10 team can push
  patches for review by ISC to be included in BIND 10, or where
  developers can share other code.

The combination of "Developer R/W" and "Public R/O" gives us the same
openness that we have today, but also allows us to handle security
issues without having the bad guys know about the problem before the
good guys can apply patches.


Stage 1: What we have today, but with Git
-----------------------------------------
There are some pieces in the proposed layout that do not exist today.
To avoid delay, we will not implement those in our first stage, but
rather go forward trying to implement something like what we have
today, except using Git.

This means we need to create the "Developer R/W" and the "Public R/O"
repositories.

Areas of concern:

  * Repository Conversion
    I think this can be safely handled with svn2git.
  * Revision to Git ID conversion.
    We need to switch from the nice rXXXX format that is revisions to
    a Git ID, which is a SHA1 hash. We also need to convert these
    where they appear in the wiki, tickets, and everywhere else on the
    site. We also need to update the ChangeLog file to use Git IDs.
  * Trac integration
    We need to make Trac work nicely with Git. This is apparently a
    solved problem and it looks nice and pretty like the Subversion
    integration (although I haven't actually used it). (We should
    probably also convert from the SQLite database underlying our site
    to MySQL or PostgreSQL to avoid our current locking contention.)
  * Tool changes
    We need to migrate our Subversion hooks, and also update our
    automatic build/test systems to use Git.
  * Public repository
    We need to set up the public repository, and make tools to push to
    it.
    

Proposed Plan
-------------
I suggest we have the following tasks:

1. Test svn2git.
   Run manually, make sure it is sane.
2. Set up Developer R/W and Public R/O repositories.
   Initially set these up as on-the-fly copies, with all changes to
   Subversion pushed to them. (So the Developer R/W would actually be
   R/O in practice.)
3. Update our tools to use Git.
   We need to change our tools mentioned above to use Git. We also
   migrate our pre-commit and post-commit hooks. They can use the Git
   repositories we set up in step #2.
4. Set up a test Trac site using Git.
   This can also run on the bind10.isc.org computer, but using a
   different domain, like "bind10-git-test.isc.org". This would use
   the appropriate Trac plugins, and then we could do some scripts to
   find & replace the references to "rXXXX" with Git IDs.

Once this is in place, we would pick a date for conversion. On that
date, we would:

* Turn off the Subversion to Git copy.
* Shut down the Subversion repository.
* Update Apache to disable the bind10.isc.org site and rename
  bind10-git-test.isc.org to bind10.isc.org.

At that point we should be living in our new Git world.


----------------------------------------------------------------------
(1) Google can find lots of sites. This one seemed compelling:

http://thinkvitamin.com/code/why-you-should-switch-from-subversion-to-git/

--
Shane