New tradindexed overview ready, I believe

[Russ Allbery]

Expiration has now been tested for the new overview (and it's good that I
did, since it would have mangled overview completely).  It's now working
correctly.

The new overview has been tested with the test suite and by running
expireover on a single group.  I've also tested rebuilding the overview
for a group, both into an empty overview database and into a live database
while the server is running, and verified that it produces exactly the
same overview as the old code.  I've added a -F option to tdx-util to run
the internal audit and repair code in the new overview implementation, and
repaired the problems with it.  I've run tdx-util -F against a live
overview database on a running server and let it correct problems,
although I did it with the server throttled (which shouldn't be needed).

(Doing that turns up a lot of places where the old overview code didn't do
exactly the right thing, including one instance of putting a group index
entry into two different hash chains at the same time.)

I think the code is ready to go.  I'm going to install it on a
quasi-production server probably sometime later this week.

Should I go ahead and do the switchover in CVS so that other people can
easily play as well?  I've finished my internal checklist on the things
that needed to be done before I did that, now that building and installing
of auxiliary programs out of the storage tree works properly.  The only
thing that I've not done yet is put a full server load on it, because I
won't be set up to do that until after I do the big upgrade here.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.