Rebuilding group.index

[Lars Magne Ingebrigtsen]

I have a major problem at news.gmane.org, which is still running inn
2.3.2.

Today I discovered that I hadn't compiled inn with largefile
support.  inn stopped working when the history file reached 2GB.
Oops.  I recompiled with largefile support (I had to tweak the
configure script slightly, but otherwise there were no problems.)

I then makedbz'd the history dbz files (since inn didn't want to
start with the old files; the of_t size in the index file changed
from 10 to 14.  Ok, that didn't take long.

I started up inn again, and it inflooped.  After poking around a bit,
I discovered that the group.index file was the problem.  If I
understand things correctly, there's an off_t in there that turned
into a loff_t, which understandably threw things off.

So I moved the old group.index file out of the way.

And then I started inn, and things seemed to work OK.  I mean, I
could connect to the server, get an article list, etc.

But then I tried issuing a GROUP command, and all it returns is
"0 0 0" for every group.

I've had a try at poking at makehistory for a couple of hours.  The
first thing it does is go through the active file and call

  OVgroupadd(line, lo, hi, q+1)

for each group, which is just what I'd want, I would have thought.

I'm using tradindexed, so this is what the function looks like, just
to remind y'all about this old software.  :-)

BOOL tradindexed_groupadd(char *group, ARTNUM lo, ARTNUM hi, char *flag) {
    unsigned int        i;
    HASH                grouphash;
    GROUPLOC            loc;
    GROUPENTRY          *ge;

    loc = GROUPfind(group);
    if (!GROUPLOCempty(loc)) {
      GROUPentries[loc.recno].flag = *flag;
      return TRUE;
    }

    grouphash = Hash(group, strlen(group));
    memcpy(&i, &grouphash, sizeof(i));
    i = i % GROUPHEADERHASHSIZE;
    GROUPlockhash(LOCK_WRITE);
    loc = GROUPnewnode();
    ge = &GROUPentries[loc.recno];
    ge->hash = grouphash;
    if (lo != 0)
        ge->low = lo;
    ge->high = hi;
    ge->deleted = ge->low = ge->base = ge->count = 0;
    ge->flag = *flag;
    ge->next = GROUPheader->hash[i];
    GROUPheader->hash[i] = loc;
    GROUPlockhash(LOCK_UNLOCK);
    return TRUE;
}

Doesn't that look just like what the doctor ordered?  But
GROUPLOCempty(loc) up there is always false when this function is
called (from makehistory), so hi/lo doesn't get set.  I've tried
cheating and setting hi/lo anyway, but even if I do that, restart inn
and say "GROUP something", I get "0 0 0" back.

I've now set makehistory to regenerate all the NOV files, but it
seems to be doing only a 100 messages per second, and there's 21M
messages on Gmane, so that'll take two or three days.

Is there no way for me to regenerate the group.index file?  I still
have the old file, and I still have the entire old overview
hierarchy. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi at gnus.org * Lars Magne Ingebrigtsen