History problems, probably chained

[Jeffrey M. Vinocur]

On Mon, 14 Jun 2004, Zenon Panoussis wrote:

> That is sorta what I typed, but I was cancelling 9000
> articles. Few of them would have had expandable characters
> in the message-id but all of them were left on the server.

I find that about half a random bunch of articles I just grabbed do have a
$ sign and thus are trouble with double quotes, but regardless...


> The server works and is receiving streams as it should.
> The logs give no additional clue. expire.log has the
> same message as the news.notice mail, says only "can't
> reserve sercver". At the same point in time, news.notice
> says "innd: SERVER server mode throttled" and then notes
> the unthrottling and normal operation that followed.

Not so sure what to make of that.


> In the meanwhile, the nightly news.daily, which has
> worked fine for months on end, has run from cron, and
> now all 1.000.000+ postings on the server appear to be
> gone in the readers, except for those that arrived
> after the last news.daily run. Physically they're
> still in the spool alright, but suddenly the active
> file has (highmark-lowmark=1) on all newsgroups
> except those that received new articles.

You'd want to use grephistory to see if one of the "missing" articles is
missing from history, and then if not also look in the appropriate
overview .DAT file to see if it's missing from there.

 
> If I were to start completely fresh with a populated
> spool and nothing else, is the following procedure
> adequate to create history, overview and a correct
> active? Anything missing? Should I zero the counts in
> active too or should I leave them alone?

Um, you certainly can't zero the counts in active, or all will go to 
pieces.

 
> ctlinnd throttle "Messed history."

Honestly, I'd rather stop the server completely at this point, and make 
sure all the processes running as news are dead -- just throttling doesn't 
stop an already-running nnrpd from continuing, for example.


> rm -rf /var/spool/news/overview/*
> cd /var/lib/news

Fine.


> rm -f history*

I would have left the old history as a guide to how large a hash table to 
use, but that's just an efficiency point; will work regardless.


> touch history
> makedbz -i
> mv history.n.dir history.dir
> mv history.n.hash history.hash
> mv history.n.index history.index

This is all silly, since makehistory is going to create a new history.


> rm -rf /var/spool/news/overview/*

You already did this above, but no harm done in doing it again, I suppose.


> makehistory -a -O -b -e

The -a flag is redundant since you've deleted the previous history.  And I 
always use -F and a sane argument to -l, but both of those are also just 
efficiency type things.

Depending on what version of INN you have, you may need to run makedbz 
after running makehistory.  (In some versions, makehistory calls makedbz 
automatically; in others it doesn't.)  It never hurts to run makedbz.

At this point you're missing a critical step, which is to rename the .n. 
out of the middle of the history.n.* files.  Since you didn't do that, you 
started up the server with the result of `touch history` only.

After doing a `makehistory -O`, you want to be careful to renumber the
active file.  In current versions of INN, this happens automatically the
next time INN starts after rebuilding overview, but if you're not sure
then as soon as innd starts you want to do `ctlinnd renumber ""` to
renumber all of the groups.

Note that if new articles arrive before you renumber, everything will go 
to pieces and thus if you have speedy peers, you might want to comment out 
all of incoming.conf, start INN, do the renumber, uncomment incoming.conf, 
and run `ctlinnd reload incoming.conf foo`.


-- 
Jeffrey M. Vinocur
jeff at litech.org