Corruption Of Tradspool

[David T. Ashley]

Hi Russ,

Thank you very much.

That was one of the first pieces of information I encountered, and it did
not help.

That alone was not helpful enough.

Please check out this URL:

http://kalman.eng.wayne.edu/archived_docs/ht_inn_tradspool_db_repair/index2.
htm

Note at the end that there is still a subtle numbering problem remaining.  I
finally fixed it by running the nightly cron manually--that did something,
and it fixed the subtle numbering problem described at the URL above.  I
will add that information (the nightly cron info and other stuff) to the web
page above shortly.

I would like to make the following points with regard to the FAQ you posted
and which I encountered earlier:

a)It did not solve my particular problem.  I think the missing step involved
"ctlinnd renumber ...".

b)I think under this FAQ the procedure you want to post is an exhaustive
procedure that will solve just about any database corruption problem.  In
other words, you have to trade off complexity of diagnosis against the time
required to implement the solution.  So, under the FAQ, have the decisive
rebuild procedure that will cure anything.

The breakdown is like this:

1)If somebody is an expert INN user, they'll know how to fix it in minimal
time (with a more streamlined procedure), but:

2)If somebody corrupted their spools and is coming to the FAQ, they are
desperate and want a procedure--even a long one--that will fix their problem
decisively.  I think you need to cater to this group in the FAQ.  You want a
detailed procedure--even one involving 40 steps--that will fix ANY problem.
Somebody reading the FAQ is not concerned about "overkill"--they are
desperate and want to get INN working again.  The Power Users won't be
coming to the FAQ.

c)The nightly cron is doing something.  Not sure what.

Best regards, and naturally I welcome dissent if I've misspoken.

Dave.

-----Original Message-----
From: inn-workers-bounce at isc.org [mailto:inn-workers-bounce at isc.org]On
Behalf Of Russ Allbery
Sent: Friday, September 20, 2002 10:34 PM
To: inn-workers at isc.org
Subject: Re: Corruption Of Tradspool



David T Ashley <dtashley at esrg.org> writes:

> Due to a power failure, I corrupted my tradspools.  I tried various ways
> of repair mentioned on the web, using makedbz and one other program
> (forgot the name).  Nothing worked.  I still get errors related to INN
> throttled and all that.

Does this help?

URL: <http://www.eyrie.org/~eagle/faqs/inn.html#S4.4>

Subject: 4.4. tradspool: could not open ... File exists

This error generally happens after a crash or unclean shutdown of innd
using the tradspool storage method, and is caused by overview information
being out of sync with what articles are in the spool.  When innd was
restarted, it renumbered its active file (which determines the range of
existing articles in each group and therefore what article number is
assigned to new articles) based on the overview information.  If there are
newer articles already on disk that aren't mentioned in the overview
(because the overview information for those articles hasn't been flushed
to disk yet), new incoming articles will get assigned the same number as
the existing article and then innd will fail to store the article and
throttle with this error.

One way to correct this error is to rebuild the entire overview database
with:

    makehistory -O -x -F

but this takes a long time and is to some degree overkill.  A better
solution in some cases is to just remove all articles in the spool that
have higher numbers than the numbers in the active file.

Here's a Perl script that will do that.  Just save this to a file, make it
executable, and run it, giving it the path to the active file as the first
argument and the path to the top of your tradspool news spool as the
second argument:

    #!/usr/bin/perl
    die "Usage: <name> <active> <spool-path>\n" unless @ARGV == 2;
    open (ACTIVE, $ARGV[0]) or die "Can't open $ARGV[0]: $!\n";
    while (<ACTIVE>) {
        my ($group, $hi, $lo, $flag) = split;
        my $directory = $group;
        next if ($hi == 0 and $lo <= 1);
        $directory =~ tr%.%/%;
        $directory = $ARGV[1] . '/' . $directory;
        opendir (DIR, $directory) or die "Can't open $directory: $!\n";
        while (defined ($_ = readdir DIR)) {
            unlink "$directory/$_" if ($_ > $hi);
        }
        closedir DIR;
    }

INN 2.4 is expected to have better recovery tools to deal with problems
like this, and to be less vulnerable to losing overview information in
system crashes.

--
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.