History/Overview messed up with INN 2.5.0 rc1

Ray Banana rayban at banana.shacknet.nu
Sun Apr 12 06:41:14 UTC 2009

Julien ÉLIE wrote:

> After a few investigation, I did not manage to find out where a race
> condition could come from in SMstore().
> As for local postings, I do not think it changes something because
> nnrpd handles the article to innd via IHAVE, as a normal feed would do.

Meanwhile I found duplicates where both articles came in through feeds,
so it doesn't seem to bee related to local vs. remote articles.
I also noticed that there can be a time difference of more than
90 minutes between the entries in the history file (arrival date),
so it's not really a *race* condition.

> I have also checked with my own duplicated tokens.
> They appear to be only for CNFS and timecaf (which are incidentally
> the two storage methods which use bitmaps).

As I only use CNFS storage, I would notice a different behaviour for
tradspool etc.

> And another interesting thing is that my duplicated tokens only
> showed up when I am doing some testings at the beginning of the
> month (with innd segfaulting upon receiving "Distribution:   \r\n"
> and also with very frequent rc.news start/stop).
> I wonder whether the problem is not related.  I see for instance
> that CNFS uses a metacycbuff_update every 25 articles.  It flushes
> useful information about the place of articles only every 25 articles
> and, of course, on normal exit.
> However, when innd segfaults...

I don't do tests on that particular server, but I do have issues related
 to OVDB hangs or timeouts from time to time, but these seem to have
started when I updated to INN 2.5.0 rc1 and Debian Lenny (with Berkeley
DB 4.6).

My duplicate storage tokens, however, seem to have started at the end of
December 2008 (earliest duplicate is dated December 22, 2008).

> Ray, did you notice something special with your INN at the time you
> received
> the article?  The real time will be in <pathlog>/OLD/news.xx.gz and you
> should
> check the corresponding news.notice.xx.gz.
> And could you please check how many duplicates you have?

I currently have 164 duplicates in a total of 20 million articles in the
history file.

As the dupes seem to occur only particular points of time (December 22,
between 4 and 5 o'clock in the morning, January 1, 1 to 2 o'clock at
night, March 19, 7 to 8 o'clock in the evening) this would indicate that
your assumption is right. At least on January, 1, there was a major
outage on the server (kernel dropped dead at midnight UTC while trying
to insert leapsecond, OVDB partly destroyed after hard reset).

OTOH, starting in April, I can see duplicates for almost every day,
which may be related to the current instabilities of OVDB.

I will keep an eye on this issue ans see if I can gather more information.

More information about the inn-workers mailing list