History/Overview messed up with INN 2.5.0 rc1

Julien ÉLIE julien at trigofacile.com
Sat Apr 11 12:44:59 UTC 2009

Hi Ray,

> Well, it looks as if the same storage token was used twice, maybe some
> kind of a race condition allocating storage in the CNFS, as one of the
> articles came in through a feed and the other was posted locally.

After a few investigation, I did not manage to find out where a race
condition could come from in SMstore().
As for local postings, I do not think it changes something because
nnrpd handles the article to innd via IHAVE, as a normal feed would do.

I have also checked with my own duplicated tokens.
They appear to be only for CNFS and timecaf (which are incidentally
the two storage methods which use bitmaps).

And another interesting thing is that my duplicated tokens only
showed up when I am doing some testings at the beginning of the
month (with innd segfaulting upon receiving "Distribution:   \r\n"
and also with very frequent rc.news start/stop).
I wonder whether the problem is not related.  I see for instance
that CNFS uses a metacycbuff_update every 25 articles.  It flushes
useful information about the place of articles only every 25 articles
and, of course, on normal exit.
However, when innd segfaults...

Regarding timecaf, maybe a similar problem occurs (though I have not seen
where it happens -- buffering during writes in files?).  I read a comment
in caf.c which says:

    ** This means
    ** that in cases where the CAF file is inconsistent due to a crash ---
    ** the CAFTOCENT shows an article as being existent, but the header
    ** doesn't show that article as being in the currently valid range ---
    ** the header value "wins" and we assume the article does not exist.

Ray, did you notice something special with your INN at the time you received
the article?  The real time will be in <pathlog>/OLD/news.xx.gz and you should
check the corresponding news.notice.xx.gz.

And could you please check how many duplicates you have?

I for one did:

cd <pathdb>
cat history | cut -f3 | grep -v '^$' | sort > tokens
cat tokens | sort -u > tokens.unic
diff tokens tokens.unic

It normally shows duplicated tokens.
How many duplicates do you have?  What type? (if they begin with '03', it is CNFS)
Did they occur the same day?  (try sm on a few of these, or see whether the offsets
in the token are near)

For instance:


03 = CNFS
03 = the class in my storage.conf file
4652310000000000 = FR1 (the name in my storage.conf file)
001461E6 = the offset
00000002 = the cycle number

Julien ÉLIE

« A killfile on Usenet can get you peace and quiet.  A killfile in the
  real world can get you twenty years to life. » (Nils Nieuwjaar) 

More information about the inn-workers mailing list