nnrpdcheckart speedup possible?

Lars Magne Ingebrigtsen larsi at gnus.org
Sun Mar 27 06:39:54 UTC 2005


Organization: Programmerer Ingebrigtsen
User-Agent: Gnus/5.110003 (No Gnus v0.3) Emacs/21.3.50 (gnu/linux)

(Setup: inn 2.4.1 with tradspool/tradindexed.)

I've been running news.gmane.org with nnrpdcheckart set to false for
a couple of years.  Since Gmane doesn't expire much (and doesn't
accept cancels), this works quite well, but it's not exactly ideal.
Sometimes we want messages to really go away, and having them still
appear in XOVER output can be awkward.

Now, I don't know whether there's a configuration setting that'll
just fix up the .DAT NOV file when a cancel arrives.  I've peeked
around in the documentation, but haven't seen anything likely...

Running tdx-util over the groups that needs fixing once in a while
would almost be acceptable -- but some of these groups are so large
that it doesn't really make much sense.  tdx-util over a group with
300K messages takes a while.

Perhaps something that reads the directory, and then reads the .DAT
file and weeds out the articles that aren't there any more?  That
should be pretty fast.  One directory read, one (big) file read, and
then one (big) file write.  And some locking, of course.  And the
.IDX file would need touching up, I guess.

If something like this doesn't exist, and there's no configuration
option to make this happen automatically, I could take a whack at
writing something like this.  Would looking at the tdx-util source
code be the right place to start?

Another approach to take would be to have tradindex mark the NOV line
as "removed".  Looking at the code for outputting the NOV lines, it
looks like if the Xref data is blanked out, the XOVER command won't
output the lines.  The comments for tradindexed_cancel() say that
it's not implemented yet because you can't go from token to group
name, which I guess is true, but tradspool does just that by looking
at the Xref header and cancelling all the instances there.  Can't
tradindexed_cancel() use the same approach?  (And the other backends
as well, but that doesn't really interest me as much.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi at gnus.org * Lars Magne Ingebrigtsen


More information about the inn-workers mailing list