Slow expireover

[Russ Allbery]

Mike Zanker <mike-sender-6677e0 at zanker.org> writes:

> I'm using an old Sun Ultra 10 as a reader box for a handful of
> users. Performance is fine for reading (provided that nnrpdcheckart is
> set to false otherwise XOVER takes ages).

> The only problem is that expireover is taking a long, long time - 
> nearly two hours already:

> expireover start Mon May 17 03:01:08 BST 2004: ( 
> -z/usr/local/news/log/expire.rm -Z/usr/local/news/log/expire.lowmark)
>     Article lines processed  1026416
>     Articles dropped             169
>     Overview index dropped       169
> expireover end Mon May 17 04:48:14 BST 2004

That does seem kind of slow.  Usually expire is the one that takes a long
time.  But it's not all that surprising.  Here's my server, which is an
Ultra 2 with much faster A1000 SCSI disk:

expireover start Sun May 16 00:10:58 PDT 2004: ( -z/news/log/expire.rm -Z/news/log/expire.lowmark)
    Article lines processed  3952062
    Articles dropped          142897
    Overview index dropped    149292
expireover end Sun May 16 01:17:01 PDT 2004

So yours is taking about four times longer, which does seem surprising but
isn't out of the realm of possibility.

> ovmethod is tradindexed and the article spool is timehash. It's a new
> setup that has only been running for about a week - I intend to keep 14
> days of articles.

> Is there anything I can tweak to make this run faster?

No, not really.  Note that the server is fully available while expireover
is running, so it may not really be a problem.

One possibility is that you're running into memory thrashing issues, since
expireover does query the history file to see if an article is still
present in the spool, and the history file is a huge memory hog.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.