Slow expireover (again)

[Russ Allbery]

Mike Zanker <mike-sender-6677e0 at zanker.org> writes:

> ...and it has made a substantial difference already. Yesterday's
> expireover time was 6h 57m for 2525160 article lines processed, today's
> was 3h 39m for 2331680 article lines processed.

> How does expireover check for the existence of articles when you have 
> both timehash and timecaf? This is what I have in storage.conf:

Depends on the article.  If the article is in timecaf, it checks timecaf
directly; if it's in timehash, it checks history.

Okay, we've identified the source of your problems, then; the querying of
the history database from inside expireover is too slow.  Since it's
directly related to the number of articles in the spool, I'm betting it's
directly related to the size of your history file, which means that I
think I'm right about the memory thrashing.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.