slow expire with buffindexed

[Heath Kehoe]

>
>I'm just wondering what performance I should be seeing when I expire
>overviews for buffindex overviews and cnfs spool.
>
>Currently I have 10867473 lines in history, 12605786 overview data stored
>and 15 % overview space used. I have 15*2GB buffers storing the overviews
>and the cnfs buffer for non-binary articles is about 40% full. I am not
>exactly sure why the number of files in overview is greater than the
>length of the history.
>

For every article that is cross-posted to more than one group, there
will be an overview record for each crossposted group, but only
one history line.  So it's normal to have a larger overview count
than history lines.


>The article expire bit takes about 4-5 hours which I can live with,
>however the overview expire is taking at least 12 hours or more. Watching
>the expire.lowmark file it appears to not update for about 10 minutes at a
>time and then write a couple of hundred lines worth of newsgroups. This
>means that the time to cover even my 18,000 line active file is huge.
>
>Ideas I'm think are that perhaps my overview files are corrupt and should
>be rebuilt although I'm trying to avoid this since it will take the server
>down for a long time. 
>

When I was running buffindexed, I was seeing 12-hour expireover runs
on an overview with ~6000000 records, and rebuilding overview didn't
help.

Now that server runs ovdb; and its expireover takes about 5 hours.


>Is it possible that the fact that my overviews are bigger than I need slowing
>everything down? 
>

I don't think so.  Having more unused space shouldn't hinder performance.


>The rest of the server operation is faily good and pretty responsive, my
>main concern is that the expireover speed.
>

Yea, especially considering your overview storage is still growing,
which means your expireover time will be even longer.


-heath