Overview speeds and methods

[bill davidsen]

In article <E17eQzW-0007fQ-00 at batzmaru.gol.ad.jp>,
Christian Balzer  <chibi at gol.com> wrote:

| I'm wondering what particular overview method would be the fastest for
| a sizeable (160GB) cycbuff article spool. To keep things maintenance 
| free I used buffindexed overview so far. 
| 
| First I had the buffers on the same IO subsystem as the article spool,
| with them as raw devices (article buffers are regular files).
| A overview rebuild did take well over a day, with very little CPU usage,
| suggesting this to be quite IO bound. Expireover runs took 8 hours.

I've been using tradindexed because I had problems with the others. I
could never seem to find the "right" db version, although I'd love to
use that, and every time I tried buffindexed it needed more space than
I wanted to give it (I'm told that's fixed). It takes less than a hour,
but I only have about 10M articles online.

================================================================
expireover: can't expire alt.binaries.mac.games.adult
Expire messages:
expireover start Tue Aug 13 00:23:02 EDT 2002: (
-z/usr/local/news/log/expire.rm -Z/usr/local/news/log/expire.lowmark)
expireover end Tue Aug 13 01:10:23 EDT 2002
lowmarkrenumber begin Tue Aug 13 01:10:23 EDT 2002:
(/usr/local/news/log/expire.lowmark)
lowmarkrenumber end Tue Aug 13 01:10:23 EDT 2002
expire begin Tue Aug 13 01:10:53 EDT 2002: (-v1)
    Article lines processed 10326442
    Articles retained        9361291
    Entries expired           965151
    Old entries dropped      1202848
    Old entries retained     4301504
expire end Tue Aug 13 01:27:28 EDT 2002
        all done Tue Aug 13 01:27:28 EDT 2002
================================================================

Don't know if this helps, it's a data point.


-- 
bill davidsen <davidsen at tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.