Overview speeds and methods
chibi at gol.com
Tue Aug 13 02:01:02 UTC 2002
I'm wondering what particular overview method would be the fastest for
a sizeable (160GB) cycbuff article spool. To keep things maintenance
free I used buffindexed overview so far.
First I had the buffers on the same IO subsystem as the article spool,
with them as raw devices (article buffers are regular files).
A overview rebuild did take well over a day, with very little CPU usage,
suggesting this to be quite IO bound. Expireover runs took 8 hours.
Venturing that making the overview buffers regular files should gain
some speed considering that the box has 1GB of memory I moved them to
another IO subsystem. Unfortunately this storage array is substantially
slower than the main one, so expireover now takes 12 hours and a rebuild
lasts 3 days. D'oh. :)
But at least I was able to analyze that IO for overview actually accounts
for HALF of the total disk IO in that box.
I'm now bringing a much faster and dedicated IO subsystem for the overview
data online, but before more wild stabs in the dark I'd like to tap any
insights available here on the following items:
1. Which is considered the overall fastest overview method? (I read that
classic is best for readers)
2. Will buffindexed as a regular file benefit from caching as opposed to
the buffers being raw devices?
Many thanks in advance,
Christian Balzer Network Engineer Engineering
chibi at gol.com Global OnLine Japan/Exodus Communications K.K.
More information about the inn-workers