INN not keeping up with incoming news

[Russ Allbery]

Alex Kiernan <alexk at demon.net> writes:

> We've just taken to locking the history.{hash,index} in core; its worked
> wonders - we've gone from spending hours per day in HIS* to under 10
> minutes (although I've no doubt that some of the time is hidden
> elsewhere).

Here's something to try if you have a test box for it:  only mlock
history.index and see if you still get the same win.  I wouldn't be
surprised if you did.

history.hash is already mmap'd, so I think you're already seeing a lot of
the benefits there.  history.index on the other hand isn't mapped at all
and INN normally writes to it without any buffering at all.  This is a big
drawback of the new dbz code that needs to be fixed, after the new history
API, and I think it's the main reason why people are seeing history writes
be so slow.

When you lock it into memory, though, with Solaris and its unified buffer
cache, I bet all of those disk writes are staying in memory.  I wonder if
you wouldn't get similar performance by editing innd/his.c and specifying
INCORE_MMAP for opt.pag_incore.

> If anyone's interested, here's the code we're using (it doubtless has
> some Solaris assumptions, I haven't tried building it anywhere else):

Shall I drop this into contrib?

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>