Causes of NULL message-IDs (was Re: INN 2.4.0 dumps cor

Forrest J. Cavalier III mibsoft at
Fri Nov 7 17:29:13 UTC 2003

davidsen at (bill davidsen) wrote, in part:

> I haven't done this for years, but when I did I simply didn't mmap the
> files. Linux does a quite good job of keeping recently used file
> information in memory, and even of sharing it, and the whole thing
> worked fine.
> Was there some copying of data which could be avoided? Probably. But CPU
> is even less of an issue than it was when the 200MHz Pentium was my
> reader box, memory is cheap, and if turning off mmap would solve the
> problem I would certainly do so.

Bill, thank you for replying.

The tradindexed reader code in 2.4 uses either mmap or xreadv
(if tradindexedmmap is false) to get the _whole_ .DAT file, even
if it only needs part of it.  

.DAT files can be 180M bytes in large newsgroups.  
Memory is cheap, but not _that_ cheap.

Does anyone know that Linux is smart enough to not page-fault-in data
from a large read/readv until it is needed?  I know it does that for
mmap, which I thought was the point of mmap.  Will Linux also use
copy-on-write in order to share non-mmapped pages across processes?

The former seemed out of the question, but I have not tried it.  Maybe
Linux is smarter than I estimate, but I am pretty sure that readv
blocks at call time until complete, not when the page is touched.
I am asking, I do not know.

More information about the inn-workers mailing list