Question about "<news>/db/history" file size

Petr Novopashenniy pety at rusnet.ru
Thu Apr 23 11:23:44 UTC 2009


Hi to All!

On Wed, 22 Apr 2009, Russ Allbery wrote:

RA> Julien ?LIE <julien at trigofacile.com> writes:
RA> 
RA> >> Yes, you are certainly right. But I am more interested in the practical
RA> >> side of the issue. With something like this success story: "my history
RA> >> file size is ~20 GBytes, daily expire process uses ~5 GBytes of memory
RA> >> (and works ~1 hour), no problem".
RA> >
RA> > Is it makehistory which uses that amount of RAM?
RA> > (Maybe there is a memory leak somewhere?)
RA> 
RA> makehistory and innd both mmap the history file index, which I suspect
RA> is where the memory usage is coming from.

Yes, memory leak is unlikely..
My history.index is ~710 MBytes (history - 3249 MBytes), and innd uses 
only ~490 MBytes of memory.

But expire process uses ~1 GBytes of memory.

When history file was ~4,5 GBytes (two weeks ago), expire process 
uses more than 1,2 GBytes of memory. Yes, looks like "expire process size" 
= ~"history.index size".

I had planned to increase retention for text groups (approximately twice), 
but I do not have much memory for expire process. No other hardware limits 
in the near future (hard disks, CPU, etc..).

Maybe it is possible not do full memory map (at one time) for history file 
index (for expire process)? 

And, also, it is possible show number of articles in cnfs buffers (in 
"cnfsstat -a" output)?

Something like this:
Buffer news1, size:  47.7 GBytes, position:  28.3 GBytes  10.59 cycles
Articles count: xxxx?

And, if expire process not shutdown correctly, innd remains in "reserved" 
mode. How I can "ureserve" innd without restart?

RA> 
RA> -- 
RA> Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>
RA> 


--pety




More information about the inn-workers mailing list