Validate history file contents.

[Bill Davidsen]

On Tue, 2 May 2006, Christiaan den Besten wrote:

> Hi !
> 
> It is not the storage I am having issue's with .. it's the size of our
> history file. The server is running out of nmap'able memory during an
> expire. For the moment we will at least put more RAM in the box, but to
> me this seems to be a short term solution.

This does appear to be a problem with expire. We can run our target user
load nicely with only 2GB RAM, but expire takes double that and is
dog-slow even then. By stopping innd and nnrpd the expire can be run in
reasonable time, but for sites with only one server that's probably not an
option.

I would like to try the O_DIRECT option on the underlying open used for
the mmap, but it seems to be abstracted to the point of needing more
time than I have right now to find the right place to add the flag and 
not mess up other things.

> ----- Original Message ----- 
> From: "Jeffrey M. Vinocur" <jeff at litech.org>
> To: <inn-workers at isc.org>
> Sent: Tuesday, May 02, 2006 2:58 PM
> Subject: Re: Validate history file contents.
> 
> 
> On Sun, 30 Apr 2006, Christiaan den Besten wrote:
> 
> > > your CNFS buffers are needlessly large and are wasting space.
> >
> > Well .. the whopping 160Gb's of storage for the full-text server is
> > nothing compared to the space we have allocated for the binary feed, so
> > that's not realy an issue.
> 
> Okay...but then, why are you trying to expire these articles at all?  Why
> not let them sit around until the buffer rolls over, so your users have
> the bonus of a longer retention time?
> 
> 
>