History File over 2GB

[Alex Kiernan]

Russ Allbery <rra at stanford.edu> writes:

> bill davidsen <davidsen at tmr.com> writes:
> 
> > And running nnrpd as a daemon adds to the problem, it doesn't always
> > notice when expire is run, processes holding file open use tons of file
> > space in deleted but not released files, etc.
> 
> nnrpd running as a daemon is certainly supposed to notice when expire
> happens.  If it doesn't, that's a bug.  It's not that hard for nnrpd to
> figure it out.  It already checks to see if the history file has changed
> and reopens it, I thought.
> 

I've just dropped in a fix if you happen to get ESTALE back, but yes,
otherwise history checks for expire having happened.

> > I think we have to go database at some point, so the cleanup can be done
> > all the time. Going to cycbuffs eliminated trying to unlink a million
> > files a day, I think history and overview must go that way eventually.
> 
> Note that I'm still using traditional spool for all text hierarchies
> because I honestly thing it's better.  More reliable, more robust, and in
> many ways just easier to deal with for anything that doesn't have sudden,
> unpredictable usage spikes.  It takes expire about four hours to run, and
> the server is fully available to clients during the entire period and
> accepting new articles during most of that period.
> 
> I agree that the way that expire is currently being done is less than
> ideal, but I don't see it as as large of a problem as you do.
> 

Personally I'd love to drop in a real database for overviews, but for
history I'm not convinced there's any real benefit - for 16M articles
(on cnfs) an expire takes about 20 minutes.

-- 
Alex Kiernan, Principal Engineer, Development, THUS plc