Speeding tradspool expiry

[Russ Allbery]

Forrest J Cavalier <mibsoft at mibsoftware.com> writes:

> You can use ctimes to decide which files to remove.  Then you can remove
> overview based on whether the file still exists.  (Actually, you don't
> have to stat, you can just look at the group, and already know what
> files were removed.)

> Based on what you were thinking, this may sound strange, but CNFS
> already does something similar.  The CNFS buffer rolls over and the
> article disappears.  Removing files in tradspool based on ctimes will be
> the same as a CNFS buffer rolling over.

This is a *really* good idea, and the only reason why I didn't implement
this instead is because it required more thought and more coordination
with news.daily and I wanted something in place fairly quickly since I
don't have a lot of time to spend on this right now and I think it's a
major issue.  But down the road, particularly if we get to redesigning
expire like I want to do, I think this is the right approach.  Set
tradspool to be self-expiring in the storage subsystem and instead launch
a separate process that parses expire.ctl and removes tradspool articles
based on mtime, and then expireover will clean up the rest.

This probably wants a parser for expire.ctl separated out from the rest of
the code so that all the pieces that want it can use it.

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>