Natterings about history files
supersede at india.com
Wed Jan 17 03:00:11 UTC 2001
In nnfolder+Mail:inn-workers, Russ Allbery <rra at stanford.edu> wrote:
> Anyway, one thing that I think might be a win if we stay with the nightly
> expire idea for history entries would be separate, daily history files.
This kind of thing works well with a dedicated history server, no idea
how well it would hold up in a more traditional timesharing
> The lookup time would be somewhat slower, but it would save a lot of time
> with expire since there's no need to even look at the history files that
> are younger than /remember/.
It can be worthwhile to keep only remember + 1 databases around;
i.e. each night create a fresh db for today's news, append the
rememberth history to the wicked-old-history db, and expire that one
in the traditional manner. Otherwise there is just way too much
bookkeeping to do, especially if you have groups that don't expire or
have extremely long expires.
One perq of using a history server (I suppose in Unix it could be a
daemon sort of thing, but that's not as fun) is you can dispense with
writing the hash stuff to disk, just log the text entries and do all
your hashing in memory. An on-disk index can wait 'til rotation time
(assuming you've got the spare memory to run that way of course).
Lookup time can be very fast for transit clients, since old-article
lookups are less common there. This mix does change for reader
service, but readers don't rely so much on history anyway and the hits
will still tend toward the recent.
Also, I found it much easier to modify sdbm to do this sort of thing
than to start from scratch, or to try and corral dbz's vast sea of
globals and preprocessor glop into something that would accommodate
multiple files per process.
More information about the inn-workers