History File over 2GB

[bill davidsen]

In article <ylk7m92yuo.fsf at windlord.stanford.edu>,
Russ Allbery  <rra at stanford.edu> wrote:

| It's one of the history modifications that I've been interested in seeing,
| although I think we'd get the most mileage out of it in combination with a
| few other techniques, like going to a binary (but platform-independent)
| file format, which would roughly third the amount of data we're writing to
| the history file.
| 
| In other words, yes, I think so, but I also don't want to end up with a
| proliferation of minorly-tweaked history file formats.  I'd like to get to
| a point where we have three:  traditional dbz, traditional tagged hash,
| and then whatever new thing we can come up with combining all of the
| tweaks we've thought of.  (And then people who want to write completely
| new experimental things, like something that uses BerkeleyDB or something
| that uses some completely different hash layout, can of course add in
| additional ones.)

I'd like to a single access interface for all the delete functions,
since this seems to be done from cancel, buffer rollover, etc. 

Off the top of my head:
 - delete art: cancel, rollover, expire
     delete the pointer, and the article itself in art-per-file
     leaves the msg-id remembered
 - delete MID from history: expire
     also should delete art?
 - checkhistory: incoming arts
     just return seen/unseen (wwould this be faster than graphistory?)
 - grephistory: head, article, feed
     MID->SM_token (or fail)

and it would probably be better to include a flag to delete overview for
an article as well, to avoid mismatches between data and information.
Clearly this would be a problem for non-database overview, perhaps a
queue of deletes to process? Not easy to do fast, but expire is
something I'd love to avoid.

COmments?
-- 
bill davidsen <davidsen at tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.