History File over 2GB

[bill davidsen]

In article <3D6FD74A.7040201 at meowing.net>,
greg andruk  <gja at meowing.net> wrote:
| 
| bill davidsen wrote:
| 
| > Off the top of my head:
| >  - delete art: cancel, rollover, expire
| >      delete the pointer, and the article itself in art-per-file
| >      leaves the msg-id remembered
| 
| The rollover stuff might be better noticed and flagged by readers than innd.

The problem being that readers are not all smart. I'm fighting a problem
right now related to this, the article should vanish from overview or
some clients will try to fetch the article and give the user some "not
available" status. Ideally I'd like the overview and LISTGROUP to agree
and match the actualy available articles. Currently that's not the case
with most servers, and I'm not sure it could be done with INN unless you
have nnrpd read the overview and edit it, rather than grab and send.
 
| >  - delete MID from history: expire
| >      also should delete art?
| 
| I can read this as meaning maybe three different things.  MAybe an example?

At the end of the remember period the message-id becomes not known, so
it would be accepted if offered again (not likely). And for advanced
expire schemes you might want all info on the article to go away,
including the article text if still present. The alternative would be to
have article text not known to history, evil or not depending on storage
method.

| >  - checkhistory: incoming arts
| >      just return seen/unseen (wwould this be faster than graphistory?)
| 
| Definitely faster if you're only checking a small number of IDs per 
| process.  dbz can already do this, it just needs to be exposed, really.
| 
| >  - grephistory: head, article, feed
| >      MID->SM_token (or fail)
| 
| sm and grephistory could stand some small tweaks.  What I'd like to see:
| - grephistory gets an "interactive" mode kind of like what sm with no 
| command flags now.
| - sm's interactive mode prepends an rnews header (easy) or spits out a 
| dot-stuffed-and-termed article (busier and probably less useful) to make 
| it more suitable for running through a pipe.
| 
| I kind of like that the two aren't currently trying to do each other's 
| job, though.  Any more elaborate, and you might as well just fire up an 
| nnrpd.
| 
| > and it would probably be better to include a flag to delete overview for
| > an article as well, to avoid mismatches between data and information.
| > Clearly this would be a problem for non-database overview, perhaps a
| > queue of deletes to process? Not easy to do fast, but expire is
| > something I'd love to avoid.
| 
| tradinidexed in particular wants careful handling, since that and 
| tradspool are supposed to be there to support spool-based readers.  The 
| cheap and easy approach of poking an invalidation string into the middle 
| of .overview kind of loses there =(

Yes, that's the overhead of having nnrpd edit the overview before
sending. Can't be worse than overview in a database, I guess, pulling a
bunch of records.

As you might guess I have problems with users complaining when an
article is in overview and not online, or when LISTGROUP and XOVER don't
agree (and sometimes neighter is correct).
-- 
bill davidsen <davidsen at tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.