History File over 2GB

[Russ Allbery]

Mark Hittinger <bugs at pu.net> writes:

> With the INN 1.x history format we could do it right away.  Going to a
> token robbed us of a quick way to do it.

That's not entirely true; consider crossposted articles.  If it were just
a matter of pointing to one newsgroup and article number, there are a
bunch of solutions that would work; the difficulty comes in when you start
looking at crossposting and realize that there's no inherent limit as to
how many groups to which something can be crossposted.

> Today we'd have to take the message id hash out of the cancel, do an
> article lookup, and then we'd have the article headers which would get
> us to the overview data.  Last time I looked at the code this was
> expensive because we'd SMretrieve() the entire article (for example
> during expire to see if the article existed) rather than just the header
> info.

Oh, you can get just the headers already; that's part of the storage API.
It just doesn't help; hunting down and opening the article is already too
expensive.

> have a block of info at the beginning of the article that would have the
> pointers to the overview data - maybe some other flags about the article
> too.

I think this is what you'd have to do, but it breaks all existing storage
formats since none of them have metadata or a way of holding it right now.

(And you really want to assign a number to every newsgroup, which is
something I've talked about from time to time.)

-- 
Russ Allbery (rra at stanford.edu)             <http://www.eyrie.org/~eagle/>

    Please send questions to the list rather than mailing me directly.
     <http://www.eyrie.org/~eagle/faqs/questions.html> explains why.