History File over 2GB
gja at meowing.net
Thu Aug 29 20:22:06 UTC 2002
Russ Allbery wrote:
> INN should be able to handle history files in excess of 2GB if compiled
> with large file support, but be aware that an INN built with large file
> support and one without large file support saves structures on disk
> slightly differently and may change the layout of overview (at least for
> tradindexed) in an incompatible way.
Been playing with some k0deZ to sorta-kinda deal with this, since it
looks like 32-buit file stuff isn't going to die as soon as I'd like.
Simplistic, but seems okay so far. The leftmost bits of the DBZ offset
are chopped off and used as a directory number (so text becomes
1/history, 2/history...., named that way to make spanning disks easier)
and the remainder as a real offset. Lookups use lazy opens so that a
zillion texts aren't grabbed unless necessary, and readers don't really
care how many history files there are anyway (they can get that from the
offset on a need-to-know basis, or something). Writers (innd/make*) get
stuck doing a bit of stat'ing around when they start up to figure out
when the end of the world is, but hey. The expire file shuffle is
slightly more messy than with straight dbz, but the time window remains
Now that there's a history API and stuff, would this kludge be worth
whipping into presentable shape and sending in?
More information about the inn-workers