time spent writing history..
fta at sofaraway.org
Tue Jan 23 13:00:02 UTC 2001
According to Reddy Crashalott:
> :: > I wonder why innd spend so much time to write into the history.
> :: > These numbers are for a full feed (1.3M articles and 220 GB that day).
> :: Because, I think, one of the changes in INN 2.0 was to change from one
> :: text file and one memory mapped table of offsets to a memory mapped table
> :: indicating presence or absence of a particular hash and a separate table
> :: of offsets into the text file. It cuts down on memory usage and it makes
> :: lookups faster, but the table of offsets isn't memory mapped and writes
> :: are done with pwrite, which is seeking all over the disk and not buffering
> :: a bunch of writes.
> INND timer:
> Code region Time Pct Invoked ms Min ms Avg ms Max
> history grep 00:00:00.000 0.0% 0 0.000 0.000 0.000
> history lookup 00:03:30.496 0.2% 14279616 0.010 0.015 0.020
> history sync 00:00:03.281 0.0% 11574 0.023 0.283 5.818
> history write 00:04:54.013 0.3% 817600 0.149 0.360 1.022
> idle 17:10:53.713 71.6% 47914650 0.778 1.291 2.259
> Precommit cache stats:
> Precommit cache lookups Precommit cache hits Hit rate
> 36504070 23042161 63.1%
> So, total of some 36 1/2 million history lookups in a few minutes.
> History.hash and history.index file, 21 days retention.
hmm.. looks good. Time to port the precommit artifice to INN 2.*.
Russ, do think it is still possible ?
I'll try to see if I can do it but if someone already have something,
please, speak now ! :)
Fabien Tassin -+- fta at sofaraway.org
More information about the inn-workers