time spent writing history..
rra at stanford.edu
Mon Jan 22 01:36:51 UTC 2001
Fabien Tassin <fta at sofaraway.org> writes:
> Code region Time Pct Invoked Min(ms) Avg(ms) Max(ms)
> article cleanup 01:03:22.687 4.4% 1349312 1.207 2.818 4.229
> article write 03:38:32.563 15.2% 1316215 4.628 9.962 14.299
> history lookup 01:33:00.892 6.5% 15994074 0.008 0.349 2.731
> history sync 00:00:09.114 0.0% 26704 0.000 0.341 76.367
> history write 03:35:37.066 15.0% 1327232 8.507 9.747 21.893
> I wonder why innd spend so much time to write into the history.
> These numbers are for a full feed (1.3M articles and 220 GB that day).
Because, I think, one of the changes in INN 2.0 was to change from one
text file and one memory mapped table of offsets to a memory mapped table
indicating presence or absence of a particular hash and a separate table
of offsets into the text file. It cuts down on memory usage and it makes
lookups faster, but the table of offsets isn't memory mapped and writes
are done with pwrite, which is seeking all over the disk and not buffering
a bunch of writes.
I'm not sure how to improve the tradeoffs. :/
Russ Allbery (rra at stanford.edu) <http://www.eyrie.org/~eagle/>
More information about the inn-workers