makedbz and expire grow very big
Miquel van Smoorenburg
list-inn-workers at news.cistron.nl
Thu Nov 22 13:00:30 UTC 2001
In article <728zcz41qn.fsf at nd1.eng.demon.net>,
Alex Kiernan <alexk at demon.net> wrote:
>list-inn-workers at news.cistron.nl (Miquel van Smoorenburg) writes:
>> makedbz becomes quite big:
>> 345 MB? During nightly expire, it grows about this big as well. This
>> box has 1GB of memory, so it isn't that much of a problem, but I've
>> got other boxes with 192/256 MB and they will trash to death because
>> of this.
>> Looks like the history.hash or history.index files are built in-memory
>> and only later on written out ?
>Thats the way -STABLE works too (either that or I completely
>misunderstood the code when I rewrote it). You could hack line 1219 in
>hisv6.c to be INCORE_MMAP and hope your VM dtrt (Solaris 8 most
>definately doesn't, Linux I don't know).
Ok, so my guess was right ;). Looks like I never noticed it on other
machines because they run tagged hash and the .pag file is only 74 MB.
I noticed it on the feeder that runs non-tagged last night because
expire had been running for about 10 hours and was dead slow, the
box went into swap and so on. Upgraded from kernel 2.4.15-pre6 to
2.4.15-pre8 and that appears to fix it (living on the edge ... )
If it becomes a problem I'll try INCORE_MMAP to see if that helps.
Perhaps that should be tunable? OTOH INN has anough tunables as it is
BTW, I also run with largefile support enabled. That's probably
also a reason that the process grows so big because of the
64-bit file offsets, right ?
"Only two things are infinite, the universe and human stupidity,
and I'm not sure about the former" -- Albert Einstein.
The From: and Reply-To: addresses are internal news2mail gateway addresses.
Reply to the list or to miquels at cistron-office.nl (Miquel van Smoorenburg)
More information about the inn-workers