{Spam?} Re: Proposing peer Network support and software support

[Bill Davidsen]

Felix Kronlage wrote:
> On Tue, Oct 02, 2007 at 10:47:16PM -0400, Bill Davidsen wrote:
>
>   
>> Having looked at Diablo I have to conclude several things, (a) it is 
>> capable of being faster and more scalable than INN, and (b) the 
>> documentation on getting it working right suck rocks off the bottom of 
>> the ocean! INN is easy to understand and configure, and IMHO the main 
>> thing which is wrong with it is expire, which sort of drives people to 
>> Cyclone.
>>     
>
> What is wrong with the way (that differs depending on spooltype) expire is 
> done on inn? I'm not familiar with cyclone (and for that matter neither
> DNews nor diablo), what is a way of doing expire that people like more?
>   

What's wrong is that expire is done at all. No matter how or when you do 
it, it's a load spike, and while that's usually fine on a small user 
base, on an ISP level there are no off hours and no excess capacity.

Ideally the instant an article was removed, however it was done, the 
overview would reflect the fact, the file space would be reclaimed, etc. 
All overhead resulting from removing an article would be spread over the 
day, load would be even. With CNFS I can get most of that, but I still 
need expire to rebuild the dbz database and expireover to clean out the 
old articles.

Thus my interest from time to time in using a real database program 
which would allow deletions without ever having expire. I actually tried 
a proof of concept in about 1998 using a B+tree database library I was 
investigating, and it seems to be doable, but the performance was 
somewhat slower than dbz and I was on PC-class hardware for testing, and 
there were licensing issues with the library, so I wrote it off for 
production use. I also played with speeding article lookup using MPH in 
about 2001, that didn't produce a huge performance gain for read, but 
was nice for taking multiple feeds.

Anyway, it is what it is, I am certainly not going to do a rewrite now, 
since most people disagree with the need (and maybe have some magic 
tuning I never found to avoid the expire slowdown). I just feel that 
expire, introducing a load spike, is an intrinsic limitation on the 
capacity in servers running INN.

-- 
bill davidsen <davidsen at tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979