Offloading readers in INN + CNFS

Paul Tomblin ptomblin at xcski.com
Wed Oct 20 15:31:12 UTC 1999


Quoting Dan Merillat (harik at chaos.ao.net):
> 
> I've had a co-worker propose a solution to making multiple newservers.
> 
> Obviously, duplicating 10s of gigs of storage for each reader box is
> infeasable, so he proposed the following solution:
> 
> send the overview information to the reader boxes, where they maintain
> it.  They then call the storage box to grab specific article-IDs for
> the client.

Since you're grabbing articles from the server anyway, I don't see how this
would improve things over not having reader machines at all.

How about this small modification of your proposal instead:
Instead of having "reader" boxes, have boxes that run a program that caches
nntp requests.  They have an in-memory copy of the active and maybe overview
files, which they keep in-sync with the feeder machine through some sort of
RPC.  These cache boxes just have a circular buffer of articles, so when a
user requests an article the cache box attempts to satisfy that request from
its cache, and if it can't then it goes to the feeder machine.  By making the
circular cache big enough to hold a typical days worth of reader load, you can
get to a steady state where most readers are getting stuff out of the cache
unless they are the first ones to read an article on that box.

-- 
Paul Tomblin, not speaking for anybody.
SETI at Home:  Finally a *good* way to impress Jodie Foster
http://www.setiathome.ssl.berkeley.edu/


More information about the inn-workers mailing list