Reducing communications between NNTP servers ...
Don.Lewis at tsc.tdk.com
Fri Sep 17 14:26:03 UTC 1999
On Sep 16, 6:37pm, The Hermit Hacker wrote:
} Subject: Reducing communications between NNTP servers ...
} Recently, I asked about what N+1 servers does to ones bandwidth, and
} someone brought up that software like NNTPrelay(?) cache the information
} of what a host has sent you, so that you don't send back to them, even if
} its queued up...
} Has anyone looked at doing this?
} Basically, I would think this would be relatively easy to do, no?
} Extend innfeed.conf to include a 'path' field, to reflect the path of the
} machine that the feed is for, and have innd write to something like
} <spooldir>/received...have the received/path file cached in innfeed, such
} that when its processing its backlog file (or innxmit is), it checks to
} see if we've received/rejected that article from that site yet, and if so,
} don't try and feed it back...
In the case of received or (non-duplicate) rejected articles, this
is already handled by innd. That's what the path entry in newsfeeds is
for. Without this simple optimization, leaf sites would try to feed
everything they receive back upstream.
A useful optimization that INN doesn't do is to keep from offering
articles to servers that have offered it the article. If you've
got a well connected server, it will receive CHECKs for the same
message from a number of its peers within a very short period of time.
INN will respond to the first with a request for the article to be
sent, and will send deferal or refusal responses to the rest. Once
the article has been received, INN will offer the article to all of
its peers (modulo newsgroup/distribution, etc. entries in newsfeeds)
except the one it accepted the article from (due to the path check).
If INN knew the correspondence between it's incoming connections and
their corresponding newsfeed entries, then it could cache information
about which peers offered each article and not bother offering
the article to those peers. Cache entries would be purged once the
article has been written to the output channel(s).
} Doable? Or more overhead then benefit?
While it would be nice to avoid offering backlogged articles to
peers that have offered them, it would require a (possibly large)
persistent database that could handle N times the write transaction
rate that the history database has to handle. This could be difficult ...
More information about the inn-workers