INN feed autonegotation

[Kristian Köhntopp]

I am running a INN server on a typical dedicated server. The 
machine (Celeron 2400 with 512 MB RAM and 6.5 GB cycbuff 
storage) has a traffic limit of 300 GB/month (120 KB/sec), which 
is plenty if you do not carry binaries, and at the moment I have 
50+ news peers. The machine is usually in the <200 of the top 
1000.

Cycbuffs are great if you want to run a maintenance free news 
server - the system is essentially self-servicing and does not 
need the constant care and feeding that I was used to when I 
last played around with USENET news, which was about 10 years 
ago.

Maintaining and debugging 50+ peers is work, though, and I 
beliebe a great bit of this need not be there.

For example, I have a incoming.conf, where I can specify 
acceptance patterns for all feeds, groups of feeds or individual 
feeds, as well as the number of connections and similar stuff. 
The peer connecting me has matching parameters, a newsfeeds 
entry, a max-connection setting, an article size limit and the 
like. 

Unfortunately, these parameters often do not actually match. For 
the last two weeks, I have been burning 250+ KB/sec, with peak 
rates of 1250 KB/sec and more, because some of my feeds did not 
honor my article size limits, and the NNTP protocol provides 
insufficient feedback to the upstream. Also, I have dropped the 
it.* and it-alt.* hierarchies, because I do not read Italian and 
these hierarchies are much traffic. I have plenty of feeds still 
feeding me it.* and it-alt.*, because there is no in-band way to 
notify them of the change.

Gup is not a solution to this problem, because it is an 
out-of-band solution and an optional install. Many of my peers 
do not run Gup, and for those that do, I have to manage 
passwords, send mail, and manually reintegrate the results.


I propose a better, in-band solution and want to hear what you 
think about this. That solution is a NNTP extension, perhaps as 
an extended answer to "MODE STREAM" or as a separate command 
such as "FEEDPARMS" or "FEEDPARMS pattern|max-connections|
article-size". 

Upon receiving that command, the downstream server deliers a set 
of feed configuration parameters to the upstream server. The 
upstream server intersects that parameter set with its local 
configuration and generates an effective configuration from 
this. Thus, a downstream server can maintain and control what it 
is being fed simply by changing its local incoming.conf and 
reloading the configuration. Parameters negotiated are:

- the pattern of newsgroups that is being accepted at downstream 
("pattern")
- the maxmimum number of connections ("max-connections")
- the maximum article size (to be introduced parameter 
"articles-size" in incoming.conf).

The upstream server receives this parameters and creates an 
effective configuration by intersecting (minimizing) these 
parameters with its local configuration. That is:

- effective feed pattern is the intersection of what is 
configured in newsfeeds and what the downstream announced it 
accepts.

- effective max-connections is the minimum of what is configured 
in innfeed.conf and what the downstream announced it accepts.

- effective article-size limit is the minimum of what is 
configured in newsfeeds and what the downstream announced it 
accepts.

Security implications of this are nil, because the exchange is 
done inband within a connection that both parties already agreed 
to trust, and because the negotation is intersecting/minimizing 
parameters.

What do you think of this idea? Is this useful? Can it be 
implemented?

Kristian