Thoughts on a feed control system

[Kai Henningsen]

Well, it *is* in TODO.

It seems to me a generic design would be as follows:

1. Each (downstream) peer has up to N subfeeds that can be configured  
pretty much independently, corresponding to entries in newsfeeds

2. For each subfeed, there's an uwildmat [list] and possibly other  
attributes

3. Each peer can (by some transport) set, get, and modify a request for  
any of its N subfeeds detailing these attributes

4. These requests, together with local downstream and upstream peer  
configs, get munged by admin scripts (applying whatever policy is wanted)  
to produce

4a. "real" requests (newsfeeds entries)

4b. outgoing requests for upstream peers peers (to be transmitted by  
whatever transport). This step probably needs reasonable docs on how to  
make sure your peers don't overload you with binaries, and so on, and any  
default should be set up that way. And it would probably be useful to have  
a needs-manual-confirmation mode here.

Local policy would decide when 4a and 4b happen - not necessarily at the  
same time.


Now, *if* a text format for such a request could be agreed to, then any  
transport would just need to have some way to handle transferring this  
request and doing whatever authentication seems necessary.


One might want to be able to send partial requests, changing only some  
attributes, or adding some uwildmat patterns to the end of the list - GUP  
has a nice algorithm to optimize lists changed that way, dropping useless  
earlier patterns. Then again, especially for 4b-style requests, at least  
changing the whole uwildmat [list] seems useful, too.

Anyway, the format should be *able* to express anything that current  
newsservers *can* configure per-feed, but should not actually need to set  
all those attributes (most would probably be overriden by those admin  
scripts anyway).

news.software.nntp should probably be asked for thoughts if this proposal  
seems to be actually going anywhere.


There is one other point. A design such as this makes it useful to have a  
master newsgroup list that is larger than the list of newsgroups actually  
carried. In westfalen.de, this is the active file, but other designs seem  
possible. This is presumably where any grand scheme of all-hierarchies  
group lists (such as the ones at ISC) come in. Anyway, the active file  
should have the groups one is requesting from elsewhere, or one is  
probably wasting bandwidth.

MfG Kai