innfeed questions/recommendations

[bill davidsen]

In article <Pine.GSO.4.44.0309301318080.29923-100000 at vidi.ucdavis.edu>,
Kathryn Hemness  <kfhemness at ucdavis.edu> wrote:
| Greetings --
| 
| My news server is using innfeed for the distribution of articles and
| currently has an unlimited backlog which I wish to limit.
| 
| Here are my current innfeed.conf settings:
| 
| backlog-directory:              innfeed                 # relative to pathspool
| backlog-rotate-period:          60
| backlog-ckpt-period:            30
| backlog-newfile-period:         600
| backlog-limit:                  0
| backlog-factor:                 1.10
| backlog-limit-highwater:        0
| dynamic-backlog-filter:         0.7
| dynamic-backlog-low:            25.0
| dynamic-backlog-high:           50.0
| no-backlog:                     false
| 
| Is there a recommended size for backlog-limit?  Just looking at my backlog
| directory, I've determined that a 1MB *.output file contains about 300 lines.
| One of my peer.output files contains over 3 million lines.
| 
| Also, this particular peer is actually accepting most of the articles sent,
| but I have another peer which is refusing or rejecting about half of the
| articles sent (this peer's innfeed output file is about half the size of
| the 3-million-liner).  Is there any way that I can determine why this
| peer is refusing/rejecting articles so that I can optimize my newsfeeds
| file?

If you are going to do this I suggest you split the feed into several
parts, and apply a dose of common sense to each. For external peers I
split by size to <32k, <250k, big. Depending on your needs you may go
smaller on the things you assume are text, have large and huge queues,
etc. I also have a special queue for local posts, which checks the
server of origin. If it was posted internally it gets help a very long
time to be sure it gets out. Of course with 60 peers that's not really
an issue except to leaf feeds.

This also allows you to determine what the problem is. If big stuff
backlogs it's bandwidth, if small stuff backlogs it's a history lookup
bottleneck. I've seen sites which could only accept 10 srt/sec
regardless of size, this does happen.

STREAMS: when you see history backlog, using more streams might help,
particularly if you have only one! Look at the adaptive methods and
think about the fed server. Don't go hog wild, you usually hit
diminishing returns pretty fast.
| 
| 
| And one last question...backlog-rotate-period is undocumented in the
| innfeed.conf man page.  I'm assuming the value is in seconds, but what
| kind of "rotation" is it controlling?

No idea without reading source.
-- 
bill davidsen <davidsen at tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.