INN commit: trunk (10 files)
INN Commit
rra at isc.org
Wed Mar 2 18:46:54 UTC 2011
Date: Wednesday, March 2, 2011 @ 10:46:54
Author: iulius
Revision: 9189
Convert the innfeed.conf man page to POD.
* Add backlog-highwater, news-spool and input-file in the innfeed.conf
sample file.
* Add fast-exit, debug-shrinking and initial-sleep in both the documentation
and the sample file.
* To match what innfeed does by default :
- change host-queue-highwater from 200 to 10 in the sample file
- change max-connections from 5 to 2 in the sample file
- change dynamic-backlog-low from 25.0 to 20.0 in the sample file
* innfeed/host.c was expecting "backlog-limit-high" whereas innfeed/tape.c
was properly using "backlog-limit-highwater" (which is the one mentioned
in the innfeed.conf configuration file and the documentation). It then
appears that innfeed was not taking the value into account and always
using the default value of 0.
* Fix the documentation for the path to innfeed.status. It is either
pathlog or pathhttp, and not the backlog directory.
* Fix the paragraph about imapfeed: "deliver" is not a parameter.
* Remove the notion of version 0.9.3 and 1.0 for innfeed.
* Remove the mention that "in previous versions of innfeed, a value of 1
had a special meaning for <max-connections>. This is no longer the case,
1 means a maximum of 1 connection."
* Remove the mention that "in innfeed 0.9.3 and earlier, <no-check-high>
was in the range [0.0, 9.0]".
* Remove the sample innfeed.conf file from the documentation (no need to
duplicate it).
* Typos.
Added:
trunk/doc/pod/innfeed.conf.pod
Modified:
trunk/MANIFEST
trunk/doc/man/ (properties)
trunk/doc/pod/Makefile
trunk/doc/pod/innfeed.pod
trunk/innfeed/host.c
trunk/innfeed/innfeed.h
trunk/innfeed/tape.c
trunk/samples/innfeed.conf
Deleted:
trunk/doc/man/innfeed.conf.5
--------------------------+
MANIFEST | 1
doc/man/innfeed.conf.5 | 740 ---------------------------------------------
doc/pod/Makefile | 3
doc/pod/innfeed.conf.pod | 716 +++++++++++++++++++++++++++++++++++++++++++
doc/pod/innfeed.pod | 6
innfeed/host.c | 8
innfeed/innfeed.h | 2
innfeed/tape.c | 2
samples/innfeed.conf | 207 +++++-------
9 files changed, 827 insertions(+), 858 deletions(-)
Modified: MANIFEST
===================================================================
--- MANIFEST 2011-02-12 13:09:20 UTC (rev 9188)
+++ MANIFEST 2011-03-02 18:46:54 UTC (rev 9189)
@@ -266,6 +266,7 @@
doc/pod/innconfval.pod Master file for innconfval.1
doc/pod/innd.pod Master file for innd.8
doc/pod/inndf.pod Master file for inndf.8
+doc/pod/innfeed.conf.pod Master file for innfeed.conf.5
doc/pod/innfeed.pod Master file for innfeed.8
doc/pod/innmail.pod Master file for innmail.1
doc/pod/innupgrade.pod Master file for innupgrade.8
Property changes on: trunk/doc/man
___________________________________________________________________
Modified: svn:ignore
- active.5
active.times.5
actsync.8
archive.8
auth_krb5.8
batcher.8
buffchan.8
buffindexed.conf.5
ckpasswd.8
cnfsheadconf.8
cnfsstat.8
control.ctl.5
convdate.1
ctlinnd.8
cvtbatch.8
cycbuff.conf.5
distrib.pats.5
distributions.5
docheckgroups.8
domain.8
expire.ctl.5
expire.8
expireover.8
expirerm.8
fastrm.1
getlist.1
grephistory.1
ident.8
incoming.conf.5
inews.1
inn.conf.5
INN__Config.3pm
innbind.8
inncheck.8
innconfval.1
innd.8
inndf.8
innfeed.8
innmail.1
innupgrade.8
innxmit.8
libauth.3
libinnhist.3
libstorage.3
list.3
mailpost.8
makedbz.8
makehistory.8
mod-active.8
moderators.5
motd.news.5
newsfeeds.5
news.daily.8
news2mail.8
newslog.5
newsgroups.5
ninpaths.8
nnrpd.8
nntpsend.8
nntpsend.ctl.5
ovdb.5
ovdb_init.8
ovdb_monitor.8
ovdb_server.8
ovdb_stat.8
overchan.8
passwd.nntp.5
perl-nocem.8
pgpverify.1
prunehistory.8
pullnews.1
qio.3
radius.8
radius.conf.5
rc.news.8
readers.conf.5
rnews.1
sasl.conf.5
scanlogs.8
send-uucp.8
sendinpaths.8
shlock.1
simpleftp.1
sm.1
storage.conf.5
subscriptions.5
tally.control.8
tdx-util.8
tinyleaf.8
tst.3
uwildmat.3
+ active.5
active.times.5
actsync.8
archive.8
auth_krb5.8
batcher.8
buffchan.8
buffindexed.conf.5
ckpasswd.8
cnfsheadconf.8
cnfsstat.8
control.ctl.5
convdate.1
ctlinnd.8
cvtbatch.8
cycbuff.conf.5
distrib.pats.5
distributions.5
docheckgroups.8
domain.8
expire.ctl.5
expire.8
expireover.8
expirerm.8
fastrm.1
getlist.1
grephistory.1
ident.8
incoming.conf.5
inews.1
inn.conf.5
INN__Config.3pm
innbind.8
inncheck.8
innconfval.1
innd.8
inndf.8
innfeed.conf.5
innfeed.8
innmail.1
innupgrade.8
innxmit.8
libauth.3
libinnhist.3
libstorage.3
list.3
mailpost.8
makedbz.8
makehistory.8
mod-active.8
moderators.5
motd.news.5
newsfeeds.5
news.daily.8
news2mail.8
newslog.5
newsgroups.5
ninpaths.8
nnrpd.8
nntpsend.8
nntpsend.ctl.5
ovdb.5
ovdb_init.8
ovdb_monitor.8
ovdb_server.8
ovdb_stat.8
overchan.8
passwd.nntp.5
perl-nocem.8
pgpverify.1
prunehistory.8
pullnews.1
qio.3
radius.8
radius.conf.5
rc.news.8
readers.conf.5
rnews.1
sasl.conf.5
scanlogs.8
send-uucp.8
sendinpaths.8
shlock.1
simpleftp.1
sm.1
storage.conf.5
subscriptions.5
tally.control.8
tdx-util.8
tinyleaf.8
tst.3
uwildmat.3
Deleted: doc/man/innfeed.conf.5
===================================================================
--- doc/man/innfeed.conf.5 2011-02-12 13:09:20 UTC (rev 9188)
+++ doc/man/innfeed.conf.5 2011-03-02 18:46:54 UTC (rev 9189)
@@ -1,740 +0,0 @@
-.\" -*- nroff -*-
-.\"
-.\" Author: James A. Brister <brister at vix.com> -- berkeley-unix --
-.\" Start Date: Sun, 21 Jan 1996 00:47:37 +1100
-.\" Project: INN -- innfeed
-.\" File: innfeed.conf.5
-.\" RCSId: $Id$
-.\" Description: Man page for innfeed.conf(5)
-.\"
-.TH innfeed.conf 5
-.SH NAME
-innfeed.conf \- configuration file for innfeed
-.SH DESCRIPTION
-.PP
-This man page describes the configuration file for version 1.0 of
-innfeed. This format has changed dramatically since version 0.9.3.
-.PP
-The file
-.B innfeed.conf
-is used to control the innfeed(1) program. It is a fairly free-format file
-that consists of three types of entries: \fIkey/value\fP, \fIpeer\fP and
-\fIgroup\fP.
-Comments are from the hash character ``#'' to the end of the line.
-.PP
-\fIKey/value\fP entries are a keyword and a value separated by a colon
-(which can itself be surrounded by whitespace). For example:
-.PP
-.RS
-.nf
-max-connections: 10
-.fi
-.RE
-.PP
-A legal
-key starts with a letter and contains only letters, digits, and ``_'',
-``-''.
-.LP
-There are 5 different type of values: integers, floating-point numbers,
-characters, booleans, and strings. Integer and floating point numbers are
-as to be expected except that exponents in floating point numbers are not
-supported. A boolean value is either ``true'' or ``false'' (case is not
-significant). A character value is a single-quoted character as defined by
-the C-language. A string value is any other sequence of characters. If the
-string needs to contain whitespace, then it must be quoted with double
-quotes, and uses the same format for embedding non-printing characters as
-normal C-language string.
-.PP
-Peer entries look like:
-.PP
-.RS
-.nf
-peer <name> {
- # body ...
-}
-.fi
-.RE
-.PP
-The word ``peer'' is required. The ``<name>'' is the same as the site name
-in INN's newsfeeds file. The body of a peer entry contains some number
-(possibly zero) of key/value entries.
-.PP
-Group entries look like:
-.PP
-.RS
-.nf
-group <name> {
- # body
-}
-.fi
-.RE
-.PP
-The word ``group'' is required. The ``<name>'' is any string valid as a
-key. The body of a group entry contains any number of the three types of
-entries. So key/value pairs can be defined inside a group, and peers can be
-nested inside a group, and other groups can be nested inside a group.
-.PP
-Key/value entries that are defined outside of all peer and group entries
-are said to be at ``global scope''. There are global key/value entries that
-apply to the process as a whole (for example the location of the backlog
-file directory), and there are global key/value entries that act as
-defaults for peers. When innfeed looks for a specific value in a peer entry
-(for example, the maximum number of connections to set up), if the value is
-not defined in the peer entry, then the enclosing groups are examined for
-the entry (starting at the closest enclosing group). If there are no
-enclosing groups, or the enclosing groups don't define the key/value, then
-the value at global scope is used.
-.PP
-A small example could be:
-.PP
-.RS
-.nf
-# Global value applied to all peers that have
-# no value of their own.
-max-connections: 5
-
-# A peer definition. ``uunet'' is the name used by innd in
-# the newsfeeds file.
-peer uunet {
- ip-name: usenet1.uu.net
-}
-
-peer vixie {
- ip-name: gw.home.vix.com
- max-connections: 10 # override global value.
-}
-
-# A group of two peers who can handle more connections
-# than normal
-group fast-sites {
- max-connections: 15
-
- # Another peer. The ``max-connections'' value from the
- # ``fast-sites'' group scope is used. The ``ip-name'' value
- # defaults to the peer's name.
- peer data.ramona.vix.com {
- }
-
- peer bb.home.vix.com {
- max-connections: 20 # he can really cook.
- }
-}
-.fi
-.RE
-.PP
-Given the above configuration file, the defined peers would have the
-following values for the ``max-connections'' key.
-.PP
-.RS
-.nf
-uunet 5
-vixie 10
-data.ramona.vix.com 15
-bb.home.vix.com 20
-.fi
-.RE
-.PP
-Innfeed ignores key/value pairs it is not interested in. Some config file
-values can be set via a command line option, in which case that setting
-overrides the settings in the file.
-.PP
-Config files can be included in other config files via the syntax:
-.sp 1
-.nf
-.RS
-$INCLUDE filename
-.RE
-.fi
-.sp 1
-There is a maximum nesting depth of 10.
-.PP
-For a fuller example config file, see the supplied \fIinnfeed.conf\fP.
-.SH "GLOBAL VALUES"
-.PP
-The following listing show all the keys that apply to the process as
-whole. These are not required (compiled-in defaults are used where needed).
-.TP
-.B news-spool
-This key requires a pathname value. It specifies where the top of the
-article spool is. This corresponds to the ``\fI\-a\fP'' command-line
-option.
-.TP
-.B input-file
-This key requires a pathname value. It specifies the pathname (relative to
-the \fBbacklog-directory\fP) that should be read in funnel-file mode. This
-corresponds to giving a filename as an argument on the command-line (i.e.
-its presence also implies that funnel-file mode should be used).
-.TP
-.B pid-file
-This key requires a pathname value. It specifies the pathname (relative to
-the \fBbacklog-directory\fP) where the pid of the innfeed process should be
-stored. This corresponds to the ``\fI\-p\fP'' command-line option.
-.TP
-.B debug-level
-This key defines the debug level for the process. A non-zero number
-generates a lot of messages to stderr, or to the config-defined ``log-file''.
-This corresponds to the ``\fI\-d\fP'' command-line option.
-.TP
-.B use-mmap
-This key requires a boolean value. It specifies whether mmaping should be
-used if innfeed has been built with mmap support. If article data on disk
-is not in NNTP-ready format (CR/LF at the end of each line), then after
-mmaping the article is read into memory and fixed up, so mmaping has no
-positive effect (and possibly some negative effect depending on your
-system), and so in such a case this value should be \fIfalse\fP. This
-corresponds to the ``\fI\-M\fP'' command-line option.
-.TP
-.B log-file
-This key requires a pathname value. It specifies where any logging messages
-that couldn't be sent via syslog(3) should go (such as those generated when
-a positive value for ``\fBdebug-value\fP'', is used). This corresponds to
-the ``\fI\-l\fP'' command-line option. A relative pathname is relative to
-the ``\fBbacklog-directory\fP'' value.
-.TP
-.B log-time-format
-This key requires a format string suitable for strftime(3). It is used for
-messages sent via syslog(3) and to the \fBstatus-file\fP. Default value
-is "%a %b %d %H:%M:%S %Y".
-.\" .TP
-.\" .B initial-sleep
-.\" This key requires a positive integer value. It specifies how many seconds
-.\" innfeed should sleep at startup before attempting to take out its locks. On
-.\" fast machines and with innfeed handling many connections, it can take too
-.\" long for innfeed to recognise that its input has been closed, and that it
-.\" should release any locks it holds.
-.\"..................................................
-.TP
-.B backlog-directory
-This key requires a pathname value. It specifies where the current innfeed
-process should store backlog files. This corresponds to the ``\fI\-b\fP''
-command-line option.
-.TP
-.B backlog-highwater
-This key requires a positive integer value. It specifies how many articles
-should be kept on the backlog file queue before starting to write new
-entries to disk.
-.TP
-.B backlog-ckpt-period
-This key requires a positive integer value. It specifies how many seconds
-between checkpoints of the input backlog file. Too small a number will mean
-frequent disk accesses, too large a number will mean after a crash innfeed
-will re-offer more already-processed articles than necessary.
-.TP
-.B backlog-newfile-period
-This key requires a positive integer value. It specifies how many seconds
-before each checks for externally generated backlog files that are to be
-picked up and processed.
-.TP
-.B backlog-rotate-period
-This key requires a positive integer value. It specifies how many seconds
-elapse before
-.B innfeed
-checks for a manually created backlog file and moves the output backlog
-file to the input backlog file.
-.\"..................................................
-.TP
-.B dns-retry
-This key requires a positive integer value. It defines the number of seconds
-between attempts to re-lookup host information that previous failed to be
-resolved.
-.TP
-.B dns-expire
-This key requires a positive integer value. It defines the number of seconds
-between refreshes of name to address DNS translation. This is so long-running
-processes don't get stuck with stale data, should peer ip addresses change.
-.TP
-.B close-period
-This key requires a positive integer value. It is the maximum number of
-seconds a connection should be kept open. Some NNTP servers don't deal well
-with connections being held open for long periods.
-.TP
-.B gen-html
-This key requires a boolean value. It specifies whether the
-\fBstatus-file\fP should be HTML-ified.
-.TP
-.B status-file
-This key requires a pathname value. It specifies the pathname (relative to
-the \fBbacklog-directory\fP) where the periodic status of the innfeed
-process should be stored. This corresponds to the ``\fI\-S\fP''
-command-line option.
-.TP
-.B connection-stats
-This key requires a boolean value. If the value is true, then whenever the
-transmission statistics for a peer are logged, then each active connection
-logs its own statistics. This corresponds to the ``\fI\-z\fP''
-command-line option.
-.TP
-.B host-queue-highwater
-This key requires a positive integer value. It defines how many articles
-will be held internally for a peer before new arrivals cause article
-information to be spooled to the backlog file.
-.TP
-.B stats-period
-This key requires a positive integer value. It defines how many seconds
-innfeed waits between generating statistics on transfer rates.
-.TP
-.B stats-reset
-This key requires a positive integer value. It defines how many seconds
-innfeed waits before resetting all internal transfer counters back to zero
-(after logging one final time). This is so a innfeed-process running more
-than a day will generate ``final'' stats that will be picked up by logfile
-processing scripts.
-.\"..................................................
-.TP
-.B initial-reconnect-time
-This key requires a positive integer value. It defines how many seconds to
-first wait before retrying to reconnect after a connection failure. If the
-next attempt fails too, then the reconnect time is approximately doubled
-until the connection succeeds, or \fBmax-reconnection-time\fP is reached.
-.TP
-.B max-reconnect-time
-This key requires an integer value. It defines the maximum number of
-seconds to wait between attempt to reconnect to a peer. The initial value
-for reconnection attempts is defined by \fBinitial-reconnect-time\fP, and
-it is doubled after each failure, up to this value.
-.TP
-.B stdio-fdmax
-This key requires a non-negative integer value. If the value is greater
-than zero, then whenever a network socket file descriptor is created and
-it has a value \fIless than\fP this, the file descriptor will be dup'ed to
-bring the value up greater than this. This is to leave lower numbered file
-descriptors free for stdio. Certain systems, Sun's in particular, require
-this. SunOS 4.1.x usually requires a value of 128 and Solaris requires a
-value of 256. The default if this is not specified, is 0.
-.\"..................................................
-.SH "GLOBAL PEER DEFAULTS"
-.PP
-All the key/value pairs mentioned in this section must be specified at
-global scope. They may also be specified inside a group or peer
-definition. Note that when peers are added dynamically (i.e. when
-innfeed receives an article for an unspecified peer), it will add
-the peer site using the parameters specified at global scope.
-.TP
-.B article-timeout
-This key requires a non-negative integer value. If no articles need to be
-sent to the peer for this many seconds, then the peer is considered idle
-and all its active connections are torn down.
-.TP
-.B response-timeout
-This key requires a non-negative integer value. It defines the maximum
-amount of time to wait for a response from the peer after issuing a
-command.
-.TP
-.B initial-connections
-This key requires a non-negative integer value. It defines the number of
-connections to be opened immediately when setting up a peer binding. A
-value of 0 means no connections will be created until an article needs to
-be sent.
-.TP
-.B max-connections
-This key requires positive integer value. It defines the maximum number of
-connections to run in parallel to the peer. A value of zero specifies an
-unlimited number of maximum connections. In general use of an unlimited
-number of maximum connections is not recommended. Do not ever set
-\fBmax-connections\fP to zero with \fBdynamic-method\fP 0 set, as this will
-saturate peer hosts with connections. [ Note that in previous versions
-of innfeed, a value of 1 had a special meaning. This is no longer the case,
-1 means a maximum of 1 connection ].
-.TP
-.B dynamic-method
-This key requires an integer value between 0 and 3. It controls how connections
-(up to max-connections) are opened, up to the maximum specified by
-\fBmax-connections\fP. In general (and specifically, with \fBdynamic-method\fP
-0), a new connection is opened when the current number of connections is
-below \fBmax-connections\fP, and an article is to be sent while no current
-connections are idle. Without further restraint (i.e. using
-\fBdynamic-method\fP 0), in practice this means that \fBmax-connections\fP
-connections are established while articles are being sent. Use of other
-\fBdynamic-method\fP settings imposes a further limit on the amount of
-connections opened below that specified by \fBmax-connections\fP. This
-limit is calculated in different ways, depending of the value of
-\fBdynamic-method\fP.
-Users should note that adding additional connections is not always
-productive - just because opening twice as many connections results
-in a small percentage increase of articles accepted by the remote peer,
-this may be at considerable resource cost both locally and at the remote
-site, whereas the remote site might well have received the extra articles
-sent from another peer a fraction of a second later. Opening large
-numbers of connections is considered antisocial.
-The meanings of the various settings are:
-.RS
-.TP
-.B 0 no method
-Increase of connections up to \fBmax-connections\fP is unrestrained.
-.TP
-.B 1 maximize articles per second
-Connections are increased (up to \fBmax-connections\fP) and decreased so as
-to maximize the number of articles per second sent, while using the fewest
-connections to do this.
-.TP
-.B 2 set target queue length
-Connections are increased (up to \fBmax-connections\fP) and decreased so as
-to keep the queue of articles to be sent within the bounds set by
-\fBdynamic-backlog-low\fP and \fBdynamic-backlog-high\fP,
-while using the minimum resources possible.
-As the queue will tend to fill if the site is not keeping up, this method
-ensures that the maximum number of articles are offered to the peer
-while using the minimum number of connections to achieve this.
-.TP
-.B 3 combination
-This method uses a combination of methods 1 and 2 above. For sites
-accepting a large percentage of articles, method 2 will be used to
-ensure these sites are offered as complete a feed as possible. For sites
-accepting a small percentage of articles, method 1 is used, to minimize
-remote resource usage. For intermediate sites, an appropriate combination
-is used.
-.RE
-.TP
-.B dynamic-backlog-low
-This key requires an integer value between 0 and 100. It represents (as a
-percentage) the low water mark for the host queue. If the host queue falls
-below this level while using \fBdynamic-method\fP 2 or 3, and if 2 or more
-connections are open, innfeed will attempt to drop connections to the host.
-An IIR filter is applied to the value to prevent connection flap (see
-\fBdynamic-filter\fP). A value of 25.0 is recommended. This value
-must be smaller than \fBdynamic-backlog-high\fP.
-.TP
-.B dynamic-backlog-high
-This key requries an integer value between 0 and 100. It represents (as a
-percentage) the high water mark for the host queue. If the host queue rises
-above this level while using \fBdynamic-method\fP 2 or 3, and if less than
-\fBmax-connections\fP are open to the host, innfeed will attempt
-to open further connections to the host. An IIR filter is applied to the value
-to prevent connection flap (see \fBdynamic-filter\fP). A value of 50.0
-is recommended. This value must be larger than \fBdynamic-backlog-low\fP.
-.TP
-.B dynamic-backlog-filter
-This key requires a floating-point value between 0 and 1. It represents the
-filter coefficient used by the IIR filter used to implement
-\fBdynamic-method\fP 2 and 3.
-The recommended value of this filter is 0.7, giving a time
-constant of 1/(1-0.7) articles. Higher values will result in slower
-response to queue fullness changes, lower values in faster response.
-.TP
-.B max-queue-size
-This key requires a positive integer value. It defines the maximum number
-of articles to process at one time when using streaming to transmit to a
-peer. Larger numbers mean more memory consumed as articles usually get
-pulled into memory (see the description of \fBuse-mmap\fP).
-.TP
-.B streaming
-This key requires a boolean value. It defines whether streaming commands
-are used to transmit articles to the peers.
-.TP
-.B no-check-high
-This key requires a floating-point number which must be in the range [0.0,
-100.0]. When running transmitting with the streaming commands, innfeed
-attempts an optimization called ``no-CHECK'' mode. This involves \fInot\fP
-asking the peer if it wants the article, but just sending it. This
-optimization occurs when the percentage of the articles the peer has
-accepted gets larger than this number. If this value is set to 100.0, then
-this effectively turns off no-CHECK mode, as the percentage can never get
-above 100.0. If this value is too small, then the number of articles the
-peer rejects will get bigger (and your bandwidth will be wasted). A value
-of 95.0 usually works pretty well. NOTE: In innfeed 0.9.3 and earlier this
-value was in the range [0.0, 9.0].
-.TP
-.B no-check-low:
-This key requires a floating-point number which must be in the range [0.0,
-100.0), and it must be smaller that the value for \fBno-check-high\fP. When
-running in no-CHECK mode, as described above, if the percentage of articles
-the remote accepts drops below this number, then the no-CHECK optimization
-is turned off until the percentage gets above the \fBno-check-high\fP value
-again. If there is small difference between this and the
-\fBno-check-high\fP value (less than about 5.0), then innfeed may
-frequently go in and out of no-CHECK mode. If the difference is too big,
-then it will make it harder to get out of no-CHECK mode when necessary
-(wasting bandwidth). Keeping this to between 5.0 and 10.0 less than
-\fBno-check-high\fP usually works pretty well.
-.TP
-.B no-check-filter
-This is a floating point value representing the time constant, in articles,
-over which the CHECK / no-CHECK calculations are done. The recommended
-value is 50.0 which will implement an IIR filter of time constant 50. This
-roughly equates to making a decision about the mode over the previous
-50 articles. A higher number will result in a slower response to changing
-percentages of articles accepted; a lower number will result in a faster
-response.
-.TP
-.B bindaddress
-This key requires a string value. It specifies which outgoing IPv4 address
-innfeed should bind the local end of its connection to.
-Must be an IPv4 address in dotted-quad format (nnn.nnn.nnn.nnn), "any",
-or "none". If not set or set to "any", innfeed defaults
-to letting the kernel choose this address.
-If set to "none", innfeed will not use IPv4 for outgoing connections
-to peers in this scope (i.e. it forces IPv6).
-If not set in innfeed.conf, innfeed defaults to the value of
-\fBsourceaddress\fP from inn.conf(5) (which by default is unset).
-.TP
-.B bindaddress6
-This key requires a string value. It behaves like \fBbindaddress\fP except
-for outgoing IPv6 connections. Must be in numeric IPv6 format (note
-that a value containing colons must be enclosed in double quotes), "any",
-or "none". If set to "none", innfeed will not use IPv6 for outgoing
-connections to peers in this scope.
-If not set in innfeed.conf, innfeed defaults to the value of
-\fBsourceaddress6\fP from inn.conf(5) (which by default is unset).
-.TP
-.B port-number
-This key requires a positive integer value. It defines the tcp/ip port
-number to use when connecting to the remote.
-.TP
-.B force-ipv4
-This key requires a boolean value. By default it is set to false.
-Setting it to true is the same as setting "bindaddress6: none"
-and removing "bindaddress: none" if it was set.
-.TP
-.B drop-deferred
-This key requires a boolean value. By default it is set to false. When
-set to true, and a peer replies with code 431 or 436 (try again later) just
-drop the article and don't try to re-send it. This is useful for some
-peers that keep on deferring articles for a long time to prevent innfeed
-from trying to offer the same article over and over again.
-.TP
-.B min-queue-connection
-This key requires a boolean value. By default it is set to false. When
-set to true, innfeed will attempt to use a connection with the least queue
-size (or the first empty connection). If this key is set to true, it is
-recommended that \fBdynamic-method\fP be set to 0. This allows for article
-propagation with the least delay.
-.TP
-.B no-backlog
-This key requires a boolean value. It specifies whether spooling should
-be enabled (false, the default) or disabled (true). Note that when no-backlog
-is set, articles reported as "spooled" are actually silently discarded.
-.TP
-.B backlog-limit
-This key requires a non-negative integer value. If the number is 0 then
-backlog files are allowed to grown without bound when the peer is unable to
-keep up with the article flow. If this number is greater than 0 then it
-specifies the size (in bytes) the backlog file should get truncated to when
-the backlog file reaches a certain limit. The limit depends on whether
-\fBbacklog-factor\fP or \fBbacklog-limit-highwater\fP is used.
-.TP
-.B backlog-factor
-This key requires a floating point value, which must be larger than 1.0. It
-is used in conjunction with the peer key \fBbacklog-limit\fP. If
-\fBbacklog-limit\fP has a value greater than zero, then when the backlog
-file gets larger than the value \fBbacklog-limit * backlog-factor\fP, then
-the backlog file will be truncated to the size \fBbacklog-limit\fP. For
-example if \fBbacklog-limit\fP has a value of 1000000, and
-\fBbacklog-factor\fP has a value of 2.0, then when the backlogfile gets to
-be larger than 2000000 bytes in size, it will be truncated to 1000000 bytes.
-The front
-portion of the file is removed, and the trimming happens on line boundaries,
-so the final size may be a bit less than this number. If
-\fBbacklog-limit-highwater\fP is defined too, then \fBbacklog-factor\fP
-takes precedence.
-.TP
-.B backlog-limit-highwater
-This key requires a positive integer value that must be larger than the
-value for \fBbacklog-limit\fP. If the size of the backlog file gets larger
-than this value (in bytes), then the backlog file will be shrunk down to
-the size of \fBbacklog-limit\fP. If both \fBbacklog-factor\fP and
-\fBbacklog-limit-highwater\fP are defined, then the value of
-\fBbacklog-factor\fP is used.
-.TP
-.B backlog-feed-first
-This key requires a boolean value. By default it is set to false. When set
-to true, the backlog is fed before new files. This is intended to enforce
-in-order delivery, so setting this to true when initial-connections or
-max-connections is more than 1 is inconsistent.
-.TP
-.B username
-This key requires a string value. If the value is defined, then innfeed
-tries to authenticate by ``AUTHINFO USER'' and this value used for user name.
-\fBpassword\fP must also be defined, if this key is defined.
-.TP
-.B password
-This key requires a string value. The value is the password
-used for ``AUTHINFO PASS''.
-\fBusername\fP must also be defined, if this key is defined.
-.TP
-.B deliver
-This key is used with imapfeed to authenticate to a remote host. It is optional.
-There are several parameters that must be included with deliver:
-.RS
-.TP
-.B deliver-authname
-The authname is who you want to authenticate as.
-.TP
-.B deliver-password
-This is the appropriate password for authname.
-.TP
-.B deliver-username
-The username is who you want to "act" as, that is, who is actually
-going to be using the server.
-.TP
-.B deliver-realm
-In this case, the "realm" is the realm in which the specified authname
-is valid. Currently this is only needed by the DIGEST-MD5 SASL
-mechanism.
-.TP
-.B deliver-rcpt-to
-A printf-style format string for creating the envelope recipient address.
-The pattern MUST include a single string specifier which will be
-replaced with the newgroup (e.g "bb+%s"). The default is "+%s".
-.TP
-.B deliver-to-header
-An optional printf-style format string for creating a To: header to be
-prepended to the article. The pattern MUST include a single string
-specifier which will be replaced with the newgroup (e.g
-"post+%s at domain"). If not specified, the To: header will not be
-prepended.
-.RE
-.\"..................................................
-.SH "PEER VALUES"
-As previously explained, the peer definitions can contain redefinitions of
-any of the key/value pairs described in the \fBGLOBAL PEER DEFAULTS\fP
-section above. There is one key/value pair that is specific to a peer
-definition.
-.TP
-.B ip-name
-This key requires a word value. The word is the host's FQDN, or the dotted
-quad ip-address. If this value is not specified then the name of the peer
-is taken to also be its ip-name. See the entry for
-data.ramona.vix.com in the example below.
-.\"..................................................
-.SH RELOADING
-.PP
-If innfeed gets a SIGHUP signal, then it will reread the config file. All
-values at global scope except for ``\fBbacklog-directory\fP'' can be
-changed (although note that ``\fBbindaddress\fP'' and
-``\fBbindaddress6\fP'' changes will only affect new connections). Any new
-peers are added and any missing peers have their connections closed.
-.\"..................................................
-.SH EXAMPLE
-.PP
-Below is the sample innfeed.conf file.
-.RS
-.nf
-#
-# innfeed.conf file. See the comment block at the
-# end for a fuller description.
-#
-
-##
-## Global values. Not specific to any peer. These
-## are optional, but if used will override the
-## compiled in values. Command-line options used
-## will override these values.
-##
-
-pid-file: innfeed.pid
-debug-level: 0
-use-mmap: false
-log-file: innfeed.log
-stdio-fdmax: 0
-
-backlog-directory: innfeed
-backlog-rotate-period: 60
-backlog-ckpt-period: 30
-backlog-newfile-period: 600
-
-dns-retry: 900
-dns-expire: 86400
-close-period: 3600
-gen-html: false
-status-file: innfeed.status
-connection-stats: false
-host-queue-highwater: 200
-stats-period: 600
-stats-reset: 43200
-
-max-reconnect-time: 3600
-initial-reconnect-time: 30
-
-
-##
-## Defaults for all peers. These must all exist at
-## global scope. Any of them can be redefined
-## inside a peer or group definition.
-##
-
-article-timeout: 600
-response-timeout: 300
-initial-connections: 1
-max-connections: 5
-max-queue-size: 25
-streaming: true
-no-check-high: 95.0
-no-check-low: 90.0
-no-check-filter: 50.0
-port-number: 119
-backlog-limit: 0
-backlog-factor: 1.10
-backlog-limit-highwater:0
-dynamic-method: 3
-dynamic-backlog-filter: 0.7
-dynamic-backlog-low: 25.0
-dynamic-backlog-high: 50.0
-no-backlog: false
-
-##
-## Peers.
-##
-peer decwrl {
- ip-name: news1.pa.dec.com
-}
-
-peer uunet {
- ip-name: news.uunet.uu.net
- max-connections: 10
-}
-
-peer data.ramona.vix.com {
- # ip-name defaults to data.ramona.vix.com
- streaming: false
-}
-
-peer bb.home.vix.com {
- ip-name: 192.5.5.33
-}
-
-
-
-# Blank lines are ignored. Everything after a '#'
-# is ignored too.
-#
-# Format is:
-# key : value
-#
-# See innfeed.conf(5) for a description of
-# necessary & useful keys. Unknown keys and their
-# values are ignored.
-#
-# Values may be a integer, floating-point, c-style
-# single-quoted characters, boolean, and strings.
-#
-# If a string value contains whitespace, or
-# embedded quotes, or the comment character
-# (``#''), then the whole string must be quoted
-# with double quotes. Inside the quotes, you may
-# use the standard c-escape sequence
-# (\\t,\\n,\\r,\\f,\\v,\\",\\').
-#
-# Examples:
-# eg-string: "New\\tConfig\\tfile\\n"
-# eg-long-string: "A long string that goes
-# over multiple lines. The
-# newline is kept in the
-# string except when quoted
-# with a backslash \\
-# as here."
-# eg-simple-string: A-no-quote-string
-# eg-integer: 10
-# eg-boolean: true
-# eg-char: 'a'
-# eg-ctrl-g: '\\007'
-
-.fi
-.RE
-.SH HISTORY
-Written by James Brister <brister at vix.com> for InterNetNews.
-.de R$
-This is revision \\$3, dated \\$4.
-..
-.R$ $Id$
-.SH SEE ALSO
-innfeed(1), newsfeeds(5)
Modified: doc/pod/Makefile
===================================================================
--- doc/pod/Makefile 2011-02-12 13:09:20 UTC (rev 9188)
+++ doc/pod/Makefile 2011-03-02 18:46:54 UTC (rev 9189)
@@ -22,7 +22,7 @@
MAN5 = ../man/active.5 ../man/active.times.5 ../man/buffindexed.conf.5 \
../man/control.ctl.5 ../man/cycbuff.conf.5 ../man/distrib.pats.5 \
../man/distributions.5 ../man/expire.ctl.5 ../man/incoming.conf.5 \
- ../man/inn.conf.5 ../man/moderators.5 \
+ ../man/inn.conf.5 ../man/innfeed.conf.5 ../man/moderators.5 \
../man/motd.news.5 ../man/newsfeeds.5 ../man/newsgroups.5 \
../man/newslog.5 ../man/nntpsend.ctl.5 ../man/ovdb.5 \
../man/passwd.nntp.5 ../man/radius.conf.5 ../man/readers.conf.5 \
@@ -96,6 +96,7 @@
../man/expire.ctl.5: expire.ctl.pod ; $(POD2MAN) -s 5 $? > $@
../man/incoming.conf.5: incoming.conf.pod ; $(POD2MAN) -s 5 $? > $@
../man/inn.conf.5: inn.conf.pod ; $(POD2MAN) -s 5 $? > $@
+../man/innfeed.conf.5: innfeed.conf.pod ; $(POD2MAN) -s 5 $? > $@
../man/moderators.5: moderators.pod ; $(POD2MAN) -s 5 $? > $@
../man/motd.news.5: motd.news.pod ; $(POD2MAN) -s 5 $? > $@
../man/newsfeeds.5: newsfeeds.pod ; $(POD2MAN) -s 5 $? > $@
Added: doc/pod/innfeed.conf.pod
===================================================================
--- doc/pod/innfeed.conf.pod (rev 0)
+++ doc/pod/innfeed.conf.pod 2011-03-02 18:46:54 UTC (rev 9189)
@@ -0,0 +1,716 @@
+=head1 NAME
+
+innfeed.conf - Configuration file for innfeed
+
+=head1 DESCRIPTION
+
+The configuration file F<innfeed.conf> in I<pathetc> is used to control
+the innfeed(8) program. It is a fairly free-format file that consists
+of three types of entries: I<key>:I<value>, I<peer> and I<group>.
+Comments are from the hash character C<#> to the end of the line.
+
+I<key>:I<value> entries are a keyword and a value separated by a colon
+(which can itself be surrounded by whitespace). For example:
+
+ max-connections: 10
+
+A legal I<key> starts with a letter and contains only letters, digits,
+and the C<_> and C<-> characters. There are 5 different types of values:
+integers, floating-point numbers, characters, booleans, and strings.
+
+Integer and floating-point numbers are as to be expected, except that
+exponents in floating-point numbers are not supported. A boolean value
+is either C<true> or C<false> (case is not significant). A character
+value is a single-quoted character as defined by the C-language. A string
+value is any other sequence of characters. If the string needs to contain
+whitespace, then it must be quoted with double quotes, and uses the same
+format for embedding non-printing characters as normal C-language string.
+
+Peer entries look like:
+
+ peer <name> {
+ # body ...
+ }
+
+The word C<peer> is required. The I<< <name> >> is the same as the site
+name in INN's F<newsfeeds> configuration file. The body of a peer entry
+contains some number (possibly zero) of I<key>:I<value> entries.
+
+Group entries look like:
+
+ group <name> {
+ # body ...
+ }
+
+The word C<group> is required. The I<< <name> >> is any string valid as
+a key. The body of a group entry contains any number of the three types of
+entries. So I<key>:I<value> pairs can be defined inside a group, and peers
+can be nested inside a group, and other groups can be nested inside a group.
+
+I<key>:I<value> entries that are defined outside of all I<peer> and I<group>
+entries are said to be at "global scope". There are global I<key>:I<value>
+entries that apply to the process as a whole (for example the location of
+the backlog file directory), and there are global I<key>:I<value> entries
+that act as defaults for peers. When B<innfeed> looks for a specific value
+in a peer entry (for example, the maximum number of connections to set up),
+if the value is not defined in the peer entry, then the enclosing groups
+are examined for the entry (starting at the closest enclosing group).
+If there are no enclosing groups, or the enclosing groups do not define
+the I<key>:I<value>, then the value at global scope is used.
+
+A small example could be:
+
+ # Global value applied to all peers that have
+ # no value of their own.
+ max-connections: 5
+
+ # A peer definition. "uunet" is the name used by innd
+ # in the newsfeeds configuration file.
+ peer uunet {
+ ip-name: usenet1.uu.net
+ }
+
+ peer vixie {
+ ip-name: gw.home.vix.com
+ max-connections: 10 # Override global value.
+ }
+
+ # A group of two peers which can handle more connections
+ # than normal.
+ group fast-sites {
+ max-connections: 15
+
+ # Another peer. The "max-connections" value from the
+ # "fast-sites" group scope is used. The "ip-name" value
+ # defaults to the peer's name.
+ peer data.ramona.vix.com {
+ }
+
+ peer bb.home.vix.com {
+ max-connections: 20 # He can really cook.
+ }
+ }
+
+Given the above configuration file, the defined peers would have the
+following values for the I<max-connections> key:
+
+ uunet 5
+ vixie 10
+ data.ramona.vix.com 15
+ bb.home.vix.com 20
+
+B<innfeed> ignores I<key>:I<value> pairs it is not interested in.
+Some configuration file values can be set via a command-line option,
+in which case that setting overrides the settings in the file.
+
+Configuration files can be included in other configuration files via
+the syntax:
+
+ $INCLUDE filename
+
+There is a maximum nesting depth of 10.
+
+For a fuller example configuration file, see the supplied F<innfeed.conf>.
+
+=head1 GLOBAL VALUES
+
+The following listing show all the keys that apply to the process as
+whole. These are not required (compiled-in defaults are used where needed).
+
+=over 4
+
+=item I<news-spool>
+
+This key requires a pathname value and defaults to I<patharticles>
+in F<inn.conf>. It specifies where the top of the article spool is.
+This corresponds to the B<-a> command-line option.
+
+=item I<input-file>
+
+This key requires a pathname value. It specifies the pathname (relative to
+the I<backlog-directory> value) that should be read in funnel-file mode.
+This corresponds to giving a filename as an argument on the command-line
+(i.e. its presence also implies that funnel-file mode should be used).
+
+The default is unset; B<innfeed> then runs in channel or batch mode.
+
+=item I<pid-file>
+
+This key requires a pathname value and defaults to F<innfeed.pid>.
+It specifies the pathname (relative to the I<backlog-directory> value)
+where the pid of the B<innfeed> process should be stored. This corresponds
+to the B<-p> command-line option.
+
+=item I<debug-level>
+
+This key defines the debug level for the process. Default is C<0>.
+A non-zero number generates a lot of messages to stderr, or to the
+config-defined I<log-file>. This corresponds to the B<-d> command-line
+option.
+
+=item I<debug-shrinking>
+
+This key requires a boolean value and defaults to false (the debug file is
+allowed to grow without bound). If set to true, this file is truncated when
+its size reaches a certain limit. See I<backlog-limit> for more details.
+
+=item I<fast-exit>
+
+This key requires a boolean value and defaults to false. If set to true,
+when B<innfeed> receives a SIGTERM or SIGQUIT signal, it will close its
+listeners as soon as it can, even if it means dropping articles.
+
+=item I<use-mmap>
+
+This key requires a boolean value. It specifies whether mmaping should be
+used if B<innfeed> has been built with mmap(2) support. If article data
+on disk is not in NNTP-ready format (CR/LF at the end of each line), then
+after mmaping, the article is read into memory and fixed up, so mmaping
+has no positive effect (and possibly some negative effect depending on
+your system), and so in such a case this value should be C<false>, which
+is the default value. This corresponds to the B<-M> command-line option.
+
+=item I<log-file>
+
+This key requires a pathname value and defaults to F<innfeed.log>.
+It specifies where any logging messages that could not be sent via syslog(3)
+should go (such as those generated when a positive value for I<debug-value>
+is used). This corresponds to the B<-l> command-line option.
+
+A relative pathname is relative to the I<backlog-directory> value.
+
+=item I<log-time-format>
+
+This key requires a format string suitable for strftime(3). It is used
+for messages sent via syslog(3) and to the I<status-file>. Default value
+is C<%a %b %d %H:%M:%S %Y>.
+
+=item I<backlog-directory>
+
+This key requires a pathname value and defaults to F<innfeed>. It specifies
+where the current B<innfeed> process should store backlog files.
+This corresponds to the B<-b> command-line option.
+
+=item I<backlog-highwater>
+
+This key requires a positive integer value and defaults to C<5>.
+It specifies how many articles should be kept on the backlog file queue
+before starting to write new entries to disk.
+
+=item I<backlog-ckpt-period>
+
+This key requires a positive integer value and defaults to C<30>.
+It specifies how many seconds elapse between checkpoints of the input
+backlog file. Too small a number will mean frequent disk accesses; too
+large a number will mean after a crash, B<innfeed> will re-offer more
+already-processed articles than necessary.
+
+=item I<backlog-newfile-period>
+
+This key requires a positive integer value and defaults to C<600>.
+It specifies how many seconds elapse before each check for externally
+generated backlog files that are to be picked up and processed.
+
+=item I<backlog-rotate-period>
+
+This key requires a positive integer value and defaults to C<60>.
+It specifies how many seconds elapse before B<innfeed> checks for a
+manually created backlog file and moves the output backlog file to the
+input backlog file.
+
+=item I<dns-retry>
+
+This key requires a positive integer value and defaults to C<900>.
+It defines the number of seconds between attempts to re-lookup host
+information that previously failed to be resolved.
+
+=item I<dns-expire>
+
+This key requires a positive integer value and defaults to C<86400>.
+It defines the number of seconds between refreshes of name to address
+DNS translation. This is so long-running processes do not get stuck with
+stale data, should peer IP addresses change.
+
+=item I<close-period>
+
+This key requires a positive integer value and defaults to C<86400>. It is
+the maximum number of seconds a connection should be kept open. Some NNTP
+servers do not deal well with connections being held open for long periods.
+
+=item I<gen-html>
+
+This key requires a boolean value and defaults to false. It specifies
+whether the I<status-file> should be HTML-ified.
+
+=item I<status-file>
+
+This key requires a pathname value and defaults to F<innfeed.status>.
+It specifies the pathname (relative to I<pathhttp> when I<gen-html> is true;
+otherwise, I<pathlog> as set in F<inn.conf>) where the periodic status of
+the B<innfeed> process should be stored. This corresponds to the B<-S>
+command-line option.
+
+=item I<connection-stats>
+
+This key requires a boolean value and defaults to false. If the value
+is true, then whenever the transmission statistics for a peer are logged,
+each active connection logs its own statistics. This corresponds to the
+B<-z> command-line option.
+
+=item I<host-queue-highwater>
+
+This key requires a positive integer value and defaults to C<10>.
+It defines how many articles will be held internally for a peer before
+new arrivals cause article information to be spooled to the backlog file.
+
+=item I<stats-period>
+
+This key requires a positive integer value and defaults to C<600>.
+It defines how many seconds B<innfeed> waits between generating statistics
+on transfer rates.
+
+=item I<stats-reset>
+
+This key requires a positive integer value and defaults to C<43200>.
+It defines how many seconds B<innfeed> waits before resetting all internal
+transfer counters back to zero (after logging one final time). This is
+so a B<innfeed> process running more than a day will generate "final"
+stats that will be picked up by logfile processing scripts.
+
+=item I<initial-reconnect-time>
+
+This key requires a positive integer value and defaults to C<30>.
+It defines how many seconds to first wait before retrying to reconnect
+after a connection failure. If the next attempt fails too, then the
+reconnect time is approximately doubled until the connection succeeds,
+or I<max-reconnection-time> is reached.
+
+=item I<max-reconnect-time>
+
+This key requires an integer value and defaults to C<3600>. It defines
+the maximum number of seconds to wait between attempt to reconnect
+to a peer. The initial value for reconnection attempts is defined by
+I<initial-reconnect-time>, and it is doubled after each failure, up to
+this value.
+
+=item I<stdio-fdmax>
+
+This key requires a non-negative integer value and defaults to C<0>.
+If the value is greater than zero, then whenever a network socket file
+descriptor is created and it has a value I<less than> this, the file
+descriptor will be dup'ed to bring the value up greater than this. This is
+to leave lower numbered file descriptors free for stdio. Certain systems,
+Sun's in particular, require this. SunOS 4.1.x usually requires a value
+of 128 and Solaris requires a value of 256. The default if this is not
+specified, is C<0>.
+
+=back
+
+=head1 GLOBAL PEER DEFAULTS
+
+All the I<key>:I<value> pairs mentioned in this section must be specified at
+global scope. They may also be specified inside a group or peer definition.
+Note that when peers are added dynamically (i.e. when B<innfeed> receives
+an article for an unspecified peer), it will add the peer site using the
+parameters specified at global scope.
+
+=over 4
+
+=item I<article-timeout>
+
+This key requires a non-negative integer value. A common value is C<600>.
+If no articles need to be sent to the peer for this many seconds, then
+the peer is considered idle and all its active connections are torn down.
+
+=item I<response-timeout>
+
+This key requires a non-negative integer value. A common value is C<300>.
+It defines the maximum amount of time to wait for a response from the peer
+after issuing a command.
+
+=item I<initial-sleep>
+
+This key requires a positive integer. A common value is C<2>. It defines
+the number of seconds to wait when B<innfeed> (or a fork) starts, before
+beginning to open connections to remote hosts.
+
+=item I<initial-connections>
+
+This key requires a non-negative integer value. A common value is C<1>.
+It defines the number of connections to be opened immediately when setting
+up a peer binding. A value of C<0> means no connections will be created
+until an article needs to be sent.
+
+=item I<max-connections>
+
+This key requires a positive integer value. A common value is C<2> but
+may be increased if needed or for large feeds. It defines the maximum
+number of connections to run in parallel to the peer. A value of C<0>
+specifies an unlimited number of maximum connections. In general,
+use of an unlimited number of maximum connections is not recommended.
+Do not ever set I<max-connections> to zero with I<dynamic-method> 0 set,
+as this will saturate peer hosts with connections.
+
+=item I<dynamic-method>
+
+This key requires an integer value between 0 and 3. A common value is C<3>.
+It controls how connections are opened, up to the maximum specified by
+I<max-connections>. In general (and specifically, with I<dynamic-method> 0),
+a new connection is opened when the current number of connections is below
+I<max-connections>, and an article is to be sent while no current connections
+are idle. Without further restraint (i.e. using I<dynamic-method> 0), in
+practice this means that I<max-connections> connections are established
+while articles are being sent. Use of other I<dynamic-method> settings
+imposes a further limit on the amount of connections opened below that
+specified by I<max-connections>. This limit is calculated in different
+ways, depending of the value of I<dynamic-method>.
+
+Users should note that adding additional connections is not always productive
+S<-- just> because opening twice as many connections results in a small
+percentage increase of articles accepted by the remote peer, this may
+be at considerable resource cost both locally and at the remote site,
+whereas the remote site might well have received the extra articles sent
+from another peer a fraction of a second later. Opening large numbers of
+connections is considered antisocial.
+
+The meanings of the various settings are:
+
+=over 2
+
+=item B<0> (no method)
+
+Increase of connections up to I<max-connections> is unrestrained.
+
+=item B<1> (maximize articles per second)
+
+Connections are increased (up to I<max-connections>) and decreased so as
+to maximize the number of articles per second sent, while using the fewest
+connections to do this.
+
+=item B<2> (set target queue length)
+
+Connections are increased (up to I<max-connections>) and decreased
+so as to keep the queue of articles to be sent within the bounds set
+by I<dynamic-backlog-low> and I<dynamic-backlog-high>, while using the
+minimum resources possible. As the queue will tend to fill if the site is
+not keeping up, this method ensures that the maximum number of articles
+are offered to the peer while using the minimum number of connections to
+achieve this.
+
+=item B<3> (combination)
+
+This method uses a combination of methods 1 and 2 above. For sites accepting
+a large percentage of articles, method 2 will be used to ensure these sites
+are offered as complete a feed as possible. For sites accepting a small
+percentage of articles, method 1 is used, to minimize remote resource usage.
+For intermediate sites, an appropriate combination is used.
+
+=back
+
+=item I<dynamic-backlog-low>
+
+This key requires an integer value between 0 and 100. It represents (as
+a percentage) the low water mark for the host queue. If the host queue
+falls below this level while using I<dynamic-method> 2 or 3, and if 2 or
+more connections are open, B<innfeed> will attempt to drop connections to
+the host. An Infinite Impulse Response (IIR) filter is applied to the value
+to prevent connection flap (see I<dynamic-filter>). A value of C<20.0>
+is recommended. This value must be smaller than I<dynamic-backlog-high>.
+
+=item I<dynamic-backlog-high>
+
+This key requires an integer value between 0 and 100. It represents
+(as a percentage) the high water mark for the host queue. If the host
+queue rises above this level while using I<dynamic-method> 2 or 3, and
+if less than I<max-connections> are open to the host, B<innfeed> will
+attempt to open further connections to the host. An Infinite Impulse
+Response (IIR) filter is applied to the value to prevent connection flap
+(see I<dynamic-filter>). A value of C<50.0> is recommended. This value
+must be larger than I<dynamic-backlog-low>.
+
+=item I<dynamic-backlog-filter>
+
+This key requires a floating-point value between 0 and 1. It represents
+the filter coefficient used by the Infinite Impulse Response (IIR) filter
+used to implement I<dynamic-method> 2 and 3. The recommended value of
+this filter is C<0.7>, giving a time constant of 1/(1-0.7) articles.
+Higher values will result in slower response to queue fullness changes;
+lower values in faster response.
+
+=item I<max-queue-size>
+
+This key requires a positive integer value. A common value is C<5>.
+It defines the maximum number of articles to process at one time when using
+streaming to transmit to a peer. Larger numbers mean more memory consumed as
+articles usually get pulled into memory (see the description of I<use-mmap>).
+
+=item I<streaming>
+
+This key requires a boolean value. It is usually set to true. It defines
+whether streaming commands are used to transmit articles to the peers.
+
+=item I<no-check-high>
+
+This key requires a floating-point number which must be in the range
+[0.0, 100.0]. When running transmitting with the streaming commands,
+B<innfeed> attempts an optimization called "no-CHECK mode". This involves
+I<not> asking the peer if it wants the article, but just sending it.
+This optimization occurs when the percentage of the articles the peer has
+accepted gets larger than this number. If this value is set to C<100.0>,
+then this effectively turns off no-CHECK mode, as the percentage can never
+get above 100.0. If this value is too small, then the number of articles
+the peer rejects will get bigger (and your bandwidth will be wasted).
+A value of C<95.0> usually works pretty well.
+
+=item I<no-check-low>
+
+This key requires a floating-point number which must be in the range
+[0.0, 100.0], and it must be smaller that the value for I<no-check-high>.
+When running in no-CHECK mode, as described above, if the percentage
+of articles the remote server accepts drops below this number, then the
+no-CHECK optimization is turned off until the percentage gets above the
+I<no-check-high> value again. If there is small difference between this
+and the I<no-check-high> value (less than about 5.0), then B<innfeed> may
+frequently go in and out of no-CHECK mode. If the difference is too big,
+then it will make it harder to get out of no-CHECK mode when necessary
+(wasting bandwidth). Keeping this to between 5.0 and 10.0 less than
+B<no-check-high> usually works pretty well.
+
+=item I<no-check-filter>
+
+This is a floating-point value representing the time constant, in articles,
+over which the CHECK/no-CHECK calculations are done. The recommended
+value is C<50.0>, which will implement an Infinite Impulse Response (IIR)
+filter of time constant 50. This roughly equates to making a decision
+about the mode over the previous 50 articles. A higher number will result
+in a slower response to changing percentages of articles accepted; a lower
+number will result in a faster response.
+
+=item I<bindaddress>
+
+This key requires a string value. It specifies which outgoing IPv4 address
+B<innfeed> should bind the local end of its connection to. It must be an
+IPv4 address in dotted-quad format (nnn.nnn.nnn.nnn), C<any>, or C<none>.
+If not set or set to C<any>, B<innfeed> defaults to letting the kernel
+choose this address. If set to C<none>, B<innfeed> will not use IPv4 for
+outgoing connections to peers in this scope (i.e. it forces IPv6).
+
+If not set in F<innfeed.conf>, B<innfeed> defaults to the value of
+I<sourceaddress> from F<inn.conf> (which by default is unset).
+
+=item I<bindaddress6>
+
+This key requires a string value. It behaves like I<bindaddress> except
+for outgoing IPv6 connections. It must be in numeric IPv6 format (note
+that a value containing colons must be enclosed in double quotes), C<any>,
+or C<none>. If set to C<none>, B<innfeed> will not use IPv6 for outgoing
+connections to peers in this scope.
+
+If not set in F<innfeed.conf>, B<innfeed> defaults to the value of
+I<sourceaddress6> from F<inn.conf> (which by default is unset).
+
+=item I<port-number>
+
+This key requires a positive integer value. It defines the TCP/IP port
+number to use when connecting to the remote. Usually, port number 119
+is used.
+
+=item I<force-ipv4>
+
+This key requires a boolean value. By default, it is set to false.
+Setting it to true is the same as setting I<bindaddress6> to C<none>
+and removing I<bindaddress> from C<none> if it was set.
+
+=item I<drop-deferred>
+
+This key requires a boolean value. By default, it is set to false.
+When set to true, and a peer replies with code 431 or 436 (try again later),
+B<innfeed> just drops the article and does not try to re-send it. This is
+useful for some peers that keep on deferring articles for a long time to
+prevent B<innfeed> from trying to offer the same article over and over again.
+
+=item I<min-queue-connection>
+
+This key requires a boolean value. By default, it is set to false.
+When set to true, B<innfeed> will attempt to use a connection with the
+least queue size (or the first empty connection). If this key is set to
+true, it is recommended that I<dynamic-method> be set to 0. This allows
+for article propagation with the least delay.
+
+=item I<no-backlog>
+
+This key requires a boolean value. It specifies whether spooling should
+be enabled (false, the default) or disabled (true). Note that when
+I<no-backlog> is set, articles reported as spooled are actually silently
+discarded.
+
+=item I<backlog-limit>
+
+This key requires a non-negative integer value. If the number is C<0>,
+then backlog files are allowed to grow without bound when the peer is unable
+to keep up with the article flow. If this number is greater than 0, then
+it specifies the size (in bytes) the backlog file should get truncated
+to when the backlog file reaches a certain limit. The limit depends on
+whether I<backlog-factor> or I<backlog-limit-highwater> is used.
+
+This parameter also applies to the debug file when I<debug-shrinking>
+is set to true, and has the same effect on this file as the one has on
+backlog files.
+
+=item I<backlog-factor>
+
+This key requires a floating-point value, which must be larger than
+1.0. It is used in conjunction with the peer key I<backlog-limit>.
+If I<backlog-limit> has a value greater than zero, then when the backlog
+file gets larger than the value I<backlog-limit> * I<backlog-factor>,
+then the backlog file will be truncated to the size I<backlog-limit>.
+
+For example, if I<backlog-limit> has a value of C<1000000>, and
+I<backlog-factor> has a value of C<2.0>, then when the backlog file gets to
+be larger than 2000000 bytes in size, it will be truncated to 1000000 bytes.
+The front portion of the file is removed, and the trimming happens on
+line boundaries, so the final size may be a bit less than this number.
+If I<backlog-limit-highwater> is defined too, then I<backlog-factor>
+takes precedence.
+
+This parameter also applies to the debug file when I<debug-shrinking>
+is set to true, and has the same effect on this file as the one has on
+backlog files.
+
+=item I<backlog-limit-highwater>
+
+This key requires a positive integer value that must be larger than the value
+for I<backlog-limit>. If the size of the backlog file gets larger than this
+value (in bytes), then the backlog file will be shrunk down to the size of
+I<backlog-limit>. If both I<backlog-factor> and I<backlog-limit-highwater>
+are defined, then the value of I<backlog-factor> is used.
+
+This parameter also applies to the debug file when I<debug-shrinking>
+is set to true, and has the same effect on this file as the one has on
+backlog files.
+
+=item I<backlog-feed-first>
+
+This key requires a boolean value. By default it is set to false. When set
+to true, the backlog is fed before new files. This is intended to enforce
+in-order delivery, so setting this to true when I<initial-connections>
+or I<max-connections> is more than 1 is inconsistent.
+
+=item I<username>
+
+This key requires a string value. If the value is defined, then B<innfeed>
+tries to authenticate by AUTHINFO USER and this value used for user name.
+I<password> must also be defined, if this key is defined.
+
+=item I<password>
+
+This key requires a string value. The value is the password used for
+AUTHINFO PASS. I<username> must also be defined, if this key is defined.
+
+=item Special keys for B<imapfeed>
+
+The following keys are used with B<imapfeed> to authenticate to a remote
+host. They are optional.
+
+Several parameters must be included:
+
+=over 2
+
+=item I<deliver-authname>
+
+The authname is who you want to authenticate as.
+
+=item I<deliver-password>
+
+This is the appropriate password for authname.
+
+=item I<deliver-username>
+
+The username is who you want to "act" as, that is, who is actually going
+to be using the server.
+
+=item I<deliver-realm>
+
+In this case, the "realm" is the realm in which the specified authname
+is valid. Currently this is only needed by the DIGEST-MD5 SASL mechanism.
+
+=item I<deliver-rcpt-to>
+
+A printf(3)-style format string for creating the envelope recipient address.
+The pattern MUST include a single string specifier which will be replaced
+with the newgroup (e.g. C<bb+%s>). The default is C<+%s>.
+
+=item I<deliver-to-header>
+
+An optional printf(3)-style format string for creating a To: header field
+to be prepended to the article. The pattern MUST include a single string
+specifier which will be replaced with the newgroup (e.g. C<post+%s at domain>).
+If not specified, the To: header field will not be prepended.
+
+=back
+
+=back
+
+=head1 PEER VALUES
+
+As previously explained, the peer definitions can contain redefinitions
+of any of the I<key>:I<value> pairs described in the section about global
+peer defaults above. There is one I<key>:I<value> pair that is specific
+to a peer definition.
+
+=over 4
+
+=item I<ip-name>
+
+This key requires a word value. The word is the host's FQDN, or the dotted
+quad IP-address. If this value is not specified, then the name of the
+peer in the enclosing I<peer> block is taken to also be its I<ip-name>.
+
+=back
+
+=head1 RELOADING
+
+If B<innfeed> gets a SIGHUP signal, then it will reread the configuration
+file. All values at global scope except for I<backlog-directory> can be
+changed (although note that I<bindaddress> and I<bindaddress6> changes
+will only affect new connections).
+
+Any new peers are added and any missing peers have their connections closed.
+
+=head1 EXAMPLE
+
+For a comprehensive example, see the sample F<innfeed.conf> distributed
+with INN and installed as a starting point.
+
+Here are examples of how to format values:
+
+ eg-string: "New\tconfig\tfile\n"
+ eg-long-string: "A long string that goes
+ over multiple lines. The
+ newline is kept in the
+ string except when quoted
+ with a backslash \
+ as here."
+ eg-simple-string: A-no-quote-string
+ eg-integer: 10
+ eg-boolean: true
+ eg-char: 'a'
+ eg-ctrl-g: '\007'
+
+=head1 HISTORY
+
+Written by James Brister <brister at vix.com> for InterNetNews. Converted to
+POD by Julien Elie.
+
+Earlier versions of B<innfeed> (up to 0.10.1) were shipped separately;
+B<innfeed> is now part of INN and shares the same version number.
+Please note that the F<innfeed.conf> format has changed dramatically since
+S<version 0.9.3>.
+
+$Id$
+
+=head1 SEE ALSO
+
+inn.conf(5), innfeed(8), newsfeeds(5).
+
+=cut
Property changes on: trunk/doc/pod/innfeed.conf.pod
___________________________________________________________________
Added: svn:keywords
+ Author Date Id Revision
Added: svn:eol-style
+ native
Modified: doc/pod/innfeed.pod
===================================================================
--- doc/pod/innfeed.pod 2011-02-12 13:09:20 UTC (rev 9188)
+++ doc/pod/innfeed.pod 2011-03-02 18:46:54 UTC (rev 9189)
@@ -457,8 +457,12 @@
=head1 HISTORY
-Written by James Brister <brister at vix.com> for InterNetNews. Converted to POD by Julien Elie.
+Written by James Brister <brister at vix.com> for InterNetNews. Converted to
+POD by Julien Elie.
+Earlier versions of B<innfeed> (up to 0.10.1) were shipped separately;
+B<innfeed> is now part of INN and shares the same version number.
+
$Id$
=head1 SEE ALSO
Modified: innfeed/host.c
===================================================================
--- innfeed/host.c 2011-02-12 13:09:20 UTC (rev 9188)
+++ innfeed/host.c 2011-03-02 18:46:54 UTC (rev 9189)
@@ -1351,7 +1351,7 @@
host->params->lowPassFilter) ;
fprintf (fp,"%s backlog-limit : %d\n",indent,
host->params->backlogLimit) ;
- fprintf (fp,"%s backlog-limit-high : %d\n",indent,
+ fprintf (fp,"%s backlog-limit-highwater : %d\n",indent,
host->params->backlogLimitHigh) ;
fprintf (fp,"%s backlog-factor : %2.1f\n",indent,
host->params->backlogFactor) ;
@@ -2786,11 +2786,11 @@
}
if (findValue (s,"backlog-factor",inherit) == NULL &&
- findValue (s,"backlog-limit-high",inherit) == NULL)
+ findValue (s,"backlog-limit-highwater",inherit) == NULL)
{
logOrPrint (LOG_ERR,fp,
"ME config: must define at least one of backlog-factor"
- " and backlog-limit-high. Adding %s: %f", "backlog-factor",
+ " and backlog-limit-highwater. Adding %s: %f", "backlog-factor",
LIMIT_FUDGE) ;
addReal (s,"backlog-factor",LIMIT_FUDGE) ;
rv = 0 ;
@@ -2814,7 +2814,7 @@
" to peer, but backlog-feed-first is set");
}
- GETINT(s,fp,"backlog-limit-high",0,LONG_MAX,NOTREQNOADD,p->backlogLimitHigh, inherit);
+ GETINT(s,fp,"backlog-limit-highwater",0,LONG_MAX,NOTREQNOADD,p->backlogLimitHigh, inherit);
GETREAL(s,fp,"backlog-factor",1.0,DBL_MAX,NOTREQNOADD,p->backlogFactor, inherit);
GETINT(s,fp,"dynamic-method",0,3,REQ,p->dynamicMethod, inherit);
Modified: innfeed/innfeed.h
===================================================================
--- innfeed/innfeed.h 2011-02-12 13:09:20 UTC (rev 9188)
+++ innfeed/innfeed.h 2011-03-02 18:46:54 UTC (rev 9189)
@@ -49,7 +49,7 @@
#define GEN_HTML false /* gen-html */
#define INNFEED_STATUS "innfeed.status" /* status-file */
#define LOG_CONNECTION_STATS 0 /* connection-stats */
-#define HOST_HIGHWATER 10 /* host-highwater */
+#define HOST_HIGHWATER 10 /* host-queue-highwater */
#define STATS_PERIOD (60 * 10) /* stats-period */
#define STATS_RESET_PERIOD (60 * 60 * 12) /* stats-reset-period */
Modified: innfeed/tape.c
===================================================================
--- innfeed/tape.c 2011-02-12 13:09:20 UTC (rev 9188)
+++ innfeed/tape.c 2011-03-02 18:46:54 UTC (rev 9189)
@@ -415,7 +415,7 @@
if (!getInteger (s,"backlog-limit-highwater",
&nt->outputHighLimit,INHERIT))
{
- warn ("%s no backlog-factor or backlog-high-limit",
+ warn ("%s no backlog-factor or backlog-limit-highwater",
nt->peerName) ;
nt->outputLowLimit = 0 ;
nt->outputHighLimit = 0 ;
Modified: samples/innfeed.conf
===================================================================
--- samples/innfeed.conf 2011-02-12 13:09:20 UTC (rev 9188)
+++ samples/innfeed.conf 2011-03-02 18:46:54 UTC (rev 9189)
@@ -1,136 +1,123 @@
-# $Revision$
-#
-# Sample innfeed config file. See the comment block at the
-# end for a fuller description of the format, and innfeed.conf(5) for a
-# description of the entries.
-#
-
+## $Id$
##
-## Global values. Not specific to any peer. These are optional, but if
-## used will override the compiled in values.
+## innfeed.conf - Configuration file for innfeed
##
+## Format:
+## key: value
+##
+## Values may be an integer, a floating-point number,
+## C-style single-quoted characters, boolean, and strings.
+##
+## If a string value contains whitespace, or embedded quotes,
+## or the comment character ("#"), then the whole string must
+## be quoted with double quotes. Inside the quotes, standard
+## C-escape sequences can be used (\t, \n, \r, \f, \v, \", \').
+##
+## Blank lines are ignored. Everything after a '#' is ignored too.
+##
+##
+## See the innfeed.conf(5) man page for a full description
+## of the format.
-pid-file: innfeed.pid # relative to pathrun
-debug-level: 0
-use-mmap: false
-log-file: innfeed.log # relative to pathlog
-stdio-fdmax: 0
+## Global values. Not specific to any peer. These are optional,
+## but if used, will override the compiled-in values.
+
+#news-spool: # Default is <patharticles>.
+#input-file: # Default is unset (channel mode).
+pid-file: innfeed.pid # Relative to <pathrun>.
+debug-level: 0
+debug-shrinking: false
+fast-exit: false
+use-mmap: false
+log-file: innfeed.log # Relative to <pathlog>.
+stdio-fdmax: 0
log-time-format: "%a %b %d %H:%M:%S %Y"
-## Uncomment the next line to include the contents
-## of ``testfile'' at this point.
+## Uncomment the next line to include the contents
+## of "testfile" at this point.
#$INCLUDE testfile
-backlog-directory: innfeed # relative to pathspool
-backlog-rotate-period: 60
-backlog-ckpt-period: 30
-backlog-newfile-period: 600
+backlog-directory: innfeed # Relative to <pathspool>.
+backlog-highwater: 5
+backlog-rotate-period: 60
+backlog-ckpt-period: 30
+backlog-newfile-period: 600
-dns-retry: 900
-dns-expire: 86400
-close-period: 86400
-gen-html: false # if true, status-file is relative to pathhttp
-status-file: innfeed.status # otherwise, it is relative to pathlog
-connection-stats: false
-host-queue-highwater: 200
-stats-period: 600
-stats-reset: 43200
+dns-retry: 900
+dns-expire: 86400
+close-period: 86400
+gen-html: false # If true, status-file is relative to <pathhttp>;
+status-file: innfeed.status # otherwise, it is relative to <pathlog>.
+connection-stats: false
+host-queue-highwater: 10
+stats-period: 600
+stats-reset: 43200
-max-reconnect-time: 3600
-initial-reconnect-time: 30
+initial-reconnect-time: 30
+max-reconnect-time: 3600
-##
-## Defaults for all peers. These must all exist at
-## global scope. Any of them can be redefined
-## inside a peer or group definition.
-##
+## Defaults for all peers. These must all exist at
+## global scope. Any of them can be redefined
+## inside a peer or group definition.
-article-timeout: 600
-response-timeout: 300
-initial-connections: 1
-max-connections: 5
-max-queue-size: 5
-streaming: true
-no-check-high: 95.0
-no-check-low: 90.0
-no-check-filter: 50.0
-port-number: 119
-force-ipv4: false
-drop-deferred: false
-min-queue-connection: false
-backlog-limit: 0
-backlog-factor: 1.10
-backlog-limit-highwater: 0
-dynamic-method: 3
-dynamic-backlog-filter: 0.7
-dynamic-backlog-low: 25.0
-dynamic-backlog-high: 50.0
-no-backlog: false
+article-timeout: 600
+response-timeout: 300
+initial-sleep: 2
+initial-connections: 1
+max-connections: 2
+max-queue-size: 5
+streaming: true
+no-check-high: 95.0
+no-check-low: 90.0
+no-check-filter: 50.0
+#bindaddress:
+#bindaddress6:
+port-number: 119
+force-ipv4: false
+drop-deferred: false
+min-queue-connection: false
+backlog-limit: 0
+backlog-factor: 1.10
+backlog-limit-highwater: 0
+dynamic-method: 3
+dynamic-backlog-filter: 0.7
+dynamic-backlog-low: 20.0
+dynamic-backlog-high: 50.0
+no-backlog: false
backlog-feed-first: false
-##
-## Peers.
-##
+## Peers.
+
#peer decwrl {
-# ip-name: news1.pa.dec.com
+# ip-name: news1.pa.dec.com
#}
#peer uunet {
-# ip-name: news.uunet.uu.net
-# max-connections: 10
+# ip-name: news.uunet.uu.net
+# max-connections: 10
#}
-##
-## Group peers together to give second level defaults.
-##
+## Group peers together to give second level defaults.
+
#group fast-sites {
-# max-connections: 7
+# max-connections: 7
#
-# peer data.ramona.vix.com {
-# # ip-name defaults to data.ramona.vix.com
-# streaming: false
-# }
+# peer data.ramona.vix.com {
+# # ip-name defaults to "data.ramona.vix.com".
+# streaming: false
+# }
#
-# peer bb.home.vix.com {
-# ip-name: 192.5.5.33
-# }
-#}
-
-
-
-# Blank lines are ignored. Exerything after a '#'
-# is ignored too.
+# peer bb.home.vix.com {
+# ip-name: 192.5.5.33
+# username: john
+# password: myPassword
+# }
#
-# Format is:
-# key : value
-#
-# See innfeed.conf(5) for a description of
-# necessary & useful keys. Unknown keys and their
-# values are ignored.
-#
-# Values may be a integer, floating-point, c-style
-# single-quoted characters, boolean, and strings.
-#
-# If a string value contains whitespace, or
-# embedded quotes, or the comment character
-# (``#''), then the whole string must be quoted
-# with double quotes. Inside the quotes, you may
-# use the standard c-escape sequence
-# (\t,\n,\r,\f,\v,\",\').
-#
-# Examples:
-# eg-string: "New\tConfig\tfile\n"
-# eg-long-string: "A long string that goes
-# over multiple lines. The
-# newline is kept in the
-# string except when quoted
-# with a backslash \
-# as here."
-# eg-simple-string: A-no-quote-string
-# eg-integer: 10
-# eg-boolean: true
-# eg-char: 'a'
-# eg-ctrl-g: '\007'
+# # If imapfeed is used, the following parameters can
+# # be set for a peer:
+# # deliver-authname, deliver-password, deliver-username,
+# # deliver-realm, deliver-rcpt-to, deliver-to-header.
+#}
More information about the inn-committers
mailing list