INN commit: branches/2.5 (12 files)

INN Commit rra at isc.org
Sun Jul 17 19:09:27 UTC 2011


    Date: Sunday, July 17, 2011 @ 12:09:26
  Author: iulius
Revision: 9275

check all the default values in innfeed.conf (documentation, sample, source code)

* Change the default dynamic method from 0 to 3.  It already was the
recommended value in the documentation, and the default value in the sample
innfeed.conf file.

* Change the default use-mmap setting from true to false.  It already was
the recommended value in the documentation.  The value of this parameter
is only used when innfeed is given file names to send instead of storage
API tokens, which is a fairly rare use case.

* No longer generate an error message (logged in news.err) when a parameter
is not defined in innfeed.conf.  It has a default value, so no need to
warn the user.  (Also updated in innreport.)

* Add a "noreturn" hint to the compiler for the dump_core() function.

* Homogeneize tape.c with host.c for the default values of backlog-limit,
backlog-factor and backlog-limit-highwater.

* Log a few more parameters in innfeed.status:  "Stats period", "Stats
reset", "Debug shrinking", "Fast exit", "stdio-fdmax", "reconnection time",
"max reconnection time", "DNS retry period", "DNS expire period", "port num",
"force IPv4", "backlog highwater", "highwater queue".

* Documentation fixes:
- "pid-file" is relative to "pathrun", and not "backlog-directory";
- "log-file" is relative to "pathlog", and not "backlog-directory";
- "initial-sleep" is a global value, and not per-peer;
- "close-period" is a per-peer value, and not a global value;
- "max-reconnect-time" is mispelled once in the documentation;
- specify that *no* keys are mandatory.  They are all optional, and have
a default value (all of them are now mentioned in the documentation).

* Documentation additions:
- If a file named "innfeed.debug" exists in the "pathlog" directory,
then "debug-level" is automatically set to 1.  This is a cheap way of
avoiding continual reloading of the "newsfeeds" file when debugging.
Note that debug messages still go to "log-file";
- "backlog-directory" is relative to "pathspool";
- "status-file" can be set to an absolute path.

Modified:
  branches/2.5/doc/pod/innfeed.conf.pod
  branches/2.5/innfeed/connection.c
  branches/2.5/innfeed/endpoint.c
  branches/2.5/innfeed/host.c
  branches/2.5/innfeed/imap_connection.c
  branches/2.5/innfeed/innlistener.c
  branches/2.5/innfeed/main.c
  branches/2.5/innfeed/misc.c
  branches/2.5/innfeed/misc.h
  branches/2.5/innfeed/tape.c
  branches/2.5/samples/innfeed.conf
  branches/2.5/scripts/innreport_inn.pm

---------------------------+
 doc/pod/innfeed.conf.pod  |  140 +++++++++++++++++++++++---------------------
 innfeed/connection.c      |    4 -
 innfeed/endpoint.c        |    2 
 innfeed/host.c            |   67 ++++++++++++---------
 innfeed/imap_connection.c |    4 -
 innfeed/innlistener.c     |    2 
 innfeed/main.c            |   20 +++---
 innfeed/misc.c            |    5 -
 innfeed/misc.h            |    5 -
 innfeed/tape.c            |   15 ++--
 samples/innfeed.conf      |   98 +++++++++++++++---------------
 scripts/innreport_inn.pm  |    2 
 12 files changed, 197 insertions(+), 167 deletions(-)

Modified: doc/pod/innfeed.conf.pod
===================================================================
--- doc/pod/innfeed.conf.pod	2011-07-17 19:08:55 UTC (rev 9274)
+++ doc/pod/innfeed.conf.pod	2011-07-17 19:09:26 UTC (rev 9275)
@@ -115,7 +115,7 @@
 =head1 GLOBAL VALUES
 
 The following listing show all the keys that apply to the process as
-whole. These are not required (compiled-in defaults are used where needed).
+whole.  These are not required (compiled-in defaults are used where needed).
 
 =over 4
 
@@ -137,7 +137,7 @@
 =item I<pid-file>
 
 This key requires a pathname value and defaults to F<innfeed.pid>.
-It specifies the pathname (relative to the I<backlog-directory> value)
+It specifies the pathname (relative to I<pathrun> in F<inn.conf>)
 where the pid of the B<innfeed> process should be stored.  This corresponds
 to the B<-p> command-line option.
 
@@ -148,12 +148,23 @@
 config-defined I<log-file>.  This corresponds to the B<-d> command-line
 option.
 
+If a file named F<innfeed.debug> exists in the I<pathlog> directory (as
+set in F<inn.conf>), then I<debug-level> is automatically set to C<1>.
+This is a cheap way of avoiding continual reloading of the F<newsfeeds>
+file when debugging.  Note that debug messages still go to I<log-file>.
+
 =item I<debug-shrinking>
 
 This key requires a boolean value and defaults to false (the debug file is
 allowed to grow without bound).  If set to true, this file is truncated when
 its size reaches a certain limit.  See I<backlog-limit> for more details.
 
+=item I<initial-sleep>
+
+This key requires a positive integer.  The default value is C<2>.  It defines
+the number of seconds to wait when B<innfeed> (or a fork) starts, before
+beginning to open connections to remote hosts.
+
 =item I<fast-exit>
 
 This key requires a boolean value and defaults to false.  If set to true,
@@ -177,7 +188,7 @@
 should go (such as those generated when a positive value for I<debug-value>
 is used).  This corresponds to the B<-l> command-line option.
 
-A relative pathname is relative to the I<backlog-directory> value.
+This pathname is relative to I<pathlog> in F<inn.conf>.
 
 =item I<log-time-format>
 
@@ -191,6 +202,8 @@
 where the current B<innfeed> process should store backlog files.
 This corresponds to the B<-b> command-line option.
 
+This pathname is relative to I<pathspool> in F<inn.conf>.
+
 =item I<backlog-highwater>
 
 This key requires a positive integer value and defaults to C<5>.
@@ -231,12 +244,6 @@
 DNS translation.  This is so long-running processes do not get stuck with
 stale data, should peer IP addresses change.
 
-=item I<close-period>
-
-This key requires a positive integer value and defaults to C<86400>.  It is
-the maximum number of seconds a connection should be kept open.  Some NNTP
-servers do not deal well with connections being held open for long periods.
-
 =item I<gen-html>
 
 This key requires a boolean value and defaults to false.  It specifies
@@ -245,10 +252,10 @@
 =item I<status-file>
 
 This key requires a pathname value and defaults to F<innfeed.status>.
-It specifies the pathname (relative to I<pathhttp> when I<gen-html> is true;
-otherwise, I<pathlog> as set in F<inn.conf>) where the periodic status of
-the B<innfeed> process should be stored.  This corresponds to the B<-S>
-command-line option.
+An absolute pathname can be used.  It specifies the pathname (relative
+to I<pathhttp> when I<gen-html> is true; otherwise, I<pathlog> as set in
+F<inn.conf>) where the periodic status of the B<innfeed> process should
+be stored.  This corresponds to the B<-S> command-line option.
 
 =item I<connection-stats>
 
@@ -283,7 +290,7 @@
 It defines how many seconds to first wait before retrying to reconnect
 after a connection failure.  If the next attempt fails too, then the
 reconnect time is approximately doubled until the connection succeeds,
-or I<max-reconnection-time> is reached.
+or I<max-reconnect-time> is reached.
 
 =item I<max-reconnect-time>
 
@@ -308,7 +315,7 @@
 
 =head1 GLOBAL PEER DEFAULTS
 
-All the I<key>:I<value> pairs mentioned in this section must be specified at
+All the I<key>:I<value> pairs mentioned in this section can be specified at
 global scope.  They may also be specified inside a group or peer definition.
 Note that when peers are added dynamically (i.e. when B<innfeed> receives
 an article for an unspecified peer), it will add the peer site using the
@@ -316,36 +323,37 @@
 
 =head2 Required keys
 
+No keys are currently required.  They all have a default value, if not
+present in the configuration file.
+
+=head2 Optional keys
+
+The following keys are optional:
+
 =over 4
 
 =item I<article-timeout>
 
-This key requires a non-negative integer value.  A common value is C<600>.
+This key requires a non-negative integer value.  The default value is C<600>.
 If no articles need to be sent to the peer for this many seconds, then
 the peer is considered idle and all its active connections are torn down.
 
 =item I<response-timeout>
 
-This key requires a non-negative integer value.  A common value is C<300>.
+This key requires a non-negative integer value.  The default value is C<300>.
 It defines the maximum amount of time to wait for a response from the peer
 after issuing a command.
 
-=item I<initial-sleep>
-
-This key requires a positive integer.  A common value is C<2>.  It defines
-the number of seconds to wait when B<innfeed> (or a fork) starts, before
-beginning to open connections to remote hosts.
-
 =item I<initial-connections>
 
-This key requires a non-negative integer value.  A common value is C<1>.
+This key requires a non-negative integer value.  The default value is C<1>.
 It defines the number of connections to be opened immediately when setting
 up a peer binding.  A value of C<0> means no connections will be created
 until an article needs to be sent.
 
 =item I<max-connections>
 
-This key requires a positive integer value.  A common value is C<2> but
+This key requires a positive integer value.  The default value is C<2> but
 may be increased if needed or for large feeds.  It defines the maximum
 number of connections to run in parallel to the peer.  A value of C<0>
 specifies an unlimited number of maximum connections.  In general,
@@ -353,9 +361,15 @@
 Do not ever set I<max-connections> to zero with I<dynamic-method> 0 set,
 as this will saturate peer hosts with connections.
 
+=item I<close-period>
+
+This key requires a positive integer value and defaults to C<86400>.  It is
+the maximum number of seconds a connection should be kept open.  Some NNTP
+servers do not deal well with connections being held open for long periods.
+
 =item I<dynamic-method>
 
-This key requires an integer value between 0 and 3.  A common value is C<3>.
+This key requires an integer value between 0 and 3.  The default value is C<3>.
 It controls how connections are opened, up to the maximum specified by
 I<max-connections>.  In general (and specifically, with I<dynamic-method> 0),
 a new connection is opened when the current number of connections is below
@@ -411,13 +425,13 @@
 
 =item I<dynamic-backlog-low>
 
-This key requires a floating-point value between 0 and 100.  It represents (as
-a percentage) the low water mark for the host queue.  If the host queue
+This key requires a floating-point value between 0 and 100.  It represents
+(as a percentage) the low water mark for the host queue.  If the host queue
 falls below this level while using I<dynamic-method> 2 or 3, and if 2 or
 more connections are open, B<innfeed> will attempt to drop connections to
-the host.  An Infinite Impulse Response (IIR) filter is applied to the value
-to prevent connection flap (see I<dynamic-filter>).  A value of C<20.0>
-is recommended.  This value must be smaller than I<dynamic-backlog-high>.
+the host.  An Infinite Impulse Response (IIR) filter is applied to the
+value to prevent connection flap (see I<dynamic-filter>).  The default
+value is C<20.0>.  This value must be smaller than I<dynamic-backlog-high>.
 
 =item I<dynamic-backlog-high>
 
@@ -427,28 +441,28 @@
 if less than I<max-connections> are open to the host, B<innfeed> will
 attempt to open further connections to the host.  An Infinite Impulse
 Response (IIR) filter is applied to the value to prevent connection flap
-(see I<dynamic-filter>).  A value of C<50.0> is recommended.  This value
-must be larger than I<dynamic-backlog-low>.
+(see I<dynamic-filter>).  The default value is C<50.0>.  This value must
+be larger than I<dynamic-backlog-low>.
 
 =item I<dynamic-backlog-filter>
 
 This key requires a floating-point value between 0 and 1.  It represents
-the filter coefficient used by the Infinite Impulse Response (IIR) filter
-used to implement I<dynamic-method> 2 and 3.  The recommended value of
-this filter is C<0.7>, giving a time constant of 1/(1-0.7) articles.
+the filter coefficient used by the Infinite Impulse Response (IIR)
+filter used to implement I<dynamic-method> 2 and 3.  The default value
+of this filter is C<0.7>, giving a time constant of 1/(1-0.7) articles.
 Higher values will result in slower response to queue fullness changes;
 lower values in faster response.
 
 =item I<max-queue-size>
 
-This key requires a positive integer value.  A common value is C<5>.
+This key requires a positive integer value.  The default value is C<5>.
 It defines the maximum number of articles to process at one time when using
 streaming to transmit to a peer.  Larger numbers mean more memory consumed as
 articles usually get pulled into memory (see the description of I<use-mmap>).
 
 =item I<streaming>
 
-This key requires a boolean value.  It is usually set to true.  It defines
+This key requires a boolean value.  Its default value is true.  It defines
 whether streaming commands are used to transmit articles to the peers.
 
 =item I<no-check-high>
@@ -462,7 +476,7 @@
 then this effectively turns off no-CHECK mode, as the percentage can never
 get above 100.0.  If this value is too small, then the number of articles
 the peer rejects will get bigger (and your bandwidth will be wasted).
-A value of C<95.0> usually works pretty well.
+The default value of C<95.0> usually works pretty well.
 
 =item I<no-check-low>
 
@@ -476,23 +490,23 @@
 frequently go in and out of no-CHECK mode.  If the difference is too big,
 then it will make it harder to get out of no-CHECK mode when necessary
 (wasting bandwidth).  Keeping this to between 5.0 and 10.0 less than
-B<no-check-high> usually works pretty well.
+B<no-check-high> usually works pretty well.  The default value is C<90.0>.
 
 =item I<no-check-filter>
 
 This is a floating-point value representing the time constant, in articles,
-over which the CHECK/no-CHECK calculations are done.  The recommended
-value is C<50.0>, which will implement an Infinite Impulse Response (IIR)
-filter of time constant 50.  This roughly equates to making a decision
-about the mode over the previous 50 articles.  A higher number will result
-in a slower response to changing percentages of articles accepted; a lower
+over which the CHECK/no-CHECK calculations are done.  The default value is
+C<50.0>, which will implement an Infinite Impulse Response (IIR) filter
+of time constant 50.  This roughly equates to making a decision about
+the mode over the previous 50 articles.  A higher number will result in
+a slower response to changing percentages of articles accepted; a lower
 number will result in a faster response.
 
 =item I<port-number>
 
 This key requires a positive integer value.  It defines the TCP/IP port
 number to use when connecting to the remote.  Usually, port number 119
-is used.
+is used, which is the default value.
 
 =item I<force-ipv4>
 
@@ -513,7 +527,7 @@
 This key requires a boolean value.  By default, it is set to false.
 When set to true, B<innfeed> will attempt to use a connection with the
 least queue size (or the first empty connection).  If this key is set to
-true, it is recommended that I<dynamic-method> be set to 0.  This allows
+true, it is recommended that I<dynamic-method> be set to C<0>.  This allows
 for article propagation with the least delay.
 
 =item I<no-backlog>
@@ -525,12 +539,12 @@
 
 =item I<backlog-limit>
 
-This key requires a non-negative integer value.  If the number is C<0>,
-then backlog files are allowed to grow without bound when the peer is unable
-to keep up with the article flow.  If this number is greater than 0, then
-it specifies the size (in bytes) the backlog file should get truncated
-to when the backlog file reaches a certain limit.  The limit depends on
-whether I<backlog-factor> or I<backlog-limit-highwater> is used.
+This key requires a non-negative integer value.  If the number is C<0>
+(the default), then backlog files are allowed to grow without bound when the
+peer is unable to keep up with the article flow.  If this number is greater
+than 0, then it specifies the size (in bytes) the backlog file should get
+truncated to when the backlog file reaches a certain limit.  The limit
+depends on whether I<backlog-factor> or I<backlog-limit-highwater> is used.
 
 This parameter also applies to the debug file when I<debug-shrinking>
 is set to true, and has the same effect on this file as the one has on
@@ -550,7 +564,7 @@
 The front portion of the file is removed, and the trimming happens on
 line boundaries, so the final size may be a bit less than this number.
 If I<backlog-limit-highwater> is defined too, then I<backlog-factor>
-takes precedence.
+takes precedence.  The default value of I<backlog-factor> is C<1.1>.
 
 This parameter also applies to the debug file when I<debug-shrinking>
 is set to true, and has the same effect on this file as the one has on
@@ -559,11 +573,13 @@
 =item I<backlog-limit-highwater>
 
 This key requires a positive integer value that must be larger than the value
-for I<backlog-limit>.  If the size of the backlog file gets larger than this
-value (in bytes), then the backlog file will be shrunk down to the size of
-I<backlog-limit>.  If both I<backlog-factor> and I<backlog-limit-highwater>
-are defined, then the value of I<backlog-factor> is used.
+for I<backlog-limit>.  The default value is C<0>.
 
+If the size of the backlog file gets larger than this value (in bytes),
+then the backlog file will be shrunk down to the size of I<backlog-limit>.
+If both I<backlog-factor> and I<backlog-limit-highwater> are defined,
+then the value of I<backlog-factor> is used.
+
 This parameter also applies to the debug file when I<debug-shrinking>
 is set to true, and has the same effect on this file as the one has on
 backlog files.
@@ -575,14 +591,6 @@
 in-order delivery, so setting this to true when I<initial-connections>
 or I<max-connections> is more than 1 is inconsistent.
 
-=back
-
-=head2 Optional keys
-
-The following keys are optional:
-
-=over 4
-
 =item I<bindaddress>
 
 This key requires a string value.  It specifies which outgoing IPv4 address

Modified: innfeed/connection.c
===================================================================
--- innfeed/connection.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/connection.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -216,8 +216,8 @@
 
 static Connection gCxnList = NULL ;
 static unsigned int gCxnCount = 0 ;
-static unsigned int max_reconnect_period = MAX_RECON_PER ;
-static unsigned int init_reconnect_period = INIT_RECON_PER ;
+unsigned int max_reconnect_period = MAX_RECON_PER ;
+unsigned int init_reconnect_period = INIT_RECON_PER ;
 #if 0
 static bool inited = false ;
 #endif

Modified: innfeed/endpoint.c
===================================================================
--- innfeed/endpoint.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/endpoint.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -106,7 +106,7 @@
 
 static EndPoint mainEndPoint ;
 static bool mainEpIsReg = false ;
-static unsigned int stdioFdMax = MAX_STDIO_FD ;
+unsigned int stdioFdMax = MAX_STDIO_FD ;
 
 time_t  PrivateTime;
 

Modified: innfeed/host.c
===================================================================
--- innfeed/host.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/host.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -70,6 +70,9 @@
 extern int h_errno;
 #endif
 
+extern unsigned int init_reconnect_period;
+extern unsigned int max_reconnect_period;
+
 /* the host keeps a couple lists of these */
 typedef struct proc_q_elem 
 {
@@ -347,7 +350,7 @@
 static unsigned int maxIpNameLen = 0 ;
 static unsigned int maxPeerNameLen = 0 ;
 
-static unsigned int hostHighwater = HOST_HIGHWATER ;
+unsigned int hostHighwater = HOST_HIGHWATER ;
 static time_t start ;
 static char startTime [30] ;    /* for timeToString */
 static pid_t myPid ;
@@ -526,7 +529,7 @@
       params->portNum=PORTNUM;
       params->forceIPv4=FORCE_IPv4;
       params->closePeriod=CLOSE_PERIOD;
-      params->dynamicMethod=METHOD_STATIC;
+      params->dynamicMethod=METHOD_COMBINED;
       params->wantStreaming=STREAM;
       params->dropDeferred=false;
       params->minQueueCxn=false;
@@ -2753,20 +2756,20 @@
   val = (bv ? true : false);
 
   inherit = isDefault?NO_INHERIT:INHERIT;
-  GETINT(s,fp,"article-timeout",0,LONG_MAX,REQ,p->articleTimeout, inherit);
-  GETINT(s,fp,"response-timeout",0,LONG_MAX,REQ,p->responseTimeout, inherit);
-  GETINT(s,fp,"close-period",0,LONG_MAX,REQ,p->closePeriod, inherit);
-  GETINT(s,fp,"initial-connections",0,LONG_MAX,REQ,p->initialConnections, inherit);
-  GETINT(s,fp,"max-connections",0,LONG_MAX,REQ,p->absMaxConnections, inherit);
-  GETINT(s,fp,"max-queue-size",1,LONG_MAX,REQ,p->maxChecks, inherit);
-  GETBOOL(s,fp,"streaming",REQ,p->wantStreaming, inherit);
-  GETBOOL(s,fp,"drop-deferred",REQ,p->dropDeferred, inherit);
-  GETBOOL(s,fp,"min-queue-connection",REQ,p->minQueueCxn, inherit);
-  GETREAL(s,fp,"no-check-high",0.0,100.0,REQ,p->lowPassHigh, inherit);
-  GETREAL(s,fp,"no-check-low",0.0,100.0,REQ,p->lowPassLow, inherit);
-  GETREAL(s,fp,"no-check-filter",0.1,DBL_MAX,REQ,p->lowPassFilter, inherit);
-  GETINT(s,fp,"port-number",0,LONG_MAX,REQ,p->portNum, inherit);
-  GETINT(s,fp,"backlog-limit",0,LONG_MAX,REQ,p->backlogLimit, inherit);
+  GETINT(s,fp,"article-timeout",0,LONG_MAX,NOTREQ,p->articleTimeout, inherit);
+  GETINT(s,fp,"response-timeout",0,LONG_MAX,NOTREQ,p->responseTimeout, inherit);
+  GETINT(s,fp,"close-period",0,LONG_MAX,NOTREQ,p->closePeriod, inherit);
+  GETINT(s,fp,"initial-connections",0,LONG_MAX,NOTREQ,p->initialConnections, inherit);
+  GETINT(s,fp,"max-connections",0,LONG_MAX,NOTREQ,p->absMaxConnections, inherit);
+  GETINT(s,fp,"max-queue-size",1,LONG_MAX,NOTREQ,p->maxChecks, inherit);
+  GETBOOL(s,fp,"streaming",NOTREQ,p->wantStreaming, inherit);
+  GETBOOL(s,fp,"drop-deferred",NOTREQ,p->dropDeferred, inherit);
+  GETBOOL(s,fp,"min-queue-connection",NOTREQ,p->minQueueCxn, inherit);
+  GETREAL(s,fp,"no-check-high",0.0,100.0,NOTREQ,p->lowPassHigh, inherit);
+  GETREAL(s,fp,"no-check-low",0.0,100.0,NOTREQ,p->lowPassLow, inherit);
+  GETREAL(s,fp,"no-check-filter",0.1,DBL_MAX,NOTREQ,p->lowPassFilter, inherit);
+  GETINT(s,fp,"port-number",0,LONG_MAX,NOTREQ,p->portNum, inherit);
+  GETINT(s,fp,"backlog-limit",0,LONG_MAX,NOTREQ,p->backlogLimit, inherit);
 
   GETBOOL(s,fp,"force-ipv4",NOTREQ,p->forceIPv4,inherit);
   if (p->forceIPv4)
@@ -2787,10 +2790,13 @@
   if (findValue (s,"backlog-factor",inherit) == NULL &&
       findValue (s,"backlog-limit-highwater",inherit) == NULL)
     {
-      logOrPrint (LOG_ERR,fp,
+      /* This is not an error.  A default value will be used.
+       *
+       * logOrPrint (LOG_ERR,fp,
                   "ME config: must define at least one of backlog-factor"
-                  " and backlog-limit-highwater. Adding %s: %f", "backlog-factor",
+                  " and backlog-limit-highwater.  Adding %s: %f", "backlog-factor",
                   LIMIT_FUDGE) ;
+       */
       addReal (s,"backlog-factor",LIMIT_FUDGE) ;
       rv = 0 ;
     }
@@ -2816,10 +2822,10 @@
   GETINT(s,fp,"backlog-limit-highwater",0,LONG_MAX,NOTREQNOADD,p->backlogLimitHigh, inherit);
   GETREAL(s,fp,"backlog-factor",1.0,DBL_MAX,NOTREQNOADD,p->backlogFactor, inherit);
 
-  GETINT(s,fp,"dynamic-method",0,3,REQ,p->dynamicMethod, inherit);
-  GETREAL(s,fp,"dynamic-backlog-filter",0.0,DBL_MAX,REQ,p->dynBacklogFilter, inherit);
-  GETREAL(s,fp,"dynamic-backlog-low",0.0,100.0,REQ,p->dynBacklogLowWaterMark, inherit);
-  GETREAL(s,fp,"dynamic-backlog-high",0.0,100.0,REQ,p->dynBacklogHighWaterMark, inherit);
+  GETINT(s,fp,"dynamic-method",0,3,NOTREQ,p->dynamicMethod, inherit);
+  GETREAL(s,fp,"dynamic-backlog-filter",0.0,DBL_MAX,NOTREQ,p->dynBacklogFilter, inherit);
+  GETREAL(s,fp,"dynamic-backlog-low",0.0,100.0,NOTREQ,p->dynBacklogLowWaterMark, inherit);
+  GETREAL(s,fp,"dynamic-backlog-high",0.0,100.0,NOTREQ,p->dynBacklogHighWaterMark, inherit);
 
   l=p->lowPassLow;
   h=p->lowPassHigh;
@@ -3288,6 +3294,8 @@
 
       fprintf (fp,"innfeed from %s\npid %d started %s\n\nUpdated: %s\n",
                INN_VERSION_STRING,(int) myPid,startTime,timeString) ;
+      fprintf (fp,"Stats period: %-5ld    Stats reset: %ld\n",
+               (long) statsPeriod, (long) statsResetPeriod);
       fprintf (fp,"(peers: %d active-cxns: %d sleeping-cxns: %d idle-cxns: %d)\n\n",
                peerNum, actConn, slpConn,(maxcon - (actConn + slpConn))) ;
 
@@ -3328,9 +3336,16 @@
       fprintf(fp,"   response timeout: %-5d         max connections: %d\n",
 	    defaultParams->responseTimeout,
 	    defaultParams->absMaxConnections) ;
+      fprintf(fp,"  reconnection time: %-5d   max reconnection time: %d\n",
+              init_reconnect_period, max_reconnect_period);
       fprintf(fp,"       close period: %-5d              max checks: %d\n",
 	    defaultParams->closePeriod,
 	    defaultParams->maxChecks) ;
+      fprintf(fp,"   DNS retry period: %-5d       DNS expire period: %d\n", 
+              dnsRetPeriod, dnsExpPeriod);
+      fprintf(fp,"           port num: %-5d              force IPv4: %s\n",
+              defaultParams->portNum,
+              defaultParams->forceIPv4 ? "true " : "false");
       fprintf(fp,"     want streaming: %-5s          dynamic method: %d\n",
 	    defaultParams->wantStreaming ? "true " : "false",
 	    defaultParams->dynamicMethod) ;
@@ -3349,7 +3364,7 @@
       fprintf(fp," backlog limit high: %-7d         min-queue-cxn: %s\n",
 	    defaultParams->backlogLimitHigh,
 	    defaultParams->minQueueCxn ? "true " : "false");
-      fprintf(fp,"  backlog feed first: %s\n",
+      fprintf(fp," backlog feed first: %s\n",
            defaultParams->backlogFeedFirst ? "true " : "false");
       fprintf(fp,"     backlog factor: %1.1f\n\n",
 	    defaultParams->backlogFactor);
@@ -3922,7 +3937,7 @@
         }
       else
         logOrPrint (LOG_INFO,fp,
-                    "ME config: adding missing key/value %s: %ld",name
+                    "ME config: adding default value for key %s: %ld",name
                     ,setval) ;
     }
   else if (v != NULL && v->type != intval)
@@ -3976,7 +3991,7 @@
         }
       else
         logOrPrint (LOG_INFO,fp,
-                    "ME config: adding missing key/value %s: %f",name,
+                    "ME config: adding default value for key %s: %f",name,
                     setval) ;
     }
   else if (v != NULL && v->type != realval)
@@ -4027,7 +4042,7 @@
         }
       else
         logOrPrint (LOG_INFO,fp,
-                    "ME config: adding missing key/value %s: %s",name,
+                    "ME config: adding default value for key %s: %s",name,
                     (setval ? "true" : "false")) ;
     }
   else if (v != NULL && v->type != boolval)

Modified: innfeed/imap_connection.c
===================================================================
--- innfeed/imap_connection.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/imap_connection.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -366,8 +366,8 @@
 
 static Connection gCxnList = NULL ;
 static unsigned int gCxnCount= 0 ;
-static unsigned int max_reconnect_period = MAX_RECON_PER ;
-static unsigned int init_reconnect_period = INIT_RECON_PER;
+unsigned int max_reconnect_period = MAX_RECON_PER ;
+unsigned int init_reconnect_period = INIT_RECON_PER;
 
 typedef enum {
     RET_OK = 0,

Modified: innfeed/innlistener.c
===================================================================
--- innfeed/innlistener.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/innlistener.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -54,7 +54,7 @@
 static long droppedCount = 0 ;
 static int droppedFileCount = 0 ;
 static char *dropArtFile = NULL ;
-static bool fastExit = false ;
+bool fastExit = false ;
 
 extern const char *pidFile ;
 extern const char *InputFile ;

Modified: innfeed/main.c
===================================================================
--- innfeed/main.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/main.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -52,7 +52,9 @@
 char *configFile = NULL ;
 bool RollInputFile = false ;
 char *pidFile = NULL ;
-bool useMMap = false ;
+bool useMMap = false ; /* Even if HAVE_MMAP.  Only used when innfeed is given
+                        * file names to send instead of storage API tokens,
+                        * which is a fairly rare use case. */
 void (*gPrintInfo) (void) ;
 char *dflTapeDir;
 /* these are used by imapfeed */
@@ -69,6 +71,9 @@
 extern int optind;
 #endif
 extern bool genHtml ;
+extern bool debugShrinking;
+extern bool fastExit;
+extern unsigned int stdioFdMax;
 
 extern void openInputFile (void);
 
@@ -136,12 +141,6 @@
       exit(1);
   }
 
-#if defined (HAVE_MMAP)
-  useMMap = true ;
-#else
-  useMMap = false ;
-#endif
-
   message_handlers_die (2, error_log_stderr_date, message_log_syslog_err) ;
   message_handlers_warn (1, message_log_syslog_warning);
   message_handlers_notice (1, message_log_syslog_notice) ;
@@ -971,7 +970,10 @@
   fprintf (fp,"    News spool: %s\n",newsspool) ;
   fprintf (fp,"      Pid file: %s\n",pidFile) ;
   fprintf (fp,"      Log file: %s\n",(logFile == NULL ? "(none)" : logFile));
-  fprintf (fp,"   Debug level: %2ld                Mmap: %s\n",
-           (long)loggingLevel,boolToString(useMMap)) ;
+  fprintf (fp,"   Debug level: %-5u        Debug shrinking: %s\n",
+           loggingLevel, boolToString(debugShrinking));
+  fprintf (fp,"     Fast exit: %-5s            stdio-fdmax: %d\n",
+           boolToString(fastExit), stdioFdMax);
+  fprintf (fp,"          Mmap: %s\n", boolToString(useMMap));
   fprintf (fp,"\n") ;
 }

Modified: innfeed/misc.c
===================================================================
--- innfeed/misc.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/misc.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -67,7 +67,7 @@
 
 /* If desired, print out the state of innfeed, call a cleanup function, and
    then dump core.  Used as an exit handler for die. */
-int
+void
 dump_core(void)
 {
 #if SNAPSHOT_ON_DIE
@@ -82,9 +82,6 @@
   
     sleep(5);
     abort();
-
-    /* Not reached. */
-    return 1;
 }
 
 /* An alternate version of die, used when we don't want to dump core.  This

Modified: innfeed/misc.h
===================================================================
--- innfeed/misc.h	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/misc.h	2011-07-17 19:09:26 UTC (rev 9275)
@@ -75,11 +75,12 @@
 void error_log_stderr_date(int len, const char *fmt, va_list args, int err);
 
 /* Do cleanup and then abort, for use with die. */
-int dump_core(void);
+void dump_core(void)
+    __attribute__ ((noreturn));
 
 /* Alternate die that doesn't invoke an error handler. */
 void logAndExit (int exitVal, const char *fmt, ...)
-    __attribute__((__format__(printf, 2, 3)));
+    __attribute__((noreturn, __format__(printf, 2, 3)));
 
 /* return true of the file exists and is a regular file */
 bool fileExistsP (const char *filename) ;

Modified: innfeed/tape.c
===================================================================
--- innfeed/tape.c	2011-07-17 19:08:55 UTC (rev 9274)
+++ innfeed/tape.c	2011-07-17 19:09:26 UTC (rev 9275)
@@ -52,6 +52,7 @@
 
 extern char *dflTapeDir;
 extern bool genHtml ;
+extern unsigned int hostHighwater;
 
 #if 0
 /* a structure for temporary storage of articles. */
@@ -395,9 +396,9 @@
   nt->lossage = 0 ;
 
   nt->noBacklog = false ;
-  nt->backlogFactor = 0.0 ;
-  nt->outputLowLimit = 0 ;
-  nt->outputHighLimit = 0 ;
+  nt->outputLowLimit = BLOGLIMIT;
+  nt->outputHighLimit = BLOGLIMIT_HIGH;
+  nt->backlogFactor = LIMIT_FUDGE;
 
   if (!talkToSelf)
     {
@@ -417,9 +418,9 @@
                 {
                   warn ("%s no backlog-factor or backlog-limit-highwater",
                         nt->peerName) ;
-                  nt->outputLowLimit = 0 ;
-                  nt->outputHighLimit = 0 ;
-                  nt->backlogFactor = 0.0 ;
+                  nt->outputLowLimit = BLOGLIMIT;
+                  nt->outputHighLimit = BLOGLIMIT_HIGH;
+                  nt->backlogFactor = LIMIT_FUDGE;
                 }
             }
           else
@@ -495,6 +496,8 @@
   fprintf (fp,"    rotate period: %-3ld seconds\n",(long) rotatePeriod) ;
   fprintf (fp,"checkpoint period: %-3ld seconds\n",(long) tapeCkPtPeriod) ;
   fprintf (fp,"   newfile period: %-3ld seconds\n",(long) tapeCkNewFilePeriod);
+  fprintf (fp,"backlog highwater: %u\n", tapeHighwater);
+  fprintf (fp,"  highwater queue: %u\n", hostHighwater);
   fprintf (fp,"\n") ;
 }
 

Modified: samples/innfeed.conf
===================================================================
--- samples/innfeed.conf	2011-07-17 19:08:55 UTC (rev 9274)
+++ samples/innfeed.conf	2011-07-17 19:09:26 UTC (rev 9275)
@@ -24,69 +24,71 @@
 
 #news-spool:                                            # Default is <patharticles>.
 #input-file:                                            # Default is unset (channel mode).
-pid-file:                       innfeed.pid             # Relative to <pathrun>.
-debug-level:                    0
-debug-shrinking:                false
-fast-exit:                      false
-use-mmap:                       false
-log-file:                       innfeed.log             # Relative to <pathlog>.
-stdio-fdmax:                    0
-log-time-format:                "%a %b %d %H:%M:%S %Y"
+#pid-file:                       innfeed.pid            # Relative to <pathrun>.
+#debug-level:                    0
+#debug-shrinking:                false
+#fast-exit:                      false
+#use-mmap:                       false
+#log-file:                       innfeed.log            # Relative to <pathlog>.
+#stdio-fdmax:                    0
+#log-time-format:                "%a %b %d %H:%M:%S %Y"
 
 ##  Uncomment the next line to include the contents
 ##  of "testfile" at this point.
 
 #$INCLUDE testfile
 
-backlog-directory:              innfeed                 # Relative to <pathspool>.
-backlog-highwater:              5
-backlog-rotate-period:          60
-backlog-ckpt-period:            30
-backlog-newfile-period:         600
+##  Other global parameters.
 
-dns-retry:                      900
-dns-expire:                     86400
-close-period:                   86400
-gen-html:                       false                   # If true, status-file is relative to <pathhttp>;
-status-file:                    innfeed.status          # otherwise, it is relative to <pathlog>.
-connection-stats:               false
-host-queue-highwater:           10
-stats-period:                   600
-stats-reset:                    43200
+#backlog-directory:              innfeed                # Relative to <pathspool>.
+#backlog-highwater:              5
+#backlog-rotate-period:          60
+#backlog-ckpt-period:            30
+#backlog-newfile-period:         600
 
-initial-reconnect-time:         30
-max-reconnect-time:             3600
+#dns-retry:                      900
+#dns-expire:                     86400
+#close-period:                   86400
+#gen-html:                       false                  # If true, status-file is relative to <pathhttp>;
+#status-file:                    innfeed.status         # otherwise, it is relative to <pathlog>.
+#connection-stats:               false
+#host-queue-highwater:           10
+#stats-period:                   600
+#stats-reset:                    43200
 
+#initial-sleep:                  2
+#initial-reconnect-time:         30
+#max-reconnect-time:             3600
 
-##  Defaults for all peers.  These must all exist at
+
+##  Defaults for all peers.  These all exist at
 ##  global scope.  Any of them can be redefined
 ##  inside a peer or group definition.
 
-article-timeout:                600
-response-timeout:               300
-initial-sleep:                  2
-initial-connections:            1
-max-connections:                2
-max-queue-size:                 5
-streaming:                      true
-no-check-high:                  95.0
-no-check-low:                   90.0
-no-check-filter:                50.0
+#article-timeout:                600
+#response-timeout:               300
+#initial-connections:            1
+#max-connections:                2
+#max-queue-size:                 5
+#streaming:                      true
+#no-check-high:                  95.0
+#no-check-low:                   90.0
+#no-check-filter:                50.0
 #bindaddress:
 #bindaddress6:
-port-number:                    119
-force-ipv4:                     false
-drop-deferred:                  false
-min-queue-connection:           false
-backlog-limit:                  0
-backlog-factor:                 1.10
-backlog-limit-highwater:        0
-dynamic-method:                 3
-dynamic-backlog-filter:         0.7
-dynamic-backlog-low:            20.0
-dynamic-backlog-high:           50.0
-no-backlog:                     false
-backlog-feed-first:             false
+#port-number:                    119
+#force-ipv4:                     false
+#drop-deferred:                  false
+#min-queue-connection:           false
+#backlog-limit:                  0
+#backlog-factor:                 1.10
+#backlog-limit-highwater:        0
+#dynamic-method:                 3
+#dynamic-backlog-filter:         0.7
+#dynamic-backlog-low:            20.0
+#dynamic-backlog-high:           50.0
+#no-backlog:                     false
+#backlog-feed-first:             false
 
 ##  Peers.
 

Modified: scripts/innreport_inn.pm
===================================================================
--- scripts/innreport_inn.pm	2011-07-17 19:08:55 UTC (rev 9274)
+++ scripts/innreport_inn.pm	2011-07-17 19:09:26 UTC (rev 9275)
@@ -1059,6 +1059,8 @@
       return 1 if $left =~ m/ preparing to roll /o;
       return 1 if $left =~ m/ reached EOF in /o;
       return 1 if $left =~ m/ opened /o;
+      # when optional parameters are not present in innfeed.conf
+      return 1 if $left =~ m/ config: adding default value for key /o;
     }
     # hostChkCxn - now: x.xx, prev: x.xx, abs: xx, curr: x
     return 1 if $left =~ m/ hostChkCxn - now/o;




More information about the inn-committers mailing list