Call for inncheck testing

Julien ÉLIE julien at trigofacile.com
Sat Jul 9 16:54:20 UTC 2011


Hi Florian,

> I've done a little digging in the innfeed sources and have come to
> the conclusion that the whole required-key-thing is bogus: *all*
> innfeed configuration options have a default value!

Hmm, yes :-)



> So what I propose is to delete the notion of required keys from the 
> manpage (leaving only the differentiation between global-only
> options and those that can appear everywhere) as well as from
> inncheck. And then I think it would make things easier for admins of
> small sites if we ship a sample innfeed.conf with everything
> commented out, in order to symbolize that nothing needs to be touched
> unless there's a special need.

OK, just done.  (Still not committed.)

I have a question about the "use-mmap" keyword.
We currently have in our documentation:

“This key requires a boolean value.  It specifies whether mmaping should be
used if innfeed has been built with mmap(2) support.  If article data
on disk is not in NNTP-ready format (CR/LF at the end of each line), then
after mmaping, the article is read into memory and fixed up, so mmaping
has no positive effect (and possibly some negative effect depending on
your system), and so in such a case this value should be "false", which
is the default value.  This corresponds to the "-M" command-line option.”

However, when innfeed is built with mmap support, the default value is "true".
What should we do?  Fix the documentation and say that it is set to "true"
by default; or change the default value to "false" in the source code?



>> I can check that if you want and reorganize the man page. I am
>> waiting for your confirmation or refusal, so that we do not do the
>> work twice!
> 
> yes please re-check the "everything has defaults, actually" thing; 
> and I'd be happy to leave the manpage to you or do it when I'm back

Check done.

Changes in the pending commit:

* Change the default dynamic method from 0 to 3.  It already was the
recommended value in the documentation, and the default value in the
sample innfeed.conf file.

* No longer generate an error message (logged in news.err) when a parameter
is not defined in innfeed.conf.  It has a default value, so no need to warn
the user.

* Log a few more parameters in innfeed.status.

* Documentation fixes:
- "pid-file" is relative to "pathrun", and not "backlog-directory";
- "log-file" is relative to "pathlog", and not "backlog-directory";
- "initial-sleep" is a global value, and not per-peer;
- "close-period" is a per-peer value, and not a global value;
- "max-reconnect-time" is mispelled once in the documentation;
- specify that *no* keys are mandatory.  They are all optional, and have
a default value (all of them are now mentioned in the documentation).

* Documentation additions:
- If a file named "innfeed.debug" exists in the "pathlog" directory,
then "debug-level" is automatically set to 1.  This is a cheap way of
avoiding continual reloading of the "newsfeeds" file when debugging.
Note that debug messages still go to "log-file";
- "backlog-directory" is relative to "pathspool";
- "status-file" can be set to an absolute path.




My current innfeed.status logs:


innfeed from INN 2.6.0 (20110624 prerelease)
pid 15038 started Sat Jul 09 16:49:55 2011

Updated: Sat Jul 09 17:21:27 2011
Stats period: 600      Stats reset: 43200
(peers: 11 active-cxns: 10 sleeping-cxns: 1 idle-cxns: 0)

Configuration file: /home/news/etc/innfeed.conf

Global configuration parameters:
          Mode: Channel
    News spool: /home/news/spool/articles
      Pid file: /var/run/news/innfeed.pid
      Log file: /var/log/news/innfeed.log
   Debug level: 0            Debug shrinking: false
     Fast exit: false            stdio-fdmax: 0
          Mmap: true

Listener Status:
    Dropped article file: /home/news/spool/innfeed/innfeed-dropped.A015038
   Dropped article count: 0

Default peer configuration parameters:
    article timeout: 600       initial connections: 1
   response timeout: 300           max connections: 2
  reconnection time: 30      max reconnection time: 3600
       close period: 86400              max checks: 5
   DNS retry period: 900         DNS expire period: 86400
           port num: 119                force IPv4: false
     want streaming: true           dynamic method: 3
        no-check on: 95.0%     dynamic backlog low: 20.0%
       no-check off: 90.0%    dynamic backlog high: 50.0%
    no-check filter: 50.0   dynamic backlog filter: 0.7
  backlog limit low: 0               drop-deferred: false
 backlog limit high: 0               min-queue-cxn: false
 backlog feed first: false
     backlog factor: 1.1

Backlog file global values:
        directory: /home/news/spool/innfeed
    rotate period: 60  seconds
checkpoint period: 30  seconds
   newfile period: 600 seconds
backlog highwater: 5
  highwater queue: 10



As you see, I have added "Stats period", "Stats reset", "Debug shrinking",
"Fast exit", "stdio-fdmax", "reconnection time", "max reconnection time",
"DNS retry period", "DNS expire period", "port num", "force IPv4",
"backlog highwater", "highwater queue".

There are other variables that could be added.  But I wonder whether it is
worthwhile adding everything.  A few parameters like the "log time format"
are not necessarily interesting in this status file.



> Even the ip-name of a peer defaults to the name set in newsfeeds,
> so many people don't really need to touch innfeed.conf at all - or am I
> missing something here?

You're right.



> Mind I haven't had a chance to test any of this yet - I'm on beach
> vacation until the end of the month, with only a netbook and a mobile
> internet connection...

Geek'attitude :-)

http://29.media.tumblr.com/tumblr_lnkh2ffVOp1qb752so1_500.jpg



> Some useful notes from my digging around:
>      - what the manpage says are "common" values are usually the actual
>        defaults
>      - dynamic-method defaults to 0, not 3 as the manpage suggests
>      - no-check-low defaults to 90.0
>      - backlog-limit and backlog-factor default to 0,
>        backlog-limit-highwater to 1.10 (the combination of which results
>        in... what behaviour?)

OK, thanks.  All spotted.  I think 3 is better for dynamic-method.
backlog-factor defaults to 1.10 and backlog-limit-highwater to 0.  (I think
there is a discrepancy between host.c and tape.c — it needs being homogenized).

                  params->backlogLimit=BLOGLIMIT;              # 0
                  params->backlogLimitHigh=BLOGLIMIT_HIGH ;    # 0
                  params->backlogFactor=LIMIT_FUDGE ;          # 1.10

                  nt->outputLowLimit = 0 ;
                  nt->outputHighLimit = 0 ;
                  nt->backlogFactor = 0.0 ;

To answer your question, as backlog-limit is 0 and backlog-factor is also set
(to 1.10), backlog-limit-highwater is computed to 0 x 1.10 = 0.
Then:

  if (tape->outputHighLimit > 0 && tape->outputSize >= tape->outputHighLimit)
    {
      shrinkfile (tape->outFp,tape->outputLowLimit,tape->outputFilename,"a+");
    }

The condition is not met.  Therefore, no shrinking is done.
The man page mentions:

backlog-limit:  If the number is C<0>
(the default), then backlog files are allowed to grow without bound when the
peer is unable to keep up with the article flow.

-- 
Julien ÉLIE

« Oublie les injures, n'oublie jamais les bienfaits. »



More information about the inn-workers mailing list