BIND 10 #421: Requirements for Logging Framework

[BIND 10 Development]

#421: Requirements for Logging Framework
--------------------------+-------------------------------------------------
      Reporter:  stephen  |        Owner:  UnAssigned
          Type:  task     |       Status:  reviewing 
      Priority:  minor    |    Milestone:            
     Component:  logging  |   Resolution:            
      Keywords:           |    Sensitive:  0         
Estimatedhours:  0.0      |        Hours:  0         
      Billable:  1        |   Totalhours:  0         
      Internal:  0        |  
--------------------------+-------------------------------------------------

Comment(by stephen):

 > It is not clear to me how logging sources work exactly. Can a log
 message belong to
 > multiple logging sources? That is, can it belong to both the
 authoritative server and the
 > DNS library? It seems like this is the case to me, but it should be
 explicit.

 The idea was that a message is associated with just one component - the
 module where it is generated, e.g. NSAS, DNS library, xfrout etc.  In
 practice though, the message would have two sources; the module it is
 generated and the program it is used (e.g. we use the DNS library in both
 the authoritative server and recursive server): so in answer to your
 question, I guess that we should specify that a message can belong to
 these two sources.

 > I think that whether a given source is enabled or disabled should be
 considered
 > configuration, and managed by the config manager.

 You are right, it is configuration.

 > It is not clear to me that severity is per source. I think it should be.

 I see your point, e.g. I only want error messages apart from this
 component about which I am suspicious and for which I want all messages.

 > Note that if there are multiple sources per message then severity
 becomes tricky. For
 > instance, what > if I want debugging for the DNS library but only for
 the authoritative
 > server? For simplicity I think we should not specify this, but maybe
 note that it is
 > possible for future work.

 I guess in practice that if we disabled/enabled messages we would do it
 only on a per-program basis or a per-program/per-module basis.  E.g.  we
 might want to enable all possible messages from the authoritative server
 or enable all possible DNS library messages within the authoritative
 server; but I think it is unlikely that someone would want to enable all
 possible DNS library messages regardless of the program that was running.

 So maybe the answer here is to distinguish between instances, e.g.
 auth.dnslib and recurse.dnslib, which would allow us to do things such as
 setting "auth.dnslib = INFO" and "recurse.dnslib = CRITICAL".

 > Should we define some rough indication of what the debugging levels are,
 or offer any
 > guidance as to how they should be used? (I am reminded of the habit in
 BASIC of numbering
 > lines by 10 so that you can add lines in between if necessary...)

 Probably a good idea.  I'll try to come up with some guidelines.

 > We should use file sizes as maximum, and rotate before we hit the
 defined size. I know it
 > doesn't really matter whether we have 1000003 or 999912 byte log files,
 but as an
 > administrator I am always annoyed by "best effort" maximums.

 We could do that - just before "write" we check the file size, file size
 limit and message size and rotate the file if appropriate.

 > File rotation based on time might also be useful. One tricky bit if we
 do decide to do
 > that is that one cannot always guarantee a log message occurs in a given
 time interval.

 It depends how we are going to do file-based logging.  If we are sending
 all messages through a central logging process, then a configuration
 option within that process that sets up a repeating timer appears to be
 the way to go.

 > Whether we include rotation based on time, we will probably want some
 sort of way to
 > define what file names look like for users.
 "/var/log/bind10/bind10-%Y%m%d.log" or similar
 > (we can probably steal from some advanced syslog).

 Agreed.

 > It might be nice to set different logging levels per output. I think
 BIND 9 allows this
 > now. So what I think we really have is something like this:
 > :
 > Each log output can be connected to any number of log sources
 (authoritative server,
 > recursive server, DNS library, ...). Each log source is set to a
 specific severity. Each
 > logging call (in the source code) can be connected to any number of
 sources.

 Do we really want any message to be connected to any number of sources, or
 are the two suggested above (facility and program) sufficient?

 In summary, in doing this we have:
  * Source filter: sets levels of messages that are generated (call this
 Ls)
  * Output filter: sets levels of messages that are logged (call this Lo)
 Note that the filter is a single level, with DEBUG the lowest and CRITICAL
 the highest.  All messages at or above that level are output/logged.  This
 means that if Lo > Ls, not all generated messages will be output.

 > If can't write due to full disk, then pause, and try again to write same
 message.

 Do this once then consider it to be an error and send to the secondary
 destination...

 > Actually a generic "fallback log output" might be good. Each log output
 could define one
 > (although we'd have to be careful not to log the same message too many
 times in case of a
 > full disk or other
 > error with logging).

 OK. For each logging output specify a primary and secondary destination.
 Messages are written to the primary destination.  If there is a problem,
 an error message is sent to the secondary destination (no more than one
 message every <N> minutes, N being a configuration option and defaulting
 to one hour) and the log output written there as well.

-- 
Ticket URL: <http://bind10.isc.org/ticket/421#comment:7>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development