[kea-dev] Suggestions for DHCP packets logs

Tomek Mrugalski tomasz at isc.org
Fri Jun 12 18:32:54 UTC 2015


On 12/06/15 11:11, Chaigneau, Nicolas wrote:
> The following are recommendations for implementing the logging of
> DHCP packets.
>
> These originate from functional requirements, and from performance
> tests I've run (on our implementation of packets logging, handled
> through a hook).
> 
> Discussion is welcome as always!
Thanks for sharing those. I assume that your suggestions are addressed
at some future packet logging implementation, not the current Kea code,
right?

While we do have code for logging contents of each packet (including
parsed options) in current Kea, it is on very high debugging level and
is not intended to be used during normal operation. It is there just in
case we need to debug an issue.

> 1) don't use log4cplus
> 
> There is a very significant performance penalty when logging a large
> volume of data through log4cplus.
We intend to keep minimal amount of logging per packet in the default
logging setup. Ideally that should be a single line for each incoming
and outgoing packet.

> Packets should not be logged through log4cplus. Packets should be
> written directly through a fstream. (I've measured about 30%
> improvement in DORA/s capacity, as compared to using log4cplus).
We can possibly consider adding such a feature, but a single line for
each packet will be logged through log4cplus, at least in the
foreseeable future. log4cplus is takes care of all the logging
destinations: file, syslog, stdout, stderr. That logging flexibility is
something we think is very useful for users.

We certainly can implement a dedicated mechanism for writing whole
received packets. As Francis suggested, better name for it is packet
dumping, rather than logging. It seems most convenient to dump them in
pcap format, so anything that can handle wireshark of tcpdump output,
could be used here. If you and other users find it useful, we may
consider it implementing it some time after Kea 1.0.

> 2) enabling packets logs
> 
> A parameter should allow to enable or disable the packets logs.
If we implement packet dumping, it will surely be disabled by default
and there will be a parameter to enable it.

> 3) flushing of packets logs
> 
> There is also a performance benefit in allowing to disable explicit
> flushing (ie each time a line is logged) of the log file. In this
> case, automatic flushing is handled by the system and happens
> whenever the output buffer is full (or when the file is closed).
I vaguely remember that there's flush parameter in log4cplus, but I
can't seem to find any uses of it in the code. Apparently we don't touch
flush switch and keep it in the default state, whatever it is.

> This is not necessarily desirable for everyone. A parameter should be
> provided to control whether packets logs are flushed explicitly or
> automatically.
No, but there are very good reasons why flush may be called. Imagine a
server crash. You really want to every log entry before the crash. If
they are not flushed, the data in buffers may be never logged when the
signal comes.

But I agree that having the ability to disable flush is useful.

> 4) handle packets logs rotation
> 
> Rotation of the packets logs should not be delegated to an external
> task such as logrotate. This should be handled directly by Kea.
We expose several log4cplus parameters that are dedicated to rotating
logs, but I think we never tested this. See Kea User's guide, sections
13.1.2.2.1 (maxsize) that specifies maximum log file size and 13.1.2.2.2
(maxver) that specifies number of old log files to keep around.

> A solution is to have Kea directly handle the file rotation. This can
> be done on several criteria, for instance: - on a time frame basis
> (eg. hourly packets log files), - when file size exceeds a given
> limit, - or a number of logged packets, ...
Please see what we already have there (my comments above). If that's not
sufficient, can you take a look at log4cplus and check whether there are
features that are not exposed by Kea yet, but would solve your problem.
If yes, we can add those parameters easily.

If not, we'll have to think about implementing something. I must
honestly admit that our schedules up to 1.0 are very tight, so any
log4cplus replacement is out of question.

Angelo mentioned google glog. I took at quick look at it. One major
thing missing, as compared to log4cplus, is ability to have multiple
destinations. IMHO this is a significant benefit of log4plus - you can
have different levels and outputs configured for each logger. This
ability to increase or decrease logging on per component basis is very
useful when you want to debug an issue.

Tomek


More information about the kea-dev mailing list