[bind10-dev] Tracking queries and other data

[Michal 'vorner' Vaner]

Hello

On Fri, Dec 03, 2010 at 11:59:11AM +0000, Stephen Morris wrote:
> > I think filtering it in the logging framework would be more flexible and allowed
> > future features. What comes into my mind would be multiple logging targets
> > (file, pipe, etc) with each one having different events/queries logged/traced or
> > a ringbuffer of last few events that could be output as a result of some other
> > event (print all events generated by query resulting in error). And I think
> > there could be many more.
> 
> The problem I addressed is a specific case of filtering.  My suggestion put the identification of packets to trace in the application logic itself as the identification means marking the packet in some way (in the case I described, by setting a query ID).  If we try to do it wholly in the logging framework (with the implication that the framework will not modify the packets or other application data), then the framework will need to somehow identify and remember the packets it is interested in. I think this will be a more difficult option to implement.

Maybe we mean a different think here. Assume we have some context that is
carried with the query (passed trough each of the functions).

The first one (filtering in the application) would assume there's a boolean
value „marked“ (yes, I probably simplify here) and the code to trace something
would look like:

if (context->marked) {
  trace(query, "I'm doing foo to the query");
}

In the second case, the application logic does not care at all if the query is
marked or not. It just calls:

trace(query, "I'm doing foo to the query", context);

and the condition is inside. The advantage I see here is that the lot of
application code does not care by which condition it is decided, providing
encapsulation and ability to change the condition in future. And it is shorter.

I do not much care who or how sets the „marked“ variable (or any other mean of
enabling logging for that particular query), because it is only a single place.

> Most logging frameworks have the things you describe and I would think that the BIND10 one will have many of those characteristics.  But we need to define exactly what we want; for example, we may want trace multiple queries at once, but would we ever want the output from each going to a different destination?  And the ring buffer idea is nice, but is it a single server-wide buffer, or would each query have its own ring buffer?

What I mean by ringbuffer is single buffer (with configurable size) for the
whole server. It could be just a logging backend from the point of the rest of
the application. The advantage I see there is it is cheap to write there (much
cheaper than log files) and we do not want to spam the log files (there are two
problems of log files, that they contain too little and that they contain too
much). So we do not log many of what we are doing (it is impossible, with 100
000 queries per second). But then something interesting happens and we hope we
had that kind of information we did not log.

With the ringbuffer, we still have the information and we can find them later,
in the rare occasion when it is needed. Example of that occasion might be
something configured by the user (some kind of rare condition he is trying to
hunt down) or server crash (the ringbuffer would be stored with core file, it
could try to dump it to file within a SIGSEGV handler, etc). Of course, the
ringbuffer might turn to be too small when we want to find the info.

However, I wanted to give it as an example mostly, of how flexible the logging
and filtering should be.

Have a nice day

-- 
Why TODO list in haskell? It handles infinite data structures.

Michal 'vorner' Vaner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind10-dev/attachments/20101203/df08d6dc/attachment.bin>