approach on parsing the query-log file

[Chris Dew]

You may be interested in using circular buffers, instead of a log file.

http://www.finalcog.com/replace-logs-emlog-circular-buffer

I've used emlog successfully in the past and been very pleased with
it's performance.

Hope this is useful.

Chris.

2009/4/29 Scott Haneda <talklists at newgeo.com>:
> I have read the other posts here, and it looks like you are setting on tail,
> or a pipe, but that log rotation is causing you headaches.
>
> I have had to deal with things like this in the past, and took a different
> approach.  Here are some ideas to think about.
>
> Since you mentioned below you wanted this in real time, and that parsing an
> old log file is out, what about setting up a second log in named, of the
> same data, but do not rotate the log at all?
>
> This gives you a log that you can run tail on.  It probably is going to grow
> too large.  I solved this for a different server in the past, by telling the
> log that was a clone to be be limited in size.  In this way, it was not
> rolled out, but rather, truncated.
>
> I am not sure how named would do this.  If it will not truncate it, you can
> write a small script to do it for you.  Now that you have a log that is
> maintained at a fixed size that is manageable, you can do your tail business
> on it.
>
> I also seem to remember, tail has some flags that may help you with dealing
> with the log ration issues.  I only remember them vaguely, as they were not
> applicable to what I was doing at the time.
>
> Hope this helps some.
>
> On Apr 27, 2009, at 10:26 PM, Jonathan Petersson wrote:
>
>> Hi all,
>>
>> I'm thinking of writing a quick tool to archive the query-log in a
>> database to allow for easier reports.
>>
>> The obvious question that occurs is; What would be what's the best
>> approach to do this?
>>
>> Running scripts that parses through the query-log would cause locking
>> essentially killing BIND on a heavy loaded server and only parsing
>> archived files wouldn't allow real-time information, also re-parsing
>> the same set of data over and over again until the log has rotated
>> would cause unnecessary I/O load. I'm guessing the best would be to
>> have BIND write directly to a script that dumps the data where-ever it
>> makes sense to.
>>
>> I've used BIND statistics and found it highly useful but then again it
>> doesn't allow me to make breakdowns based on host/query.
>>
>> If anyone has done something like this or having pointers on how this
>> could achieved any information is welcome!
>
> --
> Scott * If you contact me off list replace talklists@ with scott@ *
>
> _______________________________________________
> bind-users mailing list
> bind-users at lists.isc.org
> https://lists.isc.org/mailman/listinfo/bind-users
>



-- 

http://www.finalcog.com/