[bind10-dev] Enabling/disabling logging on per-message basis

[Michal 'vorner' Vaner]

Hello

On Mon, Mar 26, 2012 at 11:25:40AM -0700, JINMEI Tatuya / 神明達哉 wrote:
> > So, I measured the impact of adding the map there, in ticket #1752. It turns if
> > there's a tight loop doing only logging of messages that are not output, the
> > performance drops to 73% (from 86114991 messages per second to 62564871). Jinmei
> > proposed further optimisation, but I believe that with the amount logging we do
> > in real code (we do more of the other stuff than the logging, we definitely
> > don't do 86 million messages per second), the impact would be pretty small.
> 
> To me, it's not so obvious.  We have quite a lot of debug level log
> messages in performance sensitive path, so a downgrade of 27% looks
> still alarming, even if I understand other stuff should dominate the
> overall process.
> 
> Personally, I'd still keep this overhead minimum (e.g. within 5%
> range) unless the optimization requires hundreds of additional code.

OK, I made another small experiment. I configured bind10 here with full debug
output and counted 7 messages per successful query, 7 per NXDOMAIN and 3 when
no such zone (for in-memory). This can differ by amount of additional data and
like that, so lets count 10 per query to be safe and have easier mathematics.

Looking at the emails Jeremy sent, we have something 40k QPS per core. OK, count
50k, again to be on the safe side.

As measured by me, the orders of logging messages is above 50M MPS (again error
to the safe side). That means our .5M messages output for that form 1% of
the time of the answering process. Even by making the process of determining the
message should not be logged twice as slow would be 1% difference in the total
performance.

Therefore, I don't think we would have to worry too much about the performance
cost of the feature. Considering both you and Stephen provided some additional
optimisation ideas (while my measurement was just first and the most
straightforward implementation), we can get the performance difference under the
size of benchmark errors.

So I'm stating it again: we don't have to worry about it making the server too
slow. Now we need to worry if the feature provides enough benefits (both in
usability and „marketing“) to be worth the work we would put into implementing
it and the one more item in the Logging configuration module. I think the
feature is worth it.

With regards

-- 
Please stay calm. There is no use both of us being hysterical.

Michal 'vorner' Vaner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <https://lists.isc.org/pipermail/bind10-dev/attachments/20120327/1a6883ae/attachment.bin>