[bind10-dev] some profiling on the new in-memory data source

[JINMEI Tatuya / 神明達哉]

At Mon, 24 Sep 2012 13:37:20 +0200,
Jelte Jansen <jelte at isc.org> wrote:

> I used the root zone, a simple query input of www.example.net A, and
> 100000 queries through query_bench. My callgrind outputs can be found
> at http://tjeb.nl/opendir/profiling/ and can be interactively
> inspected with kcachegrind)
> 
> 1. Most time is spent in LabelSequence::compare(). This is quite a big
> change (it was 1.7% of the calls and it now 15.6%) and could perhaps
> partly or completely be expected, but right now it seems it is called
> rather excessively often (about 50 times per query).
> 2. The next method where a lot of time is spent is the LabelSequence
> constructor itself (when creating from buffer data), called even more
> often (most of which because of calls to DomainTreeNode::getLabels())
> 3. find() is called 2700000 times; 2600000 times of those are from
> getAdditionalAddrs (which itself is called 1300000 times, which makes
> sense) (i.e. 13 root addresses, 2 lookups per address). find() itself
> then calls the construct/compare (mentioned in 1 and 2) 48500000 times
> (about 17 times per call to itself, on average, in domaintree.h line 1527)
> 
> I'll leave further interpretation of these numbers as an exercise for
> the reader for now ;)

I guess all of these differences largely come from the fact that the
new version doesn't have shortcut links from the NS name of each NS
RDATA to the corresponding zone tree node.  This means more calls to
find(), which also means more LabelSequence constructions and more
calls to compare().

This is an architectural bottleneck.  While we can improve it by
re-implementing the shortcut framework (my very original proposal
http://bind10.isc.org/wiki/InMemoryZoneDesign included it), it would
take quite some time.

There are some other, more easy-to-fix bottlenecks in the new
implementation.  The first one was addressed in #2282, which should
substantially mitigate the performance drop in the root zone +
www.example.net scenario.

Other major issues are summarized in #2283.  It won't be very
difficult to fix these cases.

At that point I expect the new version will work pretty fast in
practice, especially for signed zone and DNSSEC-enabled (which is
common today) queries.  The internal representation of the new
in-memory data is optimized for faster message rendering, and some of
the optimizations proposed in #2283 include algorithmic improvements
for DNSSEC handling.

So, I'd first complete these tasks and then measure the performance
again.  If we still want (need) to make it faster, we can then
consider re-implementing additional shortcuts.  It'd also be a good
idea to perform profiling at that point to find other code-level
bottlenecks.

---
JINMEI, Tatuya