[bind10-dev] Caching and other performance enhancements

[Michael Graff]

There has been some discussion on making our server faster.  Some of
this happened in Jabber, but it's likely time to move it to mail.

I agree that we need to look carefully at the SQL data source, and make
it more efficient.  I think we need to be very careful here though and
not make schema changes (like using BLOBs) which remove a lot of what
people use SQL for.  This topic probably needs its own email thread.

I'd like to comment a bit on caching here.

I tend to compare DNS to the web world a lot.  This is because I tend to
think the DNS server world has finally reached the technological
problems the web server world has run into about 10 years ago.  This is
mostly why I remain surprised when people think we need to do something
special with TCP because it puts too much of a load on a machine.  Meh.
 But that's also another topic.

I also compare them because a lot of how performance problems are
handled in the web server world apply directly to DNS.  For instance,
most web sites today touch a database in some way to render page
contents.  Sure, there is static content too, but that's boring.

In the web world, various types of caching are used to try to provide
the best performance possible.  In general, the more you cache in a
"block" the best you are.  So, in terms of caching performance, the
preferred order is "page", "fragment", and "query."

Page caching is just that, the whole page is cached and rendered to any
number of clients.  This is often used for "landing pages" which may
change rarely.  It is also used for forum postings where the only
changes occur through comments being added.  In all cases, the page is
rendered once and then served many times.

Fragment caching is the next best caching win.  Here, parts of the page
are rendered per-hit and other parts are rendered from the cache.  This
would be used, for instance, in a page with sidebars or other fairly
constant data.  The menus won't change, so they are rendered once and
cached.

Query caching is the lowest level of improvement and often yields the
least overall gain.  Here, database queries are cached and thus avoided.
 However, the rest of the application needs to still render the page, so
CPU time is consumed even if disk access is not needed.

In BIND 10 and to a lesser extent in BIND 9, a "DNS message" is more or
less a page (with slight exceptions), a "DNS section" is more or less a
fragment, and a "database lookup" is the same as it is in the web world.

Using this model, and focusing only on caching, I believe BIND 10 would
benefit most from adding a query cache.

The message IDs change, but overall the remaining part of the message is
entirely able to be cached.  Negative responses can even be cached when
DNSSEC proof is required if we wish to add that later.

The only really tricky part here is EDNS0, or specifically packet sizes.
I think this is overall an easy problem to solve, we just need to
remember how large the response was and if it ran into size limits or
not.  Bracketing this should be straight-forward.

Hmm, perhaps this is a blog post waiting to happen.

--Michael