[bind10-dev] More ideas about high-performance data-source backend (warning: long text)

[JINMEI Tatuya / 神明達哉]

At Thu, 7 Oct 2010 15:43:01 +0200,
Michal 'vorner' Vaner <michal.vaner at nic.cz> wrote:

> But before I start, I would like to ask few questions, because the rest is quite
> long and I do not expect everyone to keep reading until the very end.
> • Jinmey, do you think I could use some of your code of the experiment you
>   showed on face2face meeting? Or, could we cooperate somehow on it?

I don't think my experimental code is so useful for your purpose
because
- it has other features (such as offset-based pointers to support
  mmap-based loading), which would be just noisy for you
- other than that, the tree structure itself is a simpler port of BIND
  9's rbt.  so you might rather want to see the original
  implementation.

But, of course, if you are interested, please feel free to play with
it and make your private branch from it.

As for cooperation, I'm generally happy about that.  How much we can
spend for the cooperation will depend on overall project plans,
though.

> • Do you think it makes sense to write this at all? It is little bit more
>   complicated than usual tree, but I think it should be faster and allow realtime
>   updates without much speed penalty on the lookup side. Is the lookup the
>   bottleneck of authoritative server speed, or should we be optimising somewhere
>   else?

These are not easy to answer.  From my experiences,
- lookup is certainly "a" major bottleneck
- but in terms of overall server performance, it also depends on how
  many times we need to perform lookups for a single query.  if we can
  minimize that (NSD and BIND 9's "acache" take that approach) the
  lookup bottleneck would become relatively minor.
- also, there are other major bottlenecks, such as name compression.
  if we can tweak the DB to make it faster (again, NSD takes that
  approach) it would make more significant performance improvement
  even if the resulting structure is suboptimal for lookup per se.

As for your specific proposal, I think we need a figure:-)

One quick thought that occurred to me is whether we can (efficiently)
update this DB without keeping the good property of locality.

---
JINMEI, Tatuya
Internet Systems Consortium, Inc.