[bind10-dev] new in-memory zone data design

[JINMEI Tatuya / 神明達哉]

At Thu, 21 Jun 2012 17:39:20 +0200,
Shane Kerr <shane at isc.org> wrote:

> This is a minor variant of Michal's concerns about complication. The
> RdataEncodeSpec is a bit hard to read to me. I don't have any real
> proposals to improve it, but it seems a bit unfriendly to coders:
> 
> GENERIC_NS_ENCODE_SPEC = {1, 1, {{NAME, COMPRESSIBLE | ADDITIONAL, 0}}};
> 
> I miss Python's named parameters:
> 
> GENERIC_NS_ENCODE_SPEC = { num_fields=1, num_names=1,
>     {{type=NAME, attributes=COMPRESSIBLE | ADDITIONAL, len=0}}};

Yeah this is not very readable, and I'm not even sure if the above is
a valid C/C++ statement.  Maybe I should have described them in any
readable structured format like json.

> RdataSet optimizations?
> ----
> I find the idea of using the "many_rrs" interesting. In fact, I wonder
> if we can use uint8_t rather than uint16_t, with the observation that
> very few real-world RRSET have more than 63 RR (I've never seen one).

Hmm, I think I previously heard of a reverse-lookup zone that has a
very large number of PTR RRs for a single IP address.  I don't
remember how many, but it's quite likely to be more than 63.  Such a
configuration would still be considered one of "very few", but I'm
afraid it may not so uncommon.

> Another possible optimization is to take advantage of the fact that
> almost every zone has only a very few number of TTLs that it uses. I
> was thinking we could perhaps introduce the concept of a "zone TTL",
> possibly stealing another bit from the num_rrs field:
> 
> struct RdataSet {
>     offset_ptr<RdataSet> next; // for linked list
>     uint16_t type; // RR type
>     uint8_t is_signed : 1; // If this set contains RRSIGs
>     uint8_t custom_ttl : 1; // TTL different from zone default 
>     uint8_t many_rrs : 1;  // If the number RRs >= 2^5 = 32
>     uint8_t num_rrs  : 5; // # of RRs, up to 31
> };
> // Data that follows:
> // [uint32_t // TTL, if custom_ttl == 1]
> // [uint16_t // # of RRs, if many_rrs == 1]
[...] 
> Optimizing this way will probably only save a few percentage of memory
> footprint though, so perhaps not worth the effort?

It's hard to tell the cost-benefit balance, but I have some
observations:

- (excluding next) the size of this version of RdataSet is 3, but in
  this context it will effectively mean 4 or even 8 (depending on the
  architecture) due to the necessary padding.  I also suspect it's
  more likely to be a 64-bit machine for large scale operators where
  this level of optimization really matters.  If so, any further
  optimization that cannot save in the unit of 8-byte blocks wouldn't
  be that effective.
- For 32-bit environment, if we can save the space for TTLs in most of
  the RRsets that may be nice.  One difficulty is that how to know the
  "zone TTL".  If $TTL is defined at the top of a textual zone file,
  that may work as a hint, but it may not always be provided.
  Besides, the in-memory implementation wouldn't directly refer to
  that level of information.  So, the only reliable source would be
  the TTL of any RRset that is first given to the zone building code.
  Maybe in common cases most of the others RRsets have the same TTL,
  but I'm not sure how common it is.

> Wildcards, DNAME
> ----
> Ah, wildcards. I'm quite happy if zones using wildcards are less
> efficient than "normal" zones. I don't think we need to kill ourselves
> to optimize for a feature not used *so* commonly, and not at all for
> performance-critical zones, AFAIK.
> 
> DNAME... well, okay. DNAME in principle can be awesomely useful, for
> people who need it. :(

Same for DNSSEC? :-)

---
JINMEI, Tatuya