[bind10-dev] writable datasources 1/3; data_source.h

[JINMEI Tatuya / 神明達哉]

At Mon, 10 May 2010 10:41:38 +0200,
Jelte Jansen <jelte at isc.org> wrote:

> It seems to be quite lacking in the form of doxygen comments.

Right, and my understanding is that this is a big TODO for this module
(there may even be a trac ticket).

> Speaking of should and should not override, there is a comment in there:
> 
>     // 'High-level' methods.  These will be implemented by the
>     // general DataSrc class, and SHOULD NOT be overwritten by subclasses.
> 
> I thought the idea was that these functions don't *need* to be
> overwritten, but *could be* if the implementor wants to (for instance to
> make use of specific backend features). That SHOULD NOT there doesn't
> really seem to imply that :)

I don't know the original intent of the "SHOULD NOT", but one thing
I'm pretty sure about is that it depends on where's the
customization/optimization point.  And, my understanding is that we
vaguely expected that our highest performance query handler (to be
seen:-) would optimize things at this level.  For example, it might
have pre-compiled wire format "data source" that might even allow
bypassing the default (not very efficient) name compression, and might
non-selectively use that source for all queries.

On the other hand, if this "SHOULD NOT" is actually more like "MUST
NOT", I think we should enforce the idea through the class (or API in
general) design, rather than just noting that in comment lines
(comments/documents about this are good for reference, per se).

These points lead to my meta level concerns about the current API.
IMO (I know opinions may vary on this point), the current design of
the DataSrc class is too monolithic, containing too many things.
One direct consequence of this is the "SHOULD NOT be overwritten"
methods.  It's pretty clear that we'd need multiple classes (probably
derived from a single abstract class) for specific types of data
sources (sqlite3, mysql, etc), but if we also give high level
responsibility to that class we'll face an issue of how to handle the
high level features in the derived classes while these features are
generally expected to be common regardless of the data source type.

So, although I've not fully considered it (or it's not in my short
term assignments), I'd refactor the design like this:
- define separate (unrelated) classes for high/medium (if any)/low
  level interfaces.
- the highest level class is responsible for the generic query handle
  logic.  it's composed of lower level class objects (among other
  necessary members).  if we basically assume one common variation at
  this level, we can simply define it as a leaf (only) class; if we
  want to customize the behavior at this level, we can define an
  abstract base class with inheritance.
- the lowest level class is responsible for retrieving (and now
  modifying) data from/in a specific data source.  we'll probably use
  inheritance here: there's a single abstract base class, and we
  define a derived class per data source type.
- I'm not sure if we need medium level, but if we do the same idea
  would apply.

> I'm also wondering whether we should make the low-level functions
> private. Do we expect these to be used directly? (in which case, I'm
> wondering whether we even need some of the high-level ones that I
> currently have for the writable part, as I suspect tools like loadzone
> might want to use the low-level ones so they can read/store one rr at a
> time instead of loading an entire zone into memory, just to name something).

I guess we should at least have public interface for "findRRset()" or
something equivalent to that.  My understanding is that we'd even
allow an external developer to implement their own version of higher
level query module that calls our lowest module data source API.  For
example, someone might want to write a python implementation of
authoritative server that uses a collection of BIND 10 data sources.
(btw if we want to allow this it would be another reason for
separating classes per the processing level).

---
JINMEI, Tatuya