[bind10-dev] Ideas for modularity & hooks

[Michal 'vorner' Vaner]

Hello

On Thu, Feb 03, 2011 at 11:10:31AM -0800, Jerry Scharf wrote:
> First, this sounds really exciting, I hope something like this comes to 

Well, about excitement, may I suggest to keep it calm? ;-) We just talk about it
now, it is not decided we will do something like this and it is a long way to
go, let's keep the excitement at the time we see it working.

Anyway, it seems to me such a feature would be quite unique in a DNS server.
But it exists elsewhere (apache, quite common in similar ways in IM clients,
every other bigger software now has plugins). So, I ask myself, why it is unique
in DNS? Is it that no one thought of it (unlikely)? Or, is it that no one dared
(Does it look complicated? Is it?)? Or, some tried and failed (Then, why?)?

> pass. I am going to use some clumsy words here, but good terms for what 
> the place where a list exists and what the elements of the list are 
> called. I called them processing points and processing elements, which I 

OK, whatever they are called. I like lists more, because it captures that they
have ordering, while point suggests something singular. But naming it is
premature now.

> hope capture the concept. The terms I would use for the return states 
> would be: finish, fail, continue and restart. Restart seems like a 
> "weapon of self destruction", but it sounds like Apache has survived 
> with the functionality.

Sure, people need to be careful about it. But perl survived with stuff like the
$[ variable (where you can set at runtime which is the first index of arrays,
combined with the fact that if you index by negative, it is counted from the end
and setting it to -1, you can have fun). And many people just enjoy having big
guns, while other just silently avoid them. And I think that there are actually
good reasons to have restart (for example, you can set a view in ACL and go
trough the ACLs again, but you could probably do it with recursion as well).

> This draws my mind to how one would configure this type of capability. 
> For a general case, there would be a tree of processing points to 
> control (a processing element loaded could introduce processing points 
> within it.) It also seems clear from the description that there are data 
> elements that processing elements can manipulate. I am going to posit 
> that there is nothing in the configuration that is needed for the data 
> elements, just the lists. (these data elements may drive dependencies)
> 
> To specify an processing element, we need to specify the name of the 
> element, possibly a version, the source from which the element is loaded 
> and the position of this element in the list. There was a discussion of 
> dependency for processing elements, so that would also need to be 
> addressed. If a processing element was added as a dependency, we want to 
> tag that so that when the primary element that needed it goes away the 
> element installed as a dependency goes away as well.

I find it hard to imagine how it would work. Because, one extension (one block
of functionality) might require more elements in various places (both different
processing points or different positions in a single one). And it doesn't make
sense to plug any element to whatever point (it is both logically and parameter
incompatible to put a configuration hook into an ACL processing point). So I
think presenting this to user is dangerous and confusing.

What I propose is having some kind of spec file for each extension. It would
contain name, description, version, url to help, name of the person you want to
sue in case it kills your kitten… and the important stuff ‒ dependencies (and
blocks) on other extensions, the .so library to load and names of initialization
and finalization functions.

User would just be presented with a list of available/installed extensions and
would click (or choose in other way) on the one that shall be loaded or
unloaded.


So, an example from real life. I have computer in school dormitories with
strange admin and strange rules. It is forbidden to talk to any DNS server
except his resolver. But that DNS server doesn't know DNSSEC (and, after some
discussion on the dormitory mailing list, it seems it will never do so). So, the
need for me was to have my own resolver, but it can send queries only to that
one resolver, to get the data.

This is a perfect place for an extension ‒ it's rare, no need to confuse people
to configure that thing. But it is also a valid functionality need. The
extension would work like this:

• It is enabled by user.
• The spec is read, the deps show that only the recursive server itself is
  needed and it is already loaded, so nothing to load first.
• The .so is loaded.
• The init function is run:
  ◦ It hooks the processing point for sending upstream queries (places itself in
    the front, to forward all queries there). This allows modifying the query be
    sent to different location.
  ◦ It needs to hook the configuration processing point (because when there's
    update to the configuration, it wants to know about it).
• Somehow some configuration elements are added (let's say the extension
  notifies the configuration manager it should merge another configuration
  specification into the current module, or whatever else).
• I can happily configure the address of the admin's resolver (or write a hook
  script that plugs into my dhcp client to configure it each time the address is
  assigned to me by the dhcp server).

This requires three actions from the user: getting the extension installed,
enabling it and configuring it. No need to think about what gets plugged where
(look into Firefox and it's extensions ‒ if you want a better download manager,
you don't care a bit, where inside it hooks itself).

From the programmer point of view, it seems easier to just plug the parts
together by the code.

Of course, I'm not against a way for a user to examine (and, for debugging
purposes) to manipulate the processing points. Let's say we could write an
extension for it ;-).

> There almost certainly needs to be a default list for each control point 
> so the system can come up working before the configuration is modified. 
> If it is determined that this is only for exception processing, the 
> default becomes null.

Well, my idea here was that we would have a single „core“ that knows how to read
the spec files, load extensions and so on. And we would create several simple
wrappers around that core, passing list of basic extensions that implement the
default behaviour, like this:

#include <plugins/core.h>

int main(int argc, char* argv[]) {
  isc::plugins::Core core;
  core.setName("auth");
  core.addPluginPath(DEFAULT_PLUGIN_PATH);
#ifdef BIND_MINIMAL
  core.loadExtension("config/standalone");
#else
  core.loadExtension("msgbus");
  core.loadExtension("config/msgbus");
#endif
  core.loadExtension("listener");
  core.loadExtension("auth");
  return (core.run());
}

> Questions:
> Can we assume that the lists will be short, say <10 elements?

I wouldn't put a strict size boundary on it, for we don't know how many
extensions a user might want, but yes, it should be short, <10 elements seems
like reasonable estimate for most real-life needs.

> What happens to an element list when the processing point it's 
> associated with goes away?
> What exactly drives element dependencies?

The dependencies are determined by both logic (it makes no sense to load
extensions for dynamically created auth data, if we don't have an auth extension
loaded), the need for an existing processing point to plug into (the auth
extension needs the listener loaded, because that one creates the processing
point used to process incoming packet somehow) and used functions (quite
anything will use some functions of the configuration manager to read the
config, so it will link against it's .so library).

> Can you get into order dependency problems?

As with anything, circular dependencies, versioning problems (A wants C in
version <=1, B wants it in >=5).

> how best to present/manage the insert at position x?

The processing point would be just a (somehow encapsulated) vector to examine
and modify?

> How will element versions and version management fit into this?

The spec file can have anything needed.

> Will this get extended to include a matching clause like a general ACL?

Sorry, I don't understand what you mean. If you're referring if we want to be
able to write ACLs in simple way in config, then, sure, yes, we should have a
default ACL implementation that understands them. If you talk about some kind of
ACLs to describe what „request“ should go into which element of the point, then
not without a huge performance penalty I guess and the elements themselfs can
decide if they want to handle the request or let it go trough. They can, of
course, use any functions (including some with ACL-like configuration) to decide
that, if it makes sense, or we can have an extension that would wrap another
call into itself. If you refer to any configuration syntax to determine which
extensions should be loaded and how, then I consider that a implementation
detail too far in future to be solved.

Have a nice day

-- 
In the name of kernel, compiler and holy penguin

Michal 'vorner' Vaner
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <https://lists.isc.org/pipermail/bind10-dev/attachments/20110205/03cfcb5b/attachment.bin>