[bind10-dev] config data api

[Jelte Jansen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


OK so I promised to send round a proposal for updating the
configuration data API, This was already getting quite long, so I
decided to send this round now; it is just a set of requirements (some
of which are not new) and an approach as to how to implement them
without needing to break everything to get there. Most of it is not
fully formed yet, but in the spirit of release-early-release-often
it?s probably better to get some feedback now.

Note, this is just about the data API, not the communications and
tie-in to ModuleCCSession, though parts of this text may touch that.
It is also not about config wizards and such (but once again, some
additions here might make that easier to do too).

The biggest reason to think about this now is that we are running into
all kinds of small problems in bindctl, and modules have trouble
easily finding some values (they need access to the full
moduleccsession object to get defaults, or, if we go with jinmei?s
proposal, which i wouldn?t mind, they need access to the configdata
object. I want the code to have direct access to the value, whether it
is custom or default, or null).



So, initial requirements braindump:

- - need to be able to represent ?changes? (modifications, additions,
removals)
- - need to be able to count on content structure (i.e. data must be
validated according to .spec file, and to check whether it has been
validated)
- - need to be able to easily access ?actual? data (i.e. custom setting
or default)
- - need to be able to check whether the value was default
- - need to be able to override some values for in-source use
- - need to be able to access ?deeper? parts of the structure, both
directly as children from an element and an ?absolute? and relative
?path? in the data (e.g. the current identifier functions, and
Element::find())
- - identifiers must support (nested) maps, and (nested) lists
- - possibly: case insensitivity in identifiers?
- - we need extensible/additional ?types? (e.g. IP address, domain name)
- - while we are at it, we should clarify or redefine ?optional? and
?default? values


on the bindctl side:
- - must have easy way to enumerate items, and values (not necessarily
by the objects themselves, but the api must at least allow for it);
both ?possible items? (i.e. spec), and ?actual values? (i.e. list and
named-set entries). This is not only necessary for ?config show?-like
behaviour, but also for tab-completion (which is currently broken
inside a list element)
- - should be able to just send the actual changes (it does so partly
now, but not really well)

on module side:
- - when a (set of) changes come in, it could either check the changes
directly for non-syntactic validity, or merge them with current config
and check the whole set of config data. This merge should not be final
until after validation succeeds (as told by the module, in fact, once
we have this maybe we can implement a three-phase commit approach and
a full history of changes in a non-hackish way)



Proposal:

Currently, we are working mostly with ?raw types? in Python; on disk
settings and specs are represented in JSON, and these are parsed and
then used directly. Because a lot of operations depend on the kind of
data they operate on this has made the additions we have made a bit
messy, to say the least. On the C++ side we have the Element set of
classes, which is a little bit better, but can still be improved quite
a bit.

So the proposal is to extend the Element classes, and make a similar
set (perhaps through python/C) for the python side (though i am
thinking we may want stuff in python we don?t need in c++, on top of
the basics).

The general idea is, that once you have such an Element class, you can
check that it has been validated, and then rely on the structure of
the data. Validation also ties the relevant specification for an
element to the instance, so you can get to default values without the
hassle you need now.

So for starters, we need classes that are direct representations of
whatever is in the spec files; i.e. not just a wrapper for the entire
specification, (ModuleSpec), but ElementSpecification (containing
things like name, type, optional, default value), and depending on
type, certain operations (e.g. getChild, enumerate).

Given an ElementSpecification, we need a way to validate data, and for
nested structures such as lists and maps, it should validate
recursively. When validation was successful, a pointer to the
ElementSpecification is set in the Element itself.

An ElementSpecification should also be able to produce an Element,
with default data (for instance when you add an element to a list).
(note that this is different than the default value of the
ElementSpecification itself, which in the case of a list could be an
empty list. this is also an area that has been confusing at times).

Some ?raw? types can conform to multiple Element types; for instance,
both domain names and strings are represented as strings ?on the wire?
(in json format). Names just have additional syntactic requirements.



Proposed approach:

- - add ElementSpecification classes (a base class and subclasses for
that), the current ModuleSpec would still be a container for the
several specification parts of a module, but instead of returning
ElementPtrs for the different gets it would return
ElementSpecificationPtrs.
- - create generator function (from json) for those
- - add a ptr to Element linking to the above classes when validated

(we can do this without changing any of the existing code, btw, as
long as we don?t reuse method names, although once it?s there we can
start using it in the way we use the raw data itself)

We also need to create python wrappers for this, or do the same in the
python implementation.

With those, we can start fixing bindctl for real, it should now be
able to use these classes instead of mucking around with multiple sets
of raw data (the first thing i?d redo is the tab completion part,
which would still not be trivial btw, that is some weird interfacing
in readline).

Then we can
- - define and create ?difference? classes for those element types that
need it (for basic elements, a ?difference? would just be a new value,
but for lists and maps, it would be additions, removals, and updates)
- - define a representation format for those
- - extend the Element classes to ?handle? them (e.g. applyChange(), as
opposed to the currently scary ?merge? static function)
- - start extending our types, and maybe even add custom syntactic
requirements to elements from the specification (this has been a plan
before, but never materialized)
- - be all happy and fluffy about all the cool things this will allow.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk+G6M8ACgkQ4nZCKsdOncUkIACgt6RHQvzPKG0DstNucGnew2+8
ckEAoKplfTFty6J3WgJevR5I2g0T3PrR
=LbYt
-----END PGP SIGNATURE-----