[bind10-dev] Datasource API

[Shane Kerr]

On Wed, 2010-01-06 at 00:00 +0100, Jelte Jansen wrote:
> On 01/05/2010 08:51 PM, Shane Kerr wrote:
> > 
> > Right. We also may want to implement read-only back-ends (for example).
> > Programs that want to use the API to update data sources need to have a
> > way to check whether imports/updates are possible. (Is there a
> > difference? Can we reasonably envision a data source that we can import
> > to but not then subsequently update?)
> > 
> 
> nope, but i can certainly think of a few where both are done externally

Sure. Again, we need a way for the API to tell programs using it that a
given data source supports updates or not in this case.

> >> ps. we'll probably need a more advanced getRRset(), and i have not
> >> thought about how we could do history for ixfr at all yet
> > 
> > We spent quite a few cycles talking about this at the first BIND 10
> > face-to-face meeting. Fundamentally we need something like a transaction
> > for any data source that can be updated.
> > 
> 
> i am not as daunted by having a transaction model as i am by needing
> some form of change history for outbound ixfr...

Not all sources will support IXFR. Ones that do can implement this in
various ways, depending on what is most efficient. I'm guessing that a
diff will make sense in most cases.

Actually, maybe it makes sense to make an abstract IXFR-diff
implementation that we can use in the generic sense? There's really not
that much information:

class ixfrDiff {
private:
    serial_number_t new_serial;
    vector<namePlusRR> delRecords;
    vector<namePlusRR> addRecords;
    time_t creation_timestamp;    // needed for time-based cleanup, maybe interesting to admins 
public:
    // some accessor functions, or just make the above public?
};

Each domain would have an list of these, which could be stored as an
array, in an SQL table, or whatever.

We would probably want to have limits on how many versions to store,
based on serial number, wall clock age, number of records, memory used,
or some other heuristic. I don't know that all would apply to each data
source, but it may be worth it for simplicity sake to do so.

> > An API could look a lot like BIND 9, which basically creates an update
> > context and locks the database AIUI. This can be BEGIN/COMMIT in an SQL
> > database... although probably we want to insure we have the correct
> > transaction isolation level (serializable is best, but possibly other
> > levels are acceptable and give much improved performance... SQLite does
> > serializable by default, PostgreSQL and MySQL need a bit of convincing):
> > 
> > http://www.sqlite.org/sharedcache.html
> > http://www.postgresql.org/docs/8.4/interactive/transaction-iso.html
> > http://dev.mysql.com/doc/refman/5.5/en/set-transaction.html
> > 
> > We can make serials either implicit or explicit in this. I think keeping
> > them implicit makes sense as it allows concurrent processing, although
> > we may then need to add some special handling if the SOA itself is
> > modified.
> > 
> 
> implicit as in automatically-changed? we'll need explicits then anyway

I mean implicit as automatically-changed. So I can say:

* begin transaction
* add foo.bar A 1.2.3.4
* add foo.bar AAAA abba::
* commit transaction

And not have to care about what the serial of the zone is.

I think we only need to worry about the serial if we are updating the
SOA (like when we are receiving an AXFR or IXFR).
  
> > We need some sort of transaction, as mentioned above. We may want to
> > always have this for every operation, although conceivably some
> > shortcuts are nice. So, basically a factory method as part of the
> > DataSource:
> > 
> >     Transaction beginTrans();
> > 
> > In this case, we move almost all operations out of the DataSource class
> > and into the Transaction class...
> > 
> 
> i'm trying to aim for a single interface that needs implementation here,
> if possible...

I don't think it buys you much. If you hold the transaction information
in the DataSource class, then you can only have a single outstanding
transaction, and users of the API need to instantiate multiple objects
to allow parallel processing. If you keep the information about
transactions in a handle or something, then users of the API need to
pass that in to all method calls.

I think it's cleaner to return a transaction object, and execute methods
on that. Does that make sense?

Of course, I guess instantiating multiple DataSource objects may make
sense... that kind of matches how SQL libraries work.

> > We also need an iterator of some kind, so we can figure out the names in
> > the data source, right?
> > 
> > I am assuming fully-qualified names throughout the code - we don't have
> > add/remove zones, as to do that one would either add records of the
> > appropriate name, or remove them all.
> > 
> 
> err, yes, perhaps having non-fqdns and a zone context in which it
> operates might be better, but atm we don't have non-fqdns yet :)

Nah. I like relying on the FQDN nature of this... it seems quite clean
to me. :)

--
Shane