[bind10-dev] Ruminations on difference normalization (#1456)

[Jelte Jansen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hey,

I was just jotting down some notes on how to implement #1456 (Create
diff-normalizer utility functions, as needed for DDNS). I had some
ideas, and we had a bit of discussion on Jabber, but so far the results
are inconclusive.

Since I won't be starting the implementation today anyway, I thought
this would be a nice thing to share :)

(Edit: ok this email turned out to be quite long, sorry for that, please
bear with me)

First off, the requirements;

- - In order to support IXFR-out, we need to feed our current ZoneUpdater
  data in a fixed format (old SOA, deletions, new SOA, additions,
  repeat last two as applicable)
- - This is certainly needed for DDNS, but ideally, we would have such
  information optionally generated on AXFR-in and b10-loadzone as well.
- - If so, and the updated data does not update the SOA record, we need
  to do it automatically with an incremented serial value.

So the original approach I had in mind was to create a class that we can
give additions and deletions, and ask whether those update the SOA or
not. Then ask it to give us the changes in the format that our
ZoneUpdater requires to support IXFR-out. Or give it a ZoneUpdater and
tell it to apply all that it knows.

And, keeping the future in mind, it could perhaps also just be given a
full new zone and an updater, and it could figure out what the
differences between them are itself, and apply those as one update
(giving us ixfr-from-differences functionality).

I was jotting down the interface for this, and couldn't help but notice
that it actually looks a lot like the ZoneUpdater interface itself.

So we got to discussing; what if we don't add an external structure for
this, but put this functionality in the ZoneUpdater itself?

It would not do any changes to the current diffs table directly, but
keep track of all the additions and deletions, and apply them all
correctly when commit() is called.

This would of course require that the update is only for 1 SOA change,
so for IXFR-in it would need slightly different behaviour (i.e. the
current behaviour).

So in this approach, we would extend ZoneUpdater with the following
functionality (i'm naming tables below as the current implementation
lives in database.cc, and it is a bit clearer for me);

- - it would get a 'single-update' mode (or perhaps the other way around,
it would get an IXFR-mode in which case it would have the current behaviour)
- - when in single-update mode, and if journaling is supported, and if
'replace' is false, it would apply the additions and deletions to the
*records* table directly, but it would keep the changes in a data
structure instead of applying them directly to the *diffs* table
- - when commit is called, *then* it would create all the table
modifications for the diffs table, and apply them
- - if all is ok, it would perform the actual commit.

And we either need to extend it with a way to ask it if the SOA has been
updated, or give it yet another option (an update-soa-policy, there are
several ways you can update serial values in)

And for the ixfr-from-differences approach; when replace is true, it
needs to be even smarter (for each addition, it needs to see if the
record existed. And for any record that existed in the old data but not
in the new it needs to create a deletion record, etc.

Compared to the original separate structure idea, afaict this has the
following pros and cons;



Separate Structure:

pro:
- - easier to build right now. Especially if we do it in Python.

con:
- - from a high level, the interfaces would look very similar
- - unnecessary steps; looping over the same data we just read to create
the Differences structure, to 'transfer' it from the Differences
structure into the Updater.
- - when done in C++, yet another thing we'll need to wrap



Extending ZoneUpdater:

pro:
- - any tool or module that uses the updater API can do it whilst
supporting IXFR-out, without any requirements whatsoever (on the
'user's' side that is, naturally the datasource would need to support
it, etc.). Immediate thoughts would be b10-ddns (and with the
from-differences approach, AXFR-in and b10-loadzone as well). But also
whatever we will build for auto-dnssec later, and dhcp-updates, and any
other cool module someone comes up with later.

con:

- - this may make the ZoneUpdater smarter than it necessarily needs to be.
It already has a few options that change its behaviour and this would
add at least two more (and maybe more).
- - the current zoneupdater is also database-api only so depending on what
internal structure we make for this we may find that the 'direct'
implementations (i.e. in-memory) need a bit more work as well, when we
add this there.



There is, of course, another alternative. We could also add a second
type of Updater to DataSourceClient; one that does all the above, and
then calls the 'real' updater. Let's call it SmartUpdater for now.
Some modules would call the current 'raw' updater, and give it its
information in the correct form. Other, 'lazy', modules would call the
SmartUpdater, which would have a pretty similar interface, but would not
modify any data directly. When it is told to commit, it would call the
'real' updater.

pro:
- - no changes necessary to ZoneUpdater (also no extra smartness necessary
there)

con:
- - external datasource interface gets larger, and new externally visible
classes (by lack of a better name, the SmartUpdater)
- - need wrappers for added classes

Any thoughts?


Jelte
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7ozEIACgkQ4nZCKsdOncVMPQCeKcsAsj2/JNIdHOFhKaPzY3BH
iQUAn0jU37ZFOFAQxBRs4h0q1mlkAtPO
=HqY1
-----END PGP SIGNATURE-----