BIND 10 #1212: Implement IXFR-in

[BIND 10 Development]

#1212: Implement IXFR-in
-------------------------------------+-------------------------------------
                   Reporter:         |                 Owner:
  stephen                            |                Status:  new
                       Type:  task   |             Milestone:
                   Priority:  major  |  Sprint-20111011
                  Component:  xfrin  |            Resolution:
                   Keywords:         |             Sensitive:  0
            Defect Severity:  N/A    |           Sub-Project:  DNS
Feature Depending on Ticket:         |  Estimated Difficulty:  12
        Add Hours to Ticket:  0      |           Total Hours:  0
                  Internal?:  0      |
-------------------------------------+-------------------------------------

Comment (by jinmei):

 Replying to [comment:3 jelte]:
 > Whoever takes up this ticket should start out with reviewing the current
 code and RFC1995 and make a quick (and we mean quick) breakdown of the
 tasks involved, as discussed on the call today.

 I've read RFC1995 and BIND 9's xfrin implementation.

 First off, I propose a couple of simplifications toward the next
 release:
 - Don't support UDP.  In fact, if I understand it correctly even BIND
   9 doesn't support IXFR-in over UDP.
 - Don't support IXFR to AXFR fallback for now.  To help environment
   where IXFR doesn't work, introduce a (possibly short term)
   workaround of an option to disable IXFR (see below)

 Here's my proposed task breakdown:

 1. Add support for the "PRESERVE_ORDER" option in libdns++ (and
    pydnspp) Message::fromWire().
 2. A basic framework to handle RR diffs (add/delete).  This will
    eventually be used for dynamic update support.
 3. Main code logic for IXFR protocol: send IXFR query, receive
    response, and handle them (it would be nicer if we could break down
    this further, but I cannot think of any reasonable subtasking for
    this part)
 4. A small configuration extension: "disable-ixfr" option
    (preferably per zone basis)

 3 depends on 1, 2, and 4, but it should be possible to start work on 3
 without waiting for the others to be fully completed.  There should be
 no other dependencies.

 Some details:

 As for 1, see the DNS_MESSAGEPARSE_PRESERVEORDER option of BIND 9's
 dns_message_parse().  To quote the important part: "If
 DNS_MESSAGEPARSE_PRESERVEORDER is set, a separate dns_name_t object
 will be created for each RR in the message.".  This feature was
 necessary for AXFR, too, but we've worked around it with a heuristics
 hack so far.  It seems to be the time to address it.

 As for 2, I propose porting BIND 9's ixfr_putdata(), ixfr_apply(), and
 ixfr_commit() (implemented in lib/dns/xfrin.c), and
 dns_difftuple_create() and dns_diff_apply() (lib/dns/diff.c).  We
 introduce a notion of "RR diff", which (conceptually) consists of an
 operation mode (add or delete) and an RR (which would be an RRset
 object containing only one RDATA in our API).  dns_diff_apply()
 accepts a list of RR diffs, combining them into RRset diffs (for
 efficiency reasons), then make the corresponding changes on the zone
 DB.  ixfr_putdata() takes an operation (add or delete) and RR from the
 main IXFR logic, creates an RR diff object, and maintains it in a
 list.  For every 100 diffs, it calls ixfr_apply().  ixfr_apply()
 starts a new transaction (if it has already started transaction,
 continue it) and makes changes for the RR diff list accumulated so far
 under the transaction.  ixfr_commit() calls ixfr_apply() to apply any
 remaining diff and commit the transaction.

 As for 3, we'll update the do_xfrin() function of the current
 implementation:
 {{{
             if ret == XFRIN_OK:
                 logger.info(XFRIN_AXFR_TRANSFER_STARTED, self._zone_name)
                 self._send_query(RRType.AXFR())
                 isc.datasrc.sqlite3_ds.load(self._db_file,
 self._zone_name,
                                             self._handle_xfrin_response)
 }}}

 Unless IXFR is disabled, send IXFR query instead of AXFR, and instead
 of doing the sqlite3 specific stuff, implement and perform a
 completely separate code logic, where:
 - receive IXFR messages (until the xfrin process completes), parse it
   with PRESERVE_ORDER mode
 - handle each RR in the responses according to RFC1995.  call
   ixfr_putdata() for each RR, and ixfr_commit() for each
   "difference" (a single set of deletes and adds for one SOA change).
   I also suggest checking BIND 9's lib/dns/xfrin.c:xfr_rr() to know
   any corner cases and hints for tests.

 Some additional notes:
 - In our current API add/delete RRsets doesn't check data integrity.
   For example, "delete" doesn't check if the specified RRset exists.
   It shouldn't cause a problem as long as this ixfr context is the
   only writer to the zone and the remote primary server implementation
   behaves correctly, which wouldn't be unreasonable assumption in
   practice (in fact, even BIND 9 seems to rely on the accurate
   behavior of the primary server to some extent).  So at least at the
   moment I propose ignoring the error cases where given "delete" data
   doesn't exist, etc.
 - The above implementation logic uses a separate transaction for a
   single IXFR "diff".  So if the IXFR message consists of multiple
   diffs and there's another writer to the DB, the integrity may be
   broken.  We may want to discuss this point separately, but for now I
   propose implementing it this way.  Like the previous point, it will
   normally not be a problem in practice.  Also, BIND 9 seems to use
   the same logic here (using separate "transactions" for different
   diffs of a single IXFR message).

-- 
Ticket URL: <https://bind10.isc.org/ticket/1212#comment:4>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development