BIND 10 #1460: Define system-level tests for DDNS

[BIND 10 Development]

#1460: Define system-level tests for DDNS
-------------------------------------+-------------------------------------
                   Reporter:  jelte  |                 Owner:  UnAssigned
                       Type:  task   |                Status:  reviewing
                   Priority:         |             Milestone:
  medium                             |  Sprint-20120612
                  Component:  DDNS   |            Resolution:
                   Keywords:         |             Sensitive:  0
            Defect Severity:  N/A    |           Sub-Project:  DNS
Feature Depending on Ticket:  DDNS   |  Estimated Difficulty:  5
        Add Hours to Ticket:  0      |           Total Hours:  6
                  Internal?:  0      |
-------------------------------------+-------------------------------------

Comment (by jinmei):

 Replying to [comment:6 jelte]:

 > I am willing to do the adds/deletes and all their special cases at some
 point if we think it is worth it, but these things are also tested in the
 unit tests. Because of that, perhaps, if we are short on time, we may only
 want to test the inter-process parts (i.e. the bind10-specific tests), and
 the TCP test and the IXFR test (which perhaps should be considered
 b10-specific too). But for now, I've left them out before I'm spending
 more than a full day on just this document :)

 I think it makes sense to focus more on tests as the entire system
 (e.g. behavior involving multiple components) and let unit tests be
 responsible for corner case checks.  Technically, unit tests and
 system (integration) tests are conceptually different and it's not a
 good practice if we omit cases just because they are covered in a
 different type of tests, but realistically we are not a very large
 organization with dedicated QA stuff, so that would be a reasonable
 compromise.

 Comments on the wiki description follow.  This kind of task can never
 be perfect, so I suggest just fixing some trivial errors and then
 closing this ticket.

 '''General'''

 I've noticed some earlier test descriptions miss corresponding RFC
 sections.

 BIND 9's system tests have their own update cases.  From a quick look,
 I've not found so many we need to incorporate from them, but you may
 also want to take a look at it.  It's
 bind9/bin/tests/system/nsupdate/tests.sh.  One thing that drew my
 attention is this:

 {{{
 # inserting an NS record without a corresponding A or AAAA record should
 fail
 $NSUPDATE -l -p 5300 -k ns1/session.key > nsupdate.out 2>&1 << END &&
 status=1
 update add other.nil. 600 in ns ns3.other.nil.
 send
 END
 grep REFUSED nsupdate.out > /dev/null 2>&1 || status=1
 }}}

 although the underlying implementation uses the generic zone
 management module, so in our case it's probably more reasonable to
 hold off until we do generic zone loading with post-load checks.

 '''Module tests'''

 - Is there any reason why we use in-memory over DB (sqlite3 in practice)?
 - note: Tests 1 will need #1986.  With #1986 completed, I think the
   RCODE can be of implementator's choice.  I'd slightly prefer NOTIMP,
   but REFUSED would also be okay considering it's one form of 'reject
   all' ACL.
 - We may want to test what should happen if a running b10-ddns is
   stopped.

 '''SOA serial tests'''

 - test 4: "e.g." should be add_1_4?

 - test 6: I think it can be implementation dependent, but BIND 9 seems
   to avoid SOA increment if the diff ends up being empty.
 {{{#!c
         /*
          * If any changes were made, increment the SOA serial number,
          * update RRSIGs and NSECs (if zone is secure), and write the
 update
          * to the journal.
          */
 }}}
 - We may want to test this, although this level of thing should also
   be covered in unit tests:
 {{{
    7.11. A zone's SOA SERIAL should never be set to zero (0) due to
    interoperability problems with some older but widely installed
    implementations of DNS.  When incrementing an SOA SERIAL, if the
    result of the increment is zero (0) (as will be true when wrapping
    around 2**32), it is necessary to increment it again or set it to one
    (1).  See [RFC1982] for more detail on this subject.
 }}}
 - There can be a case where the zone doesn't even have an SOA for
   DB-based data sources (in-memory should ensure that it never
   happens).

 '''Zone section tests'''
 - There's another case: the specified zone is a secondary.

 '''Prerequisite section tests'''

 From a quick look there doesn't seem to be anything obviously wrong,
 although I have to admit I didn't check the tests are comprehensive
 (and, in practice, we may not necessarily need to be 100%
 comprehensive in system tests as long as they are covered in unit
 tests) or they don't have any minor errors.

 '''Update section tests'''

 - it doesn't seem to contain post-prescan things.  is the intent that
   they should be covered in other cases?
 - what about special cases such as deleting apex NS?

 '''Prescan failures'''

 - Shouldn't the "additional section"s be "update section"? (the update
   section is ddns-equivalent of the answer section, btw)
 - (maybe it's more about unit-level tests) what about MAILA/MAILB?

 '''DDNS over TCP tests'''

 - note: we may need to develop our own tool for this test.

 - and, if possible, we might want to check what happens if the test
   client tries to keep the TCP connection for multiple DDNS
   transactions (I suspect b10-ddns won't support such an operation
   even after we complete #2003)

-- 
Ticket URL: <http://bind10.isc.org/ticket/1460#comment:7>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development