BIND 10 #2598: DHCP Testing: Start/Stop

[BIND 10 Development]

#2598: DHCP Testing: Start/Stop
-------------------------------------+-------------------------------------
            Reporter:  stephen       |                        Owner:
                Type:  task          |  jwright
            Priority:  medium        |                       Status:
           Component:  dhcp          |  reviewing
            Keywords:                |                    Milestone:
           Sensitive:  0             |  Sprint-DHCP-20130214
         Sub-Project:  DHCP          |                   Resolution:
Estimated Difficulty:  0             |                 CVSS Scoring:
         Total Hours:  0             |              Defect Severity:  N/A
                                     |  Feature Depending on Ticket:
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------
Changes (by tmark):

 * owner:  tmark => jwright


Comment:

 --------------------------------------------------------------------------------

 General Comments:

 Since v4.basic.start-stop.* and v6.basic.start-stop*  tests are
 essentially identical, comments below apply to both sets of tests
 even though only v4 tests are cited.

 The following comments apply to several of the tests:

 G1 Lack of runtime artifacts make reviewing the tests as they stand
 incomplete.
    Test procedures do not instruct the tester what artifacts should be
 collected.

 G2 Several tests mention starting bind10 via bindctl. This actually can't
 be done.

 G3 Tests say to verify something has started or stopped but do not
 indicate how.
    Verify by chekcing bind10 output, verify by using "ps"?

 G4 I think these tests could be consolidated:

         v4.basic.start-stop.stop.txt
         v4.basic.start-stop.start.txt
         v4.basic.start-stop.start-stop-repeatedly.txt

 should be one test, v4.basic.start-stop.1.txt, with the primary test to
 start/stop via bindctl, and a an additional test case of repeatedly
 start/stop
 via bindctl.

 and these tests:

         v4.basic.start-stop.start-w-bind.txt
         v4.basic.start-stop.start-stop-w-bind-repeatedly.txt
         v4.basic.start-stop.stop-w-bind.txt

 should be one test, v4.basic.start-stop.2.txt, with the primary test to
 start/stop with bind, and a an additional test case of repeatedly
 start/stop
 with bind.

 --------------------------------------------------------------------------------
     Specific Test reviews:
 --------------------------------------------------------------------------------

 v4.basic.start-stop.start-stop-repeatedly.txt

 Outcome:

     Test should be sufficient to test the intended functionality.
     No artifacts to verify.

 Comments:

     Test setup mentions "script available to repeatedly start/stop" but
     does not supply the script.

     Test procedure has steps to start/stop Dhcp4 after exectuing the
     script. Why?

     Review in light of general comments.

 --------------------------------------------------------------------------------
 v4.basic.start-stop.start-stop-w-bind-repeatedly.txt

 Outcome:

     Test should be sufficient to test the intended functionality.
     No artifacts to verify.

 Comments:

     Test setup mentions "script available to repeatedly start/stop" but
     does not supply the script.

     Test procedure has steps to start/stop Dhcp4 via bindclt after
     exectuing the script. Why?

     Review in light of general comments.

 --------------------------------------------------------------------------------

 v4.basic.start-stop.start-w-bind

 Outcome:

     Test should be sufficient to test the intended functionality.
     No artifacts to verify.

 Comments:

     The test description states:

     "Verify that the DHCPv4 server starts with the whole BIND10, if
 configured
     to do so"

     You cannot configure the Dhcp4 server to not start with BIND10. It is
 either
     configured and will be started or it isn't configured at all.  What
 you mean
     is:

     "Verify that, when the DHCPv4 server is configured,  BIND10 starts the
 server
      as part of the BIND10 startup process"

     Review in light of general comments.

 --------------------------------------------------------------------------------
 v4.basic.start-stop.stop-w-bind

 Outcome:

     Test should be sufficient to test the intended functionality.
     No artifacts to verify.

 Comments:

     The test description states:

     "Verify that the DHCPv4 server stops with the whole BIND10, if
 configured to
     do so"

     You cannot configure the Dhcp4 server to stop with BIND10. It is
 either
     configured and running or it isn't. What you mean is:

     "Verify that, when the DHCPv4 server is configured,  BIND10 stops the
 server
     as part of the BIND10 shutdown process"

     Review in light of general comments.

 --------------------------------------------------------------------------------
 v4.basic.start-stop.start

 Outcome:

     Test should be sufficient to test the intended functionality.
     No artifacts to verify.

 Comments:

     Review in light of general comments.

 --------------------------------------------------------------------------------
 v4.basic.start-stop.stop

 Outcome:

     Test should be sufficient to test the intended functionality.
     No artifacts to verify.

 Comments:

     Review in light of general comments.

-- 
Ticket URL: <http://bind10.isc.org/ticket/2598#comment:7>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development