BIND 10 #2607: DHCP Extended Testing

[BIND 10 Development]

#2607: DHCP Extended Testing
-------------------------------------+-------------------------------------
            Reporter:  stephen       |                        Owner:  tmark
                Type:  task          |                       Status:
            Priority:  medium        |  reviewing
           Component:  dhcp          |                    Milestone:
            Keywords:                |  Sprint-DHCP-20130328
           Sensitive:  0             |                   Resolution:
         Sub-Project:  DHCP          |                 CVSS Scoring:
Estimated Difficulty:  0             |              Defect Severity:  N/A
         Total Hours:  0             |  Feature Depending on Ticket:
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------
Changes (by marcin):

 * owner:  marcin => tmark


Comment:

 Reviewed commit 11bbfbe186125f95b691719221dbcfe9f941375a.

 '''v4.stability.1.txt''':

 The test procedure operates on !''example!'' which complicates the
 procedure quite a bit. I suggest that the test procedure mandates certain
 settings. This would simplify the text (so as it is easier to follow) and
 it will be consistent with the configuration file which has these values
 hardcoded. If we want to keep the test procedure more generic for any
 reason, we could maybe use explicit values and add a warning note in the
 Test Setup that !''if for any reason the subnet configuration in the test
 procedure is in conflict with the test environment, use different subnet
 settings and modify the configuration file accordingly!''

 (1.3.) !''For KEA see the file: kea.v4.stability.cfg!'': First of all it
 should be !''kea.v4.stability.1.cfg!''. Secondly, it instructs to
 configure the server while the server is not turned on (the 1.2 only
 instructed to install the server which is different). Note that kea
 support online configuration. Thirdly, there is no reference to the
 instructions how to configure the server. Perhaps, adding a reference to
 the doc/kea-test-config.txt would solve this problem.

 (4.) I suggest that we avoid shorts like !''sever lease
 statistics/logs!''. Note that it can be read as !''capture lease log!''
 instead of !''capture server log!''. I suggest splitting !''statistics!''
 and !''logs!'' into separate lines.

 (5.) The step to configure the client's interface is missed. Since
 interface configuration is explicitly mentioned for relay and server it
 should be probably also mentioned for client.

 Why not to give the instruction to run the client in the debug mode (-d)
 and redirect its output to a file on the beginning of this step? If a test
 executor fails to read the whole procedure before running the test, he may
 start the client in the non-debug mode and then realize that he has done
 wrong. This may be frustrating... especially that he may need to remove
 the lease file, perhaps restart the whole test procedure, or at least he
 doesn't know what to do.

 (6.) How do I ensure that I received leases? If I am using dhclient I have
 at least two options:
 - Check that the IPv4 address has been assigned to the interface
 - Check the lease file.

 I would prefer to have a look into lease file because dhclient may receive
 a lease but do not configure the interface as the script to configure the
 interface is not specified in the command line used to run the client.
 This step could probably give some guidance as how to check that.

 '''kea.v4.stability.1.cfg''':
 If I purely follow the instructions in the test procedure and use this
 config file I am probably going to end-up running Kea using the Memfile
 backend. In some tests I have overcome this problem by including the kea
 configuration for data base from tests/include. This file configures the
 database to use kea/kea/kea as dbname, user and password so it may not
 well fit into test lab configuration but it may be tweaked to fit. The
 backend selection is important step in the server configuration so it
 should be mentioned somewhere.

 '''v4.general.junk-reject'''
 The test identifier is !''v4.general.junk-reject.1!'' while the file name
 is !''v4.junk-reject.1.txt!''. Shouldn't they be consistent?

 (Test Setup): Perhaps !''dig!'' could be taken into double quotes so as it
 is clear it is a name of the tool. Also, the !''Dhcp6!'' should be
 replaced with !''Dhcp4!''.

 Test procedure:
 (1.1.) !''...on which test to test!'' - this is confusing.

 (2.) You may want to consider use of scripts/kea/kea-start-fresh-root.sh
 and scripts/kea/kea.py to start the server and configure it using the
 specified configuration file. This would also sort out the problem of
 collecting the bidn10 log and kea log.
 Also, the server is started but not configured.

 (3.) Everything below !''Use dig to generate DNS query packets!'' better
 fits to pass criteria rather than test procedure section.

 (Pass criteria): The second point says that there are no errors emitted
 which is contrary to the fact that we expect server to report (error) that
 it was unable to parse the packet.

 Gerenal comment: again the, server configuration does not select MySQL
 backend to store leases. It may not be the problem because this test
 should not result in creation of any lease but I would recommend running
 the test against the configuration that will be used in the production
 environment. Otherwise, we may test slightly different paths in the code
 and who knows how that may affect the test results one day.

 '''v6.stability.1.txt'''
 The same comments as above for !''v4.stability.1.txt!''

 '''v6.junk-reject.1.txt'''
 The same comments as for !''v4.junk-reject.1.txt!''

 '''both.stability.1.txt'''
 It is good approach in my opinion to make use of the other test procedures
 in the composite test procedure. This eliminates the need to maintain two
 documents having the same content. A few minor issues...

 (Test setup & Test procedure): The document says that the SAME server is
 used to run Kea. It is worth to mention that clients may or may not run on
 the same machine.

 (References): The document refers to other test procedures. Thus, they
 could be listed in the !''References!'' section.

-- 
Ticket URL: <http://bind10.isc.org/ticket/2607#comment:6>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development