BIND 10 #2607: DHCP Extended Testing

[BIND 10 Development]

#2607: DHCP Extended Testing
-------------------------------------+-------------------------------------
            Reporter:  stephen       |                        Owner:  tmark
                Type:  task          |                       Status:
            Priority:  medium        |  reviewing
           Component:  dhcp          |                    Milestone:
            Keywords:                |  Sprint-DHCP-20130328
           Sensitive:  0             |                   Resolution:
         Sub-Project:  DHCP          |                 CVSS Scoring:
Estimated Difficulty:  0             |              Defect Severity:  N/A
         Total Hours:  0             |  Feature Depending on Ticket:
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------

Comment (by tmark):

 >
 >  '''v4.stability.1.txt''':
 >
 >  The test procedure operates on !''example!'' which complicates the
 >  procedure quite a bit. I suggest that the test procedure mandates
 certain
 >  settings. This would simplify the text (so as it is easier to follow)
 and
 >  it will be consistent with the configuration file which has these
 values
 >  hardcoded. If we want to keep the test procedure more generic for any
 >  reason, we could maybe use explicit values and add a warning note in
 the
 >  Test Setup that !''if for any reason the subnet configuration in the
 test
 >  procedure is in conflict with the test environment, use different
 subnet
 >  settings and modify the configuration file accordingly!''

 Procedure restructed to mandate values and warning included.
 >
 >  (1.3.) !''For KEA see the file: kea.v4.stability.cfg!'': First of all
 it
 >  should be !''kea.v4.stability.1.cfg!''. Secondly, it instructs to
 >  configure the server while the server is not turned on (the 1.2 only
 >  instructed to install the server which is different). Note that kea
 >  support online configuration. Thirdly, there is no reference to the
 >  instructions how to configure the server. Perhaps, adding a reference
 to
 >  the doc/kea-test-config.txt would solve this problem.
 >

 Done.

 >  (4.) I suggest that we avoid shorts like !''sever lease
 >  statistics/logs!''. Note that it can be read as !''capture lease log!''
 >  instead of !''capture server log!''. I suggest splitting
 !''statistics!''
 >  and !''logs!'' into separate lines.
 >

 Clarified.

 >  (5.) The step to configure the client's interface is missed. Since
 >  interface configuration is explicitly mentioned for relay and server it
 >  should be probably also mentioned for client.

 Added Client section.

 >
 >  Why not to give the instruction to run the client in the debug mode
 (-d)
 >  and redirect its output to a file on the beginning of this step? If a
 test
 >  executor fails to read the whole procedure before running the test, he
 may
 >  start the client in the non-debug mode and then realize that he has
 done
 >  wrong. This may be frustrating... especially that he may need to remove
 >  the lease file, perhaps restart the whole test procedure, or at least
 he
 >  doesn't know what to do.
 >
 >  (6.) How do I ensure that I received leases? If I am using dhclient I
 have
 >  at least two options:
 >  - Check that the IPv4 address has been assigned to the interface
 >  - Check the lease file.
 >
 >  I would prefer to have a look into lease file because dhclient may
 receive
 >  a lease but do not configure the interface as the script to configure
 the
 >  interface is not specified in the command line used to run the client.
 >  This step could probably give some guidance as how to check that.

 Done.

 >
 >  '''kea.v4.stability.1.cfg''':
 >  If I purely follow the instructions in the test procedure and use this
 >  config file I am probably going to end-up running Kea using the Memfile
 >  backend. In some tests I have overcome this problem by including the
 kea
 >  configuration for data base from tests/include. This file configures
 the
 >  database to use kea/kea/kea as dbname, user and password so it may not
 >  well fit into test lab configuration but it may be tweaked to fit. The
 >  backend selection is important step in the server configuration so it
 >  should be mentioned somewhere.
 >

 Added section under Server configuration to address this.



 >  '''v4.general.junk-reject'''
 >  The test identifier is !''v4.general.junk-reject.1!'' while the file
 name
 >  is !''v4.junk-reject.1.txt!''. Shouldn't they be consistent?

 Corrected.

 >
 >  (Test Setup): Perhaps !''dig!'' could be taken into double quotes so as
 it
 >  is clear it is a name of the tool. Also, the !''Dhcp6!'' should be
 >  replaced with !''Dhcp4!''.
 >
 >  Test procedure:
 >  (1.1.) !''...on which test to test!'' - this is confusing.

 Corrected.

 >
 >  (2.) You may want to consider use of scripts/kea/kea-start-fresh-
 root.sh
 >  and scripts/kea/kea.py to start the server and configure it using the
 >  specified configuration file. This would also sort out the problem of
 >  collecting the bidn10 log and kea log.
 >  Also, the server is started but not configured.


 I will leave this to be addressed as part of the automation effort. In the
 meantime I added text to describe how to start kea to capture the output.

 >
 >  (3.) Everything below !''Use dig to generate DNS query packets!''
 better
 >  fits to pass criteria rather than test procedure section.

 Program output content moved to pass criteria section.
 >
 >  (Pass criteria): The second point says that there are no errors emitted
 >  which is contrary to the fact that we expect server to report (error)
 that
 >  it was unable to parse the packet.

 Amended the statement with "other than those expected"...

 >
 >  Gerenal comment: again the, server configuration does not select MySQL
 >  backend to store leases. It may not be the problem because this test
 >  should not result in creation of any lease but I would recommend
 running
 >  the test against the configuration that will be used in the production
 >  environment. Otherwise, we may test slightly different paths in the
 code
 >  and who knows how that may affect the test results one day.

 Added configuration step to setup MySQL backend.


 >
 >  '''v6.stability.1.txt'''
 >  The same comments as above for !''v4.stability.1.txt!''

 Done.


 >
 >  '''v6.junk-reject.1.txt'''
 >  The same comments as for !''v4.junk-reject.1.txt!''

 Done.

 >
 >  '''both.stability.1.txt'''
 >  It is good approach in my opinion to make use of the other test
 procedures
 >  in the composite test procedure. This eliminates the need to maintain
 two
 >  documents having the same content. A few minor issues...
 >
 >  (Test setup & Test procedure): The document says that the SAME server
 is
 >  used to run Kea. It is worth to mention that clients may or may not run
 on
 >  the same machine.


 Done

-- 
Ticket URL: <http://bind10.isc.org/ticket/2607#comment:7>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development