BIND 10 #2488: Create DHCP Test Plan

[BIND 10 Development]

#2488: Create DHCP Test Plan
-------------------------------------+-------------------------------------
            Reporter:  stephen       |                        Owner:
                Type:  task          |  stephen
            Priority:  medium        |                       Status:
           Component:                |  reviewing
  documentation                      |                    Milestone:
            Keywords:                |  Sprint-DHCP-20130122
           Sensitive:  0             |                   Resolution:
         Sub-Project:  DHCP          |                 CVSS Scoring:
Estimated Difficulty:  0             |              Defect Severity:  N/A
         Total Hours:  0             |  Feature Depending on Ticket:
                                     |          Add Hours to Ticket:  0
                                     |                    Internal?:  0
-------------------------------------+-------------------------------------
Changes (by tomek):

 * owner:  tomek => stephen


Comment:

 Replying to [comment:9 stephen]:
 > The tickets that will run the tests include the creation of the
 appropriate documentation.  Once the tests are complete, the test
 documents will be collected together into a single document.
 Do we

 > Following the [wiki:SprintPlanningDHCP20130103 sprint planning meeting]
 of 3 January 2013, ticket #2596 has been created for this.
 Thanks. I'll take care of it. It is reasonably simple thing to do.

 > I've changed this to "requested options".  We are checking that the
 server sends the configured options and does not send the unconfigured
 ones.
 Ok.

 > > Performance tool: ability to simulate a number of clients.
 > > Need concrete values here. "A number" may mean "more than one" or "a
 million".
 > I have set that to 100 clients.  Small enough to check, large enough to
 be reasonably certain that larger numbers are supported.
 That's a reasonable number. It also defines a nice, concrete pass
 criteria. "Somewhat passed in sense" or "it depends" is not a test status.
 "Passed" or "failed" is. Having a concrete number helps a lot here.

 > > Pool exhaustion is not supported too well. Currently server will give
 up after it tries to allocate an address after 100 tries. This parameter
 is not configurable yet.
 > Does it log this? If so, the test for a debug message is OK.
 Yes, there is.

 > >> Check that a reconfiguration removing a particular address pool
 causes clients with leases in that pool to be refused a renewal and to
 obtain another address.
 > > Kea doesn't work that way. If the client gets back to us after the
 lease expired and the lease was not assigned to anyone else, we will still
 give it back to the original client.
 > So even if we say that we no longer want to allocate that particular
 address, Kea will still allocate it if a client that had previously had
 the address comes back and asks for it?
 I'm afraid so. There is no code in the reconfiguration that checks for
 existing leases and removes those that are no longer in scope. I think we
 should add a ticket for it. It is unlikely that we will be able to
 implement it before Jan 23rd, so that makes a good bullet on "known
 limitations" list in our release notes. We may update/reuse sections 16.4
 ("DHCPv4 Server Limitations") and 17.4 ("DHCPv6 Server Limitations") from
 BIND10 Guide for that purpose.

 > >> Check that the server rejects all but DISCOVER and REQUEST messages.
 > > Kea also accepts RELEASE. The code also responds with empty packets to
 all other message types. This part of the code is still what we used to
 call a skeleton.
 > "all other message types": Does this mean that it will respond to
 messages meant for the client?  The idea of the test is to check that the
 server will not respond to messages that by the protocol are not meant to
 be sent to the server.
 I should be more precise. All other messages that server is supposed to
 receive from client: DECLINE and INFORM. The complete list is in
 Dhcpv4Srv::run() method in src/bin/dhcp4/dhcpv4_srv.cc

 > >> Check that the server rejects malformed DISCOVER and REQUEST
 messages.
 > > That will be interesting to watch. :)
 > Probably as dull as watching paint dry actually.  I was thinking that we
 could use Scapy for this.
 There are so many creative ways of malforming a packet. I agree that Scapy
 is probably the way to go here. As a side effect, this will be most likely
 one of the first automated tests we'll get.

 > >> DHCPv6 features: Set up multiple subnets and pools: using different
 client characteristics, check that leases are allocated from the correct
 pool.
 > > See my comment above.
 > Should be addressed by #2596.
 Agree.

 > >> Check that a reconfiguration removing a particular address pool
 causes clients with leases in that pool to be refused a renewal and to
 obtain another address.
 > > Not supported yet.
 > Same question as before.
 Same answer as before. We can also implement this. It is not overly
 complicated, but requires implementing one extra LeaseMgr query (select
 all leases from a given subnet). Doable before 23rd, but will take away
 precious testing time.

 > > Is the perfdhcp part of the official delivery? If not, we should
 definitely trim down its validation and focus on server.
 > It is a formal part of the software.  Given the refactoring that took
 place this year, a quick check (and this is all that the check is) that it
 is working is justified.
 Ok.

 > > Two big things are missing:
 > > explicit list of clients we want to test inter-op with
 > For now, we will just say ISC DHCP clients.
 The text does not say so. It currently uses vague "if the server is able
 to inter-operate with common DHCP clients such as the ISC DHCP Client and
 the Windows DHCP client.". It will be ok if we remove "such as" words. We
 should also say what we have in mind saying ISC DHCP: 4.1-ESV, 4.2.5?
 Both?. Also, which windows version? Randomly selected one? From XP? From
 Windows 7? Vista? etc.

 > > the actual list of tests (currently this document only lists
 requirements, there are no test descriptions).
 > Test descriptions will be built up as part of the test tickets.
 Ok.

 > > For example of actual test descriptions, please take a look at Section
 5 of doc/dhcp-test-plan.txt from dhcp-val repository.
 > Will do.  We should use that as a template as we create the tests for
 the different parts of the server.
 I will send out a mail about that shortly.

 > The [DhcpTestPlan test plan] has been modified: the new version is
 version 4.
 Thanks.

-- 
Ticket URL: <http://bind10.isc.org/ticket/2488#comment:10>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development