BIND 10 #1263: Finish current experimental DHCP benchmarking work

[BIND 10 Development]

#1263: Finish current experimental DHCP benchmarking work
-------------------------------------+-------------------------------------
                   Reporter:         |                 Owner:  johnd
  stephen                            |                Status:  accepted
                       Type:  task   |             Milestone:  Sprint-
                   Priority:  major  |  DHCP-20111026
                  Component:  dhcp   |            Resolution:
                   Keywords:         |             Sensitive:  0
            Defect Severity:  N/A    |           Sub-Project:  DHCP
Feature Depending on Ticket:         |  Estimated Difficulty:  0
        Add Hours to Ticket:  0      |           Total Hours:  0
                  Internal?:  0      |
-------------------------------------+-------------------------------------

Comment (by tomek):

 I have couple of comments. In DHCPv6 we need 3 transmission modes:

 - sent to All_DHCP_Relay_Agents_and_Servers link-local multicast address.
 This is the default mode for clients and so it should be the default for
 this tool.
 - sent to All_DHCP_Servers. This is the address that relays forward
 packets to. Server that supports relays will also listen on this address.
 This should be the second (in order of importance) address.
 - sent to unicast address. Servers may allow unicast configuration.

 There is also rapid-commit option that, when supported by both server and
 client, will cause SOLICIT to be answered immediately with REPLY. That is
 not needed in first version, but it is something that we should plan to
 implement later.

 There's one significant concern about how -r option will work. In
 particular that it will require interruption to finish operation. That
 will make any automated tests infeasible. That is ok if we also implement
 method to run test automatically (i.e. execute and conclude without any
 intervention, so it could be used from a script or fancier test
 environment)

 Regarding the -r option, it is useful, but it is not enough. Users will
 want to answer several different questions. This option will answer only
 "Can my server handle X requests per second?". The other often asked
 question is "How long will it take to do X?". This may or may not be
 achieved with -n option, depending on the interpretation of its
 description.

 I think the following usage scenarios are reasonable:
 - I want to send X exchanges per second and see if server can handle that
 for Y seconds/minutes/hours
 - I want to send X exchanges and see how long it will take server to
 handle them

 To meet those usages, -time (or -t) option should be added that specifies
 duration. -r, -n and -t should be allowed to be used together (or in
 different combinations).

 Other things that we should consider at a later date is turning this into
 stress testing. Let's call it --torture or similar. It starts sending data
 at some rate and increses it slowly until server starts dropping. That is
 the maximum rate the server can handle.

 There should be option to conclude (fail) the test if there is a single
 drop. We don't want to wait 12 hours to see that 5 seconds after test
 started something broke. Not sure how to implement this in the most
 convenient way. Maybe --drop-threshold that specified acceptable amount of
 dropped traffic? It seems useful to have it specified in both percentage
 and absolute numbers.

 Besides of using dhcperf as manual tool, it will also be used as automated
 test. In that case it should have clearly state if specified pass criteria
 are met or not. Something that could be easily parsed by automated
 environements.

 Make sure that return code will specify status.

 For automated test tools it is very convenient to print out command-line
 parameters. That's a practical experience. I received many logs that were
 useless because it was not possible to reproduce the problem due to
 missing information about used parameters.

 There is no --version parameter. Tool should also print out its version
 when started. See above comment about reproduction concerns.

 Another feature that could make this tool much more powerful is the
 ability to specify additional options. While it would be great to have
 custom option definition framework, for now we can do something much
 simpler. A command-line option that specifies extra payload that is
 appended to the message. A proper warning "it is user's responsibility to
 take care a proper format". For example, to specify that I want to send
 option type 100 with length 2 contaiing 0xabcd, I could do: --extra-data
 00:64:00:02:ab:cd. This seems simple enough (parse command-line + a single
 memcpy will do the trick)

 Another useful thing would to be to specify which options client should
 request. That is also not too difficult. This is just adding 8 bit(v4) or
 16bit (v6) integers to PRL or ORO, respectively. Usage could be simple:
 --option 45 --option 5.

 It would be useful to elaborate on reply verification. V4 server
 responding with NACK is ok or not? What about v6 server sending REPLY with
 status-code=no-addrs-avail? That is another thing we could eventually add
 as a feature. In some scenarios negative response as considered a proper
 one (test passed) and in others it is not (test failed). Make sure that
 the verification could be tuneable. For now it can be simple, but it will
 be more complex later.

-- 
Ticket URL: <http://bind10.isc.org/ticket/1263#comment:2>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development