DHCP Validation project - testing framework

[Włodzimierz Wencel]

Hello,
first I want to apologize for so long silent time, too much work on 
university.  I just returned to work over this project. Tomorrow I will 
update wiki.
>
> Hi,
>
> A few quick comments following the other thread that you have with
> Stephen....
>
> *Design*
> I think that *Requirements* will be a better name for the introductory
> section like this. Design is more about how you want to fulfil requirements.
>
> Extensibility: It is indeed not for DHCPv6 only. It should work for
> whatever DHCP implementation.
>
> *Test Schema*
> I am seeeing some weakness in the example you have given. In fact, from
> the client perspective the server behaves correctly: it sent a couple of
> solicits and got no reponse which suggests that server complies with the
> RFC. However, if the server crashed after receiving first solicit
> without DUID (because programmer made a wrong assumption that DUID
> should be there and tried to read it without checking that it is
> present) test should be marked as FAILED. If it is marked as PASSED and
> you fire-up another tests using a dead server you end-up having all
> subsequent tests failed while in fact only the first one was FAILED.
> This brings another question, how do you validate that the server is
> running before you run each test case. You should probably send some
> valid message and see if it replies. This makes simple test a bit more
> complex but the good think about it is that you can use the same check
> methodology for each test.
Yes, I thought about that.
> I think we can have multiple tests covering specific requirements within
> a section. Thus if you want to run the test by specifying "RFC section"
> you have to be prepared that multiple tests will have to be executed.
Of course, I never assumed that RFC section means one test.
> *Running tests*
> The --SAVE option makes the script "save the user configuration" but it
> is not explain what the configuration is. Is it a dump of the command
> line? How do I reuse this dumped configuration when running test again.
>
> I can see that there is a note about "config files" under "modes:"
> bullet but it would be nice to have an example or a template of a config
> file. This is a good occasion to think through what you really want to
> make configurable for the user.
>
> Since you already refer to the specific options that are exposed by the
> command line, maybe it is worth to add separate section describing the
> (preliminary) command line in the form of "man page" or output from
> "--help command"?
Certainly I need to re-write most of the wiki, I will try to be write 
more clearly.
> *Adding new tests*
> What atcually it is to "generate test file with blank space"? Is this a
> piece of code being generated like "function name, parameters and empty
> body" and the function body is where you have to put the test logic in
> python? I am wondering if it is really needed feature given that there
> are large number of tests to be written and those obviously have higher
> priority.
Yes, that's for future use, and still optional.


-- 
Goal of Open Source is to *Share *
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.isc.org/pipermail/bind10-dhcp/attachments/20121205/aecafdd5/attachment.html>