<html>
<head>
<meta content="text/html; charset=UTF-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix"><br>
</div>
<blockquote cite="mid:50A50B75.6090800@isc.org" type="cite"><br>
<pre wrap="">With DHCP, a test may well require multiple clients. (And multiple
servers if you are testing DHCP failover.) In this case, it would be
good to avoid the need to log into multiple systems to start the tests.
I think that a central console that connects to the servers would be
the way to go. It does not have to be exclusively interactive: it
could also operate as a program taking arguments, e.g.
$ ./test-console
test-console> start test_45
Starting test_45 on server1 ... started
Starting test_45 on client1 ... started
Starting test_45 on client2 ... started
test-console>
or
$ ./test-console "start test_45"
Starting test_45 on server1 ... started
Starting test_45 on client1 ... started
Starting test_45 on client2 ... started
$
</pre>
</blockquote>
for now I prefer second option ;)<br>
<blockquote cite="mid:50A50B75.6090800@isc.org" type="cite">
<pre wrap="">I would advise doing something quick and easy first and getting a
first version of the framework working before doing anything clever
like an interactive creator. This would suggest some form of file.</pre>
</blockquote>
I need to learn sth about lettuce and then I will be able to create
and present first version. (for now I have just part that controls
DHCP server and generator for fake massages)<br>
<blockquote cite="mid:50A50B75.6090800@isc.org" type="cite">
<pre wrap="">Perhaps another use-case is "What needs to be in a test". Taking a
simple test, we might need the following sequence of steps:
install server(commit id 1234...) on server1
install client(commit id 1234...) on client1
install server-configuration(test45) on server1
install client-configuration(test45) on server2
start server(using test45 server command-line switches)
start client(using test45 client command-line switches)
wait 10 minutes
kill client
kill server
copy client-logs to repository
copy server-logs to repository
This seems to suggest that even a simple test is going to require some
form of sequencing. It also suggests that a test may comprise a
series of steps, and each step itself may comprise a series of
substeps (e.g. "install server" may include copying the tarball to the
server, unpacking it, configuring it, running "make", then installing it.)
</pre>
</blockquote>
Apart from the actual details of the program, first assumption, made
by Tomek Mrugalski was:<br>
<blockquote>starting server on one virtual machine<br>
on the other machine program generate fake messages (using scapy)
and check if answer is correct. <br>
testing DHCP don't bring real client to process (client like
dhclient)<br>
</blockquote>
Is it necessary to bring clients to testing environment? It would be
easier to make new tests, but in that case lettuce/scapy are useless
(and in fact that would change my thesis little bit to much). I
think that server validation should be performed on messages level
not on logs. And it's much more faster (worst case scenario we wait
just time of max_retry_attempts * timeout_value) <br>
<blockquote> </blockquote>
I didn't think that installation should be performed by program, but
it's good idea. Also concept provides that it will be only one
configuration file (or several) and test that needs different config
will just contain something to add to config file. Not one test =
one config file.<br>
<blockquote cite="mid:50A50B75.6090800@isc.org" type="cite">
<pre wrap="">In fact, for a given commit ID, the test may not be unique. It could
be that the user wants to re-run a test on the same commit ID (perhaps
anomalous results were obtained and it is suspected that something
else was running on the test machine at the same time). Under these
circumstances a date/time needs to be included.</pre>
</blockquote>
That was reason for making two different files. Tomorrow I will make
some 'example test report', it will be easier. <br>
<blockquote cite="mid:50A50B75.6090800@isc.org" type="cite">
<pre wrap="">
This is beginning to suggest that using a version number, commit ID or
a date/time for a test identification may not be the right way to go.
Perhaps a unique number (results-1, results-2 ...) with an associated
database/file listing the attributes of the tests? When accessing a
test by version/date/commit-id, a query is made on the database to
identify the relevant directory/directories.
</pre>
<pre wrap="">This would be the advantage of XML - the information about the test
could also be placed in the XML file, and what is displayed depends on
the XSLT used.
</pre>
</blockquote>
So it's time to think again about details, and rewrite some parts of
concept.<br>
<br>
Włodek Wencel<br>
<br>
<div class="moz-signature">-- <br>
Goal of Open Source is to <b> Share </b></div>
</body>
</html>