BIND 10 #2601: DHCP Testing: V6 features - general
BIND 10 Development
do-not-reply at isc.org
Wed Mar 6 18:38:04 UTC 2013
#2601: DHCP Testing: V6 features - general
-------------------------------------+-------------------------------------
Reporter: stephen | Owner: tmark
Type: task | Status:
Priority: medium | reviewing
Component: dhcp | Milestone:
Keywords: | Sprint-DHCP-20130328
Sensitive: 0 | Resolution:
Sub-Project: DHCP | CVSS Scoring:
Estimated Difficulty: 0 | Defect Severity: N/A
Total Hours: 0 | Feature Depending on Ticket:
| Add Hours to Ticket: 0
| Internal?: 0
-------------------------------------+-------------------------------------
Changes (by marcin):
* owner: marcin => tmark
Comment:
Replying to [comment:4 tmark]:
> Comments on kea-test-config.txt:
>
> One of the premises set for in "kea-test-config.txt" is the bind10 is
always installed
> --prefix=/opt/bind10. This is contradictory to the goal of supporting
multiple machines within
> a test bed, where each machine may be part of more than one test bed,
and hosts more than
> one installation. Lab topology has generally be stated as each test
machine has more than one
> test area:
>
> var/test_1
> var/test_2
> var/test_n
>
> and so on. The intent being that each area could be supporting
different versions and or software under
> test. Pinning the tests to /opt/bind10 becomes an issue.
> Test tools/scripts should parameterize these types of things.
I have updated the kea-test-config.txt to reflect that BIND10 can be
installed in different locations and that different database, user name
etc. can be used in MySQL. I also added the -p parameter to kea.py which
can be used to specify the bind10 install dir.
>
> The same applies with MySQL configuration. The database name should be
flexible. It is always "Kea" than
> a single machine can only support a single instance of bind10 under
test. I suggest the name of the database
> track with the installation directory (i.e. test area).
That's valid point. However, until we automate these tests, we have a set
of hardcoded configuration files where we need to explicitly give the
database credentials. So, I left 3x kea in these configuration files and
if somebody wants to run that wherever else, he will need to modify them.
>
> ---------------------------------------------------------------
>
> General Comments on tests themselves:
>
> The following comments apply to several of the tests:
>
> G1. Is it necessary to use "the latest version" of dhclient or is there
a minimum version?
> I think specifying a minimum is perhaps better. Theoretically it
shouldn't matter other
> than command line option support of -6 and -r etc.
Well, if you say that minimum version is 4.2.5 than if 4.2.6 is released
you need to update the test descriptions. Obviously, you could leave 4.2.5
but why not to use the latest version which usually is better? I am in
favor of testing software using latest available versions of it.
>
> It is a little unclear as to the intent of phrase such as:
>
> " ...file which name matches the "*.v6.general.release.1.cfg" pattern.
"
>
> Why the "*" and not the actual file name? "kea.v6.general.release.1.cfg"
I was trying to create a test description which is relevant for either Kea
of DHCP4. Thus, I used a kind of pseudo code that maps to the regexp in
Python or perl. The text directly translates to the Python if we wanted to
automate that. However, I admit that it is a bit weird sentence for the
manual test procedure so I changed that as suggested.
>
>
> G2. The test procedure does not actually instruct you to start the
server. I suppose
> to "configure it" one must start it but this is only true for Kea. I
was under the impression,
> the test descriptions should be product agnostic for the most part,but
include specifics for kea where
> they apply. (Obviously not all tests apply to both).
Yes, it is only true for Kea but the opposite is only true for DHCP4. :-)
It is sometimes hard to create a common test procedure for the two
products that are so much different...
I added the bullet that instructs to run the server if it supports online
configuration before configuring it. And another one to start the server
if configuration is done and the server is not running. This is not
perfect but hopefully better than it used to be...
>
> G3. Pass critieria for tests that call of client lease file
verification,
> should perhaps also require server lease table dumps.
I added a bullet in pass criteria to check the tables in the server side.
>
> G4. Test procedures that instruct user to use the -d option when running
dhclient do
> include instructions to capture client output. Client output is present
for some tests
> and not others. It is very helpful to have the client output as
further validation.
> Would recommend amending the client invocation line to include something
like:
>
> " 2>&1 | tee -a client.out"
Added.
>
> Some test cases collected it, others didn't. I think all of them should.
Exceptions
> might be tests that run for very long periods of time and generate large
output.
>
>
> G5. It may be helpful to increase Server logging levels. Currently what
is captured demonstrates only
> that nothing catastrophic occurred
I don't know. Kea logs errors, warnings and infos by default. I don't
think it is really necessary to turn on debug mode for each test.
>
> ---------------------------------------------------------------
> Specific Test reviews:
> ---------------------------------------------------------------
> v6.general.negative -
>
> Outcome:
>
>
> Recorded output for Case 1 is missing, and output for Case 3 seems
incorrect.
>
> I ran both tests using bindctl and capture the following output
> which verifies these two cases are handled correctly.
>
> Captured using case 1:
>
> execute file v6.general.negative1.cfg
> Error: Configuration parsing failed: failed to parse pool
definition:.
> Does not contain - (for min-max) nor / (prefix/len)
> Configuration not committed
Sorry, I did not catch why you claim that the output is missing? The
reported error logs good to me.
>
>
> Captured bindctl output using case 3:
>
> execute file v6.general.negative3.cfg
> Error: Configuration parsing failed: Mandatory subnet definition in
subnet missing
> Configuration not committed
Why is this invalid in your opinion?
>
> The remaining cases all check out ok.
> The test verifies the behavior as intended.
>
> Comments:
>
> What about subnet/options? Am assuming that subnet options by
themselves cannot invalidate a subnet.
I thought that subnet/options should be checked with other test cases
covering options.
>
> Review in light of general comments.
>
>
---------------------------------------------------------------------------------
> v6.general.release.1
>
> Outcome:
>
> Question: In the dhcp6.leases file, the starts value is identical in
both entries.
> Is this correct?
It is, because one entry is created when a lease is acquired, another one
when the SAME lease is released. They both refer to the same lease so it
should be ok that starts is the same.
>
> Comments:
>
> Review in light of general comments.
>
>
-------------------------------------------------------------------------------
>
> v6.general.renew.1
>
> Outcome:
>
> This test verifies the behavior as intended.
>
> Comments:
>
> Under pass criteria, it states:
>
> "- All the entries have the same values, except that the field
> 'starts' differs for each lease entry."
>
> You may wish to amend this to say "start times should be
successively by approximately
> the renew timer value"
Changed as suggested.
>
> Review in light of general comments.
>
>
-------------------------------------------------------------------------------
> 6.general.subnet.single
>
> This test verifies the behavior as intended.
>
> Outcome:
>
> Pass Criteria states that the lease file and backup lease file
should match, but starts value
> is different for between .lease .backup.lease (all three test
cases). Is the correct?
> If so, the pass critieria should state that.
The Pass Criteria only says that addresses should match, which is true.
>
> Comments:
>
> Under "Cases", the text implies two additional runs of the test
using kea.v6.general.subnet.2.cfg.
> It isn't completely clear that there are two distinct, additional
cases, perhaps number them
> case 2, case 3....
Added.
>
> They should also specify any modifications/additions to the Pass
Critieria.
> The second and third test cases will result in lease data that does
not match the "Pass Critieria"
> for the first test case.
I changed the config file so as the pool is the same.
>
> In the test artifacts collected, the interface was "eth2" in all
three test cases. There is
> no way to know how the server came to use eth2. Perhaps capturing
bind10 config.db or using
> a different interface between case 1 and case 2 would be help.
What are the artifacts you're referring to? The interface name is specific
to the machine where the test is being run.
>
>
> Review in light of general comments.
>
>
>
---------------------------------------------------------------------------------------
> v6.general.subnets.two
>
>
> 1. v6.general.subnets.two.1
>
> Outcome:
>
> This test verifies the behavior as intended.
>
> Comments:
>
> Review in light of general comments.
>
>
>
> 2. v6.general.subnets.two.2
>
> Outcome:
>
> The test verifies the behavior as intended.
>
>
> Comments:
> Pass Criteria states there will 10 leases in the lease files, when
in fact
> there are 12. (12 clients, 12 leases).
I think it is because the physical interface (which has 5 pseudo
interfaces on it) also gets a lease. I modified the procedure to generate
only 4 pseudo interfaces.
>
> The configuration uses nearly identical subnets on two "interfaces"
but client
> logs show them both as "eth2". I would recommend requiring the use
of two different
> interfaces and/or altering the subnet/address pools so that are
visually more distinct.
> It would make the lease files a easier to verify.
I am not sure I understand the issue. I don't think that making two pools
like 2001:db8:1::/64 and 2001:db8:123::/64 would make it much clearer. On
the other hand the 2001:db8::/64 is the prefix that we usually use for
testing so I didn't want to use anything else for any of them.
>
> Review in light of general comments.
>
>
---------------------------------------------------------------------------------------
> v6.general.timers
>
> These tests are straight forward.
>
> 1. v6.general.timers.1
>
> Outcome:
>
> The test verifies the behavior as intended.
>
> Comments:
>
> Review in light of general comments.
>
> 2. v6.general.timers.2
>
> Outcome:
>
> The test verifies the behavior as intended.
>
> Comments:
>
> Review in light of general comments.
>
> 3. v6.general.timers.3
>
> Outcome:
>
> The test verifies the behavior as intended.
>
> Comments:
>
> Review in light of general comments.
>
>
>
>
--
Ticket URL: <http://bind10.isc.org/ticket/2601#comment:5>
BIND 10 Development <http://bind10.isc.org>
BIND 10 Development
More information about the bind10-tickets
mailing list