BIND 10 trac1290, updated. 46adf014f18c6b3f9a685b8f0fdd0775a583a7c5 [1290] added a written out version of the demo as tutorial
BIND 10 source code commits
bind10-changes at lists.isc.org
Thu Oct 27 09:38:18 UTC 2011
The branch, trac1290 has been updated
via 46adf014f18c6b3f9a685b8f0fdd0775a583a7c5 (commit)
from 9b76badecd4b688c95c5f83ecdc168922e48680b (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 46adf014f18c6b3f9a685b8f0fdd0775a583a7c5
Author: Jelte Jansen <jelte at isc.org>
Date: Thu Oct 27 11:37:52 2011 +0200
[1290] added a written out version of the demo as tutorial
-----------------------------------------------------------------------
Summary of changes:
tests/lettuce/README | 48 +++++++++++++-
tests/lettuce/README.tutorial | 147 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 192 insertions(+), 3 deletions(-)
create mode 100644 tests/lettuce/README.tutorial
-----------------------------------------------------------------------
diff --git a/tests/lettuce/README b/tests/lettuce/README
index df90ff9..00ea623 100644
--- a/tests/lettuce/README
+++ b/tests/lettuce/README
@@ -74,7 +74,49 @@ These files *will* be overwritten or deleted if the same scenarios are run
again, so if you want to inspect them after a failed test, either do so
immediately or move the files.
-Extending tests
----------------
+Adding and extending tests
+--------------------------
+
+If you want to add tests, it is advisable to first go through the examples to
+see what is possible, and read the documentation on http://www.lettuce.it
+
+There is also a README.tutorial file here.
+
+We have a couple of conventions to keep things manageable.
+
+Configuration files go into the configurations/ directory.
+Data files go into the data/ directory.
+Step definition go into the features/terrain/ directory (the name terrain is
+chosen for the same reason Lettuce chose terrain.py, this is the place the
+tests 'live' in).
+Feature definitions go directly into the features/ directory.
+
+These directories are currently not divided further; we may want to consider
+this as the set grows. Due to a (current?) limitation of Lettuce, for
+feature files this is currently not possible; the python files containing
+steps and terrain must be below or at the same level of the feature files.
+
+Long-running processes should be started through the world.RunningProcesses
+instance. If you want to add a process (e.g. bind9), create start, stop and
+control steps in terrain/<base_name>_control.py, and let it use the
+RunningProcesses API (defined in terrain.py). See bind10_control.py for an
+example.
+
+For sending queries and checking the results, steps have been defined in
+terrain/querying.py. These use dig and store the results split up into text
+strings. This is intentionally not parsed through our own library (as that way
+we might run into a 'symmetric bug'). If you need something more advanced from
+query results, define it here.
+
+Some very general steps are defined in terrain/steps.py.
+Initialization code, cleanup code, and helper classes are defined in
+terrain/terrain.py.
+
+If you need to add steps that are very particular to one test, create a new
+file with a name relevant for that test in terrain. We may want to consider
+creating a specific subdirectory for these, but at this moment it is unclear
+whether we need to.
+
+We should try to keep steps as general as possible, while not making them to
+complex and error-prone.
-[TODO]
diff --git a/tests/lettuce/README.tutorial b/tests/lettuce/README.tutorial
new file mode 100644
index 0000000..eca2cd0
--- /dev/null
+++ b/tests/lettuce/README.tutorial
@@ -0,0 +1,147 @@
+Quick tutorial and overview
+---------------------------
+
+Lettuce is a framework for doing Behaviour Driven Development (BDD).
+
+The idea behind BDD is that you first write down your requirements in
+the form of scenarios, then implement their behaviour.
+
+We do not plan on doing full BDD, but such a system should also help
+us make system tests. And, hopefully, being able to better identify
+what exactly is going wrong when a test fails.
+
+Lettuce is a python implementation of the Cucumber framework, which is
+a ruby system. So far we chose lettuce because we already need python
+anyway, so chances are higher that any system we want to run it on
+supports it. It only supports a subset of cucumber, but more cucumber
+features are planned. As I do not know much details of cucumber, I
+can't really say what is there and what is not.
+
+A slight letdown is that the current version does not support python 3.
+However, as long as the tool-calling glue is python2, this should not
+cause any problems, since these aren't unit tests; We do not plan to use
+our libraries directly, but only through the runnable scripts and
+executables.
+
+-----
+
+Features, Scenarios, Steps.
+
+Lettuce makes a distinction between features, scenarios, and steps.
+
+Features are general, well, features. Each 'feature' has its own file
+ending in .feature. A feature file contains a description and a number
+of scenarios. Each scenario tests one or more particular parts of the
+feature. Each scenario consists of a number of steps.
+
+So let's open up a simple one.
+
+-- example.feature
+Feature: showing off BIND 10
+ This is to show BIND 10 running and that it answer queries
+
+ Scenario: Starting bind10
+ # steps go here
+--
+
+I have predefined a number of steps we can use, as we build test we
+will need to expand these, but we will look at them shortly.
+
+This file defines a feature, just under the feature name we can
+provide a description of the feature.
+
+The one scenario we have no has no steps, so if we run it we should
+see something like:
+
+-- output
+> lettuce
+Feature: showing off BIND 10
+ This is to show BIND 10 running and that it answer queries
+
+ Scenario: Starting bind10
+
+1 feature (1 passed)
+1 scenario (1 passed)
+0 step (0 passed)
+--
+
+Let's first add some steps that send queries.
+
+--
+ A query for www.example.com should have rcode REFUSED
+ A query for www.example.org should have rcode NOERROR
+--
+
+Since we didn't start any bind10, dig will time out and the result
+should be an error saying it got no answer. Errors are in the
+form of stack traces (trigger by failed assertions), so we can find
+out easily where in the tests they occurred. Especially when the total
+set of steps gets bigger we might need that.
+
+So let's add a step that starts bind10.
+
+--
+ When I start bind10 with configuration example.org.config
+--
+
+This is not good enough; it will fire of the process, but setting up
+b10-auth may take a few moments, so we need to add a step to wait for
+it to be started before we continue.
+
+--
+ Then wait for bind10 auth to start
+--
+
+And let's run the tests again.
+
+--
+> lettuce
+
+Feature: showing off BIND 10
+ This is to show BIND 10 running and that it answer queries
+
+ Scenario: Starting bind10
+ When I start bind10 with configuration example.org.config
+ Then wait for bind10 auth to start
+ A query for www.example.com should have rcode REFUSED
+ A query for www.example.org should have rcode NOERROR
+
+1 feature (1 passed)
+1 scenario (1 passed)
+4 steps (4 passed)
+(finished within 2 seconds)
+--
+
+So take a look at one of those steps, let's pick the first one.
+
+A step is defined through a python decorator, which in essence is a
+regular expression; each captured group will be passed as an argument
+to the function we define. For bind10, i defined a configuration file,
+a cmdctl port, and a process name. The first two should be
+self-evident, and the process name is an optional name we give it,
+should we want to address it in the rest of the tests. This is most
+useful if we want to start multiple instances. In the next step (the
+wait for auth to start), I added a 'of <instance>'. So if we define
+the bind10 'as my_bind10', we can specify that one here as 'of my
+bind10'.
+
+But notice how we needed two steps, which we probably always need (but
+not entirely always)? We can also combine steps; for instance:
+
+--
+ at step('have bind10 running(?: with configuration ([\w.]+))?')
+def have_bind10_running(step, config_file):
+ step.given('start bind10 with configuration ' + config_file)
+ step.given('wait for bind10 auth to start')
+--
+
+Now we can replace the two steps with one:
+
+--
+ Given I have bind10 running
+--
+
+That's it for the quick overview. For some more examples, with comments,
+take a look at features/example.feature. You can read more about lettuce and
+its features on http://www.lettuce.it, and if you plan on adding tests and
+scenarios, please consult the last section of the main README first.
More information about the bind10-changes
mailing list