[bind10-dev] Resolver testing

[Shane Kerr]

Michal,

On Tue, 2010-11-02 at 19:01 +0100, Michal 'vorner' Vaner wrote:
> On Tue, Nov 02, 2010 at 02:30:22PM +0100, Shane Kerr wrote:
> >     # define the packets in a simple A lookup of www.domain-1.test
> >     # target(s)         time   query/answer  contents
> >     a.root,b.root       *      q             www.domain-1.test a
> >     ns1.test,ns2.test   *      q             www.domain-1.test a
> >     $client             *      a             www.domain-1.test a 10.0.0.1
> > 
> >     # define the packets with a non-responding name server
> >     # target(s)                time   query/answer  contents
> >     a.root,b.root              *      q             domain-2.test a
> >     ns1.test,ns2.test          *      q             domain-2.test a
> >     $last                      100    q             domain-2.test a
> >     (ns1.test,ns2.test)-$last  100    q             domain-2.test a
> >     $client                    *      a             domain-2.test a 10.0.0.2
> 
> There are three things that crossed my mind, which might be useful as well as
> complete nonsense:
> • Will our DNS library allow us to send broken packets? If not, we need some
>   other (broken) library that does, at last for this task.

Possibly true. We may be able to convert to wire format and munge that
directly too. Or we may want to make a variant of our library that
allows brokenness. (Probably also useful for nmap at some point). ;)

> • To properly schedule things like multiple concurrent unanswered queries, we
>   might want to run the whole machinery as one process (so we can schedule to
>   send a query after the server contacted root zone, and org zone, but before it
>   got answer ‒ it could be hard with only timing).

This is a good point. For testing purposes this makes sense - and is
simpler than what I had envisioned!

> • If we have our own language to describe the test, we need to write our own
>   parser and our own interpretter. Furthermore, it might come out as not enough
>   expressive (or, on the other hand, completely overcomplicated). And we might
>   want to send the same packet during many tests, so some kind of „functions“
>   might be useful.
> 
>   Wouldn't it be better to use some kind of existing language (python with some
>   kind of event-handling library, perl with AnyEvent and Coro, lua…) and provide
>   it with the testing framework? Functions like „Set up an authoritative
>   nameserver with this zone“ or „Set up an authoritative nameserver and when a
>   query comes, call this function to decide what to answer“ and „Send this query
>   packet and do this with the answer, or do this on timeout“?

(What you are describing is actually quite similar to what Michael Graff
built as a prototype DNS tester, DNS Cucumber.)

My main goal is to make it easy to define tests. Ideally this would
involve mostly tables of data that anyone can look at and see how the
test is supposed to run, without reading code.

I do agree that a generic ability to invoke a Python function in tests
seems like it could be useful, especially if there is a module covering
most of the normal use cases.

Perhaps the best approach to this is for whoever ends up doing this work
to start by writing tests and seeing how much expressiveness is needed? 

Cheers,

--
Shane