[bind10-dev] How to test command-line output, was Re: Weekly updates: 2010-w19

[Shane Kerr]

Ting Ting,

On Thu, 2010-05-13 at 10:05 +0800, tingting shen wrote:
> OK, until now, i finished several tasks:
> 1. finished task 25, which is b10-loadzone or python module fixed.

Cool.

> 2. finished task 86, which is $INCLUDE fixes. Support optional
> (origin, comment) and $TTL (comment) (RFC compliance). I tested the
> loadzone daemon, it works well.

Cool.

> 3. yesterday night, I finished the loadzone`s verbose option function.
> when I run loadzone, the show of the terminal is like this:

How is this being tested?

I ask for two reasons:

     1. To make sure that we have started "test driven development"
        where we write the tests first and then code them. ;)
     2. Because I think we probably want to discuss testing output in
        general.

For normal output testing, it is straightforward: redirect to a file and
see if the contents there are correct (we can save a file containing the
"correct" output to use for testing):

$ python test.py > test.out
$ diff test.out test.correct


For output testing where we need a TTY (like with the status update), we
can use something like the "script" command:

$ cat test.py
import os
import sys

if os.isatty(sys.stdout.fileno()):
    sys.stderr.write("stdout is a tty\n")
else:
    sys.stderr.write("stdout is NOT a tty\n")

$ python test.py
stdout is a tty
$ python test.py > /dev/null
stdout is NOT a tty
$ script -c "python test.py" test.out > /dev/null
$ cat test.out
Script started on 2010-05-14T14:55:46 CST
stdout is a tty

Script done on 2010-05-14T14:55:46 CST
$ diff test.out test.correct

If we use "script" (or something else, this is just the first thing that
came to mind for me) we can test TTY-oriented code in the same way that
we test any text output - by redirecting to a file and comparing with
"known good" values.

Note that "script" captures all output, so end of line is "\r\n" rather
than just "\n" - this is actually good for the b10-loadzone case because
we want to see all those "\r" characters. :)

--
Shane