[bind10-dev] unit test problems

[JINMEI Tatuya / 神明達哉]

At Mon, 23 Aug 2010 09:37:25 +0800,
"zhanglikun" <zlkzhy at gmail.com> wrote:

> I will redirect the output to /dev/null to avoid the noisy output, any
> better solution?

That's one solution, but for minor noise such as "b10-xfrin" log
messages...

> > Running test: xfrin_test.py
> > .[b10-xfrin] test error

..., I'd leave them for now and just wait for a cleaner solution with
the generic logging framework.

The noisier traceback is (IMO) a completely different problem:

> > During handling of the above exception, another exception occurred:
> > 
> > Traceback (most recent call last):
> >   File "/usr/pkg/lib/python3.1/threading.py", line 509, in _bootstrap_inner
> >     self.run()
> >   File "/usr/pkg/lib/python3.1/threading.py", line 462, in run
> >     self._target(*self._args, **self._kwargs)
> >   File "/backup/isc-work-svn/svn/branches/trac316/src/bin/xfrin/xfrin.py", line
> > 331, in process_xfrin
> >     ret = conn.do_xfrin(check_soa)
> >   File "/backup/isc-work-svn/svn/branches/trac316/src/bin/xfrin/xfrin.py", line
> > 197, in do_xfrin
> >     self._handle_xfrin_response)
[...]

This means something that neither the main code nor the test expected
happened during the test, which is basically a bug.  We should fix the
bug, rather than redirecting the output to /dev/null and ignoring it
(even if it's possible).  In fact, in that sense it could be even
harmful to naively redirect all stdout/stderr output to /dev/null (if
that's what you meant).

Another related problem is that the test exited successfully
(i.e. exit code of 0) despite the existence of a bug so we cannot
notice it via the automated build/test bot.  I don't know why that
happened and how difficult to fix it, but in general we should make
sure the test stop with a non 0 exit code when it encounters a bug.

---
JINMEI, Tatuya
Internet Systems Consortium, Inc.