[bind10-dev] NetBSD timing test failure, was Re: failed unittests

[Yoshitaka Aharen]

Hi Shane-san,

On Thu, 26 May 2011 10:08:31 +0200
Shane Kerr <shane at isc.org> wrote:

> All,
> 
> We had a build failure on our NetBSD box:
> 
> On Wed, 2011-05-25 at 23:25 +0000, BIND 10 Tests wrote:
> > http://bind10.isc.org/~tester/builder//BIND10/20110525215001-NetBSD5-amd64/logs/unittests.out
> > 
> > http://bind10.isc.org/~tester/builder//builder.html
> > 
> > 1306365902 
> 
> I guess Jeremy has fixed most of our timing failures, but not all. Is 50
> msec a bit too tight of a time limit for a virtual machine?

Maybe it is too tight. The solutions are:
  1. Modify time limits to enough longer
  2. Modify tests not to heavily rely on accuracy of the timer

I think it is not good to tweak the test parameter for the issue which
depends on the environment, so I guess the latter is better.

There're two tests which rely on the accuracy of the timer.
  * IntervalTimerTest.startIntervalTimer
  * IntervalTimerTest.overwriteIntervalTimer

The intention of the tests is not to test asio::deadline_timer, to test
asiolink::Intervaltimer. In my opinion, the issue can be solved with the
following modifications.

  * IntervalTimerTest.startIntervalTimer
    => Test if the call back function will be called after the
        specified interval or longer instead of checking the elapsed
        time is within +/- 50 msec.

  * IntervalTimerTest.overwriteIntervalTimer
    => Test if the return value of getInterval() will be updated after
        calling setup() instead of measuring the time

Suggestions or advices to fix the issue are welcome.

Thanks,

-- 
Yoshitaka Aharen <aharen at jprs.co.jp>
Japan Registry Services Co., Ltd.