[bind10-dev] Parallel tests?

[Shane Kerr]

Michal,

On Wed, 2011-07-13 at 00:02 +0200, Michal 'vorner' Vaner wrote:
> What I propose is creating a „manifest“ file in each test directory. It would
> only list which test binaries are in the test directory and which of them can
> run in parallel with other tests. Then there would be a simple script (either
> shell or python), which would:
> • Find all the manifests
> • Run the tests (most of them in parallel, then the sequential ones which don't
>   like being run together with something else). It would store their outputs in
>   memory and output them one by one, so they don't mix into each other.
> • It could make sure if anything fails, the failing binary's output is the last
>   one output, so it is easier to read the problem.
> 
> It should probably have a way to run only in some subdirectory, called from
> within the make check in a directory (when developing, I like to run only the
> local tests).
> 
> Do you think something like this makes sense? Any comments? Should I create a
> ticket for it?

I think it makes sense. However, this seems like the kind of thing that
could be generally very useful for other projects as well. Perhaps we
should approach the Google Test developers and see what the "right" way
to integrate such functionality into the framework is.

I poked around a little and found this thread:

http://groups.google.com/group/googletestframework/browse_thread/thread/3ad0d31cc38a53c/8de56acf93f689d2?lnk=gst&q=parallel#8de56acf93f689d2

Basically it seems like there is not support for this, and probably they
haven't considered our use case (tests that take wall-clock time but can
run in parallel). They support splitting tests across machines (using
the buzzword-compliant "sharding" technology); this is not done
especially intelligently, and seems to be addressing CPU bottlenecks
rather than tests which just take real time.

--
Shane