Re: Call for regression and performance testing - 8.0

On Tue, 22 Sep 2009 11:30:27 +0100
"Robert N. M. Watson" <rwatson_at_freebsd.org> wrote:

> 
> On 13 Aug 2009, at 21:55, Arno J. Klaassen wrote:
> 
> > Robert Watson <rwatson_at_freebsd.org> writes:
> >
> >>> [ ... ]
> >>  Functional testing
> >>
> >> We actually have a sizable regression test suite in src/tools/ 
> >> regression --
> >
> > is anyone aware of some rudimentary efforts to set up an overall
> > "make && ./runtests" environment?
> 
> Resurfacing after organising conferences, and discovered I had
> missed this e-mail. I'm not aware of any such efforts, and it would
> be really good to do. Making it easy to do "runtests" would vastly
> improve the ease of testing during the release process. Right now the
> output of the tests is fairly inconsistent, and this is probably to a
> large extent my fault. It would be nice to have a "test template"
> program that people can start with when writing new regression tests
> -- I know I would find that valuable.

There's already a limited template available:
src/tools/regression/README specifies the format to use, and many of
the regression tests do use it. You can run all the tests which use the
specified format by running "prove -r" (after installing
devel/p5-Test-Harness), but there are currently quite a few failures.

Some tests don't use the template for good reason - for example
src/tools/regression/redzone9 depends on the DEBUG_REDZONE kernel
option to be present.

-- 
Bruce Cran