Fw: Call for regression and performance testing - 8.0

[forwarding to the list]
Begin forwarded message:

Date: Mon, 28 Sep 2009 01:32:36 +0000
From: Bruce Cran <brucec_at_muon.cran.org.uk>
To: Robert Watson <rwatson_at_FreeBSD.org>
Cc: Bruce Cran <bruce_at_cran.org.uk>, "Arno J. Klaassen"
<arno_at_heho.snv.jussieu.fr>, current_at_freebsd.org Subject: Re: Call for
regression and performance testing - 8.0


On Tue, Sep 22, 2009 at 01:01:34PM +0100, Robert Watson wrote:
> On Tue, 22 Sep 2009, Bruce Cran wrote:
>
>>> Resurfacing after organising conferences, and discovered I had
>>> missed this e-mail. I'm not aware of any such efforts, and it would
>>> be really good to do. Making it easy to do "runtests" would vastly 
>>> improve the ease of testing during the release process. Right now
>>> the output of the tests is fairly inconsistent, and this is
>>> probably to a large extent my fault. It would be nice to have a
>>> "test template" program that people can start with when writing new
>>> regression tests -- I know I would find that valuable.
>>
>> There's already a limited template available: 
>> src/tools/regression/README specifies the format to use, and many of 
>> the regression tests do use it. You can run all the tests which use
>> the specified format by running "prove -r" (after installing 
>> devel/p5-Test-Harness), but there are currently quite a few failures.
>>
>> Some tests don't use the template for good reason - for example  
>> src/tools/regression/redzone9 depends on the DEBUG_REDZONE kernel 
>> option to be present.
>
> However, others don't do it for a good reason -- perhaps mostly my
> tests? :-)
>
> If people want to convert them over that would be excellent.

I've updated the PR kern/138465 with a patch to convert the 
sockets/sendfile test to produce TAP output. It also fixes a bug where 
it called sendfile with a length of 0 and didn't expect to receive the 
whole buffer.  

Are there any examples of producing TAP output in 
a client/server application, and avoiding bailing when an error occurs? 
It seems quite a few tests use fork, and any output in the child 
process isn't seen by 'prove' so any errors need marshalled back to the 
main thread to print the "not ok" message. Should they be converted to 
use pthreads?

-- 
Bruce Cran