Re: FreeBSD Testing Facility

On 2013-02-21 07:04, Matthew Jacob <mjacob_at_freebsd.org> wrote:
> On 2/21/2013 5:04 AM, Mehmet Erol Sanliturk wrote:
> >Dear All ,
> >
> >During development of FreeBSD , testing is very vital .
> >...
> 
> This in general is a good suggestion. Most companies do such
> automated testing as a matter of course.
> 
> Note however that this is a volunteer effort. Were you volunteering
> to set up such an automated, possibly testzilla driven, facility? It
> would certainly help the quality, although as others have noted
> snapshots are often likely to be broken.

To the OP:

Like Matthew has said, this is a volunteer effort.  So if you have
experience with setting up testing automation, and you are willing to
help us set up something like this, please go ahead :-)

I've worked in places where the following types of tests are used:

  - Presubmit tests that check specific parts of functionality.

  - Commit-related tests that run asynchronously in the background,
    and report back later (e.g. through email).

  - Test systems that cache previous results and report a simple 'green'
    vs. 'red' status for _every_ single commit.

  - System tests that check for particular functionality, health
    criteria, etc. - some times fully automated, some other times
    requiring a token amount of manual support.

So here are two important questions, regarding the tests you mentioned:

When you speak about 'testing FreeBSD', which type of tests are you
interested in seeing?

Are you willing to help us set up something that runs the type of tests
that you want to see?