Re: Turning TESTS on by default

From: Alan Somers <asomers_at_freebsd.org>
Date: Sat, 7 Jun 2014 12:31:01 -0600
On Fri, Jun 6, 2014 at 9:33 PM, Konstantin Belousov <kostikbel_at_gmail.com> wrote:
> On Fri, Jun 06, 2014 at 03:14:52PM -0400, Julio Merino wrote:
>> Hello all,
>>
>>
>> TL;DR
>> -----
>>
>> I plan to turn the TESTS src.conf knob ON by default on Tuesday once I
>> have been able to perform enough sanity-checks of the build and all of
>> them pass.
>>
>> The impact of this is that the FreeBSD Test Suite (see tests(7)) will
>> be built and installed by default under /usr/tests/ along with the
>> private atf libraries and binaries. There should be no other changes
>> and this should be transparent to everyone.
>>
>> If this happens to break the world in any way, we can trivially roll
>> the change back to fix the fallout.
>>
>>
>> Some details
>> ------------
>>
>> TESTS was never intended to be disabled by default. However, the
>> original patches that were committed months ago related to this
>> feature broke the build and the easiest way out (instead of reverting
>> the commits) was to set the knob to disabled. Unfortunately, it stayed
>> that way even after the discovered problems were fixed.
>>
>> I am confident enough now that we have ironed out all major issues
>> that this might introduce, so it is about time to enable TESTS by
>> default again in HEAD.
>>
>> The benefits of this are that 1) we allow end users (especially
>> consumers of binary releases!) to run the tests out of the box, as it
>> has always been intended; and 2) we will be able to run the official
>> release builds through testing via Jenkins, instead of having to issue
>> custom builds.
> This is very weird and unprobable.  Users cannot care less about running
> the test suite, they use OS to run applications.  IMO enabling installation
> of the stuff that bloats the system but have no practical use for the
> system consumer should not be allowed by default.

I disagree.  Sure, some users won't care.  Probably even most users
won't care.  But some of our users are active supporters of FreeBSD.
They evangelize, they file PRs, and they help other users on the
forums.  Those users will run the tests.  Some of them will find bugs
that we didn't, because they'll be using different hardware and
different configurations.  Plus, shipping a test suite exudes an aura
of quality (if the tests pass, that is).  So I think that we should
install the tests, but in a separate installation set, just like
games.

-Alan

>
> It is the same as the debugging kernel. The INVARIANTS, WITNESS, DEBUG
> and DIAGNOSTIC options are not enabled for the user consumption. There
> was a similar argument to disable compiling the profiling static
> libraries, which probably should be reconsidered since lib*_p.a is
> absolutely useless on current toolchain and hardware.
>
>>
>> Actual change: https://phabric.freebsd.org/D188
>>
>>
>> I will update this thread when the change is committed and/or with any updates.
>>
>> Please let me know if I'm missing anything.
>>
>> Cheers
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
Received on Sat Jun 07 2014 - 16:31:04 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:49 UTC