Re: Head build unsafe for /etc today

On 2017-Nov-2, at 8:50 PM, Steve Kargl <sgk_at_troutmask.apl.washington.edu> wrote:

> On Thu, Nov 02, 2017 at 07:41:21PM -0700, Bryan Drewery wrote:
>> 
>> Are you accusing me of lying?
>> 
> 
> Nope.  I'm stating the obvious.  If you are using
> META_MODE and you do "make buildwould" that is 
> equivalent to "make -DNO_CLEAN buildworld", which
> means you did not rebuild the *world*. 

Also from a prior message of this sequence:

> If your first step isn't  'cd /usr/obj ; rm -rf *' or equivalent
> in whatever jail you use, then you're not properly testing 
> your changes to the build infrastructure.

With or without META_MODE, a rm -fr /usr/obj/*
before the build attempt forces a rebuild as far
as I know. It may be more that cleaning was
effectively not tested then rebuilding was
not tested. But always doing rm -fr /usr/obj/*
first establishes a very limited context for
testing cleaning.

WITH_META_MODE and WITHOUT_META_MODE still
might not be strictly equivalent after the
rm -fr /usr/obj/* for some other properties
in such an "empty" context. So testing those
combinations makes sense but would be
insufficient.

> When I see a commit message of the form (and I've
> haven't seen one like this in 25+ years of using
> FreeBSD (aka 386BSD+patchkit))
> 
> Author: bdrewery
> Date: Thu Nov  2 22:23:00 2017
> New Revision: 325347
> URL: https://svnweb.freebsd.org/changeset/base/325347
> 
> Log:
>  Something is very wrong
> 
> Modified:
>  head/Makefile
> 
> Modified: head/Makefile
> ==============================================================================
> --- head/Makefile	Thu Nov  2 21:58:18 2017	(r325346)
> +++ head/Makefile	Thu Nov  2 22:23:00 2017	(r325347)
> _at__at_ -1,3 +1,4 _at__at_
> +.error Bad revision, please wait for a fix in head
> 
> It suggests that whomever did the commit did not properly test
> the patch.  The use of META_MODE (or any other shortcut) when
> testing simply isn't proper testing.

I think I understand the intended point but the
actual wording for "the use of . . ." and
"[i]f your first step isn't . . ." is wrong from
what I can tell.

The testing of WITH_META_MODE is a proper form of test
but is not a sufficient category of test overall. But
omitting all tests of WITH_META_MODE would be poor
procedure in my view.

Some testing needs to be done without rm -fr /usr/obj/*
after a prior build as well. Some testing of
WITH_META_MODE after a prior build needs to be done.
Some testing of WITHOUT_META_MODE after a prior build
needs to be done. And so on. At least that would be
my view.

Any and all mistakes checked-in are examples of
insufficient testing --but always doing sufficient
testing requires establishing a much simpler, more
limited, context. To my knowledge FreeBSD is not
trying to scale back like that. (It is not under
the direction of an Edsger W. Dijkstra.)

I do not know if something might be able to be done
to make such a specific type of "clean test" mistake
less likely to happen again. Could a test context
be established where attempts to delete outside the
build tree would be rejected, with notifications of
the attempts? Could running such a test be automatic
(part of something that is run systematically) and
fast enough to not want to skip it? (Just being
illustrative. The details involved are well outside
my background knowledge. There may be nothing
easy or reasonable.)

===
Mark Millard
markmi at dsl-only.net