[SOLVED] Re: Deterministic rescue buildworld error with custom make.conf/src.conf/MAKEOBJDIRPREFIX

On 12/03/2017 13:37, Ian Lepore wrote:
> On Sun, 2017-03-12 at 13:27 +1100, Lawrence Stewart wrote:
>> Hi Ian,
>>
>> On 12/03/2017 10:29, Ian Lepore wrote:
>>>
>>> On Sun, 2017-03-12 at 10:22 +1100, Lawrence Stewart wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I'm unable to complete buildworld with 2 recent svn revs I've
>>>> tried
>>>> (r314838 and r315059). I'm building for a slightly resource
>>>> constrained
>>>> production system so am specifying custom settings and a
>>>> different
>>>> obj
>>>> tree location so I can copy it to the target system. The error
>>>> persists
>>>> after an "rm -rf /usr/obj/*", and if parallel building is
>>>> disabled.
>>>>
>>>> The underlying build system built from r314838 via simple "make
>>>> -C
>>>> /usr/src -s -j6 buildworld buildkernel" built and installed fine,
>>>> so
>>>> the
>>>> problem seems to be around the use of the build customisations.
>>>>
>>>> Any clues?
>>>>
>>>> Cheers,
>>>> Lawrence
>>>>
>>>>
>>>> root_at_builder-head-amd64:/usr/src # cat cust_make.conf
>>>> KERNCONF=GENERIC-NODEBUG
>>>> MALLOC_PRODUCTION=YES
>>>>
>>>> root_at_builder-head-amd64:/usr/src # cat cust_src.conf
>>>> WITHOUT_PROFILE=1
>>>>
>>>> root_at_builder-head-amd64:/usr/src # make
>>>> __MAKE_CONF=/usr/src/cust_make.conf
>>>> SRCCONF=/usr/src/cust_src.conf
>>>> MAKEOBJDIRPREFIX=/usr/obj/cust buildworld buildkernel
>>>> [...]
>>>> MK_AUTO_OBJ=no
>>>> MK_TESTS=no  UPDATE_DEPENDFILE=no  _RECURSING_CRUNCH=1
>>>> CC="cc -target x86_64-unknown-freebsd12.0
>>>> --sysroot=/usr/obj/cust/usr/src/tmp
>>>> -B/usr/obj/cust/usr/src/tmp/usr/bin
>>>> -O2 -pipe   -std=gnu99    -Qunused-arguments  "  CXX="c++  -
>>>> target
>>>> x86_64-unknown-freebsd12.0 --sysroot=/usr/obj/cust/usr/src/tmp
>>>> -B/usr/obj/cust/usr/src/tmp/usr/bin -O2 -pipe -Qunused-arguments
>>>> -Wno-c++11-extensions  "  make .MAKE.MODE="normal curdirOk=yes"
>>>> .MAKE.META.IGNORE_PATHS=""  -f rescue.mk exe
>>>> cc -target x86_64-unknown-freebsd12.0
>>>> --sysroot=/usr/obj/cust/usr/src/tmp
>>>> -B/usr/obj/cust/usr/src/tmp/usr/bin
>>>> -O2 -pipe   -std=gnu99    -Qunused-arguments   -nostdlib -Wl,-dc
>>>> -r
>>>> -o
>>>> cat.lo cat_stub.o
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/cat/cat.o
>>>> cc: error: no such file or directory:
>>>> '/usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/cat/cat.o'
>>>> *** Error code 1
>>>>
>>>> There appear to be a lot of missing .o files under the rescue obj
>>>> tree:
>>>>
>>>> root_at_builder-head-amd64:/usr/src # find
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr -type f
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mksyntax.o
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mksyntax
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mknodes.o
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/sh/mknodes
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/csh/sh.err.h
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/csh/tc.const.h
>>>> /usr/obj/cust/usr/src/rescue/rescue//usr/src/bin/csh/gethost
>>>>
>>>> compared with an obj tree on a different head system:
>>>>
>>>> find /usr/obj/usr/src/rescue/rescue/usr/ -type f | wc -l
>>>>     1552
>>>> _______________________________________________
>>>> freebsd-current_at_freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_fre
>>>> ebsd
>>>> .org"
>>> The MAKEOBJDIRPREFIX variable must be set in the environment, not
>>> in
>>> make.conf or on the make command line (documented in build(7)).
>> Your assertion seems at odds with my past experience and my reading
>> of
>> the man page... from build(7):
>>
>> 	The build may be controlled by defining make(1) variables
>> 	described in the ENVIRONMENT section below, and by the
>> 	variables documented in make.conf(5).
>>
>> ... which indicates they are make variables, not environment
>> variables
>> specifically. As a concrete example, TARGET and DESTDIR are listed
>> under
>> the "ENVIRONMENT" section of the man page, yet "EXAMPLES" shows:
>>
>>            make TARGET=sparc64 buildworld
>>            make TARGET=sparc64 DESTDIR=/clients/sparc64 installworld
>>
>> I've certainly always set build vars documented in the "ENVIRONMENT"
>> section of the man page on the make command line without issue.
>> Pretty
>> sure I've set MAKEOBJDIRPREFIX from the make command line also in the
>> past, though perhaps it has been working for me "by accident" and a
>> documentation tweak is in order if the distinction you make is in
>> fact
>> relevant...
>>
>> Cheers,
>> Lawrence
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd
>> .org"
> 
> You cited everything from build(7) except the part most on-point for
> the problem you're having:
> 
>     MAKEOBJDIRPREFIX
>         Defines the prefix for directory names in the tree of
>         built objects.  Defaults to /usr/obj if not defined.  This variable
>         should only be set in the environment and not via /etc/make.conf or the
>         command line.

Oh dear. I *completely* glossed over that... multiple times. Derp.

Thank you for the liberal application of clue bat and my apologies for
the noise

Cheers,
Lawrence