Re: Building with external toolchain was broken 6 months ago with r255187

Ian Lepore wrote this message on Thu, Mar 20, 2014 at 08:29 -0600:
> On Thu, 2014-03-20 at 10:08 -0400, John Baldwin wrote:
> > On Tuesday, March 18, 2014 6:20:50 pm Bjoern A. Zeeb wrote:
> > > 
> > > On 18 Mar 2014, at 22:01 , John-Mark Gurney <jmg_at_funkthat.com> wrote:
> > > 
> > > > Lev Serebryakov wrote this message on Wed, Mar 19, 2014 at 01:37 +0400:
> > > >> I did't build my NanoBSD images for almost year, and in this time our
> > > >> not-finished and fragile support for using "external" toolchain is 
> > rotten,
> > > >> due to r255187 (and, may meb, some other commits too).
> > > >> 
> > > >> I have very fresh -CURRENT (r263296)
> > > >> 
> > > >> I have these settings for my buildworld & buildkernel targets:
> > > >> 
> > > >> XCC=/usr/bin/cc
> > > >> XCXX=/usr/bin/c++
> > > >> XCPP=/usr/bin/cpp
> > > >> XAS=/usr/bin/as
> > > >> XAR=/usr/bin/ar
> > > >> XLD=/usr/bin/ld
> > > >> XNM=/usr/bin/nm
> > > >> XOBJDUMP=/usr/bin/objdump
> > > >> XRANLIB=/usr/bin/ranlib
> > > >> XSTRINGS=/usr/bin/strings
> > > >> COMPILER_TYPE=clang
> > > >> WITHOUT_CROSS_COMPILER=yes
> > > >> WITHOUT_BINUTILS=yes
> > > >> WITHOUT_CLANG=yes
> > > >> 
> > > >> It worked 7 months ago. Now it works for "buildworld" but not for
> > > >> "buildkernel:
> > > >> 
> > > >> --- aeskeys_amd64.o ---
> > > >> /usr/bin/cc --sysroot=/data/obj.nano/gateway.v2/data/src/tmp -
> > B/data/obj.nano/gateway.v2/data/src/tmp/usr/bin -O2 -pipe -fno-strict-aliasing 
> > -Werror -D_KERNEL -DKLD_MODULE -nostdinc   -DHAVE_KERNEL_OPTION_HEADERS -
> > include /data/obj.nano/gateway.v2/data/src/sys/D2500CC/opt_global.h -I. -I_at_ -
> > I_at_/contrib/altq -fno-common -g -fno-omit-frame-pointer -mno-omit-leaf-frame-
> > pointer -I/data/obj.nano/gateway.v2/data/src/sys/D2500CC  -mno-aes -mno-avx -
> > mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float  -fno-
> > asynchronous-unwind-tables -ffreestanding -fstack-protector -std=iso9899:1999 
> > -Qunused-arguments  -fstack-protector -Wall -Wredundant-decls -Wnested-externs 
> > -Wstrict-prototypes  -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual  
> > -Wundef -Wno-pointer-sign -fformat-extensions  -Wmissing-include-dirs -
> > fdiagnostics-show-option  -Wno-error-tautological-compare -Wno-error-empty-
> > body  -Wno-error-parentheses-equality -Wno-unused-function    -c 
> > /data/src/sys/modules/aesni/../../cryp
> > > > to/aesni/aeskeys_amd64.S
> > > >> --- aesni_wrap.o ---
> > > >> In file included from 
> > /data/src/sys/modules/aesni/../../crypto/aesni/aesni_wrap.c:40:
> > > >> /data/src/sys/modules/aesni/../../crypto/aesni/aesencdec.h:30:10: fatal 
> > error: 'wmmintrin.h' file not found
> > > >> #include <wmmintrin.h>
> > > >>         ^
> > > >> 1 error generated.
> > > >> *** [aesni_wrap.o] Error code 1
> > > >> 
> > > >> It could not find header file with intrinsics from "system" ("external")
> > > >> clang. I could disable building of this module with 
> > WITHOUT_MODULES=aesni,
> > > >> and it works, but what if I need this module?
> > > >> 
> > > >> Could it be fixed, pleeeeeeease?
> > > > 
> > > > Sounds like your tool chain doesn't have the necessary support for
> > > > AES-NI...  Are you using gcc as cc?  If so, do you have the necessary
> > > > tool chain work that I did in r255185 in your local tree?
> > > 
> > > 
> > > The problem is that the kernel is deepening on a compiler header which is 
> > not in the right place in objdir if the compiler is not built.  I thought I 
> > had reported this before (maybe just informally).  I have been helping myself 
> > locally using this:
> > 
> > No, the compiler should provide a working "wmmintrin.h" header in one of
> > its built-in paths if it supports the AES instructions.  This is akin to
> > saying that code that uses "stdio.h" should use -I/usr/src/include.
> 
> But it's a module, built with -nostdinc, so the appropriate -I has to be
> on the command line.

Actually, the above quoted text mixes two different files...
aeskeys_amd64 is an assembly file so doesn't need the header...

> I notice that -no-aes is also on the command line, which seems like a
> strange thing for compiling a file named aeskeys_amd64.

That's could be a bug in the assembler for allowing aes instructions
when they are turned off...  If it gets fixed, it isn't hard to enable
them for this specific file..

If you look at aesni_wrap.c's build, you'll see it looks somethingl like:
/usr/bin/cc -B/usr/obj/usr/src/tmp/usr/bin -c -O3 -pipe -fno-strict-aliasing -Werror -D_KERNEL -DKLD_MODULE -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/GENERIC/opt_global.h -I. -I_at_ -I_at_/contrib/altq -fno-common -gdwarf-2 -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -I/usr/obj/usr/src/sys/GENERIC -mno-aes -mno-avx -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fstack-protector -std=iso9899:1999 -Qunused-arguments -fstack-protector -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-unused-function -Werror   -mmmx -msse -maes /usr/src/sys/modules/aesni/../../crypto/aesni/aesni_wrap.c

where the aes instructions are turned on after they are disabled, and so
will work..

So, one option is to remove sysroot from the CC command..  This does
the equivalent of removing -nostdinc, and it passes the build using
XCC=/usr/bin/cc et al., though I'm not sure how well this will work
for environments where the cross compiler is more complicated, like
building cross platforms...

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."