Re: emulators/qemu: qemu ports failing due to compiler error on 12-CURRENT

On Thu, 2016-12-29 at 21:17 +0100, O. Hartmann wrote:
> Am Thu, 29 Dec 2016 20:26:32 +0100
> Dimitry Andric <dim_at_FreeBSD.org> schrieb:
> 
> > 
> > On 29 Dec 2016, at 17:29, O. Hartmann <ohartmann_at_walstatt.org>
> > wrote:
> > > 
> > > 
> > > Am Wed, 7 Dec 2016 23:31:01 +0100
> > > Dimitry Andric <dim_at_FreeBSD.org> schrieb:
> > >   
> > > > 
> > > > On 07 Dec 2016, at 10:42, O. Hartmann <ohartmann_at_walstatt.org>
> > > > wrote:  
> > > > > 
> > > > > 
> > > > > I try my first steps in cross compiling ports with poudriere
> > > > > and therefore I try to
> > > > > setup an appropriate jail and QEMU environment.
> > > > > 
> > > > > Well, I'm failing at the jail setup due to the non-exitence
> > > > > of any suitable QEMU
> > > > > environment and for that I tried to figure out to find some
> > > > > proper HOWTO.
> > > > > Searching via google ave some hints, but in questions which
> > > > > QEMU from ports should
> > > > > be used, all leave me alone, so I tried
> > > > > 
> > > > > emulators/qemu
> > > > > emulators/qemu-devel
> > > > > emulators/qemu-static
> > > > > 
> > > > > emulators/qemu is known for me to fail since months and the
> > > > > days of 11-CURRENT,
> > > > > there is a compiler error spit out with clang 3.8 and now
> > > > > 3.9. The very same for
> > > > > qemu-devel (both ports used with standard options, no
> > > > > extras). See also Bug 214873
> > > > > (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214873)
> > > > > and Bug 215100
> > > > > (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215100).  
> > > > I couldn't reproduce the compilation errors, it builds fine for
> > > > me until
> > > > the link phase.  
> > > Well, I face this in poudriere on the most recent 12-CURRENT, too
> > > as well as
> > > 12-CURRENT buildworld today.
> > > 
> > > On the host I'd like to run qemu for testing aarch64 binaries for
> > > a Odroid-C2
> > > project, I use a customized /etc/src.conf - but on poudriere,
> > > there is no such
> > > customisation but the failing is identical.  
> > Looking at your errors, it seems that the port has decided to
> > enable
> > rdma support.  This is normally enabled using --enable-rdma with
> > the
> > configure script, but I don't see that at all in the port Makefile.
> > 
> > On my systems, it runs a test to check for rdma support, but this
> > fails.
> > Quoting from config.log:
> > 
> >     cc -m64 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64
> > -D_LARGEFILE_SOURCE
> > -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings
> > -Wmissing-prototypes -fno-strict-aliasing -fno-common
> > -I/usr/work/share/dim/ports/emulators/qemu/work/qemu-2.6.1
> > -I/usr/local/include
> > -DPREFIX=\""/usr/local\"" -Wno-string-plus-int -Wno-initializer-
> > overrides
> > -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs 
> > -Wformat-security
> > -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-
> > definition -Wtype-limits
> > -fstack-protector-strong -I/usr/local/include
> > -I/usr/local/include/p11-kit-1
> > -I/usr/local/include -o config-temp/qemu-conf.exe config-temp/qemu-
> > conf.c -m64 -g
> > -fstack-protector -L"/usr/local/lib" -lrdmacm -libverbs config-
> > temp/qemu-conf.c:1:10:
> > fatal error: 'rdma/rdma_cma.h' file not found #include
> > <rdma/rdma_cma.h> ^
> > 
> > The minimal test program it tries to compile here is just this:
> > 
> >     #include <rdma/rdma_cma.h>
> >     int main(void) { return 0; }
> > 
> > and it attempts to link it with -lrdmacm -libverbs.  If this
> > somehow
> > succeeds on your system, then it will think rdma support is
> > available,
> > while apparently the support is not complete, if it misses the
> > rdma_getaddrinfo() function.
> > 
> > Do you have some Linux rdma or infiniband headers or libraries
> > installed
> > into /usr or /usr/local?  This might be the cause of the problems.
> No Linux, but I found these files on all of the boxes in question:
> 
> locate rdma
> 
> [...]
> /usr/include/rdma
> /usr/include/rdma/rdma_cma.h
> /usr/include/rdma/rdma_cma_abi.h
> /usr/lib/librdmacm.a
> /usr/lib/librdmacm.so
> /usr/lib/librdmacm.so.1
> 
> ll usr/include/rdma discovers:
> 
> total 44
> 322075 drwxr-xr-x   2 root  wheel  -  512B Oct  7 13:52 ./
> 240768 drwxr-xr-x  55 root  wheel  -  6.5K Dec 29 19:14 ../
> 324275 -r--r--r--   1 root  wheel  -   21K Oct  7 13:52 rdma_cma.h
> 324276 -r--r--r--   1 root  wheel  -  4.7K Oct  7 13:52
> rdma_cma_abi.h
> 
> and
> 
> ll /usr/lib/librdma*
> 804463 -r--r--r--  1 root  wheel  -   28K Dec 18 16:34
> /usr/lib/librdmacm.a
> 804127 lrwxr-xr-x  1 root  wheel  -   14B Dec 29 19:15
> /usr/lib/librdmacm.so_at_ ->
> librdmacm.so.1 804128 -r--r--r--  1 root  wheel  -   24K Dec 29
> 19:15 /usr/lib/librdmacm.so.1
> 
> 
> As you can see, the libraries are of the date of the last full
> install of the system,
> while the headers seem to be remains from October?
> 
> In my jail for poudriere, I was surprised, based on your analysis,
> that I found this:
> 
> ll /pool/poudriere/jails/head-amd64/usr/include/rdma
> total 43
> 77923 drwxr-xr-x   2 root  wheel  uarch    4B Jul 13 07:09 ./
> 77028 drwxr-xr-x  55 root  wheel  uarch  336B Dec 28 11:29 ../
> 96127 -r--r--r--   1 root  wheel  -       21K Jul 13 07:09 rdma_cma.h
> 96128 -r--r--r--   1 root  wheel  -      4.7K Jul 13 07:09
> rdma_cma_abi.h
> 
> I do buildworlds on almost a daily basis, so could this be a fallout
> of any kind?
> 
> It seems the installworld didn't got right
> 
> Thanks for your analysis - I delete, frankly, the header files as
> they reside in the base
> systems domain without a valid timestamp - they are considered
> hostile ;-)
> 
> the big scary question is now how i can prevent such a situation
> (possibly not if not
> staying with STABLE or RELEASE) or how can I clean the system without
> danger. I think
> some of my problems reported in the nearest past (half a year back)
> could be triggered by
> some nasty fallout of CURRENT - you might remember that I also had a
> strange lib path in
> the tree considered not valid and caused lots of problems.
> 
> Thank you anyway, it seems that emulators/qemu is now over the point
> of failure and I
> hope I'll get a working package now to start working on the AARCH64
> ports.
> 
> Thanks.
> 
> Kind regards,
> Oliver
> 
> 
> > 
> > If you don't want or care about rdma, you can try the following
> > patch
> > (should similarly apply to the other qemu ports):
> > 
> > Index: emulators/qemu/Makefile
> > ===================================================================
> > --- emulators/qemu/Makefile     (revision 429888)
> > +++ emulators/qemu/Makefile     (working copy)
> > _at__at_ -78,6 +78,7 _at__at_
> >                 --disable-libssh2 --enable-debug \
> >                 --prefix=${PREFIX} --cc=${CC} --enable-docs --
> > disable-kvm \
> >                 --disable-linux-user --disable-linux-aio --disable-
> > xen \
> > +               --disable-rdma \
> >                 --smbd=${LOCALBASE}/sbin/smbd --enable-debug-info
> > --python=${PYTHON_CMD} \ --extra-cflags=-I${WRKSRC}\
> > -I${LOCALBASE}/include\
> > -DPREFIX=\\\"\"${PREFIX}\\\"\"
> > 
> > -Dimitry
> > 
> 
> 

The rdma stuff is part of OFED, it comes from sys/ofed/include.  Other
parts of it are in sys/contrib/rmda and src/contrib/ofed.  Maybe it
only gets installed if you are using certain kernel options?  I'm not
sure.

-- Ian