Re: A head buildworld race visible in the ci.freebsd.org build history

On Mon, Jun 18, 2018 at 6:27 PM Bryan Drewery <bdrewery_at_freebsd.org> wrote:
>
> On 6/18/2018 1:45 PM, Konstantin Belousov wrote:
> > On Mon, Jun 18, 2018 at 12:42:46PM -0700, Bryan Drewery wrote:
> >> On 6/15/2018 10:55 PM, Mark Millard wrote:
> >>> In watching ci.freebsd.org builds I've seen a notable
> >>> number of one time failures, such as (example from
> >>> powerpc64):
> >>>
> >>> --- all_subdir_lib/libufs ---
> >>> ranlib -D libufs.a
> >>> ranlib: fatal: Failed to open 'libufs.a'
> >>> *** [libufs.a] Error code 70
> >>>
> >>> where the next build works despite the change being
> >>> irrelevant to whatever ranlib complained about.
> >>>
> >>> Other builds failed similarly:
> >>>
> >>> --- all_subdir_lib/libbsm ---
> >>> ranlib -D libbsm_p.a
> >>> ranlib: fatal: Failed to open 'libbsm_p.a'
> >>> *** [libbsm_p.a] Error code 70
> >>>
> >>> and:
> >>>
> >>> --- kerberos5/lib__L ---
> >>> ranlib -D libgssapi_spnego_p.a
> >>> --- libgssapi_spnego.a ---
> >>> ranlib -D libgssapi_spnego.a
> >>> --- libgssapi_spnego_p.a ---
> >>> ranlib: fatal: Failed to open 'libgssapi_spnego_p.a'
> >>> *** [libgssapi_spnego_p.a] Error code 70
> >>>
> >>> and so on.
> >>>
> >>>
> >>> It is not limited to powerpc64. For example, for aarch64
> >>> there are:
> >>>
> >>> --- libpam_exec.a ---
> >>> building static pam_exec library
> >>> ar -crD libpam_exec.a `NM='nm' NMFLAGS=''  lorder pam_exec.o  | tsort -q`
> >>> ranlib -D libpam_exec.a
> >>> ranlib: fatal: Failed to open 'libpam_exec.a'
> >>> *** [libpam_exec.a] Error code 70
> >>>
> >>> and:
> >>>
> >>> --- all_subdir_lib/libusb ---
> >>> ranlib -D libusb.a
> >>> ranlib: fatal: Failed to open 'libusb.a'
> >>> *** [libusb.a] Error code 70
> >>>
> >>> and:
> >>>
> >>> --- all_subdir_lib/libbsnmp ---
> >>> ranlib: fatal: Failed to open 'libbsnmp.a'
> >>> --- all_subdir_lib/ncurses ---
> >>> --- all_subdir_lib/ncurses/panelw ---
> >>> --- panel.pico ---
> >>> --- all_subdir_lib/libbsnmp ---
> >>> *** [libbsnmp.a] Error code 70
> >>>
> >>>
> >>> Even amd64 gets such:
> >>>
> >>> --- libpcap.a ---
> >>> ranlib -D libpcap.a
> >>> ranlib: fatal: Failed to open 'libpcap.a'
> >>> *** [libpcap.a] Error code 70
> >>>
> >>> and:
> >>>
> >>>
> >>> --- libkafs5.a ---
> >>> ranlib: fatal: Failed to open 'libkafs5.a'
> >>> --- libkafs5_p.a ---
> >>> ranlib: fatal: Failed to open 'libkafs5_p.a'
> >>> --- cddl/lib__L ---
> >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua/lbaselib.c:60:26: note: include the header <ctype.h> or explicitly provide a declaration for 'toupper'
> >>> --- kerberos5/lib__L ---
> >>> *** [libkafs5_p.a] Error code 70
> >>>
> >>> make[5]: stopped in /usr/src/kerberos5/lib/libkafs5
> >>> --- libkafs5.a ---
> >>> *** [libkafs5.a] Error code 70
> >>>
> >>> and:
> >>>
> >>>
> >>> --- lib__L ---
> >>> ranlib -D libclang_rt.asan_cxx-i386.a
> >>> ranlib: fatal: Failed to open 'libclang_rt.asan_cxx-i386.a'
> >>> *** [libclang_rt.asan_cxx-i386.a] Error code 70
> >>>
> >>>
> >>> (Notice the variability in what .a the ranlib's fail for.)
> >>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >> I looked at this a few days ago and don't believe it's actually a build
> >> race. I think there is something wrong with the ar/ranlib on that system
> >> or something else. I've found no evidence of concurrent building of the
> >> .a files in question.
> >
> > FWIW, I got the similar failure when I did last checks for the OFED
> > commit.  For me, it was libgcc.a.
> >
>
> If it was -lgcc_s then it's a known rare build race due to
> tools/install.sh not handling -S.

It seems a more general problem, this one:

https://ci.freebsd.org/job/FreeBSD-head-aarch64-build/8190/console

calls for libcuse_p.a, while this one:

https://ci.freebsd.org/job/FreeBSD-head-mips-build/2919/console

calls for libfifolog.a

-- 
Li-Wen Hsu <lwhsu_at_FreeBSD.org>
https://lwhsu.org