Re: A head buildworld race visible in the ci.freebsd.org build history

On Mon, Jun 18, 2018 at 12:42:46PM -0700, Bryan Drewery wrote:
> On 6/15/2018 10:55 PM, Mark Millard wrote:
> > In watching ci.freebsd.org builds I've seen a notable
> > number of one time failures, such as (example from
> > powerpc64):
> > 
> > --- all_subdir_lib/libufs ---
> > ranlib -D libufs.a
> > ranlib: fatal: Failed to open 'libufs.a'
> > *** [libufs.a] Error code 70
> > 
> > where the next build works despite the change being
> > irrelevant to whatever ranlib complained about.
> > 
> > Other builds failed similarly:
> > 
> > --- all_subdir_lib/libbsm ---
> > ranlib -D libbsm_p.a
> > ranlib: fatal: Failed to open 'libbsm_p.a'
> > *** [libbsm_p.a] Error code 70
> > 
> > and:
> > 
> > --- kerberos5/lib__L ---
> > ranlib -D libgssapi_spnego_p.a
> > --- libgssapi_spnego.a ---
> > ranlib -D libgssapi_spnego.a
> > --- libgssapi_spnego_p.a ---
> > ranlib: fatal: Failed to open 'libgssapi_spnego_p.a'
> > *** [libgssapi_spnego_p.a] Error code 70
> > 
> > and so on.
> > 
> > 
> > It is not limited to powerpc64. For example, for aarch64
> > there are:
> > 
> > --- libpam_exec.a ---
> > building static pam_exec library
> > ar -crD libpam_exec.a `NM='nm' NMFLAGS=''  lorder pam_exec.o  | tsort -q` 
> > ranlib -D libpam_exec.a
> > ranlib: fatal: Failed to open 'libpam_exec.a'
> > *** [libpam_exec.a] Error code 70
> > 
> > and:
> > 
> > --- all_subdir_lib/libusb ---
> > ranlib -D libusb.a
> > ranlib: fatal: Failed to open 'libusb.a'
> > *** [libusb.a] Error code 70
> > 
> > and:
> > 
> > --- all_subdir_lib/libbsnmp ---
> > ranlib: fatal: Failed to open 'libbsnmp.a'
> > --- all_subdir_lib/ncurses ---
> > --- all_subdir_lib/ncurses/panelw ---
> > --- panel.pico ---
> > --- all_subdir_lib/libbsnmp ---
> > *** [libbsnmp.a] Error code 70
> > 
> > 
> > Even amd64 gets such:
> > 
> > --- libpcap.a ---
> > ranlib -D libpcap.a
> > ranlib: fatal: Failed to open 'libpcap.a'
> > *** [libpcap.a] Error code 70
> > 
> > and:
> > 
> > 
> > --- libkafs5.a ---
> > ranlib: fatal: Failed to open 'libkafs5.a'
> > --- libkafs5_p.a ---
> > ranlib: fatal: Failed to open 'libkafs5_p.a'
> > --- cddl/lib__L ---
> > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/lua/lbaselib.c:60:26: note: include the header <ctype.h> or explicitly provide a declaration for 'toupper'
> > --- kerberos5/lib__L ---
> > *** [libkafs5_p.a] Error code 70
> > 
> > make[5]: stopped in /usr/src/kerberos5/lib/libkafs5
> > --- libkafs5.a ---
> > *** [libkafs5.a] Error code 70
> > 
> > and:
> > 
> > 
> > --- lib__L ---
> > ranlib -D libclang_rt.asan_cxx-i386.a
> > ranlib: fatal: Failed to open 'libclang_rt.asan_cxx-i386.a'
> > *** [libclang_rt.asan_cxx-i386.a] Error code 70
> > 
> > 
> > (Notice the variability in what .a the ranlib's fail for.)
> > 
> > 
> > 
> > 
> > 
> 
> 
> I looked at this a few days ago and don't believe it's actually a build
> race. I think there is something wrong with the ar/ranlib on that system
> or something else. I've found no evidence of concurrent building of the
> .a files in question.

FWIW, I got the similar failure when I did last checks for the OFED
commit.  For me, it was libgcc.a.