RFC: less chatty system builds

From: Luigi Rizzo <rizzo_at_iet.unipi.it>
Date: Mon, 16 Dec 2013 19:46:26 +0100
The following is a proof-of-concept patch to make system builds
less chatty.

It also has the nice side effect of showing more clearly
which rules are used during the build and possibly help
debugging the share/mk files and the individual Makefiles.

The logic is the following:
the environment/make variable SILENT (or any other name we may want
to use; linux defaults to quiet mode and uses V=1 to be as verbose
as we are), is analysed in one of the top *.mk files (i used
bsd.own.mk) to conditionally create ECHO_CC, ECHO_CXX, ... SILENT_CMD
variables.  Then individual rules in the makefiles are extended
with an extra ${ECHO_...} (non-null to print the brief form of the
command) and each command is prefix ${SILENT_CMD} on each line
(which becomes _at_ when we want to be quiet)

I could not find a less intrusive way. Ideally I would prepend a '_at_'
to ${CC} and other commands, but Make only supports
appending ( += ) but not prepending text to a variable.

The output becomes something like this:

	...
	LIB CC       /media/bsd10/usr/home/luigi/FreeBSD/head/lib/libcompiler_rt/../../contrib/compiler-rt/lib/truncdfsf2.c
	LIB CC       /media/bsd10/usr/home/luigi/FreeBSD/head/lib/libcompiler_rt/../../contrib/compiler-rt/lib/divsi3.c
	LIB CC       /media/bsd10/usr/home/luigi/FreeBSD/head/lib/libcompiler_rt/../../contrib/compiler-rt/lib/modsi3.c
	LIB CC       /media/bsd10/usr/home/luigi/FreeBSD/head/lib/libcompiler_rt/../../contrib/compiler-rt/lib/udivsi3.c
	LIB CC       /media/bsd10/usr/home/luigi/FreeBSD/head/lib/libcompiler_rt/../../contrib/compiler-rt/lib/umodsi3.c
	building static compiler_rt library
	ranlib libcompiler_rt.a
	...
	PROG MKDEP    /media/bsd10/usr/home/luigi/FreeBSD/obj_head/media/bsd10/usr/home/luigi/FreeBSD/head/tmp/media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/.depend
	PROG MKDEP    /media/bsd10/usr/home/luigi/FreeBSD/obj_head/media/bsd10/usr/home/luigi/FreeBSD/head/tmp/media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/.depend ______EXTRADEPEND________
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_main.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_clntout.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_cout.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_hout.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_parse.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_sample.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_scan.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_svcout.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_tblout.c
	PROG CC       /media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpc_util.c
	PROG LD       /media/bsd10/usr/home/luigi/FreeBSD/obj_head/media/bsd10/usr/home/luigi/FreeBSD/head/tmp/media/bsd10/usr/home/luigi/FreeBSD/head/usr.bin/rpcgen/rpcgen SRCS
	PROG INSTALL _proginstall
	...

and for the kernel

	...
	CC      /media/bsd10/usr/home/luigi/FreeBSD/head/sys/dev/virtio/virtio.c
	CC      /media/bsd10/usr/home/luigi/FreeBSD/head/sys/dev/virtio/virtqueue.c
	AWK+CC  /media/bsd10/usr/home/luigi/FreeBSD/head/sys/dev/virtio/virtio_bus_if.m
	AWK+CC  /media/bsd10/usr/home/luigi/FreeBSD/head/sys/dev/virtio/virtio_if.m
	CC      /media/bsd10/usr/home/luigi/FreeBSD/head/sys/dev/virtio/pci/virtio_pci.c
	CC      /media/bsd10/usr/home/luigi/FreeBSD/head/sys/dev/virtio/network/if_vtnet.c
	CC      /media/bsd10/usr/home/luigi/FreeBSD/head/sys/isa/syscons_isa.c
	CC      /media/bsd10/usr/home/luigi/FreeBSD/head/sys/isa/vga_isa.c
	...

which I find incredibly more readable and useful than our default.

It might still be a little too chatty (e.g. we might try to trim part
of the pathnames).

--- QUESTIONS ABOUT OUR Makefiles and share/mk files

Apart from improving readability, with this patch i noticed the
following (which may be well known to our toolchain gurus):

- there are several programs in our source tree (e.g. sh, csh,
  others) that have their own ${CC} invocation in the makefiles --
  often to do the final link phase. They clearly show up in the build
  output because of the 400-character lines

	cc -Dthis_and_that -Oall_possible_things -Imost_paths...

  Maybe we could revise these things ?

- we do not seem to have a default rule .o: to link a single
  object, should we include it ?

cheers
luigi
(patch follows)


Index: bsd.dep.mk
===================================================================
--- bsd.dep.mk	(revision 259453)
+++ bsd.dep.mk	(working copy)
_at__at_ -135,14 +135,15 _at__at_
 
 DPSRCS+= ${SRCS}
 ${DEPENDFILE}: ${DPSRCS}
-	rm -f ${DEPENDFILE}
+	${ECHO_MKDEP}
+	${SILENT_CMD} rm -f ${DEPENDFILE}
 .if !empty(DPSRCS:M*.[cS])
-	${MKDEPCMD} -f ${DEPENDFILE} -a ${MKDEP} \
+	${SILENT_CMD} ${MKDEPCMD} -f ${DEPENDFILE} -a ${MKDEP} \
 	    ${MKDEP_CFLAGS} ${.ALLSRC:M*.[cS]}
 .endif
 .if !empty(DPSRCS:M*.cc) || !empty(DPSRCS:M*.C) || !empty(DPSRCS:M*.cpp) || \
     !empty(DPSRCS:M*.cxx)
-	${MKDEPCMD} -f ${DEPENDFILE} -a ${MKDEP} \
+	${SILENT_CMD} ${MKDEPCMD} -f ${DEPENDFILE} -a ${MKDEP} \
 	    ${MKDEP_CXXFLAGS} \
 	    ${.ALLSRC:M*.cc} ${.ALLSRC:M*.C} ${.ALLSRC:M*.cpp} ${.ALLSRC:M*.cxx}
 .endif
Index: bsd.lib.mk
===================================================================
--- bsd.lib.mk	(revision 259453)
+++ bsd.lib.mk	(working copy)
_at__at_ -4,6 +4,8 _at__at_
 
 .include <bsd.init.mk>
 
+STGT = LIB
+
 # Set up the variables controlling shared libraries.  After this section,
 # SHLIB_NAME will be defined only if we are to create a shared library.
 # SHLIB_LINK will be defined only if we are to create a link to it.
_at__at_ -67,25 +69,31 _at__at_
 PO_FLAG=-pg
 
 .c.o:
-	${CC} ${STATIC_CFLAGS} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
-	${CTFCONVERT_CMD}
+	${ECHO_CC}
+	${SILENT_CMD} ${CC} ${STATIC_CFLAGS} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+	${SILENT_CMD} ${CTFCONVERT_CMD}
 
 .c.po:
-	${CC} ${PO_FLAG} ${STATIC_CFLAGS} ${PO_CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
-	${CTFCONVERT_CMD}
+	${ECHO_CC} .po
+	${SILENT_CMD} ${CC} ${PO_FLAG} ${STATIC_CFLAGS} ${PO_CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+	${SILENT_CMD} ${CTFCONVERT_CMD}
 
 .c.So:
-	${CC} ${PICFLAG} -DPIC ${SHARED_CFLAGS} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
-	${CTFCONVERT_CMD}
+	${ECHO_CC} .So
+	${SILENT_CMD} ${CC} ${PICFLAG} -DPIC ${SHARED_CFLAGS} ${CFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+	${SILENT_CMD} ${CTFCONVERT_CMD}
 
 .cc.o .C.o .cpp.o .cxx.o:
-	${CXX} ${STATIC_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+	${ECHO_CXX}
+	${SILENT_CMD} ${CXX} ${STATIC_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
 
 .cc.po .C.po .cpp.po .cxx.po:
-	${CXX} ${PO_FLAG} ${STATIC_CXXFLAGS} ${PO_CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+	${ECHO_CXX} .po
+	${SILENT_CMD} ${CXX} ${PO_FLAG} ${STATIC_CXXFLAGS} ${PO_CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
 
 .cc.So .C.So .cpp.So .cxx.So:
-	${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
+	${ECHO_CXX} .So
+	${SILENT_CMD} ${CXX} ${PICFLAG} -DPIC ${SHARED_CXXFLAGS} ${CXXFLAGS} -c ${.IMPSRC} -o ${.TARGET}
 
 .f.po:
 	${FC} -pg ${FFLAGS} -o ${.TARGET} -c ${.IMPSRC}
Index: bsd.own.mk
===================================================================
--- bsd.own.mk	(revision 259453)
+++ bsd.own.mk	(working copy)
_at__at_ -678,4 +678,15 _at__at_
 # overriden by Makefiles, but the user may choose to set this in src.conf(5).
 TESTSBASE?= /usr/tests
 
+# support silent compilations
+# STGT is the part of system (sys, lib, etc.) we are building
+.if defined(SILENT)
+ECHO_CC=   _at_echo "${STGT} CC       ${.IMPSRC}"
+ECHO_CXX=  _at_echo "${STGT} CXX      ${.IMPSRC}"
+ECHO_LD=   _at_echo "${STGT} LD       `pwd`/${.TARGET}"
+ECHO_MKDEP=_at_echo "${STGT} MKDEP    `pwd`/${.TARGET}"
+ECHO_INSTALL=_at_echo "${STGT} INSTALL ${.TARGET}"
+SILENT_CMD=_at_
+.endif
+
 .endif	# !target(__<bsd.own.mk>__)
Index: bsd.prog.mk
===================================================================
--- bsd.prog.mk	(revision 259453)
+++ bsd.prog.mk	(working copy)
_at__at_ -3,6 +3,8 _at__at_
 
 .include <bsd.init.mk>
 
+STGT= PROG
+
 .SUFFIXES: .out .o .c .cc .cpp .cxx .C .m .y .l .ln .s .S .asm
 
 # XXX The use of COPTS in modern makefiles is discouraged.
_at__at_ -86,10 +88,11 _at__at_
 ${PROG_FULL}: beforelinking
 .endif
 ${PROG_FULL}: ${OBJS}
+	${ECHO_LD} SRCS
 .if defined(PROG_CXX)
-	${CXX} ${CXXFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+	${SILENT_CMD} ${CXX} ${CXXFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
 .else
-	${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+	${SILENT_CMD} ${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
 .endif
 .if ${MK_CTF} != "no"
 	${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${OBJS}
_at__at_ -116,13 +119,14 _at__at_
 ${PROG_FULL}: beforelinking
 .endif
 ${PROG_FULL}: ${OBJS}
+	${ECHO_LD} NO-SRCS
 .if defined(PROG_CXX)
-	${CXX} ${CXXFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+	${SILENT_CMD} ${CXX} ${CXXFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
 .else
-	${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
+	${SILENT_CMD} ${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${OBJS} ${LDADD}
 .endif
 .if ${MK_CTF} != "no"
-	${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${OBJS}
+	${SILENT_CMD} ${CTFMERGE} ${CTFFLAGS} -o ${.TARGET} ${OBJS}
 .endif
 .endif # !target(${PROG})
 
_at__at_ -166,17 +170,18 _at__at_
 
 .if defined(PROG)
 _EXTRADEPEND:
+	${ECHO_MKDEP} ______EXTRADEPEND________
 .if defined(LDFLAGS) && !empty(LDFLAGS:M-nostdlib)
 .if defined(DPADD) && !empty(DPADD)
-	echo ${PROG}: ${DPADD} >> ${DEPENDFILE}
+	${SILENT_CMD} echo ${PROG}: ${DPADD} >> ${DEPENDFILE}
 .endif
 .else
-	echo ${PROG}: ${LIBC} ${DPADD} >> ${DEPENDFILE}
+	${SILENT_CMD} echo ${PROG}: ${LIBC} ${DPADD} >> ${DEPENDFILE}
 .if defined(PROG_CXX) && !defined(EARLY_BUILD)
 .if ${MK_CLANG_IS_CC} != "no" && empty(CXXFLAGS:M-stdlib=libstdc++)
-	echo ${PROG}: ${LIBCPLUSPLUS} >> ${DEPENDFILE}
+	${SILENT_CMD} echo ${PROG}: ${LIBCPLUSPLUS} >> ${DEPENDFILE}
 .else
-	echo ${PROG}: ${LIBSTDCPLUSPLUS} >> ${DEPENDFILE}
+	${SILENT_CMD} echo ${PROG}: ${LIBSTDCPLUSPLUS} >> ${DEPENDFILE}
 .endif
 .endif
 .endif
_at__at_ -200,14 +205,15 _at__at_
 realinstall: _proginstall
 .ORDER: beforeinstall _proginstall
 _proginstall:
+	${ECHO_INSTALL}
 .if defined(PROG)
-	${INSTALL} ${STRIP} -o ${BINOWN} -g ${BINGRP} -m ${BINMODE} \
+	${SILENT_CMD} ${INSTALL} ${STRIP} -o ${BINOWN} -g ${BINGRP} -m ${BINMODE} \
 	    ${_INSTALLFLAGS} ${PROG} ${DESTDIR}${BINDIR}/${PROGNAME}
 .if ${MK_DEBUG_FILES} != "no"
 .if defined(DEBUGMKDIR)
-	${INSTALL} -T debug -d ${DESTDIR}${DEBUGFILEDIR}
+	${SILENT_CMD} ${INSTALL} -T debug -d ${DESTDIR}${DEBUGFILEDIR}
 .endif
-	${INSTALL} -T debug -o ${BINOWN} -g ${BINGRP} -m ${DEBUGMODE} \
+	${SILENT_CMD} ${INSTALL} -T debug -o ${BINOWN} -g ${BINGRP} -m ${DEBUGMODE} \
 	    ${PROGNAME}.debug ${DESTDIR}${DEBUGFILEDIR}/${PROGNAME}.debug
 .endif
 .endif
Index: sys.mk
===================================================================
--- sys.mk	(revision 259453)
+++ sys.mk	(working copy)
_at__at_ -4,6 +4,8 _at__at_
 unix		?=	We run FreeBSD, not UNIX.
 .FreeBSD	?=	true
 
+STGT = SYS
+
 .if !defined(%POSIX)
 #
 # MACHINE_CPUARCH defines a collection of MACHINE_ARCH.  Machines with
_at__at_ -167,6 +169,7 _at__at_
 
 # SINGLE SUFFIX RULES
 .c:
+	${ECHO_CC} POSIX SINGLE
 	${CC} ${CFLAGS} ${LDFLAGS} -o ${.TARGET} ${.IMPSRC}
 	${CTFCONVERT_CMD}
 
_at__at_ -181,6 +184,7 _at__at_
 # DOUBLE SUFFIX RULES
 
 .c.o:
+	${ECHO_CC} POSIX
 	${CC} ${CFLAGS} -c ${.IMPSRC}
 	${CTFCONVERT_CMD}
 
_at__at_ -211,6 +215,7 _at__at_
 	mv lex.yy.c ${.TARGET}
 
 .c.a:
+	${ECHO_CC} LIB POSIX
 	${CC} ${CFLAGS} -c ${.IMPSRC}
 	${AR} ${ARFLAGS} ${.TARGET} ${.PREFIX}.o
 	rm -f ${.PREFIX}.o
_at__at_ -237,22 +242,27 _at__at_
 	    touch ${.TARGET}
 
 .c:
-	${CC} ${CFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET}
-	${CTFCONVERT_CMD}
+	${ECHO_CC} SINGLE
+	${SILENT_CMD} ${CC} ${CFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET}
+	${SILENT_CMD} ${CTFCONVERT_CMD}
 
 .c.o:
-	${CC} ${CFLAGS} -c ${.IMPSRC}
-	${CTFCONVERT_CMD}
+	${ECHO_CC}
+	${SILENT_CMD}${CC} ${CFLAGS} -c ${.IMPSRC}
+	${SILENT_CMD}${CTFCONVERT_CMD}
 
 .cc .cpp .cxx .C:
-	${CXX} ${CXXFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET}
+	${ECHO_CXX} SINGLE
+	${SILENT_CMD} ${CXX} ${CXXFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET}
 
 .cc.o .cpp.o .cxx.o .C.o:
-	${CXX} ${CXXFLAGS} -c ${.IMPSRC}
+	${ECHO_CXX}
+	${SILENT_CMD}${CXX} ${CXXFLAGS} -c ${.IMPSRC}
 
 .m.o:
-	${OBJC} ${OBJCFLAGS} -c ${.IMPSRC}
-	${CTFCONVERT_CMD}
+	${ECHO_CC} OBJC
+	${SILENT_CMD} ${OBJC} ${OBJCFLAGS} -c ${.IMPSRC}
+	${SILENT_CMD} ${CTFCONVERT_CMD}
 
 .p.o:
 	${PC} ${PFLAGS} -c ${.IMPSRC}
_at__at_ -299,6 +309,7 _at__at_
 	${LEX} -t ${LFLAGS} ${.IMPSRC} > ${.TARGET}
 
 .s.out .c.out .o.out:
+	${ECHO_CC} LINK.out
 	${CC} ${CFLAGS} ${LDFLAGS} ${.IMPSRC} ${LDLIBS} -o ${.TARGET}
 	${CTFCONVERT_CMD}
 
Received on Mon Dec 16 2013 - 17:43:34 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:45 UTC