Re: RFC: less chatty system builds

From: Luigi Rizzo <rizzo_at_iet.unipi.it>
Date: Thu, 19 Dec 2013 01:38:09 +0100
On Mon, Dec 16, 2013 at 10:35 PM, Dimitry Andric <dim_at_freebsd.org> wrote:

> On 16 Dec 2013, at 19:46, Luigi Rizzo <rizzo_at_iet.unipi.it> wrote:
> > The following is a proof-of-concept patch to make system builds
> > less chatty.
> >
> > It also has the nice side effect of showing more clearly
> > which rules are used during the build and possibly help
> > debugging the share/mk files and the individual Makefiles.
> >
> > The logic is the following:
> > the environment/make variable SILENT (or any other name we may want
> > to use; linux defaults to quiet mode and uses V=1 to be as verbose
> > as we are),
>
> I cannot imagine I am the only one that dislikes Linux's approach of not
> showing exactly what it is doing, so I have no objection, as long as it
> is not the default.  (I really hate having to hunt around for the magic
> option to enable verbose output if I want to know how a program is
> compiled...)
>

thanks for the feedback.

Sure, default should remain unchanged per POLA.

As for the 'magic option to enable verbose output', i think
it is just a matter of being familiar with the specific system.

We have far less mnemonic options in FreeBSD (MAKEOBJDIRPREFIX,
KERNFAST) than the V=1 required by linux.



> Also, if you want "silent" builds, you should use make -s instead.  That
> is much less chatty than (IMHO) useless "CC foo", "LD bar" messages.


The issue with verbosity is that depending on the circumstances
you need different levels.

Definitely there must be something moving in all cases
(progress bar or percentage or scrolling text);
then in case of errors you may need to know the exact command line,
perhaps with something more such as what rule was used etc;
and maybe an intermediate output in other cases.

Our default is extremely verbose, even without -v, as
it shows the full command line for almost every command
(the mk files have relatively few _at_ prefixes).

It is extremely hard to spot warnings in the output.

In fact, it is so verbose that make -v is very similar.
The following is the output of "make toolchain" with and without -v

> ls -l /tmp/build*
-rw-r--r--  1 luigi  wheel  13058140 Dec 19 01:07 /tmp/build-gcc-v.out
-rw-r--r--  1 luigi  wheel  12360113 Dec 19 01:23 /tmp/build-gcc.out
> wc /tmp/build-gcc.out
   24825  478479 12360113 /tmp/build-gcc.out
> wc /tmp/build-gcc-v.out
   57676  576964 13058140 /tmp/build-gcc-v.out

as you can see the difference in size is only 10% (though there
are twice as many lines).


make -s as you suggest is more silent, but in some places it can be
minutes between individual lines. Below is the output with a small
modification to print a timestamp on each ECHODIR line:

...
00:02:18 ===> lib/clang (all)
00:02:18 ===> lib/clang/libclanganalysis (all)
00:02:59 ===> lib/clang/libclangarcmigrate (all)
00:04:54 ===> lib/clang/libclangast (all)
00:07:27 ===> lib/clang/libclangbasic (all)
00:07:40 ===> lib/clang/libclangcodegen (all)
00:10:16 ===> lib/clang/libclangdriver (all)
00:10:30 ===> lib/clang/libclangedit (all)
00:10:34 ===> lib/clang/libclangfrontend (all)
00:11:29 ===> lib/clang/libclangfrontendtool (all)
00:11:30 ===> lib/clang/libclanglex (all)
00:11:55 ===> lib/clang/libclangparse (all)
00:12:33 ===> lib/clang/libclangrewritecore (all)
00:12:36 ===> lib/clang/libclangrewritefrontend (all)
00:13:00 ===> lib/clang/libclangsema (all)
00:16:35 ===> lib/clang/libclangserialization (all)
00:17:19 ===> lib/clang/libclangstaticanalyzercheckers (all)
00:20:51 ===> lib/clang/libclangstaticanalyzercore (all)
00:22:36 ===> lib/clang/libclangstaticanalyzerfrontend (all)
00:22:47 ===> lib/clang/libllvmanalysis (all)
... (and we are not done yet)...

The following patch might be of some help to indicate progress:

Index: /home/luigi/FreeBSD/head/share/mk/sys.mk
===================================================================
--- /home/luigi/FreeBSD/head/share/mk/sys.mk    (revision 259578)
+++ /home/luigi/FreeBSD/head/share/mk/sys.mk    (working copy)
_at__at_ -84,12 +84,12 _at__at_
 CPP            ?=      cpp

 .if empty(.MAKEFLAGS:M-s)
-ECHO           ?=      echo
-ECHODIR                ?=      echo
+ECHO           ?=      echo `date +%H:%M:%S `
+ECHODIR                ?=      echo `date +%H:%M:%S `
 .else
 ECHO           ?=      true
 .if ${.MAKEFLAGS:M-s} == "-s"
-ECHODIR                ?=      echo
+ECHODIR                ?=      echo `date +%H:%M:%S `
 .else
 ECHODIR                ?=      true
 .endif

So coming back to my original proposal, I was suggesting the intermediate
mode to give people a better idea of what is going on during the build,
and make warnings and error messages stand out in the output.

In any case, if anything like this is implemented, I would really prefer
> something like CMake does, e.g. give you a percentage counter that
> provides some information about how 'far' the build is progressing.
>

Sure it would be great to also have that (as another extreme,
even less verbose than -s), but I believe it is impossible to
implement it, because the build system has no idea of how big
is the dependency tree without doing a full pass through it.
(This apart from the obvious issue that it is time that matters,
not the number of commands/targets).

cheers
luigi




>
> -Dimitry
>
>


-- 
-----------------------------------------+-------------------------------
 Prof. Luigi RIZZO, rizzo_at_iet.unipi.it  . Dip. di Ing. dell'Informazione
 http://www.iet.unipi.it/~luigi/        . Universita` di Pisa
 TEL      +39-050-2211611               . via Diotisalvi 2
 Mobile   +39-338-6809875               . 56122 PISA (Italy)
-----------------------------------------+-------------------------------
Received on Wed Dec 18 2013 - 23:38:12 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:45 UTC