make -j 4 is really make -j 8 for buildworld?

From: Robert Watson <rwatson_at_FreeBSD.org>
Date: Wed, 14 Jan 2004 00:22:12 -0500 (EST)
A few times in the past, I've noticed that when I run 'make -j4
buildworld', I frequently run with load averages between 8 and 10 on one
of my boxes at work.  Recently, I actually sat around with ps -ax for a
bit, and found that frequently, 8 different compiler/assembler instances
are running at once, rather than the four I expect:

cboss:/data/p4/rwatson/trustedbsd/audit2/sys/i386/compile/AUDIT> ps ax | grep make ; ps ax | grep cc ; ps ax | grep as
22892  p2  S      0:01.92 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make -f Makefile.inc1 par-all
23772  p2  S      0:01.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/
23992  p2  S      0:01.04 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/
39254  p2  S      0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/
39265  p2  S      0:01.41 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/cc_int/
40689  p2  S      0:10.78 make -j 4 buildworld
40717  p2  S      0:06.36 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make -m /data/p4/rwatson/trustedbsd/mac/share/mk -f Makefile.inc1 buildworld
42307  p2  S      0:00.30 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.bin/
43220  p2  S      0:00.21 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.sbin/
45194  p2  R      0:00.06 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.bin/indent/
45217  p2  R      0:00.07 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=usr.sbin/config/
...
39254  p2  S      0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/
39265  p2  S      0:01.41 /usr/obj/data/p4/rwatson/trustedbsd/mac/make.i386/make all DIRPRFX=gnu/usr.bin/cc/cc_int/
45169  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45170  p2  R      0:01.51 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
45198  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -c /data/p4/rwatson/trustedbsd/mac/usr.bin/indent/indent.c
45199  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -c /data/p4/rwatson/trustedbsd/mac/usr.bin/indent/io.c
45201  p2  R      0:00.77 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_LONGLONG /data/p4/rwatson/trustedbsd/mac/usr
45204  p2  R      0:00.79 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 -D_LONGLONG /data/p4/rwatson/trustedbsd/mac/usr
45212  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45213  p2  R      0:01.12 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
45231  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/main.c
45232  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c config.c
45233  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/mkmakefile.c
45234  p2  R      0:00.42 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45236  p2  R      0:00.44 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45239  p2  R      0:00.42 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45241  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -Wsystem-headers -Werror -c /data/p4/rwatson/trustedbsd/mac/usr.sbin/config/mkheaders.c
45242  p2  R      0:00.23 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I. -I/data/p4/rwatson/trustedbsd/mac/usr.sbin/config -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEV
45245  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45246  p2  R      0:00.50 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
45249  p2  S      0:00.01 cc -pipe -O -mcpu=pentiumpro -DIN_GCC -DHAVE_CONFIG_H -DPREFIX="/usr" -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trust
45250  p2  R      0:00.04 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/libexec/cc1 -quiet -I/data/obj/data/p4/rwatson/trustedbsd/mac/gnu/usr.bin/cc/cc_int/../cc_tools -I/data/p4/rwatson/trusted
...
45171  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgcleanup.o -
45202  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o io.o -
45205  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o indent.o -
45214  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfglayout.o -
45235  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o config.o -
45237  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o main.o -
45240  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o mkmakefile.o -
45243  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o mkheaders.o -
45247  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgloop.o -
45251  p2  S      0:00.01 /usr/obj/data/p4/rwatson/trustedbsd/mac/i386/usr/bin/as -o cfgrtl.o -

I'm all for parallelism, but there's "useful parallelism" and "excessive
parallelism".  Is this the intended behavior of make -jX, or am I running
into a bit shifting bug?  Is it just our buildworld bits losing some state
on the way through and trying a bit too hard to passing on the make
arguments? 

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert_at_fledge.watson.org      Senior Research Scientist, McAfee Research
Received on Tue Jan 13 2004 - 20:23:56 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:37:37 UTC