Re: Very long SRCS list with unusually long src/ prefix

On Thu, 5 Feb 2004, Ruslan Ermilov wrote:

> On Fri, Feb 06, 2004 at 03:19:46AM +1100, Bruce Evans wrote:
> > On Thu, 5 Feb 2004, Ruslan Ermilov wrote:
> >
> > > On Fri, Feb 06, 2004 at 01:40:45AM +1100, Bruce Evans wrote:
> > > > On Thu, 5 Feb 2004, Ruslan Ermilov wrote:
> > > [...]
> > > > % +.for f in ${DPSRCS:M*.[cS]} ${DPSRCS:M*.cc} ${DPSRCS:M*.C} ${DPSRCS:M*.cpp} \
> > > > % +    ${DPSRCS:M*.cxx} ${DPSRCS:M*.m}
> > > > % +_mkdep_${f}: ${f}
> > > > % +	_at_echo ${.ALLSRC}
> > > > % +.endfor
> > > >
> > > > This seems to run echo once for each file separately.  It should be deemed
> > > > inefficient :-).
> > > >
> > > But that's what this patch basically does: it splits possibly very
> > > long ${.ALLSRC} list into pieces.  It's very inefficient only in
> > > traditional mode.  Run this with and without -j1 to see the diff:
> > >
> > > : COUNT!=	jot 1000
> > > :
> > > : all:
> > > : .for foo in ${COUNT}
> > > : 	_at_echo $$$$ >/dev/null
> > > : .endfor
> >
> > That is slow too.  I get 12.12 seconds with make, 2.36 with make -j1,
> > 1.37 with a shell loop and bash-1, 1.35 with a shell loop and /bin/sh,
> > and 0.0042 with jot 1000 >/dev/null.
> >
> Shell loops aren't an option here.  I realize that we loose speed here,
> but we gain a feature.

Ah, but I always choose speed over features (except reverse creeping ones) :-).

> > I wonder about all those little dependencies too.  make already takes
> > a noticeable amount of time to handle large subdirs (here it takes 0.48
> > seconds for a null "make depend").
                                    ^ in libc
> >
> I see the 0.18 secs increase for "make depend" in bin/cat.  I see no
> increase when .depend is not stale (since this code is not run then).
> What do you mean by a ``null "make depend"''?

0.18 seconds in bin/cat is huge.  A null "make depend" is when .depend is
not stale (an maybe all relevant disk blocks are cached).

> > > > %  ${DEPENDFILE}: ${DPSRCS}
> > > > %  	rm -f ${DEPENDFILE}
> > > > %  .if !empty(DPSRCS:M*.[cS])
> > > > % -	${MKDEPCMD} -f ${DEPENDFILE} -a ${MKDEP} \
> > > > % -	    ${CFLAGS:M-nostdinc*} ${CFLAGS:M-[BID]*} \
> > > > % -	    ${.ALLSRC:M*.[cS]}
> > > > % +	(cd ${.CURDIR}; ${MAKE} _mkdep_cS) | xargs env \
> > > > % +	    ${MKDEPCMD} -f ${DEPENDFILE} -a ${MKDEP} \
> > > > % +	    ${CFLAGS:M-nostdinc*} ${CFLAGS:M-[BID]*}
> > > >
> > > > make uses a real shell, so the env shouldn't be needed.
> > > >
> > > $ echo foo.c | xargs CC=cc mkdep
> > > xargs: CC=cc: No such file or directory
> >
> > $ echo foo.c | CC=cc xargs mkdep
> >
> Yes, but ``grep ^MKDEPCMD bsd.dep.mk'' to see what I'm talking
> about.

I know.  MKDEPCMD should be put in the environment before execing
xargs instead of after to avoid the env process.  It seems to the
only thing that needs to be put in the environment here.  See the
similar cleanup in kern.post.mk.

Bruce