Re: Second "RFC" on pkg-data idea for ports

On Wed, Apr 14, 2004 at 09:09:36PM -0400, Garance A Drosihn wrote:
> At 4:29 PM -0700 4/14/04, Kris Kennaway wrote:
> >On Wed, Apr 14, 2004, Robin Schoonover wrote:
> > >
> > > I use make -V a lot, and it's slow (every time you run it, make
> > > has to reread all the bsd.*.mk files, such as bsd.port.mk).  The
> > > speed isn't much of an issue when you only do one or two ports,
> > > but when you are examining the entire ports collection, you notice.
> > >
> >> That said, I'd still rather use a makefile based ports system anyway.
> >
> >Necessarily, *any* file format you choose will need to parse
> >auxilliary files analogous to bsd.port.mk.  There's just no getting
> >around the fact that ports rely on a lot of infrastructure and
> >conditional evaluation to set their variables (although it can be
> >optimized relative to what we have in CVS today [1]).
> >
> >Note that it's intentional that a lot of things are centralized
> >in bsd.port.mk where they may be easily maintained, instead of
> >being set in 10000 individual makefiles.
> >
> >Kris
> >
> >[1] As a test, I recently was able to cut index build times by
> >60% from 5 to a little over 2 minutes on test box with fast disks,
> >by stripping out (almost) everything non-essential from the 'make
> >describe' code path.
> 
> Personally, I think you can get quite a penalty by trying to
> perform too much string-manipulation by using make/sh variables
> combined with all kinds of fancy invocations of sed, awk, etc.
> In other situations (which are totally unrelated to ports), I
> have greatly improved performance of some operation by replacing
> some clever shell scripts with ruby or perl.  Neither of those
> are speed demons compared to C, but they make a huge difference
> for something which is using sed/awk for lots of low-level string
> manipulation.
> 
> My hope is that if I get far enough along into the pkg-data project,
> the result would be that many of the common operations would be
> faster.  However, right now I can only say "that is one of my
> goals", and I can't prove it would actually happen...

There's only a few things that execute external commands in the common
code path of port makefiles.  I've been working hard to remove or
limit them, and as I mentioned above it's possible to optimize the
current framework a lot further without too much hard work.

Kris