Re: 7+ days of dogfood

David Chisnall <David.Chisnall_at_cl.cam.ac.uk> wrote:

> On 11 Feb 2013, at 13:56, Fabian Keil <freebsd-listen_at_fabiankeil.de>
> wrote:
> 
> > real    350m42.363s
> > user    253m5.477s
> > sys     50m0.024s
> 
> These numbers look a bit wrong.  You've got 300 minutes of CPU time, but
> 350 minutes of real time.  In an ideal world, on your dual-core system
> you'd see 150 minutes of real time.  Seeing more than 300 implies that
> you're spending a lot of time waiting for I/O.  The normal
> recommendation is to use -j x where x is 1.5 times the number of cores,
> or 1x the number of GBs of RAM, whichever is smaller.  With only 2GB of
> RAM you might have linking problems with -j3, but it's still worth
> trying.

With only 2 GB of RAM (parts of which are needed elsewhere) I'm already
having linking problems without using -j at all and the I/O I'm waiting
for is the disk serving the swap partition.

I've used -j2 and occasionally -j4 when building world with gcc in
the past, but when using clang I'd risk temporarily having three
(or more) processes compete for swap space and bandwidth, so I
stopped doing that.

I'd expect that another 2 GB of RAM would prevent the swapping and
thus reduce the buildworld time quite a bit, but as I intend to
replace the system anyway I can't be bothered to investigate what
kind of RAM I'd need and where to get it.

> One of the more serious problems with our current build system is that
> it doesn't scale well to large numbers of cores.  On a 32-core system,
> with -j64, we're very rarely managing to have even 8 things able to run
> in parallel.  This should be addressed when the bmake import is fully
> integrated and we can use meta mode for better dependency tracking.  
>
> Ninja has a concept of pools, so you can say 'only run one link job at a
> time, but you can do two C++ compile jobs or 4 C compile jobs', and it
> might be interesting to look at adding something similar to bmake, as
> this can improve scalability a lot.

Unfortunately I don't have these issues (yet).

Fabian