Re: [TEST] make -j patch [take 2]

In message <20041112130548.GA86162_at_ip.net.ua>, Ruslan Ermilov writes:

>> At the very least, do not commit your patch until you have managed
>> to come up with at least one instance of real world data where it
>> is a good idea.
>>
>I gave you one example already, and Harti gave you another.
>
>"make universe" builds 6 worlds today.  By starting it with
>"make -j12", you build 6 worlds in parallel, and another 6
>jobs are made available for building these worlds.  When one
>world finishes, 7 jobs will be available for building 5 other
>worlds, etc.  This is fine as long as you're interested in
>the final result of "make universe".

You have not actually _tried_ running make universe with
my patch, have you ?

I thought so :-(

make -j 12 universe will make 12 jobs available.  The main
make will grab six of these to start the buildworlds, and
those six buildworlds compete for the remaining 6 tokens.

The result is that those six buildworlds progress _in parallel_
each of them using as many tokens as they can lay their hands on
at any one time.

If you want them to have more slots, say "-j 18" instead
and they will have 12 slots to compete for.

This is, as far as I can tell, just the thing you want to happen,
only it doesn't suffer from pointless compartementalization
of resources such as in your proposal.

The crucial point is that if at one particular instance the
i386.buildworld needs only one token, it will only claim
one token, and the other jobs can get more tokens and start
more things.  Once the i386 needs to run more things in
parallel, it will compete for tokens again and run as much
in parallel as it can get tokens for.

In your scenario, the tokens not used by i386 cannot be
used by the other buildworlds and they are wasted.

>If you're by chance familiar with how you can ask Oracle to
>optimize your queries, you can either optimize for ALL ROWS or
>for FIRST ROWS, and it's up to you to decide on this.  The
>"rows" in "make universe" are results of doing buildworld
>and buildkernel, and you have control over this.

This is not a database.

This is not even a cartesian problem space.

>Do you disagree with anything of the above?

Yes, I disagree with pretty much all of it.  It is clear that
you have not actually tried running with the code which is in
the tree.

>Can you agree to the below patch?  It doesn't change anything
>except when JFLAG is set explicitly to request another level
>of parallelism for building individual worlds and kernels?

The patch is pointless, but harmless once people learn not to set
JFLAG.

But I will not agree to this patch _until_ you show me _actual_ _real
_life_ data that shows it performs better than just running without
a JFLAG at all.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk_at_FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.