Re: FreeBSD Compiler Benchmark: gcc-base vs. gcc-ports vs. clang

From: Poul-Henning Kamp <phk_at_phk.freebsd.dk>
Date: Fri, 11 Mar 2011 16:46:18 +0000
In message <4D7A42CC.8020807_at_FreeBSD.org>, Martin Matuska writes:

>But what I can say, e.g. for the Intel Atom processor, if there are
>performance gains in all but one test (that falls 2% behind), generic
>perl code (the routines benchmarked) on this processor is very likely to
>run faster with that setup.

No, actually you cannot say that, unless you run all the tests at
least three times for each compiler(+flag), calculate the average
and standard deviation of all the tests, and see which, if any of
the results are statistically significant.

Until you do that, you numbers are meaningless, because we have no
idea what the signal/noise ratio is.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk_at_FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.
Received on Fri Mar 11 2011 - 15:46:20 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:12 UTC