Re: Let's use gcc-4.2, not 4.1 -- OpenMP

On Friday 15 December 2006 14:39, Poul-Henning Kamp wrote:
> In message <200612151250.10033.shoesoft_at_gmx.net>, Stefan Ehmann writes:
> >Settings/Compiler           | gcc-3.4 | gcc-4.1 | gcc-4.2
> >----------------------------+---------+---------+---------
> >-O2                         |  13.1bn |  13.8bn |  13.5bn
> >-O2 -funroll-loops          |   9.6bn |   9.3bn |   9.2bn
> >-O2 -march=athlon-xp -fun.. |   9.7bn |  10.6bn |  10.7bn
> >-O3                         |  11.5bn |   9.5bn |   9.6bn
> >-O3 -funroll-loops          |   8.4bn |   9.2bn |   9.4bn
> >-O3 -march=athlon-xp -fun.. |   8.8bn |  10.6bn |  11.1bn
>
> I love benchmarks.
>
> It's great when people benchmark things.
>
> Unfortunately, that is not what you have done, because you have
> not indicated what the standard deviation on your numbers are,
> so they are totally worthless.
I've done 3 runs on an otherwise pretty idle system with a maximum deviation 
of maybe 1 million instructions. So I figured that accurately calculating the 
standard deviation would overshoot the mark for this primitive test.

IMHO the much weaker point in my benchmark is using a single program and only 
instruction count. What I wanted to show is whether gcc4 can still be worse 
than gcc34 in some cases.

Sometimes performance counters can vary a lot (I've seen double the 
instructions on the p4 machine using papiex). So here are the results for 
the "-O3 -funroll-loops" row (using the output of 100 runs). Going on further 
seems pretty pointless to me.

Using a 99.7 confidence interval, I get these results:

-O3 -funroll-loops:
gcc-3.4: 8362606323 +/- 440336
gcc-4.1: 9246505378 +/- 531302
gcc-4.2: 9401195544 +/- 784106