Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

From: O. Hartmann <ohartman_at_mail.zedat.fu-berlin.de>
Date: Mon, 19 Dec 2011 14:40:13 +0100
On 12/19/11 13:21, Andreas Nilsson wrote:
> On 19 dec 2011, at 12:50, "Samuel J. Greear" <sjg_at_evilcode.net> wrote:
> 
>> 2011/12/19 Lev Serebryakov <lev_at_freebsd.org>:
>>> Hello, Samuel.
>>> You wrote 15 декабря 2011 г., 16:32:47:
>>>
>>>> Other benchmarks in the Phoronix suite and their representations are
>>>> similarly flawed, _ALL_ of these results should be ignored and no time
>>>> should be wasted by any FreeBSD committer further evaluating this
>>>> garbage. (Yes, I have been down this rabbit hole).
>>>  Here is one problem: we have choice from three items:
>>>
>>> (1) Make FreeBSD looks good on benchmarks by "fixing" FreeBSD
>>>
>>> (2) Make FreeBSD looks good on benchmarks by "fixing" Phoronix
>>> (communication with them, convincing, that they benchamrks are unfare
>>> / meaningless, ets)
>>>
>>> (3) Lose [potential] userbase.
>>>
>>>  You know, that these benchmarks are bad. I know. But potential (and
>>>  even some current!) user doesn't. And it seems, that these benchmarks
>>>  become popular over Internet.
>>>
>>> --
>>> // Black Lion AKA Lev Serebryakov <lev_at_FreeBSD.org>
>>>
>>
>> Here is where you completely derail the train, let me paste again what
>> I said before.
>>
>> ...
>> Take the first test as an example, Blogbench read. This doesn't raise
>> any red flags, right? At least not until you realize that Blogbench
>> isn't a read test, it's a read/write test. So what they have done here
>> is run a read/write test and then thrown away the write results for
>> both platforms and reported only the read results. If you dig down
>> into the actual results,
>> http://openbenchmarking.org/result/1112113-AR-ORACLELIN37 -- you will
>> see two Blogbench numbers, one for read and another for write. These
>> were both taken from the same Blogbench run, so FreeBSD optimizes
>> writes over reads, that's probably a good thing for your data but a
>> bad thing when someone totally misrepresents benchmark results.
>> ...
>>
>> FreeBSD actually does _BETTER_ (subjectively) in this test than the
>> Linux system when you look at what is really going on. FreeBSD is
>> favoring writes, which is _GOOD_. FreeBSD does not need to be fixed,
>> the benchmarks need to be fixed to represent reality rather than
>> throwing half of the results in the trash. To be quite frank, "fixing"
>> FreeBSD to look good on this benchmark will make it a worse real-world
>> OS. But you guys go ahead and foot-shoot over these ridiculous
>> benchmarks all you want.
>>
>> Sam
>>
> 
> I seem to remember that before ULE people were fleeing to Linux as the
> os to run apache on since 4BSD didn't scale all too well. That may
> have changed over time though.
> 
> However ULE could perhaps be made aware technologies like turbo-boost,
> ie with few threads higher performance might be gained by utilizing
> all virtual cores on a physical core before spreading tasks to too
> different cores.
> 
> Just my speculations though :)
> 
> Regards
> Andreas Nilsson

Such a scheduling stratey is definitely necessary on AMDs new
"Bulldozer" architecture, which seems to be very pitty about threads
locked on the same module.
Microsoft just offered a patch for Windows 7 to implant such a
"Bulldozer" awarenes but they withdraw the patch as invalid two days
after the release. The seults seem to favour FPU performance over
integer performance.

As Samuel Greear wrote, FreeBSD looks not that bad in some of the
benchmarks but there are obviosly issues, at least the fact that
Phoronix/openbenchmark.org are the only sites offering benchmarks at all.

People outside the FreeBSD realm looking for opportunities, what do you
think they will look first after?
Phoronix/Openbenchmark.org made the first step and they seem to make
FreeBSD look bad (in my opinion), whether righteous or not. Compared to
several subjective impressions I have in our heterogeneous environment
at the lab, Linux on the same hardware looks in several aspects much better.

Oliver


Received on Mon Dec 19 2011 - 12:40:21 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:22 UTC