Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

On 12/23/11 07:47, Martin Sugioarto wrote:
> Am Fri, 23 Dec 2011 02:17:00 +0100
> schrieb "O. Hartmann" <ohartman_at_zedat.fu-berlin.de>:
> 
>> Benchmarks also could lead developers to look into more details of the
>> weak points of their OS, if they're open for that. Therefore,
>> benchmarks are very useful. But not if any real fault of the OS is
>> excused by a faulty becnhmarking.
> 
> Hi,
> 
> it is important for the project to be known and I think that the
> benchmarks made by Phoronix help FreeBSD to gain popularity, even they
> look bad sometimes.
> 
> Furthermore, to make a benchmark is a lot of work and the results are
> useful, because at the end someone will look at it and will try to
> improve the results. Thank you for investing your time.
> 
> I remember that I've made some tests with different platforms i386 vs
> amd64 with simple tools like "openssl speed" some time ago and got some
> bad results for amd64 that no one cared to explain. These bad results
> weren't reflected on Linux that I tested later for comparison. And most
> people have a weird attitude to think that the tester measures wrong
> instead of taking a look at it. They forget that as a FreeBSD user you
> would rather see FreeBSD win over Linux.
> 
> I've seen that Phoronix made various benchmarks about FreeBSD compared
> to Linux and I can tell you that _subjectively_ the benchmarks reflect
> what I always thought about FreeBSD. I simply _know_ that FreeBSD is
> worse in concurrency behavior, I know that it has I/O trouble, I know
> that it is mostly faster emulating 3D games than Linux runs them
> natively. I knew this already _before_ you published the benchmark
> about the 3D performance.
> 
> I cannot see any evil intentions in these benchmarks. All I can see is
> the wrong attitude _here_. If anyone thinks that Phoronix makes bad
> benchmarks, they should do these benchmarks by themselves and publish
> the results. As long as no one tries, Phoronix stays the best reference
> for me and for everyone else.

There IS NO EVIL INTENTION, except, hypothetical, the benchmarker is of
the age were he is called a "beardless".
But: In many articles, there is a very distinguished and underlined
emphasizing of Linux that makes me feeling people have their
Linux-glasses on. Linux is not UNIX! And if today someone tells me about
the Linux-graphical subsystem X11, I turn green in my face ... X11 was,
in former days, a development made on UNIX and is adopted by Linux. Ok,
we all know that, most of all ...


And the aspect of reference: I agree. They do something and this thread
arose while they did.

> 
> And don't forget, benchmarks can never be objective enough and someone
> will always be mad about the results. Especially, when you present them
> a "versus battle".
> 
> A further thing is that I cannot understand the people here sometimes.
> I would like that the -RELEASE versions of FreeBSD perform well without
> any further optimizations. When the distribution does not compile with
> the latest compiler it's simply a bug. Why should one try to penalize
> the other distribution and downgrade their binaries? When FreeBSD has a
> bad default setup, there must be a reason for that. Tell me this reason
> and show me that it's justified in form of some other benchmark.

Well, look at the mailing list. FreeBSD is handled via this list since
we are spread around the globe and even the developers are spread worldwide.
But when it comes to detecting worse performnce and someone isn't
capable of giving a detailed insight and mostly scientific way of
investigating the fault he experienced, the discussion, if it starts,
get drowned by allegations like "bad testing", worse optimizations blabla.

Look at Steve Kargls problem. He investigated a SCHED_ULE problem in a
way that is far beyond enough! He gave tests, insights of his setup, bad
performance compared to SCHED_4BSD and what happend? We are still stuck
with this problem and more and more people realise, that FreeBSD does
have somewhere a problem and this seems to be a nasty problem not easy
to find or investigate. But look at how Steve has been silenced in the
past ...
Benchmarks, especially published ones, reveal those pits and soemone
could look into it.
Another problem is this very elite-feeling closed club. Once you managed
it getting into the club of committers or core team members, you'll
probably fight for your seat ...
I dont propose for that socialists crap Linux people tend to be like,
overcrowded townhalls full of important people with non-consense
opinions. The other extreme end of this spectrum.
I can not change this. And I do not know whether there is a real
way-in-the-middle. But I follow the illusion that if people can see what
benchmarks reveal, they start thinking and if the facts are starting to
give a heavy load load on those rejecting the facts, they migght change
their opinion or get hopefully replaced by more openminded people. A Vision.


Oliver