Re: Benchmark (Phoronix): FreeBSD 9.0-RC2 vs. Oracle Linux 6.1 Server

From: Igor Mozolevsky <igor_at_hybrid-lab.co.uk>
Date: Thu, 22 Dec 2011 09:56:55 +0000
On 22 December 2011 05:54, Daniel Kalchev <daniel_at_digsys.bg> wrote:
>
>
> On 22.12.11 00:33, Igor Mozolevsky wrote:
>>
>> Using the same argument one can say that Ferrari F430 vs Toyota Prius is a
>> meaningless comparison because the under-the-hood equipment is different.
>
>  Of course, it is meaningless, the Ferrari will lose big time in the fuel
> consumption comparison! I believe it will also lose the price comparison as
> well. Not to speak the availability comparison.

That's an oxymoron, right? The comparison cannot be meaningless---the
reality is F430 will indeed use up more fuel than Prius. If a
benchmark demonstrates a true reality, how can that benchmark be
possibly meaningless??? Same benchmark might be irrelevant to someone
who wants to know how fast they can get from A to B, but irrelevant is
not a synonym for meaningless!

> You say that comparison is meaningless, yet you intend to compare those two
> cars?

I didn't say that at all, I was demonstrating fallacy of the argument
that the comparisons were meaningless.

> Any 'benchmark' has a goal. You first define the goal and then measure how
> different contenders achieve it. Reaching the goal may have several
> measurable metrics, that you will use to later declare the winner in each.
> Besides, you need to define a baseline and be aware of what theoretical
> max/min values are possible.

Treating a benchmark as a binary win/lose is rather naive, it's not a
competition, and (I hope) no serious person ever does that. A proper
benchmark shows true strength and weaknesses so than a well-informed
intelligent decision can be taken by an individual according to that
individual's needs. The caveat, of course, is making your methodology
clear and methods repeatable!


Cheers,

--
Igor M.
Received on Thu Dec 22 2011 - 08:57:37 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:22 UTC