Re: Benchmarking

Kirk Strauser wrote:

> Prepare to be flamed (not by me, but in general).  The tests are littered
> with comments like:
> 
>     Missing data in the above table signifies operations that were too fast
>     to measure correctly by the bonnie++ program.

> People will wonder why you chose not to increase the number of files being
> tested so as to get meaningful results.  Viewers are left looking at a
> mostly-empty table where the BSDs dominate the results by a large margin,
> but are told:

Id really didn't occurr to me until it was already to late (too many 
systems were benchmarked already).

>     Linux clearly wins the IO throughput test, having a score upto about
>     130% better than nearest BSD, either by having a better SCSI driver, or
>     because the system itself is just faster.
> 
> Since you didn't *publish* any results were Linux won, I'll have to assume
> that your private data demonstrated a different conclusion.

And what do you call the results of bonnie++ sequential IO output test 
(in the table)? :) This is what I was reffering to in the cited text.

> You describe your custom FreeBSD 5 kernel as keeping the WITNESS options.

Really? Where do I say that? The RELEASE kernel of FreeBSD are without 
WITNESS, and my CUSTOM kernel was based on it. The options I gave were 
the differences between CUSTOM and GENERIC.

> In NetBSD, you say:
> 
>     Because of my clumsiness with the installer, or the presence of some
>     bugs regarding modifying a FreeBSD partition setup, I wasn"t able to
>     partition the drive as intended, but instead the default partition
>     scheme was used.
> 
> Full stop.  End of test.  You're now comparing bananas to pencils.

I admit NetBSD was sort of an afterthought, but do you really mean that 
the physical position and of the /bench slice would affect the tests in 
significant ways? Even if the bonnie++ results are completely invalid, 
other results remain valid.

> Basically, you ran some tests on divergent systems and got some results, but
> that's about the only conclusion I was able to get from it.

You're perfectly entitled to :)