Re: Increasing MAXPHYS

On Mar 21, 2010, at 10:53 AM, Ulrich Spörlein wrote:
> [CC trimmed]
> On Sun, 21.03.2010 at 10:39:10 -0600, Scott Long wrote:
>> On Mar 21, 2010, at 10:30 AM, Ulrich Spörlein wrote:
>>> On Sat, 20.03.2010 at 12:17:33 -0600, Scott Long wrote:
>>>> Windows has a MAXPHYS equivalent of 1M.  Linux has an equivalent of an
>>>> odd number less than 512k.  For the purpose of benchmarking against these
>>>> OS's, having comparable capabilities is essential; Linux easily beats FreeBSD
>>>> in the silly-i/o-test because of the MAXPHYS difference (though FreeBSD typically
>>>> stomps linux in real I/O because of vastly better latency and caching algorithms).
>>>> I'm fine with raising MAXPHYS in production once the problems are addressed.
>>> 
>>> Hi Scott,
>>> 
>>> while I'm sure that most of the FreeBSD admins are aware of "silly"
>>> benchmarks where Linux I/O seems to dwarf FreeBSD, do you have some
>>> pointers regarding your statement that FreeBSD triumphs for real-world
>>> I/O loads? Can this be simulated using iozone, bonnie, etc? More
>>> importantly, is there a way to do this file system independently?
>>> 
>> 
>> iozone and bonnie tend to be good at testing serialized I/O latency; each read and write is serialized without any buffering.  My experience is that they give mixed results, sometimes they favor freebsd, sometime linux, sometimes it's a wash, all because they are so sensitive to latency.  And that's where is also gets hard to have a "universal" benchmark; what are you really trying to model, and how does that model reflect your actual workload?  Are you running a single-instance, single threaded application that is sensitive to latency?  Are you running a multi-instance/multi-threaded app that is sensitive to bandwidth?  Are you operating on a single file, or on a large tree of files, or on a raw device?  Are you sharing a small number of relatively stable file descriptors, or constantly creating and deleting files and truncating space?
> 
> All true, that's why I wanted to know from you, which real world
> situations you encountered where FreeBSD did/does outperform Linux in
> regards to I/O throughput and/or latency (depending on scenario, of
> course).


I have some tests that spawn N number of threads and then do sequential and random i/o either into a filesystem or a raw disk.  FreeBSD gets more work done with fewer I/O's than linux when you're operating through the filesystem, thanks to softupdates and the block layer.  Linux has a predictive cache that often times will generate too much i/o in a vain attempt to aggressively prefetch blocks. So even then it's hard to measure in a simple way; linux will do more i/o, but less of it will be useful to the application, thereby increasing latency and increasing application runtime.  Sorry I can't be more specific, but you're asking for something that I explicitly say I can't provide.

Scott