ZFS honesty

Kris Kennaway wrote:
> Ivan Voras wrote:
>> Kris Kennaway wrote:
>>> Ivan Voras wrote:
>>>> Robert Watson wrote:
>>>>
>>>>> I'm not sure if anyone has mentioned this yet in the thread, but 
>>>>> another thing worth taking into account in considering the 
>>>>> stability of ZFS is whether or not Sun considers it a production 
>>>>> feature in Solaris.  Last I heard, it was still considered an 
>>>>> experimental feature there as well.
>>>>
>>>> Last I heard, rsync didn't crash Solaris on ZFS :)
>>>
>>> [Citation needed]
>>
>> I can't provide citation about a thing that doesn't happen - you don't 
>> hear things like "oh and yesterday I ran rsync on my Solaris with ZFS 
>> and *it didn't crash*!" often.
>>
>> But, with some grains of salt taken, consider this Google results:
>>
>> * searching for "rsync crash solaris zfs": 790 results, most of them 
>> obviously irrelevant
>> * searching for "rsync crash freebsd zfs": 10,800 results; a small 
>> number of the results is from this thread, some are duplicates, but 
>> it's a large number in any case.
>>
>> I feel that the number of Solaris+ZFS installations worldwide is 
>> larger than that of FreeBSD+ZFS and they've had ZFS longer.
> 
> Almost all Solaris systems are 64 bit.
> 
> Kris

So, let's be honest here.  ZFS is simply unreliable on FreeBSD/i386.
There are things that you can do mitigate the problems, and in certain
well controlled environments you might be able to make it work well
enough for your needs.  But as a general rule, don't expect it to work
reliably, period.  This is backed up by Sun's own recommendation to not
run it on 32-bit Solaris.

But let's also be honest about ZFS in the 64-bit world.  There is ample
evidence that ZFS basically wants to grow unbounded in proportion to the
workload that you give it.  Indeed, even Sun recommends basically
throwing more RAM at most problems.  Again, tuning is often needed, and
I think it's fair to say that it can't be expected to work on arbitrary
workloads out of the box.

Now, what about the other problems that have been reported in this
thread by Ivan and others?  I don't think that it can be said that the
only problem that ZFS has is with memory.  Unfortunately, it looks like
these "other" problems aren't well quantified, so I think that they are
being unfairly dismissed.  But at the same time, maybe these other
problems are rare and unique enough that they represent very special
cases that won't be encountered by most people.  But it also tells me
that ZFS is still immature, at least in FreeBSD.

The universal need for tuning combined with the poorly understood
problem reports tells me that administrators considering ZFS should
expect to spend a fair amount of timing testing and tuning.  Don't
expect it to work out of the box for your situation.  That's not to
say that it's useless; there are certainly many people who can attest to
it working well for them.  Just be prepared to spend time and possibly
money making it work, and be willing to provide good problem reports for
any non-memory related problems that you encounter.

Scott