Re: When will ZFS become stable?

Ivan Voras wrote:
> Kris Kennaway wrote:
> 
>> No, clearly it is not enough 
> 
> This looks like we're constantly chasing the "right amount". Does it 
> depend so much on CPU and IO speed that there's never a generally 
> sufficient "right amount"? So when CPU and drive speed increase, the new 
> amount will always be some bigger value?

It depends on your workload, which in turn depends on your hardware. 
The harder you can drive ZFS the more memory it will require.

>> (and you claimed previously to have done more tuning than this). 
> 
> Where? What else is there except kmem tuning (including KVA_PAGES)? IIRC 
> Pawel said all other suggested tunings don't do much.

Tuning is an interactive process.  If 512MB is not enough kmem_map, then 
increase it.  Repeat as necessary.

>> I have it set to 600MB on the i386 system with a 1.5GB KVA.  Both were 
>> necessary.
> 
> My point is that the fact that such things are necessary (1.5 GB KVA os 
> a lot on i386) mean that there are serious problems which aren't getting 
> fixed since ZFS was imported (that's over 6 months ago).

ZFS is a memory hog.  There is nothing that can really be done about 
this, and it is just not a good fit on i386 because of limitations of 
the hardware architecture.  Note that Sun does not recommend using ZFS 
on a 32-bit system either, for the same reasons.  It is unlikely this 
can really be fixed, although mitigation strategies like the vm_kern.c 
patch are possible.

> I see you've added to http://wiki.freebsd.org/ZFSTuningGuide; can you 
> please add the values that work for you to it (especially for KVA_PAGES 
> since the exact kernel configuration line is never spelled out in the 
> document; and say for which hardware are the values known to work)?

OK.

>> ZFS already tells you up front that it's experimental code and likely 
>> to have problems.  
> 
> I know it's experimental, but requiring users to perform so much tuning 
> just to get it work without crashing will mean it will get a bad 
> reputation early on. Do you (or anyone) know what are the reasons for 
> not having vm.kmem_size to 512 MB by default?

Increasing vm.kmem_size.max to 512MB by default has other implications, 
but it is something that should be considered.

 > Better yet, why not
> increase both vm.kmem_size and KVA_PAGES to (the equivalent of) 640 MB 
> or 768 MB by default for 7.0?

That is answered in the tuning guide.  Tuning KVA_PAGES by default is 
not appropriate.

>  >Users of 7.0-RELEASE should not have unrealistic
>  > expectations.
> 
> As I've said at the first post of this thread: I'm interested in if it's 
> ever going to be stable for 7.x.

This was in reply to a comment you made about the vm_kern.c patch 
affecting users of 7.0-RELEASE.

Anyway, to sum up, ZFS has known bugs, some of which are unresolved by 
the authors, and it is difficult to make it work on i386.  It is likely 
that the bugs will be fixed over time (obviously), but amd64 will always 
be a better choice than i386 for using ZFS because you will not be 
continually bumping up against the hardware limitations.

Kris