Re: When will ZFS become stable?

On Fri, Jan 04, 2008 at 12:42:28PM +0100, Ivan Voras wrote:
> Hi,
> 
> As far as I know about the details of implementation and what would it
> take to fix the problems, is it safe to assume ZFS will never become
> stable during 7.x lifetime?

To sum up this thread, let me present ZFS status as of today.

Before I do that, one explanation. I was away from FreeBSD for like 3-4
weeks, because of real life issues, etc. I hope, I'm now back for good.
Let me also use this again to invite any interested committers to help
working on ZFS (I'm inviting people to help from a day one).

Ok.

The most pressing issues currently are:
1. kmem_map exhaustion.
2. Low memory deadlocks in ZFS itself.

I believe 2nd problem is already fixed in OpenSolaris, at least that was
my impression when I made last integration, I need to double check. If
that's true, I'll try to commit the fix before 7.0-RELEASE.

The 1st problem has of course much wider audience. First of all you
need:

	http://people.freebsd.org/~pjd/patches/vm_kern.c.2.patch

The patch is not yet committed, because I was discussing better
solutions with alc_at_. I don't think we (he) will be able to come up with
something better before 7.0-RELEASE, so I'm going to ask re_at_ for
approval for this patch today. Note that it is low risk change, because
it is executed only in situation where the system will panic anyway.

Of course it is so much better to use ZFS on 64bit systems, but it also
works on i386. I'm running ZFS in production for many months on two i386
systems. One has 1GB memory and those tunning in loader.conf:

vfs.zfs.prefetch_disable=1
vm.kmem_size=671088640
vm.kmem_size_max=671088640

I've three ZFS pools in here, no UFS at all. The load is rather light,
serving large files. No panics.

The second "production" box is my laptop. I've 2GB of RAM (it worked
fine with 1GB too), but I do have 'options KVA_PAGES=512' in my kernel
config and my loader.conf looks like this:

vm.kmem_size=1073741824
vm.kmem_size_max=1073741824
vfs.zfs.prefetch_disable=1

My laptop is ZFS-only. No panics whatsoever.

The box I'm running ZFS for the longest time is amd64 system with 1GB of
RAM. This box is used for backups (ZFS snapshots are so damn handy) and
guess what, I'm using rsync for backups:) It also serves files through
NFS:

beast:root:~# showmount -e | wc -l
      31

ZFS is used heavly here:

beast:root:~# zfs list -t filesystem | wc -l
      50
beast:root:~# zfs list -t snapshot | wc -l
    1029

And loader.conf:

vm.kmem_size=629145600
vm.kmem_size_max=629145600

And again, rock stable.

All my ZFS systems use vm_kern.c.2.patch.

Of course all this doesn't mean ZFS works great on FreeBSD. No. It is
still an experimental feature. I don't agree we should deny mounting ZFS
on i386, etc. We can improve warning and even advise increasing
KVA_PAGES on i386. It's too late to increase vm.kmem_size by default, as
it can affect other parts of the system. ZFS also can't do it
automatically.

In my opinion people are panicing in this thread much more than ZFS:)
Let try to think how we can warn people clearly about proper tunning and
what proper tunning actually means. I think we should advise increasing
KVA_PAGES on i386 and not only vm.kmem_size. We could also warn that
running ZFS on 32bit systems is not generally recommended. Any other
suggestions?

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd_at_FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!