Re: Should we simply disallow ZFS on FreeBSD/i386?

On Sun, Jan 06, 2008 at 04:58:02PM -0800, Maxim Sobolev wrote:
> Adam McDougall wrote:
> >A summary of my opinion on this matter is that some i386 FreeBSD servers 
> >do have a place running zfs in a useful role, but some dedication and 
> >patience from the administrator is usually required, and the effort to 
> >tune at least kmem is nearly 
> 
> In Russian we have a good saying: "You can teach a bear to ride a 
> bicycle, but will it ever enjoy it?"

I enjoy my i386/ZFS Servers.
It is running with just 384MB RAM as the only instability it currently
has is because the / disk ist dying.
But even with this it has 71 days uptime.
And my backup server is also based on ZFS with just 196MB RAM.
This one isn't running stable, but it is stable for just doing the zfs
imports and restoring some files.
The only 64 bit machines I have at home are alpha and spac64, so no
option for ZFS right now.

I don't see a real difference between running a 2G i386 and a 2G amd64
server.
8GB amd64 boxes are still not very common.
Of course the i386 must be tuned to have enough kmem and KVA and of
course doing so reduces the application space, but it is still within
the hardware limitations and an NFS-Server doesn't need much application
address space anyway.
Considered that we seem to have a limitation on running amd64 with more
than 2G kmem there is even more to consider.

> The same is here - seemingly due to the ZFS design limitations and 
> limitations of the FreeBSD kernel you can't get ZFS to run reliably out 
> of the box on i386. Yes, you can probably do some tweaks here and there, 
> to make it more of less stable given the workload, but that's not what 
> most of the FreeBSD users expect from the file system. Unlike you, most 
> of administrators won't even bother to read tweaking documentation 
> explaining why ZFS is so tricky in i386, let alone doing actual 
> trial-and-error to determine the right set of tunables. More likely at 
> the first incident they would just dismiss FreeBSD/ZFS as a crap.

This is true however.
I'm OK with a big fat warning on i386 and/or a loader env to be set
to reenable this for persons who know (or think they do) what they are
doing.
If we say ZFS on i386 isn't supported than at least it shouldn't be
able to be configured without anyone hitting a special knob.

But in my opinion ZFS shouldn't overflow kmem storage at first.
This is not only an i386 problem as rasing kmem makes ZFS more hungry
by default, which is only good if kmem isn't used for something else.
I have an amd64 system with 4G RAM and kmem defaults to 419430400 Bytes.
On my 384MB i386 box I have (tuned of course) 335544320 Bytes kmem.
And with more RAM I could easily go over the default on the amd64 box.
Well RAM is too expensive since it is SDRAM, but there are many DDR
boxes without amd64 functionality out there, which allows adding
affordable memory in the nGB range.
I had to reduce ARC sizes to get the 384MB box stable - the OS version
is quite old and things have been modified for that in the meantime,
but considered that 2G i386 systems still panic with more kmem than
I have RAM it simply says that it wouldn't run out of the box on amd64
with the same applications either.

-- 
B.Walter                http://www.bwct.de      http://www.fizon.de
bernd_at_bwct.de           info_at_bwct.de            support_at_fizon.de