Re: samba+zfs

On 11/09/2011 08:07 AM, Daniel O'Connor wrote:
> On 09/11/2011, at 17:32, Garrett Cooper wrote
>>> dd's of large files (spooled backups going to tape) to /dev/null are as slow as Samba.
>>    - Dedupe?
> Nope.
You are probably right, but just to be sure, let's verify that with:

zpool get dedupratio <poolname>

If it is not 1.00x, then even though your dedup may be disabled, your
data is deduped.

>>    - Compression?
> On the mail spool & ports, but not on the tape spool.
>
>>    - How much RAM?
> 8GB.
Why are your memory settings set so low if you have 8 GB of memory? Do
you need 6.2 GB left over for other apps?


On 11/08/2011 10:26 AM, Dan The Man wrote:
> vm.kmem_size="1844M"
> vfs.zfs.arc_min="1024M"
> vfs.zfs.arc_max="1536M" 

On my machine with 48 GB of memory, I set:
vm.kmem_size="44g"
vm.kmem_size_max="44g"
vfs.zfs.arc_min="80m"
vfs.zfs.arc_max="42g"

And now it is very fast. My dataset is only about 10 TiB, and the total
space is around 32 TiB. In practice, it uses about 36g of ARC and 151g
of L2ARC, rather than the full 42g. Before, writes were quite slow when
doing read at the same time (such as simply using cp).

You shouldn't set those the way I did, using all for ZFS, because you
are also using UFS, which needs some memory too. Also, my machine has no
services other than ssh, nfs, and samba. I guess leave some for whatever
services you run (or does the kernel give up its memory it used when
userspace apps want more, like in Linux?).



Also, instead of
    vfs.zfs.zil_disable="1"
you should try setting "sync=disabled"
    # zfs set sync=disabled somepool/someasyncdataset


And to find out if it is a bad disk or some IO bottleneck, use gstat to
check the load % while it is doing the slow writing.

eg.

# gstat -I 5s -f "gpt/root|label/.ank|gpt/log|gpt/cache"

>>    - What debug options do you have enabled in the kernel?
> It is 8.2-GENERIC so.. no WITNESS (for example)
>
>>    I've been noticing a slowdown in some respects with NFS/SMB, but I
>> suspected it was because I have an re(4) based NIC. ZFS has also wired
>> down a lot of my system memory for the L2ARC…
>
> re isn't great but I wouldn't expect it to slow down over time.. Unless bounce buffers got used more and more or something.
>
> I have an em0 card in this system - but in any case it is slow locally (i.e. dd a large file with 64k block size).
>
> --
> Daniel O'Connor software and network engineer
> for Genesis Software - http://www.gsoft.com.au
> "The nice thing about standards is that there
> are so many of them to choose from."
>   -- Andrew Tanenbaum
> GPG Fingerprint - 5596 B766 97C0 0E94 4347 295E E593 DC20 7B3F CE8C
>
>
>
>
>
>
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"


-- 

--------------------------------------------
Peter Maloney
Brockmann Consult
Max-Planck-Str. 2
21502 Geesthacht
Germany
Tel: +49 4152 889 300
Fax: +49 4152 889 333
E-mail: peter.maloney_at_brockmann-consult.de
Internet: http://www.brockmann-consult.de
--------------------------------------------