On Tue, Nov 8, 2011 at 8:16 PM, Dan The Man <dan_at_sunsaturn.com> wrote: > > On Tue, 8 Nov 2011, Kurt Touet wrote: > >> On Tue, Nov 8, 2011 at 3:32 AM, Dan The Man <dan_at_sunsaturn.com> wrote: >>> >>> >>> Sorry I meant it was running fine on beta3 and 8.2 stable, and NOT RC1: >>> asterisk:~# uname -a >>> FreeBSD asterisk.sunsaturn.com 9.0-RC1 FreeBSD 9.0-RC1 #0: Mon Oct 31 >>> 19:46:53 CDT 2011 >>> droot_at_asterisk.sunsaturn.com:/usr/obj/usr/src/sys/MYKERNEL >>> amd64 >>> asterisk:~# >>> >>> >>> >>> Dan. >>> >>> >>> -- >>> Dan The Man >>> CTO/ Senior System Administrator >>> Websites, Domains and Everything else >>> http://www.SunSaturn.com >>> Email: Dan_at_SunSaturn.com >>> >>> On Tue, 8 Nov 2011, Dan The Man wrote: >>> >>>> >>>> Ok here is some specs: this been running fine on 8.2 stable and i was >>>> sure >>>> it was running fine on RC1 as well. I did some testing against samba 34 >>>> 35 >>>> and 36 in the ports collection all with the same slow write problems. >>>> >>>> I did further testing mounting drive in question with NFS and it did not >>>> suffer the same problem, so it seems just samba related here, where >>>> samba >>>> would actually outperform my NFS mount before, now its taking 10x as >>>> long >>>> to write anything. >>>> >>>> This is really most simplistic setup I have, all I want to do is map a >>>> network drive at the house read/write so my laptop, desktop etc all have >>>> access to it. I have played with all the smb.conf options, and can't >>>> seem >>>> to find where the issue is, further research suggests others are >>>> experiencing same problems with beta3 from following forum post: >>>> >>>> http://forums.freebsd.org/showthread.php?t=27300 >>>> >>>> Hardware this is running on: I beleive a 4 year old amd chip and board, >>>> with 2 gigs of ram, this is a home PC that serves as a NAS, it has 1 SSD >>>> as >>>> UFS boot OS filesystem, and uses ZFS in raid0 with 3 3terrabyte cheap >>>> hitachi drives for my storage space,which is mirrored nightly with rsync >>>> with another duplicate machine(cause I know someone is going to say why >>>> not >>>> use raid5-raidz) >>>> >>>> Network specs: machine currently has dedicated IPV4 and gif0 tunneled >>>> IPV6 >>>> addresses to he.net. >>>> >>>> I've played with nearly every option in smb.conf disabling, enabling etc >>>> and can't seem to find the issue: here are my current config file >>>> settings >>>> on machine that could apply to samba: >>>> >>>> asterisk:~# cat /boot/loader.conf >>>> autoboot_delay="5" >>>> accf_data_load="YES" >>>> aio_load="YES" >>>> zfs_load="YES" >>>> kern.maxbcache=64M >>>> kern.ipc.maxpipekva=4M >>>> >>>> vfs.zfs.prefetch_disable=1 >>>> vm.kmem_size="1844M" >>>> vfs.zfs.arc_min="1024M" >>>> vfs.zfs.arc_max="1536M" >>>> vfs.zfs.vdev.min_pending=2 >>>> vfs.zfs.vdev.max_pending=8 >>>> vfs.zfs.txg.timeout=5 >>>> vfs.zfs.zil_disable="1" >>>> ahci_load="YES" >>>> asterisk:~# >>>> >>>> asterisk:~# cat /usr/local/etc/smb.conf >>>> # Global parameters >>>> [global] >>>> workgroup = HOME >>>> netbios name = ASTERISK >>>> server string = "Primary backups" >>>> interfaces = sk0 >>>> #smb ports = 139 >>>> #security = USER >>>> security = SHARE >>>> encrypt passwords = Yes >>>> #socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 >>>> domain master = no >>>> wins support = yes >>>> guest account = root >>>> socket options=SO_RCVBUF=131072 SO_SNDBUF=131072 TCP_NODELAY >>>> use sendfile = no >>>> level2 oplocks = True >>>> read raw = no >>>> write cache size = 262144 >>>> min receivefile size = 16384 >>>> aio read size = 16384 >>>> aio write size = 16384 >>>> aio write behind = yes >>>> dns proxy = no >>>> max log size = 50 >>>> #log file = /dev/null >>>> log file = /var/log/samba.log >>>> debug level = 1 >>>> syslog = 0 >>>> >>>> [data] >>>> comment = "Primary backups" >>>> path = /data/public >>>> read only = No >>>> guest ok = Yes >>>> valid users = root >>>> asterisk:~# asterisk:~# cat /etc/sysctl.conf >>>> # $FreeBSD: src/etc/sysctl.conf,v 1.8.34.1 2009/08/03 08:13:06 kensmith >>>> Exp $ >>>> # >>>> # This file is read when going to multi-user and its contents piped >>>> thru >>>> # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for >>>> details. >>>> # >>>> >>>> # Uncomment this to prevent users from seeing information about >>>> processes >>>> that >>>> # are being run under another UID. >>>> #security.bsd.see_other_uids=0 >>>> >>>> #raise file descriptors on the system >>>> kern.maxfiles=204916 >>>> kern.maxfilesperproc=204916 >>>> >>>> #raise sockets we can accept >>>> kern.ipc.somaxconn=32768 >>>> >>>> #http://www-didc.lbl.gov/TCP-tuning/FreeBSD.html >>>> kern.ipc.maxsockbuf=16777216 >>>> net.inet.tcp.rfc1323=1 >>>> net.inet.tcp.sendbuf_max=16777216 >>>> net.inet.tcp.recvbuf_max=16777216 >>>> net.inet.tcp.sendspace=65536 >>>> net.inet.tcp.recvspace=131072 >>>> >>>> #NGINX webserver http://wiki.nginx.org/FreeBSDOptimizations >>>> net.inet.icmp.icmplim=500 >>>> kern.ipc.nmbjumbop=192000 >>>> kern.ipc.nmbclusters=229376 >>>> kern.ipc.maxsockets=204800 >>>> net.inet.tcp.maxtcptw=163840 >>>> #also add following to /boot/loader.conf >>>> #vm.kmem_size=1844M >>>> #kern.maxbcache=64M >>>> #kern.ipc.maxpipekva=4M >>>> >>>> #default setting of net.inet.ip.portrange.first is to low, causing us >>>> problems with bind >>>> net.inet.ip.portrange.last=65535 >>>> net.inet.ip.portrange.first=1024 >>>> >>>> #DOS protection >>>> net.inet.tcp.msl=7500 >>>> net.inet.tcp.blackhole=2 >>>> net.inet.udp.blackhole=1 >>>> net.inet.icmp.icmplim=50 >>>> net.inet.ip.accept_sourceroute=0 >>>> net.inet.ip.sourceroute=0 >>>> >>>> #some stuff for samba >>>> kern.ipc.nmbclusters=32768 >>>> kern.maxvnodes=800000 >>>> net.inet.tcp.delayed_ack=0 >>>> net.inet.tcp.inflight.enable=0 >>>> net.inet.tcp.path_mtu_discovery=0 >>>> net.inet.tcp.recvbuf_auto=1 >>>> net.inet.tcp.recvbuf_inc=524288 >>>> net.inet.tcp.sendbuf_auto=1 >>>> net.inet.tcp.sendbuf_inc=524288 >>>> net.inet.udp.maxdgram=57344 >>>> net.inet.udp.recvspace=65535 >>>> net.local.stream.recvspace=65535 >>>> net.local.stream.sendspace=65535 >>>> net.inet.tcp.mssdflt=1460 >>>> >>>> #IPSEC >>>> net.inet.ip.forwarding=1 >>>> net.inet6.ip6.forwarding=1 >>>> kern.module_path=/boot/kernel;/boot/modules;/usr/local/modules >>>> >>>> >>>> #NFS--not concerned with data integrity when playing mostly already >>>> stored >>>> movies >>>> vfs.nfsrv.async=1 >>>> >>>> #JAIL >>>> #i like to use ping etc inside jail! >>>> security.jail.allow_raw_sockets=1 >>>> asterisk:~# >>>> >>>> >>>> Here are logs of me trying to mux a DTS mkv file from samba.log on debug >>>> level 10, I get the following over and over again: >>>> >>>> [2011/11/08 03:24:00.067974, 10] ../lib/util/util.c:415(dump_data) >>>> [2011/11/08 03:24:00.067981, 3] smbd/process.c:1466(switch_message) >>>> switch message SMBwriteX (pid 64308) conn 0x805008450 >>>> [2011/11/08 03:24:00.067990, 4] smbd/uid.c:345(change_to_user) >>>> Skipping user change - already user >>>> [2011/11/08 03:24:00.068001, 10] >>>> locking/locking.c:120(strict_lock_default) >>>> is_locked: optimisation - exclusive oplock on file >>>> torrent_downloads_finished/Point.Break.1991.720p (1).mkv >>>> [2011/11/08 03:24:00.068010, 10] >>>> locking/locking.c:162(strict_lock_default) >>>> strict_lock_default: flavour = WINDOWS_LOCK brl start=83431665 >>>> len=65536 >>>> unlocked for fnum 49966 file >>>> torrent_downloads_finished/Point.Break.1991.720p (1).mkv >>>> [2011/11/08 03:24:00.068021, 10] lib/recvfile.c:65(default_sys_recvfile) >>>> default_sys_recvfile: from = 33, to = 39, offset=83431665, count = >>>> 65536 >>>> [2011/11/08 03:24:00.068995, 10] smbd/fileio.c:143(real_write_file) >>>> real_write_file (torrent_downloads_finished/Point.Break.1991.720p >>>> (1).mkv): pos = 83431665, size = 65536, returned 65536 >>>> [2011/11/08 03:24:00.069013, 3] smbd/reply.c:4639(reply_write_and_X) >>>> writeX fnum=49966 num=65536 wrote=65536 >>>> [2011/11/08 03:24:00.069038, 10] >>>> lib/util_sock.c:516(read_smb_length_return_keepalive) >>>> got smb length of 65600 >>>> [2011/11/08 03:24:00.069052, 10] >>>> smbd/reply.c:4459(is_valid_writeX_buffer) >>>> is_valid_writeX_buffer: true len = 65600, doff = 64, numtowrite = 65536 >>>> [2011/11/08 03:24:00.069062, 6] smbd/process.c:1659(process_smb) >>>> got message type 0x0 of len 0x3f >>>> [2011/11/08 03:24:00.069072, 3] smbd/process.c:1661(process_smb) >>>> Transaction 15398 of length 67 (65536 toread) >>>> [2011/11/08 03:24:00.069081, 5] lib/util.c:332(show_msg) >>>> [2011/11/08 03:24:00.069087, 5] lib/util.c:342(show_msg) >>>> size=63 >>>> smb_com=0x2f >>>> smb_rcls=0 >>>> smb_reh=0 >>>> smb_err=0 >>>> smb_flg=24 >>>> smb_flg2=51207 >>>> smb_tid=1 >>>> smb_pid=65279 >>>> smb_uid=0 >>>> smb_mid=36032 >>>> smt_wct=14 >>>> smb_vwv[ 0]= 255 (0xFF) >>>> smb_vwv[ 1]=57054 (0xDEDE) >>>> smb_vwv[ 2]=49966 (0xC32E) >>>> smb_vwv[ 3]= 4337 (0x10F1) >>>> smb_vwv[ 4]= 1274 (0x4FA) >>>> smb_vwv[ 5]=65535 (0xFFFF) >>>> smb_vwv[ 6]=65535 (0xFFFF) >>>> smb_vwv[ 7]= 0 (0x0) >>>> smb_vwv[ 8]= 0 (0x0) >>>> smb_vwv[ 9]= 1 (0x1) >>>> smb_vwv[10]= 0 (0x0) >>>> smb_vwv[11]= 64 (0x40) >>>> smb_vwv[12]= 0 (0x0) >>>> smb_vwv[13]= 0 (0x0) >>>> smb_bcc=0 >>>> [2011/11/08 03:24:00.069163, 10] ../lib/util/util.c:415(dump_data) >>>> [2011/11/08 03:24:00.069170, 3] smbd/process.c:1466(switch_message) >>>> switch message SMBwriteX (pid 64308) conn 0x805008450 >>>> [2011/11/08 03:24:00.069179, 4] smbd/uid.c:345(change_to_user) >>>> Skipping user change - already user >>>> [2011/11/08 03:24:00.069188, 10] >>>> locking/locking.c:120(strict_lock_default) >>>> is_locked: optimisation - exclusive oplock on file >>>> torrent_downloads_finished/Point.Break.1991.720p (1).mkv >>>> [2011/11/08 03:24:00.069197, 10] >>>> locking/locking.c:162(strict_lock_default) >>>> strict_lock_default: flavour = WINDOWS_LOCK brl start=83497201 >>>> len=65536 >>>> unlocked for fnum 49966 file >>>> torrent_downloads_finished/Point.Break.1991.720p (1).mkv >>>> [2011/11/08 03:24:00.069221, 10] lib/recvfile.c:65(default_sys_recvfile) >>>> default_sys_recvfile: from = 33, to = 39, offset=83497201, count = >>>> 65536 >>>> [2011/11/08 03:24:00.069987, 10] smbd/fileio.c:143(real_write_file) >>>> real_write_file (torrent_downloads_finished/Point.Break.1991.720p >>>> (1).mkv): pos = 83497201, size = 65536, returned 65536 >>>> [2011/11/08 03:24:00.070004, 3] smbd/reply.c:4639(reply_write_and_X) >>>> writeX fnum=49966 num=65536 wrote=65536 >>>> [2011/11/08 03:24:00.070030, 10] >>>> lib/util_sock.c:516(read_smb_length_return_keepalive) >>>> got smb length of 65600 >>>> [2011/11/08 03:24:00.070044, 10] >>>> smbd/reply.c:4459(is_valid_writeX_buffer) >>>> is_valid_writeX_buffer: true len = 65600, doff = 64, numtowrite = 65536 >>>> [2011/11/08 03:24:00.070053, 6] smbd/process.c:1659(process_smb) >>>> got message type 0x0 of len 0x3f >>>> [2011/11/08 03:24:00.070063, 3] smbd/process.c:1661(process_smb) >>>> Transaction 15399 of length 67 (65536 toread) >>>> [2011/11/08 03:24:00.070072, 5] lib/util.c:332(show_msg) >>>> [2011/11/08 03:24:00.070077, 5] lib/util.c:342(show_msg) >>>> size=63 >>>> smb_com=0x2f >>>> smb_rcls=0 >>>> smb_reh=0 >>>> smb_err=0 >>>> smb_flg=24 >>>> smb_flg2=51207 >>>> smb_tid=1 >>>> smb_pid=65279 >>>> smb_uid=0 >>>> smb_mid=36102 >>>> smt_wct=14 >>>> smb_vwv[ 0]= 255 (0xFF) >>>> smb_vwv[ 1]=57054 (0xDEDE) >>>> smb_vwv[ 2]=49966 (0xC32E) >>>> smb_vwv[ 3]= 4337 (0x10F1) >>>> smb_vwv[ 4]= 1275 (0x4FB) >>>> smb_vwv[ 5]=65535 (0xFFFF) >>>> smb_vwv[ 6]=65535 (0xFFFF) >>>> smb_vwv[ 7]= 0 (0x0) >>>> smb_vwv[ 8]= 0 (0x0) >>>> smb_vwv[ 9]= 1 (0x1) >>>> smb_vwv[10]= 0 (0x0) >>>> smb_vwv[11]= 64 (0x40) >>>> smb_vwv[12]= 0 (0x0) >>>> smb_vwv[13]= 0 (0x0) >>>> smb_bcc=0 >>>> >>>> >>>> Hopefully maybe someone can shine some light on this.... >>>> >>>> >>>> Dan. >>>> >>>> >>>> -- >>>> Dan The Man >>>> CTO/ Senior System Administrator >>>> Websites, Domains and Everything else >>>> http://www.SunSaturn.com >>>> Email: Dan_at_SunSaturn.com >>>> >>>> On Fri, 28 Oct 2011, Garrett Cooper wrote: >>>> >>>>> On Thu, Oct 27, 2011 at 6:42 PM, Dan <dan_at_sunsaturn.com> wrote: >>>>>> >>>>>> >>>>>> Updated from 9.0 beta3 to RC1 and using mkvmerge over samba/zfs >>>>>> its taking over an hour to just mux in things like DTS english, where >>>>>> it >>>>>> was >>>>>> 15 minutes on beta3. >>>>> >>>>> Hi Dan, >>>>> - Can you do more deterministic / scientific benchmarks? >>>>> - Did you upgrade Samba? >>>>> - What is your system's operating hardware profile? >>>>> Thanks! >>>>> -Garrett >>>>> >> >> I had noticed a similar problem with doing large writes from my >> windows workstation to my freebsd ZFS array. Previously I had been >> able to write at around 30MB/s (limited by a slow SATA controller), >> and those speeds had dropped to 3-5MB/s with long pauses between >> writes to the array (monitoring with zpool iostat). However, I just >> updated to stable/9 r227357 and the issue seems to have gone away; I'm >> back up to 30MB/s writes. >> >> Hope you find the same sort of thing, >> -kurt >> > > Yep your right, on RC2 my samba ZFS write speeds are back. > Thanks for heads up Kurt. > > > Dan. > > > -- > Dan The Man > CTO/ Senior System Administrator > Websites, Domains and Everything else > http://www.SunSaturn.com > Email: Dan_at_SunSaturn.com > I may have spoken too soon. With the speeds back up to normal, I moved a bunch of content that I'd been storing temporarily on my workstation to the server. After a few hours and moving 40gb+ of content (not all at once), I noticed the same write problems cropping up again. I rebooted the server and speeds came back to normal. The oddity that I notice is that the array seems to be delaying writes to a single disc in my raidz1 vdev, and then writing out very slowly to it (1-2MB/s). The drive that it is doing it with has absolutely zero read/write/seek errors in SMART, and works just fine after the box is rebooted (so it shouldn't be a drive issue). The only thing interesting about this drive is that it replaced a failed drive a few weeks ago. Is anyone else seeing problems like this with samba/zfs ? Perhaps it's not exclusive to samba, either? -kurtReceived on Wed Nov 09 2011 - 04:59:56 UTC
This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:20 UTC