Re: deadlock with zfs?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 21 Nov, 2008, at 02:01 , Thomas Vogt wrote:

> Hello
>
> I encounter a deadlock while running a few rsync processes mirroring  
> remote data to my local zfs pool. After a few hours my system is  
> starting more and more vsftpd sessions without closing any inactive  
> ftp sessions. I can't kill any rsync and vsftpd processes with "kill  
> -9". Even shutdown -r now does not work.
>
> I got a few hunderts vsftpd processes like this
>
> 61346 root          1  57    0  7880K  1692K zfs     1   0:00  0.00%  
> vsftpd
> 61481 root          1  68    0  7880K  1696K zfs     1   0:00  0.00%  
> vsftpd
> 61354 root          1  65    0  7880K  1692K zfs     1   0:00  0.00%  
> vsftpd
> 61480 root          1  68    0  7880K  1696K zfs     0   0:00  0.00%  
> vsftpd
> 61600 root          1  69    0  7880K  1704K zfs     1   0:00  0.00%  
> vsftpd
> 61599 root          1  68    0  7880K  1704K zfs     1   0:00  0.00%  
> vsftpd
>
> Right now i'm building a debug kernel. Whats the best way to get  
> usefull information from this deadlock? The system itself is not  
> crashing and response well to ssh and i also have a serial console.
>
> The system is running  8.0-CURRENT Thu Nov 20 00:15:46 UTC 2008  
> (64bit) with zpool version 13. I use zfs only as data pool. The base  
> system is running on ufs2:
>
> Filesystem       Size    Used   Avail Capacity  Mounted on
> /dev/da0s1a      496M    220M    236M    48%    /
> devfs            1.0K    1.0K      0B   100%    /dev
> /dev/da0s1g      169G     15G    141G     9%    /disk1
> /dev/da0s1f      3.9G     17M    3.5G     0%    /tmp
> /dev/da0s1e       29G    3.7G     23G    14%    /usr
> /dev/da0s1d       19G    7.1G     11G    40%    /var
> pool             853G      0B    853G     0%    /usr/local/data
> pool/cvsup       858G    5.7G    853G     1%    /usr/local/data/cvsup
> pool/ftp         3.3T    2.5T    853G    75%    /usr/local/data/ftp
> pool/portsnap    853G    633M    853G     0%    /usr/local/data/ 
> portsnap
> pool/www         853G     90M    853G     0%    /usr/local/data/www
>
> loader.conf:
> vm.kmem_size="1G"
> kern.maxfiles="65536"
> kern.maxproc="20480"
> net.inet.tcp.tcbhashsize="4096"
> net.inet.tcp.hostcache.hashsize="1024"
> vfs.zfs.arc_min="64M"
> vfs.zfs.arc_max="768M"
> vfs.zfs.prefetch_disable="1"
>
> I also tried to set vfs.zfs.zil_disable=1 but the problem still exist.
>
> I know there are few deadlock reports listed in the freebsd wiki.  
> Maybe we can trigger the root cause of the problem and fix it :)
>
> Regards
> Thomas
>

Hi Thomas,

from my recent experience it seems that tuning vm.kmem_size is no  
longer required,
maybe you can try without it.

- --
Regards,
Nikolay Denev




-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (Darwin)

iEYEARECAAYFAkkmc94ACgkQHNAJ/fLbfrlQNwCfYOgJoxqPMUeQ0JNsuES1Gd7a
T8QAnjf/bAj3LIUDjZlRIoFHkNl9l62q
=hVLJ
-----END PGP SIGNATURE-----