Re: kern/134011: [hang] swap_pager_getswapspace(4): failed

From: Larry Rosenman <ler_at_lerctr.org>
Date: Sat, 30 May 2009 20:51:29 -0500
On Sat, May 30, 2009 8:46 pm, Randy Bush wrote:
>> this time it actually said something interesting on console!
>>
>> for some values of 'interesting' :)
>>
>> swap_pager_getswapspace(3): failed
>> swap_pager_getswapspace(3): failed
>> swap_pager_getswapspace(3): failed
>> swap_pager_getswapspace(3): failed
>> swap_pager_getswapsp
>> a
>> lcatale t(ra1p 612): : pafgae ifaullte wdhi
>>  e sin wkearnepl m_odpe
>> acpugied =r 1_; agpiec itd s= w01a
>> afasulpt vairtcuael (ad3dre)ss  :=  0x0f
>> odfalulet dc
>>   e     s       =w saupeprv_isopr awgrieter _dagtae, ptagse nowta
>> ppressepnat
>> 0inest(ru3ct)i:on  pfoinateir   l= e0dx2
>>  :0sxfwfaffpff_ffp80a47gc25e6
>> rst_acgk eptoinsterw     a   p   s =p 0ax2c8:0exf(3ff)ff:f80 7f9fd1a680i
>> poieadme
>>    ntserw         a   p  _ =p 0xag28e:0rxf_fffgff8e07t9fsd16we0
>>  0xdep ssegpmenta               c= beas(e 30x0), :lim itf 0xaffifflf,e
>> tydpe
>>    1bs
>> w                       =a DpPL _0, ppraesg 1,e lorng_ 1g, deef3t2 s0,w
>> garapn s1
>> ledaocceseso(r 3ef)la:gs        =f ianteirrulpt eendab
>>    , rseswumae, pIOP_L p= a0
>> gcuerrren_t pgroecests  s       = w789a (psysslpogad)
>> ctreap( n9umb)er:               =  f12
>> epainilc:e pdag
>>   fsauwlta
>> pcp_uipd =a 1g
>> eUptime: 9h50m49s
>> Physical memory: 4083 MB
>> Dumping 1958 MB:
>>
>> <end. required power cycle>
>
> a bit better in last night's syslog, possibly during backup
>
> randy
>
>
> May 30 00:40:14 work0 kernel: lock order reversal:
> May 30 00:40:14 work0 kernel: 1st 0xffffff0057d019d0 ufs (ufs) _at_
> /usr/src/sys/ufs/ffs/ffs_snapshot.c:423
> May 30 00:40:14 work0 kernel: 2nd 0xffffff8052c01aa0 bufwait (bufwait) _at_
> /usr/src/sys/kern/vfs_bio.c:2556
> May 30 00:40:14 work0 kernel: 3rd 0xffffff0004b8d098 ufs (ufs) _at_
> /usr/src/sys/ufs/ffs/ffs_snapshot.c:544
> May 30 00:40:16 work0 kernel: lock order reversal:
> May 30 00:40:16 work0 kernel: 1st 0xffffff8052c01aa0 bufwait (bufwait) _at_
> /usr/src/sys/kern/vfs_bio.c:2556
> May 30 00:40:16 work0 kernel: 2nd 0xffffff00d35c7d30 snaplk (snaplk) _at_
> /usr/src/sys/ufs/ffs/ffs_snapshot.c:793
> May 30 00:50:14 work0 kernel: lock order reversal:
> May 30 00:50:14 work0 kernel: 1st 0xffffff00d35c7d30 snaplk (snaplk) _at_
> /usr/src/sys/kern/vfs_vnops.c:297
> May 30 00:50:14 work0 kernel: 2nd 0xffffff0057d019d
> May 30 00:50:14 work0 kernel: 0 ufs (ufs) _at_ /u
> May 30 00:50:14 work0 kernel: s
> May 30 00:50:15 work0 kernel: r/src/sys/
> May 30 00:50:15 work0 kernel: ufs/ffs/
> May 30 00:50:15 work0 kernel: ffs_snap
> May 30 00:50:15 work0 kernel: shot.c:
> May 30 00:50:15 work0 kernel: 1587
> May 30 01:45:21 work0 kernel:
> May 30 01:45:21 work0 kernel:
> May 30 01:45:21 work0 kernel: Fatal trap 12: page fault while in kernel
> mode
> May 30 01:45:21 work0 kernel: cpuid = 0; apic id = 00
> May 30 01:45:21 work0 kernel: fault virtual address	= 0x0
> May 30 01:45:21 work0 kernel: fault code		= supervisor write data, page
> not present
> May 30 01:45:21 work0 kernel: instruction pointer	=
> 0x20:0xffffffff8047c256
> May 30 01:45:21 work0 kernel: sta
> May 30 01:45:21 work0 kernel: c
> May 30 01:45:21 work0 kernel: k pointer	        = 0x28:0xffffff807a057680
> May 30 01:45:21 work0 kernel: frame pointer	        =
> 0x28:0xffffff807a0576e0
> May 30 01:45:21 work0 kernel: code segment		= base 0x0, limit 0xfffff,
> type 0x1b
> May 30 01:45:21 work0 kernel: = DPL 0, pres 1, long 1, def32 0, gran
> May 30 01:45:21 work0 kernel: 1
> May 30 01:45:21 work0 kernel: processor eflags	= interrup
> May 30 01:45:21 work0 kernel: t enabled, resume,
> May 30 01:45:21 work0 kernel: IOPL = 0
> May 30 01:45:21 work0 kernel: current process
> May 30 01:45:21 work0 kernel: = 9181 (nfcapd)
> May 30 02:10:04 work0 syslogd: kernel boot file is /boot/kernel/kernel

See Kip's replies in my ZFS Crash thread.  I suspect you have compression
turned on in ZFS, and there is an unbounded allocation of memory in the
ZFS (de)compression code.   Kip is working on a patch to put some bounds
on it.

I've changed my tunables to not allow the ARC to grab ALL of physmem
(vfs.zfs.arc_max) and testing a full backup now.

Waiting, patiently, for code patch from Kip.




-- 
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 512-248-2683                 E-Mail: ler_at_lerctr.org
US Mail: 430 Valona Loop, Round Rock, TX 78681-3893
Received on Sat May 30 2009 - 23:51:32 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:48 UTC