Re: VM deadlock?

On Sunday 05 November 2006 10:24, Pawel Jakub Dawidek wrote:
> Hi.
> 
> We're seeing such a deadlock, seems to be VM related very often. Servers
> are quite loaded, this is FreeBSD 6.2 on SMP Opteron system, but running
> i386 FreeBSD.
> 
> Here is some info from one of such machines:
> 
> db> ps
>   pid  ppid  pgrp   uid   state   wmesg     wchan    cmd
> 90346  4316  4316   101  L      *Giant    0xc8551ac0 qmail-smtpd
> [...]
> 89997  4316  4316   101  L      *vm page  0xcabc9600 qmail-smtpd
> [...]
>  4316  4311  4316   101  LL     *vm page  0xcabc9600 qmail-smtpd
>  4315  4311  4315   101  RL                          drwebd
> [...]
>    34     0     0     0  LL     *vm page  0xcabc9600 [pagezero]
> [...]
>    16     0     0     0  LL     *vm page  0xcabc9600 [swi1: net]
> [...]
>    14     0     0     0  LL     *tcp      0xcabc82c0 [swi4: clock sio]
> [...]
>     3     0     0     0  LL     *vm page  0xcabc9600 [g_up]
> 
> db> alltrace
> Tracing command drwebd pid 4315 tid 100105 td 0xcac19600
> sched_switch(cac19600,0,2) at sched_switch+0x14b
> mi_switch(2,0) at mi_switch+0x1ba
> critical_exit(0,cac19600,ca0b5688,8246000,0,...) at critical_exit+0x9d
> intr_execute_handlers(c844b0c8,f8211b90,34,f8211be0,c06590b3,...) at 
intr_execute_handlers+0x10c
> lapic_handle_intr(31) at lapic_handle_intr+0x2e
> Xapic_isr1() at Xapic_isr1+0x33
> --- interrupt, eip = 0xc06676c4, esp = 0xf8211bd4, ebp = 0xf8211be0 ---
> pmap_pte_quick(ca0b5688,8246000) at pmap_pte_quick+0x90
> pmap_copy(ca0b5688,c88d1ea0,81a1000,a18000,81a1000) at pmap_copy+0x27c
> vm_map_copy_entry(c88d1de0,ca0b55c8,cabc4374,cac161dc) at 
vm_map_copy_entry+0x14c
> vmspace_fork(c88d1de0,4,c854b860,c8823d00,f8211cbc,...) at 
vmspace_fork+0x22a
> vm_forkproc(cac19600,c854b860,c854c900,14) at vm_forkproc+0xb3
> fork1(cac19600,14,0,f8211cd4) at fork1+0x1169
> fork(cac19600,f8211d04) at fork+0x18
> syscall(3b,3b,bfbf003b,4,81bba00,...) at syscall+0x2bf
> Xint0x80_syscall() at Xint0x80_syscall+0x1f
> --- syscall (2, FreeBSD ELF32, fork), eip = 0x80e493f, esp = 0xbfbfdcac, ebp 
= 0xbfbfe148 ---
> 
> db> sh allchain
> chain 1:
>  thread 100260 (pid 90346, qmail-smtpd) blocked on lock 0xc07177e0 (sleep 
mutex) "Giant"
>  thread 100133 (pid 89997, qmail-smtpd) blocked on lock 0xc0722700 (sleep 
mutex) "vm page queue mutex"
>  thread 100105 (pid 4315, drwebd) is on a run queue
> chain 2:
>  thread 100098 (pid 4316, qmail-smtpd) blocked on lock 0xc0722700 (sleep 
mutex) "vm page queue mutex"
>  thread 100105 (pid 4315, drwebd) is on a run queue
> chain 3:
>  thread 100031 (pid 34, pagezero) blocked on lock 0xc0722700 (sleep 
mutex) "vm page queue mutex"
>  thread 100105 (pid 4315, drwebd) is on a run queue
> chain 4:
>  thread 100015 (pid 3, g_up) blocked on lock 0xc0722700 (sleep mutex) "vm 
page queue mutex"
>  thread 100105 (pid 4315, drwebd) is on a run queue
> chain 5:
>  thread 100002 (pid 14, swi4: clock sio) blocked on lock 0xc072146c (sleep 
mutex) "tcp"
>  thread 100000 (pid 16, swi1: net) blocked on lock 0xc0722700 (sleep 
mutex) "vm page queue mutex"
>  thread 100105 (pid 4315, drwebd) is on a run queue
> db> sh locktree
> db> sh turnstile
> 
> Process drwebd (pid 4315) doesn't seem to go any further.

How about 'show allpcpu' to see which threads are running?

-- 
John Baldwin