FreeBSD deadlock (with fork?)

From: David Naylor <naylor.b.david_at_gmail.com>
Date: Thu, 18 Sep 2008 06:31:42 +0200
Hi,

I have a program that spawns a lot of subprocesses (with pipes open) from 
multiple threads.  The problem is the program often deadlocks, but not 
consistently.  Sometimes the program can run over 5 times to competition 
without incidence and yet othertimes it locks within a few seconds.  

However if I limit the thread count to 1 the problem does not appear to be 
present.  

Here are the logs from gdb:
(gdb) info thread
  5 Thread 7021c0 (LWP 100203)  0x00000008009a2e8c in _umtx_op_err ()
    at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
  4 Thread a28480 (LWP 100174)  0x00000008009a2e8c in _umtx_op_err ()
    at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
  3 Thread a61d80 (LWP 100175)  0x00000008009a2e8c in _umtx_op_err ()
    at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
  2 Thread a61bc0 (LWP 100176)  0x00000008009a2e8c in _umtx_op_err ()
    at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
* 1 Thread a61840 (LWP 100177)  0x00000008009a2e8c in _umtx_op_err ()
    at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37


(gdb) bt
#0  0x00000008009a2e8c in _umtx_op_err () 
at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
#1  0x00000008009a1331 in cond_wait_common (cond=Variable "cond" is not 
available.
) at /usr/src/lib/libthr/thread/thr_cond.c:204
#2  0x00000000004c0573 in PyThread_acquire_lock (lock=0x70a760, waitflag=1) at 
thread_pthread.h:452
#3  0x00000000004c38b0 in lock_PyThread_acquire_lock (self=0x83f258, 
args=Variable "args" is not available.
) at ./Modules/threadmodule.c:46
#4  0x00000000004939e3 in PyEval_EvalFrameEx (f=0xa53920, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3679
#5  0x00000000004939a8 in PyEval_EvalFrameEx (f=0xb57420, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3765
#6  0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x9aab70, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#7  0x00000000004eb6bc in function_call (func=0x9d8938, arg=0x9d9990, 
kw=0xa24da0)
    at Objects/funcobject.c:524
#8  0x0000000000417add in PyObject_Call (func=0x9d8938, arg=0x9d9990, 
kw=0xa24da0)
    at Objects/abstract.c:2487
#9  0x000000000048f730 in PyEval_EvalFrameEx (f=0xb48e20, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3978
#10 0x00000000004939a8 in PyEval_EvalFrameEx (f=0xb57720, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3765
#11 0x00000000004939a8 in PyEval_EvalFrameEx (f=0xb49020, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3765
#12 0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x947cd8, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#13 0x00000000004eb5bd in function_call (func=0x9a5320, arg=0x9d9950, kw=0x0) 
at Objects/funcobject.c:524
#14 0x0000000000417add in PyObject_Call (func=0x9a5320, arg=0x9d9950, kw=0x0) 
at Objects/abstract.c:2487
#15 0x000000000041f12f in instancemethod_call (func=0x9a5320, arg=0x9d9950, 
kw=0x0)
    at Objects/classobject.c:2558
#16 0x0000000000417add in PyObject_Call (func=0x93b410, arg=0x718050, kw=0x0) 
at Objects/abstract.c:2487
#17 0x000000000048d4c6 in PyEval_CallObjectWithKeywords (func=0x93b410, 
arg=0x718050, kw=0x0)
    at Python/ceval.c:3548
#18 0x00000000004c3cbd in t_bootstrap (boot_raw=0x70ae60) 
at ./Modules/threadmodule.c:425
#19 0x000000080099ac11 in thread_start (curthread=0xa61840) 
at /usr/src/lib/libthr/thread/thr_create.c:288
#20 0x0000000000000000 in ?? ()
Error accessing memory address 0x7fffff5fc000: Bad address.


(gdb) thr 2
[Switching to thread 2 (Thread a61bc0 (LWP 100176))]#0  0x00000008009a2e8c in 
_umtx_op_err ()
    at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
37      RSYSCALL_ERR(_umtx_op)
(gdb) bt
#0  0x00000008009a2e8c in _umtx_op_err () 
at /usr/src/lib/libthr/arch/amd64/amd64/_umtx_op_err.S:37
#1  0x00000008009a1331 in cond_wait_common (cond=Variable "cond" is not 
available.
) at /usr/src/lib/libthr/thread/thr_cond.c:204
#2  0x00000000004c0573 in PyThread_acquire_lock (lock=0x70a600, waitflag=1) at 
thread_pthread.h:452
#3  0x00000000004c38b0 in lock_PyThread_acquire_lock (self=0x83f168, 
args=Variable "args" is not available.
) at ./Modules/threadmodule.c:46
#4  0x00000000004939e3 in PyEval_EvalFrameEx (f=0xb56520, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3679
#5  0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x8d5dc8, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#6  0x0000000000492c88 in PyEval_EvalFrameEx (f=0xb2de20, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3775
#7  0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x8d5288, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#8  0x00000000004eb5bd in function_call (func=0x9a8b18, arg=0xc524d0, kw=0x0) 
at Objects/funcobject.c:524
#9  0x0000000000417add in PyObject_Call (func=0x9a8b18, arg=0xc524d0, kw=0x0) 
at Objects/abstract.c:2487
#10 0x000000000041f12f in instancemethod_call (func=0x9a8b18, arg=0xc524d0, 
kw=0x0)
    at Objects/classobject.c:2558
#11 0x0000000000417add in PyObject_Call (func=0x93b370, arg=0x10331d0, kw=0x0) 
at Objects/abstract.c:2487
#12 0x0000000000464158 in slot_tp_init (self=Variable "self" is not available.
) at Objects/typeobject.c:5532
#13 0x0000000000461561 in type_call (type=0x7d9420, args=0x10331d0, kwds=0x0) 
at Objects/typeobject.c:700
#14 0x0000000000417add in PyObject_Call (func=0x7d9420, arg=0x10331d0, kw=0x0) 
at Objects/abstract.c:2487
#15 0x00000000004906f9 in PyEval_EvalFrameEx (f=0xb3a120, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3890
#16 0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x8d5eb8, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#17 0x0000000000492c88 in PyEval_EvalFrameEx (f=0xa47a20, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3775
#18 0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x8d5cd8, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#19 0x0000000000492c88 in PyEval_EvalFrameEx (f=0xb3a420, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3775
#20 0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x9aabe8, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#21 0x0000000000492c88 in PyEval_EvalFrameEx (f=0xb3a720, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3775
#22 0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x9aab70, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#23 0x00000000004eb6bc in function_call (func=0x9d8938, arg=0x9d98d0, 
kw=0xa5a660)
    at Objects/funcobject.c:524
#24 0x0000000000417add in PyObject_Call (func=0x9d8938, arg=0x9d98d0, 
kw=0xa5a660)
    at Objects/abstract.c:2487
#25 0x000000000048f730 in PyEval_EvalFrameEx (f=0xb2e020, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3978
---Type <return> to continue, or q <return> to quit---
#26 0x00000000004939a8 in PyEval_EvalFrameEx (f=0xb3aa20, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3765
#27 0x00000000004939a8 in PyEval_EvalFrameEx (f=0xb2e220, 
throwflag=Variable "throwflag" is not available.
) at Python/ceval.c:3765
#28 0x0000000000494ac1 in PyEval_EvalCodeEx (co=0x947cd8, 
globals=Variable "globals" is not available.
) at Python/ceval.c:2942
#29 0x00000000004eb5bd in function_call (func=0x9a5320, arg=0x9d9890, kw=0x0) 
at Objects/funcobject.c:524
#30 0x0000000000417add in PyObject_Call (func=0x9a5320, arg=0x9d9890, kw=0x0) 
at Objects/abstract.c:2487
#31 0x000000000041f12f in instancemethod_call (func=0x9a5320, arg=0x9d9890, 
kw=0x0)
    at Objects/classobject.c:2558
#32 0x0000000000417add in PyObject_Call (func=0x93b320, arg=0x718050, kw=0x0) 
at Objects/abstract.c:2487
#33 0x000000000048d4c6 in PyEval_CallObjectWithKeywords (func=0x93b320, 
arg=0x718050, kw=0x0)
    at Python/ceval.c:3548
#34 0x00000000004c3cbd in t_bootstrap (boot_raw=0x70aee0) 
at ./Modules/threadmodule.c:425
#35 0x000000080099ac11 in thread_start (curthread=0xa61bc0) 
at /usr/src/lib/libthr/thread/thr_create.c:288
#36 0x0000000000000000 in ?? ()
Error accessing memory address 0x7fffff7fd000: Bad address.

Apologies about the long backtraces [and thus the long message].  

Thanks

David

Received on Thu Sep 18 2008 - 02:32:16 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:39:35 UTC