Re: FreeBSD deadlock (with fork?)

On Thu, 18 Sep 2008, David Naylor wrote:

> On Thursday 18 September 2008 19:13:19 Daniel Eischen wrote:
>> On Thu, 18 Sep 2008, David Naylor wrote:
>>> On Thursday 18 September 2008 15:33:52 Tom Evans wrote:
>>>> On Thu, 2008-09-18 at 06:31 +0200, David Naylor wrote:
>>>>> Hi,
>>>>>
>>>>> I have a program that spawns a lot of subprocesses (with pipes open)
>>>>> from multiple threads.  The problem is the program often deadlocks, but
>>>>> not consistently.  Sometimes the program can run over 5 times to
>>>>> competition without incidence and yet othertimes it locks within a few
>>>>> seconds.
>>>>
>>>> Do you create threads, which then fork(), or do you fork() and then
>>>> create threads?
>>>
>>> I have many threads that then fork.  (aka forking threads).
>>>
>>>> I think the former will not work..
>>>
>>> Is there any reason for this and is this a FreeBSD limitation or a
>>> general problem?
>>
>> No, the former will work.  You can fork() from threads, only as long
>> as your forked processes only call async-signal-safe functions or
>> some form of exec().  For instance, you can't fork and then create
>> new threads from that child process.  If you are not immediately
>> exec()'ing from the child process, then you have to be careful
>> and only use async-signal-safe functions.
>>
>> Remember that you have multiple threads in the parent, so the
>> state of libthr, libc, etc may be inconsistent in a child
>> process.
>
> If you have a look at the backtrace from my original post you will see that
> all my threads are getting locked.  The program I am using is python and the
> call that is forking is Popen(['some', 'program'], stdout=PIPE,
> stderr=STDOUT).  As far as I know it just sets up the pipes and does an
> execvp.  (See
> http://svn.python.org/view/*checkout*/python/tags/r252/Lib/subprocess.py?content-type=text%2Fplain&rev=60915
> from line 981 to 1091).  [[I tried appending close_fds=True to the argument,
> it appeared to make the program deadlock less often...]]
>
> Any ideas as to what might be causing this dead lock (and why all the threads
> are stopping in amd64 specific assembler)?

No, could be a bug in python though.

-- 
DE