Re: [follow-up] Fatal trap 12 in r195146+ in netisr_queue_internal

On Tuesday 21 July 2009 9:59:45 am Kamigishi Rei wrote:
> John Baldwin wrote:
> > On Tuesday 21 July 2009 6:59:36 am Kamigishi Rei wrote:
> >   
> >> Everything goes fine until - under heavy load on an interface, usually - 
> >> we reach a point where:
> >> 1. m->mtx_lock is 4 (== MTX_UNOWNED).
> >> 2. v is assigned mtx_lock's value (4 == MTX_UNOWNED).
> >> 3. condition (v == MTX_UNOWNED) fails.
> >>     
> > This will not happen.  If you look at the disassembly you will see this 
can't 
> > happen either.  Do you have a crashdump from a crash?
> >   
> I've got about 40 crash dumps on unmodded (without debug code) kernel, 
> and 3 or 4 with debug stuff (KASSERTs added by me).
> I can reproduce this on my test server (Core2 Duo 3.0, 4GB RAM), on my 
> home PC (Core2 Quad 2.5), and in VMWare with 2 CPUs in VT-x mode on my 
> laptop.
> It can't be reproduced on single-CPU single-core (including 
> hyperthreaded) systems.
> 
> Quoting,
> 
> (kgdb) fr 6
> #6  0xffffffff80586255 in _mtx_lock_sleep (m=0xffffffff80e60823, 
> tid=18446742977255365296, opts=Variable "opts" is not available.
> ) at /usr/src/sys/kern/kern_mutex.c:407
> 407                     owner = (struct thread *)(v & ~MTX_FLAGMASK);
> 
> (kgdb) print m->mtx_lock
> $14 = 4
> (kgdb) print v
> $15 = 21946368

% printf "%x\n" 21946368
14ee000

Can you print out 'owner' as well?  You won't get a panic until you actually 
dereference 'owner' to get 'owner->td_state' even though gdb will show this 
as the faulting line (gdb can sometimes get confused by compiler 
optimization).  You are seeing these values because mtx_lock was changed (due 
to either a mtx_unlock() or a mtx_init()) while you were spinning.   That 
value of v is not what I have typically seen in these panics.  Do you also 
have the original fatal kernel trap messages?

-- 
John Baldwin