Re: kernel panic with pccard insert on recent 7.0 CURRENT

On Monday 16 July 2007 01:27:40 pm M. Warner Losh wrote:
> In message: <200707161251.09790.jhb_at_freebsd.org>
>             John Baldwin <jhb_at_freebsd.org> writes:
> : On Monday 16 July 2007 11:13:15 am M. Warner Losh wrote:
> : > In message: <200707160850.46259.jhb_at_freebsd.org>
> : >             John Baldwin <jhb_at_freebsd.org> writes:
> : > : On Sunday 17 June 2007 01:56:59 am M. Warner Losh wrote:
> : > : > In message: <20070617053746.GV4602_at_funkthat.com>
> : > : >             John-Mark Gurney <gurney_j_at_resnet.uoregon.edu> writes:
> : > : > : Warner Losh wrote this message on Sat, Jun 16, 2007 at 
21:12 -0600:
> : > : > : > In message: <20070617024935.GU4602_at_funkthat.com>
> : > : > : >             John-Mark Gurney <gurney_j_at_resnet.uoregon.edu> 
writes:
> : > : > : > : Warner Losh wrote this message on Sat, Jun 16, 2007 at 
> : 17:33 -0600:
> : > : > : > : > Also, I'm unclear on the difference between FILTER_STRAY and
> : > : > : > : > FILTER_HANDLED.
> : > : > : > : 
> : > : > : > : The interrupt filter is suppose to return one of FILTER_STRAY 
or
> : > : > : > : FILTER_HANDLED...  If you _HANDLED it return that, otherwise 
> : return
> : > : > : > : _STRAY...  If you need to schedule the ithread, return 
_HANDLED 
> : or'd
> : > : > : > : with _SCHEDULE_THREAD...
> : > : > : > 
> : > : > : > Will _HANDLED cause all the other handlers to not get called, or 
> : just
> : > : > : > the stray interrupt code from not happening?
> : > : > : 
> : > : > : It will cause the remaining (not yet called) handlers not to get 
> : called...
> : > : > 
> : > : > I'm not sure that's right, especially for edge triggered devices.\
> : > : 
> : > : They shouldn't share interrupts then.  Do we support shared interrupts 
on 
> : edge 
> : > : triggered devices?
> : > 
> : > We support sharing interrupts on edge triggered devices.  At least it
> : > has worked on FreeBSD 2.2.1 through 6.2.  We have to continue to
> : > support it, and to do that, we can't have HANDLED stop processing.
> : > 
> : > It is a sad fact of life, but we have to continue to support that.
> : > 
> : > As an aside, some ISA hardware cannot support sharing of interrupts,
> : > but simple modification of the drivers allows one to share ISA
> : > interrupts.  My company has been doing this successfully for about 12
> : > years, and using FreeBSD to do it for at least the past 10 years.
> : 
> : Are there any edge triggered interrupts that aren't hooked up to an edge 
> : triggered interrupt pin (like a pin on an 8259A that is set to be edge 
> : triggered in the ELCR)?  We know if an interrupt source is edge or level 
and 
> : can act appropriately in the low-level code.
> 
> I don't think there are any.  The only case that leaps to mind is the
> whole level vs edge stuff in pccard, but I think that reflects the
> underlying bus architecture for the bridge attachment.
> 
> So long as we can share edge triggered interrupts, and all the ISRs
> always get called, then the case I'm worried about goes away.
> 
> : > : > : intr_event_handle calls intr_filter_loop which will return on the 
> : first
> : > : > : non-_STRAY handler and return it...  Which intr_event_handle 
eoi's...
> : > : > :
> : > : > : It looks like this code is designed for level triggered interrupts 
and
> : > : > : not edge triggered...
> : > : > 
> : > : > Yes.  I'm pretty sure that's wrong.  All ISA and PC Card devices use
> : > : > edge triggered interrupts.  Also, it is inefficient for level
> : > : > triggered interrupts, since two interrupt sources on the same
> : > : > interrupt may trigger at about the same time...
> : > : 
> : > : It works fine since the second device will interrupt again and we will 
> : fall 
> : > : through to its routine on the second interrupt.  The idea is that 
> : > : simultaneous interrupts are rare enough that it is worth optimizing 
the 
> : > : common case.
> : > 
> : > Actually, PC Card devices aren't necessarily edge triggered, but can
> : > be either edge triggered or level triggered.  They can live in bridges
> : > that are either Edge triggered or level triggered depending on the
> : > topology of the bus they live on.
> : > 
> : > In any event, the current code is incorrect and needs to be fixed.
> : 
> : It's a perfectly fine optimization for machines where level interrupts are 
> : shared and so I'd rather think about this and not just throw it out.  So 
what 
> : happens when you stick a pccard in a PCI bridge whose upstream interrupt 
is a 
> : level-triggered PCI interrupt.  Is the interrupt that the OS sees edge or 
> : level?
> 
> The PC Card code sets the card to do level triggered interrupts if
> possible.  It falls back to a level trigger emulation if it can't.
> There is a register that clears once it is read to deassert the
> interrupt, so that should work...
> 
> It just seems like a pessimization for the case you want to optimize,
> but it boils down to is it faster to call all the shared interrupt
> handlers all the time, or is it faster to do a context switch,
> averaged over all the cases we have to consider.

Huh?  Assuming you have two interrupt handlers with filters A and B that share 
an interrupt and only A has an interrupt to handle the difference is this:

1) Run A's handler, it claims the interrupt, so schedule A's ithread if 
requested and be done right away.

2) Run A's handler, then run B's handler, then schedule A's ithread if 
requested.

So right away we avoid wasting time running B's handler just so it can return 
INTR_STRAY.

Another thing that 1) let's us do is give A a private ithread but still let 
the interrupt code only have to schedule (and maybe preempt to) one ithread.  
(All the handlers w/o filters end up on a separate ithread, but each handler 
that has both a filter and a threaded handler would have its own thread.)
The dedicated ithread doesn't need to enable an interrupt source when its done 
since it knows the filter has already squashed the interrupt on its own 
(either handling it or using a device-specific interrupt masking register), 
and the dedicated ithreads could in theory run concurrently (though with 
ithreads bound to the CPU they come in on this isn't likely).

-- 
John Baldwin