Re: msi broken?

On Tue, Mar 10, 2009 at 06:47:21PM -0500, Robert Noland wrote:
> On Wed, 2009-03-11 at 00:36 +0100, Arno J. Klaassen wrote:
> > John Baldwin <jhb_at_freebsd.org> writes:
> > 
> > > On Tuesday 10 March 2009 3:00:00 pm Arno J. Klaassen wrote:
> > >> John Baldwin <jhb_at_freebsd.org> writes:
> > >> 
> > >> > On Tuesday 10 March 2009 10:08:59 am Arno J. Klaassen wrote:
> > >> >> 
> > >> >> Hello,
> > >> >> 
> > >> >> when upgrading this morning from a March 1 -current, if_bge
> > >> >> stopped working (and irq256: bge0 not showing up in
> > >> >> vmstat -i ). Setting hw.pci.enable_msi="0" makes it work again.
> > >> >
> > >> > Can you get a verbose dmesg (boot -v) with MSI enabled?
> > >
> > > Ok, so you are getting MSI interrupts assigned and routed ok.  Can you try 
> > > disabling the code that sets the INTx_MASK flag in the PCI command register 
> > > in sys/dev/pci/pci.c:pci_setup_intr()?
> > 
> > grr : "rid" sure is 1 for the if_bge interrupt. Please tell me which
> > lines of code set the INTx_MASK flag. Thanx, more tomorrow.
> 
> if rid is 0, the chip should be using INTx.  if rid > 0 then it should
> be using MSI.
> 
> 
>                         }
>                         mte->mte_handlers++;
>                 }
> #if 0 /* Comment this out/*
>                 /* Make sure that INTx is disabled if we are using MSI/MSIX */
>                 pci_set_command_bit(dev, child, PCIM_CMD_INTxDIS);
> #endif
>         bad:
>                 if (error) {
>                         (void)bus_generic_teardown_intr(dev, child, irq,
>                             cookie);
>                         return (error);
> 

If my memory serve me right, some Broadcom controllers reset PCI
configuration registers in bge_reset() so bge(4) restores some
important PCI configuration registers. I didn't check bge(4) code
but it's possible that bge(4) didn't restore MSI enable bit in
bge_reset().