Re: Releasing MSI Resources in a Module Causes Kernel Panic on -CURRENT

On Monday 14 May 2007 10:27:02 pm David Christensen wrote:
> I'm working on the bce driver and loading/unloading it as a kernel
> module.  
> On an MSI capable 5708 controller I can load the module and pass traffic
> without a problem, but when I unload the driver I get the following
> panic:
> ------------------------------------------------------------------------
> ---
> bce0: <Broadcom NetXtreme II BCM5708 1000Base-T (B2)> mem
> 0xfc000000-0xfdffffff irq 57 at device 0.0 on pci74
> bce0: ASIC ID 0x57081020; Revision (B2); PCI-X 64-bit 133MHz
> bce0: Ethernet address: 00:10:18:15:f6:c2
> bce0: [ITHREAD]
> miibus0: <MII bus> on bce0
> brgphy0: <BCM5708C 10/100/1000baseTX PHY> PHY 1 on miibus0
> brgphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT,
> 1000baseT-FDX, auto
> brgphy0: detached
> miibus0: detached
> bce0: Removing interrupt handler.
> bce0: Releasing IRQ.
> panic: resource_list_release: resource entry is not busy
> cpuid = 7
> KDB: enter: panic
> [thread pid 806 tid 100182 ]
> Stopped at      kdb_enter+0x31: leave
> db> bt
> Tracing pid 806 tid 100182 td 0xffffff0208a1bd20
> kdb_enter() at kdb_enter+0x31
> panic() at panic+0x1e3
> resource_list_release() at resource_list_release+0xd8
> bus_generic_rl_release_resource() at
> bus_generic_rl_release_resource+0x86
> bus_release_resource() at bus_release_resource+0x75
> bce_release_resources() at bce_release_resources+0xe3
> bce_detach() at bce_detach+0xce
> device_detach() at device_detach+0x68
> devclass_delete_driver() at devclass_delete_driver+0xb7
> driver_module_handler() at driver_module_handler+0xec
> module_unload() at module_unload+0xa0
> linker_file_unload() at linker_file_unload+0xdb
> kern_kldunload() at kern_kldunload+0xce
> kldunloadf() at kldunloadf+0x19
> syscall() at syscall+0x38c
> Xfast_syscall() at Xfast_syscall+0xab
> --- syscall (444, FreeBSD ELF64, kldunloadf), rip = 0x80068d4cc, rsp =
> 0x7fffffffe738, rbp = 0x2 ---
> db> 
> ------------------------------------------------------------------------
> ---
> Looking at other MSI capable drivers I see they are releasing the
> interrupt
> in the same way that I am so why am I getting the panic?  I see where
> the
> panic comes from
> (http://fxr.watson.org/fxr/source/kern/subr_bus.c#L2751)
> but I don't quite understand what didn't get set.
> 
> Here's the allocation:
> 
> 	/* Allocate PCI IRQ resources. */
> 	count = pci_msi_count(dev);
> 	if (count == 1 && pci_alloc_msi(dev, &count) == 0) {
> 		rid = 1;
> 		sc->bce_flags |= BCE_USING_MSI_FLAG;
> 		DBPRINT(sc, BCE_INFO_RESET, "Allocating MSI
> interrupt.\n");
> 	} else
> 		rid = 0;
> 
> 	sc->bce_res_irq = bus_alloc_resource_any(dev, SYS_RES_IRQ, &rid,
> 	    RF_SHAREABLE | RF_ACTIVE);
> 
> And here the de-allocation:
> 
> 	if (sc->bce_res_irq != NULL) {
> 		DBPRINT(sc, BCE_WARN_RESET, "Releasing IRQ.\n");
> 		bus_release_resource(dev, SYS_RES_IRQ, sc->bce_flags &
> BCE_USING_MSI_FLAG ? 1 : 0, 
> 			sc->bce_res_irq);
> 	}
> 
> 	if (sc->bce_flags & BCE_USING_MSI_FLAG) {
> 		DBPRINT(sc, BCE_WARN_RESET, "Releasing MSI vector.\n");
> 		pci_release_msi(dev);
> 	}
> 
> What is going wrong? 

Can you print out the rid being passed to bus_release_resource()?  Also, 
having 'debug.bootverbose' on (sysctl) during the kldload/kldunload would be 
helpful.

-- 
John Baldwin