Re: mfi timeouts at boot on ASUS Z97 motherboard with LSI 9240-4i

On Tue, Apr 07, 2015 at 12:04:44PM -0700, Garrett Cooper wrote:
> Hi,
> 	I just tried to upgrade my system from a Nehalem style era CPU/motherboard (a W series Xeon/P6T-WS) to a Haswell style CPU/motherboard (an E-series Haswell/Z97 motherboard), and I?ve run into some fun issues with my LSI 9240-4i controller. In particular, it?s timing out because of issues noted similar to here: https://lists.freebsd.org/pipermail/freebsd-stable/2011-October/064404.html . I?ve done the following so far to try and diagnose the issue:
> 	- Upgraded the firmware on the card and in the BIOS
> 	- Tweaked with the PCI-E settings (2.0/3.0; disabled power savings mode in the BIOS; etc)
> 	- Turned off 4GB ?remapping? on the PCM.
> 	- Booted USB key fob-based images running: 9.3-RELEASE, 10.1-RELEASE, 10-STABLE, 11-CURRENT
> 	- Booted my original drive running 9.3-RELEASE
> 	- Booted with hw.mfi.msi=0 set in loader.
> 	- Booted with boot -v.
One more thing to try, is to put the card into PCIe slot handled by the
south bridge instead of the CPU slot.

I have two Drake Skinnies card which I cannot use, since all my machines
are desktop class, and the only available PCIe 8x+ slots are connected
to CPU.  I was never able to get a satisfactory explanation why the
thing does not like CPU' PCIe.

> 	Every time I run into MFI send command timeouts that eventually turn into 
> 	Here?s the message I got (transcribed) with 11-CURRENT with boot -v:
> 
> mfi0: <Drake Skinny> port 0xe000-0xe0ff mem 0xdfc60000-0x?. on pci2
> mfi0: attempting to allocate 1 MSI vectors (1 supported)
> pci: routing MSI IRQ 264 to local APIC 0 vector 59
> mfi0: using IRQ 264 for MSI
> mfi0: using MSI
> mfi0: Megaraid SAS driver Ver 4.23
> mfi0: Frame 0xfffffe07f719f280 timed out command 0x1010000
> Error 60 in callback from mfi_send_frame
> 
> 	The complete screenshot for that timeout can be found here: https://people.freebsd.org/~ngie/mfi-timeout-boot-verbose.jpg
> 	What I haven?t done yet:
> 	- Exporting the volume with the old motherboard and importing it with the new motherboard (I need to do some reading before I do this though because I don?t want to toast my data by accident :(?).
> 	- Booted either Linux or Windows to make sure the controller works with the motherboard
> 	- Tried clearing events with the controller
> 	Does anyone have any thoughts or hints on this issue?
> Thanks!