Re: Enabling NUMA in BIOS stop booting FreeBSD

On Wed, Dec 14, 2016 at 09:03:49PM +0200, Konstantin Belousov wrote:

> On Wed, Dec 14, 2016 at 06:26:27PM +0300, Slawa Olhovchenkov wrote:
> > On Wed, Dec 14, 2016 at 03:13:36PM +0300, Slawa Olhovchenkov wrote:
> > 
> > > On Wed, Dec 14, 2016 at 01:39:27PM +0200, Konstantin Belousov wrote:
> > > 
> > > > In other words, it is almost certainly the hang and not a fault causing
> > > > hang. This means that the machine is not compliant with the IA32
> > > > architecture, in particular, the region reported as normal memory by
> > > > E820 BIOS service does not behave as normal memory.
> > > > 
> > > > Since regardless of the option setting, the memory map is same, and
> > > > bootstrap page table only depend on the memory map, we use the same page
> > > > table when hanging and when operating correctly. We do not fault or hang
> > > > when the option is turned off, which together with the improved early
> > > > fault handling in the patch, makes it almost certain that the problem is
> > > > in hardware configuration and not in our early setup.
> > > > 
> > > > Of course, the most puzzling part is that memory test makes the hang
> > > > go away, while repeating memory test operation only on the msgbuf region
> > > > does not. msgbuf is special in that it is located at TOHM (top of high
> > > > memory). It spans 128KB from below it to the last byte of the last
> > > > physical segment.
> > > > 
> > > > The only ideas I have right now is that there is either a bug in the
> > > > Caching Agent/Home agent/IMC configuration in BIOS, in which case there
> > > > is nothing OS can do to mitigate it.  Or it might be that the memory
> > > > map reported by CMS is wrong (you said that you use legacy boot, right
> > > > ?).  This is not too surprising if true, because non-EFI boot code path
> > > > definitely get less and less testing.
> > > > 
> > > > For the later case (potential bug in CMS), could you switch to EFI boot
> > > > mode and see whether the issue magically healths itself ?  You could boot
> > > > from USB stick in EFI mode without reinstalling for test.
> > > 
> > > I can't boot from USB stick -- this is remote DC and IPMI allow only
> > > CDROM emulation.
> > > 
> > > OK, I am boot in UEFI 12.0 snapshot ISO.
> > > Boot ok.
> > 
> > Sorry. Overload bu work and test wrong combination (NUMA=ON,
> > interleave=OFF)
> > 
> > snapshot iso don't boot with NUMA=ON interleave=ON
> Ok.
> 
> > 
> > For test hardware setup (NUMA+interleave), what ISO I can try to boot?
> Didn't you already tried ?

Different from FreeBSD.
For sure about firmware problem and complains to Supermicro.
I think FreeBSD problem don't be accepted by Supermicro.