Re: Enabling NUMA in BIOS stop booting FreeBSD

On Sun, Dec 11, 2016 at 11:47:09PM +0300, Slawa Olhovchenkov wrote:
> Booting...
> ESC[01;00H8+0x8+0xe9bdc]                                                                  KDB: debugger backends: ddb
> KDB: current backend: ddb
> exit from kdb_init
> KDB: enter: Boot flags requested debugger
> [ thread pid 0 tid 0 ]
- remove any video consoles from the HEAD kernel config, i.e. sc/vt and
  vga/efi,
- do not use boot -d,
- use serial console (IPMI SOL qualifies),
- set late console to 0,
and show me the verbose dmesg of such boot with the BIOS options
which cause troubles.

> 
> > Hm, might be also show the output of the 'smap' and 'memmap' output from
> > the loader.  If any of them worked, could be useful to see the same output
> > with the NUMA option disabled as well.
> 
> NUMA disabled:
> OK smap
> SMAP type=01 base=0000000000000000 len=0000000000099c00 attr=01
> SMAP type=02 base=0000000000099c00 len=0000000000006400 attr=01
> SMAP type=02 base=00000000000e0000 len=0000000000020000 attr=01
> SMAP type=01 base=0000000000100000 len=000000007906b000 attr=01
> SMAP type=02 base=000000007916b000 len=000000000093a000 attr=01
> SMAP type=04 base=0000000079aa5000 len=0000000000505000 attr=01
> SMAP type=02 base=0000000079faa000 len=0000000002056000 attr=01
> SMAP type=01 base=0000000100000000 len=0000001f80000000 attr=01
> SMAP type=02 base=000000007c000000 len=0000000014000000 attr=01
> SMAP type=02 base=00000000fed1c000 len=0000000000029000 attr=01
> SMAP type=02 base=00000000ff000000 len=0000000001000000 attr=01
> NUMA enabled:
> OK smap
> SMAP type=01 base=0000000000000000 len=0000000000099c00 attr=01
> SMAP type=02 base=0000000000099c00 len=0000000000006400 attr=01
> SMAP type=02 base=00000000000e0000 len=0000000000020000 attr=01
> SMAP type=01 base=0000000000100000 len=000000007906b000 attr=01
> SMAP type=04 base=0000000079aa1000 len=0000000000509000 attr=01
> SMAP type=02 base=0000000079faa000 len=0000000002056000 attr=01
> SMAP type=01 base=0000000100000000 len=0000001f80000000 attr=01
> SMAP type=02 base=000000007c000000 len=0000000014000000 attr=01
> SMAP type=02 base=00000000fed1c000 len=0000000000029000 attr=01
> SMAP type=02 base=00000000ff000000 len=0000000001000000 attr=01
> OK memmap
> memmap not found

I.e. you use CMS in BIOS/legacy BIOS boot, right ?
Can you double-check that the smap output is indeed same for all three
cases (no-NUMA, NUMA without <4G interleave, NUMA with 4G interleave) ?

msgbuf setup is very early activity and for most practical purposes it
only depends on the physical segments layout of the machine.  So I do
expect that smap layout changes when the options are frobbed.