Re: Enabling NUMA in BIOS stop booting FreeBSD

From: Slawa Olhovchenkov <slw_at_zxy.spb.ru>
Date: Sun, 11 Dec 2016 22:16:26 +0300
On Sun, Dec 11, 2016 at 09:21:11PM +0300, Slawa Olhovchenkov wrote:

> On Sat, Nov 26, 2016 at 05:57:47PM +0200, Konstantin Belousov wrote:
> 
> > On Sat, Nov 26, 2016 at 12:21:24PM +0300, Slawa Olhovchenkov wrote:
> > > I am try to enable NUMA in bios and can't boot FreeBSD.
> > > Boot stoped after next messages:
> > > 
> > > ===
> > > Booting...
> > > KDB: debugger backends: ddb
> > > KDB: current backend: ddb
> > So at least the hammer_time() has a chance to initialize the console.
> > Do you have serial console ?  Set the loader tunable debug.late_console
> > to 1 and see if any NMI reaction appear.
> > 
> > > ===
> > > 
> > > This is verbose boot.
> > > No reaction to ~^B, NMI.
> > > 
> > > Same for head and 10.3-RELEASE.
> > > 
> > > Hardware is Supermicro X10DRi, Dual E5-2650v4, 256GB RAM.
> > Is there a BIOS option for 'on-chip cluster' or 'HPC computing' ?
> > What if you try to frob it ?
> > 
> > > 
> > > On slight different hardware
> > > (Supermicro X10DRi w/ old BIOS, Dual E5-2640v3, 128GB RAM)
> > > 10.3 boot ok w/ BIOS NUMA enabled.
> > 
> > I think the only way to debug this is to add printf() lines to hammer_time()
> > to see where does it break.  Note that amd64_kdb_init() call succeeded,
> > so you can start bisect the code from there.
> > 
> 
> Hang in next two lines:
> 
>         msgbufinit(msgbufp, msgbufsize);
> 	fpuinit();

[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c10 = msgbuf_reinit:     pushq   %rbp
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c11 = msgbuf_reinit+0x1: movq    %rsp,%rbp
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c14 = msgbuf_reinit+0x4: pushq   %r14
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c16 = msgbuf_reinit+0x6: pushq   %rbx
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c17 = msgbuf_reinit+0x7: movl    %edx,%r8d
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c1a = msgbuf_reinit+0xa: movq    %rdi,%r14
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c1d = msgbuf_reinit+0xd: movq    0x8(%r14),%rax
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c21 = msgbuf_reinit+0x11:        cmpl    $0x63062,%eax
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538c26 = msgbuf_reinit+0x16:        jnz     0xffffffff80538d37 = msgbuf_reinit+0x127
db> 
[ thread pid 0 tid 0 ]
Stopped at      0xffffffff80538d37 = msgbuf_reinit+0x127:       movq    %rsi,(%r14)
db> 
[ thread pid 0 tid 0 ]
Stopped atKDB: reentering
KDB: stack backtrace:
      db_trace_self_wrapper() at 0xffffffff8032de4b = db_trace_self_wrapper+0x2b/frame 0xffffffff80bb9370
kdb_reenter() at 0xffffffff8053670e = kdb_reenter+0x8e/frame 0xffffffff80bb9420
trap() at 0xffffffff807c0a31 = trap+0x51/frame 0xffffffff80bb9630
calltrap() at 0xffffffff807a5011 = calltrap+0x8/frame 0xffffffff80bb9630
--- trap 0x1c, rip = 0xffffffff8032db33, rsp = 0xffffffff80bb9700, rbp = 0xffffffff80bb9730 ---
X_db_search_symbol() at 0xffffffff8032db33 = X_db_search_symbol+0x53/frame 0xffffffff80bb9730
db_printsym() at 0xffffffff80330f30 = db_printsym+0x70/frame 0xffffffff80bb97a0
db_print_loc_and_inst() at 0xffffffff8032c0a3 = db_print_loc_and_inst+0x13/frame 0xffffffff80bb97c0
db_trap() at 0xffffffff8032df7f = db_trap+0xcf/frame 0xffffffff80bb9850
kdb_trap() at 0xffffffff80536b43 = kdb_trap+0x193/frame 0xffffffff80bb98e0
trap() at 0xffffffff807c0c3c = trap+0x25c/frame 0xffffffff80bb9af0
calltrap() at 0xffffffff807a5011 = calltrap+0x8/frame 0xffffffff80bb9af0
--- trap 0xa, rip = 0xffffffff80538d3a, rsp = 0xffffffff80bb9bc0, rbp = 0xffffffff80bb9bd0 ---
msgbuf_reinit() at 0xffffffff80538d3a = msgbuf_reinit+0x12a/frame 0xffffffff80bb9bd0
msgbufinit() at 0xffffffff8053dd31 = msgbufinit+0x21/frame 0xffffffff80bb9be0
hammer_time() at 0xffffffff807aa5fa = hammer_time+0xf8a/frame 0xffffffff80bba070
btext() at 0xffffffff8028fc34 = btext+0x24
db> show registers
cs                        0x20
ds                        0x28
es                        0x28
fs                        0x28
gs                        0x28
ss                        0x28
rax         0xffffffffffffffff
rcx                          0
rdx                    0x17fb8
rbx                        0x7
rsp         0xffffffff80bb9bc0  __stop_set_pcpu+0xb48
rbp         0xffffffff80bb9bd0  __stop_set_pcpu+0xb58
rsi         0xfffff8207ffe8000
rdi         0xfffff8207fffffb8
r8                     0x17fb8
r9          0xffffffff80bb9818  __stop_set_pcpu+0x7a0
r10                          0
r11                          0
r12                          0
r13               0x2080000000
r14         0xfffff8207fffffb8
r15         0xffffffff81800000
rip         0xffffffff80538d3a  msgbuf_reinit+0x12a
rflags                    0x82
0xffffffff80538d3a = msgbuf_reinit+0x12a:       movl    %r8d,0xc(%r14)
Received on Sun Dec 11 2016 - 18:16:29 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:09 UTC