Re: Enabling NUMA in BIOS stop booting FreeBSD

On Tue, Dec 13, 2016 at 05:34:01PM +0300, Slawa Olhovchenkov wrote:
> On Tue, Dec 13, 2016 at 05:11:14PM +0300, Slawa Olhovchenkov wrote:
> 
> > On Tue, Dec 13, 2016 at 03:57:59PM +0200, Konstantin Belousov wrote:
> > 
> > > On Tue, Dec 13, 2016 at 03:49:32PM +0300, Slawa Olhovchenkov wrote:
> > > > > Boot with NUMA enabled and interleave off.
> > > > 
> > > > Already with patched kernel
> > > > 
> > > > > Patch kernel with the 'if (1 || ...)' patch.
> > > > > Reboot, enter BIOS setup and enable interleave there.
> > > > > Try to boot - does it boot ?
> > > > 
> > > > No.
> > > > 
> > > > > If it did not booted, power machine off for 10 minutes.
> > > > 
> > > > OK
> > > > 
> > > > > Power it on, try to boot (with the same patched kernel).
> > > > > Does the machine boot now ?
> > > > 
> > > > Don't boot.
> > > 
> > > I am really puzzled.  In other words, touching all memory causes the
> > > msgbuf to not hang.
> > 
> > yes
> > 
> > > Can you try one more experiment ?
> > > Take the patch below, apply it.
> > > >From the config where interleave is disabled, install new kernel.
> > > Reboot, enter BIOS setup and enable interleave.
> > > Set late_console to zero in loader.
> > > Do not enable memory test.
> > > Boot the patched kernel.
> > > Kernel must hang, according to your previous reports.
> > > I want to see the console log.
> > 
> > Hmm. I am [already] show output from ddb, and guess kernel will be
> > hang at first wirte to *mbp, i.e. you don't see any in console log.
> > 
> > OK, anyway I am try this pacth.
> 
> KDB: debugger backends: ddb
> KDB: current backend: ddb
> SMAP type=01 base=0000000000000000 len=0000000000099c00
> SMAP type=02 base=0000000000099c00 len=0000000000006400
> SMAP type=02 base=00000000000e0000 len=0000000000020000
> SMAP type=01 base=0000000000100000 len=000000007906b000
> SMAP type=02 base=000000007916b000 len=0000000000936000
> SMAP type=04 base=0000000079aa1000 len=0000000000509000
> SMAP type=02 base=0000000079faa000 len=0000000002056000
> SMAP type=01 base=0000000100000000 len=0000001f80000000
> SMAP type=02 base=000000007c000000 len=0000000014000000
> SMAP type=02 base=00000000fed1c000 len=0000000000029000
> SMAP type=02 base=00000000ff000000 len=0000000001000000
> XXX1
> YYY1
> ZZZ1

Ok, please do exactly the same testing with the following patch.
If the patched kernel boots succesfully, or if the patched kernel
boots further, I will provide one more, last patch, to test.

diff --git a/sys/kern/subr_msgbuf.c b/sys/kern/subr_msgbuf.c
index f275aef3b4f..1be7a629f65 100644
--- a/sys/kern/subr_msgbuf.c
+++ b/sys/kern/subr_msgbuf.c
_at__at_ -67,14 +67,19 _at__at_ msgbuf_init(struct msgbuf *mbp, void *ptr, int size)
 	mbp->msg_ptr = ptr;
 	mbp->msg_size = size;
 	mbp->msg_seqmod = SEQMOD(size);
+printf("YYY1\n");
 	msgbuf_clear(mbp);
+printf("YYY2\n");
 	mbp->msg_magic = MSG_MAGIC;
 	mbp->msg_lastpri = -1;
 	mbp->msg_flags = 0;
+printf("YYY3\n");
 	bzero(&mbp->msg_lock, sizeof(mbp->msg_lock));
 	mtx_init(&mbp->msg_lock, "msgbuf", NULL, MTX_SPIN);
+printf("YYY4\n");
 }
 
+
 /*
  * Reinitialize a message buffer, retaining its previous contents if
  * the size and checksum are correct. If the old contents cannot be
_at__at_ -85,8 +90,10 _at__at_ msgbuf_reinit(struct msgbuf *mbp, void *ptr, int size)
 {
 	u_int cksum;
 
-	if (mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) {
+	if (1 || mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) {
+printf("XXX1\n");
 		msgbuf_init(mbp, ptr, size);
+printf("XXX2\n");
 		return;
 	}
 	mbp->msg_seqmod = SEQMOD(size);
_at__at_ -117,10 +124,12 _at__at_ void
 msgbuf_clear(struct msgbuf *mbp)
 {
 
+printf("ZZZ1\n");
 	bzero(mbp->msg_ptr, mbp->msg_size);
 	mbp->msg_wseq = 0;
 	mbp->msg_rseq = 0;
 	mbp->msg_cksum = 0;
+printf("ZZZ2\n");
 }
 
 /*
diff --git a/sys/kern/subr_prf.c b/sys/kern/subr_prf.c
index e78863830c7..435412d55ea 100644
--- a/sys/kern/subr_prf.c
+++ b/sys/kern/subr_prf.c
_at__at_ -998,6 +998,8 _at__at_ msgbufinit(void *ptr, int size)
 	char *cp;
 	static struct msgbuf *oldp = NULL;
 
+printf("TTT1 %p %p %x\n", ptr, (char *)ptr + size - sizeof(*msgbufp), size);
+bzero(ptr, size);
 	size -= sizeof(*msgbufp);
 	cp = (char *)ptr;
 	msgbufp = (struct msgbuf *)(cp + size);