Re: CURRENT + amd64 + user-ppp = panic

On Friday 28 October 2005 20:04, John Baldwin wrote:
> On Wednesday 26 October 2005 09:36 pm, Scott Long wrote:
> > Vladimir Kushnir wrote:
> > > Hello,
> > > For a couple of days already my -CURRENT amd64 reliably panicks
> > > whenever I'm trying to connect via ppp (nothing fancy - playn dialup,
> > > no firewall). It's 100% reproducible both with custom kernel and with
> > > GENERIC. A typescript of kgdb is attached.
> > >
> > > I'm running now on the kernel from Oct 19 which also panicks, BTW, with
> > > "kmem_map too small" on an attempt to run something like Linux
> > > OpenOffice or Mathematica (neither kern.ipc.nmbclusters nor
> > > vm.kmem_size_max tweaking helps; besides, I've only 512 MB RAM)
> > >
> > > Regards,
> > > Vladimir
> >
> > I think that this is a result of the interrupt handler changes that John
> > Baldwin made yesterday.  Can you step your source back in time and see
> > where it stops panicing?
>
> Actually, it can't be if softclock() is called directly from
> ithread_loop(). In the new code ithread_loop() calls
> ithread_execute_handlers() which would call softclock().
>
> > > #0  doadump () at pcpu.h:172
> > >
> > > 172	pcpu.h: No such file or directory.
> > >
> > > 	in pcpu.h
> > >
> > > (kgdb) where
> > >
> > > #0  doadump () at pcpu.h:172
> > > #1  0xffffffff803c65fc in boot (howto=260)
> > >     at /usr/src/sys/kern/kern_shutdown.c:399
> > > #2  0xffffffff803c609b in panic (fmt=0xffffffff805f2f46 "from
> > > debugger") at /usr/src/sys/kern/kern_shutdown.c:555
> > > #3  0xffffffff801a8a32 in db_panic (addr=0, have_addr=0, count=0,
> > > modif=0x0)
> > >     at /usr/src/sys/ddb/db_command.c:435
> > > #4  0xffffffff801a8f75 in db_command_loop ()
> > >     at /usr/src/sys/ddb/db_command.c:404
> > > #5  0xffffffff801aae83 in db_trap (type=-1794574032, code=0)
> > >     at /usr/src/sys/ddb/db_main.c:221
> > > #6  0xffffffff803e5279 in kdb_trap (type=9, code=0,
> > > tf=0xffffffff9508fb10)
> > >     at /usr/src/sys/kern/subr_kdb.c:445
> > > #7  0xffffffff8058d84e in trap_fatal (frame=0xffffffff9508fb10,
> > >     eva=18446742974715243568) at /usr/src/sys/amd64/amd64/trap.c:672
> > > #8  0xffffffff8058ddb1 in trap (frame=
> > >       {tf_rdi = 1, tf_rsi = 70876, tf_rdx = -2401050962867404578,
> > > tf_rcx = 70876, tf_r8 = 0, tf_r9 = 1, tf_rax = 5340, tf_rbx = 1, tf_rbp
> > > = -1794573296, tf_r10 = 1, tf_r11 = 4, tf_r12 = -1099511143680, tf_r13
> > > = -1099035903488, tf_r14 = -1964245152, tf_r15 = 2, tf_trapno = 9,
> > > tf_addr = 0, tf_flags = 0, tf_err = 0, tf_rip = -2143462195, tf_cs = 8,
> > > tf_rflags = 65538, tf_rsp = -1794573360, tf_ss = 16}) at
> > > /usr/src/sys/amd64/amd64/trap.c:488
> > > #9  0xffffffff8057b3bb in calltrap ()
> > >     at /usr/src/sys/amd64/amd64/exception.S:168
>
> This looks like a page fault rather than a 'kmem_map too small' panic.
>
> > > ---Type <return> to continue, or q <return> to quit---
> > >
> > > #10 0xffffffff803d5ccd in softclock (dummy=0x1)
> > >     at /usr/src/sys/kern/kern_timeout.c:220
>
> This is here:
> 		while (c) {
> 			depth++;
> 		==>	if (c->c_time != curticks) {
> 				c = TAILQ_NEXT(c, c_links.tqe);
>
> c can't be NULL due to the while loop.  Are any kernel modules being
> unloaded when this happens?

It isn't a NULL deref as "eva" is clearly non-NULL above.  This makes me think 
of a callout list inconsistency.  Most likely - due to the rest of the thread 
- this was introduced via "tn_timer_ch" in struct llinfo_nd6.  I am thinking 
of a double callout_stop() or something like that.  The callout_stop/reset() 
calls on that callout are clearly over-nested to get things from a quick 
glance :-\

The easiest seems to be to put some good old printf() debugging in 
nd6_llinfo_settimer() and see what it does.  Vladimir, could you try that?  
"Patch" attached.

> > > #11 0xffffffff803b05cc in ithread_loop (arg=0xffffff0000031780)
> > >     at /usr/src/sys/kern/kern_intr.c:662
> > > #12 0xffffffff803af3cb in fork_exit (
> > >     callout=0xffffffff803b0480 <ithread_loop>, arg=0xffffff0000031780,
> > >     frame=0xffffffff9508fc90) at /usr/src/sys/kern/kern_fork.c:789
> > > #13 0xffffffff8057b71e in fork_trampoline ()
> > >     at /usr/src/sys/amd64/amd64/exception.S:394
> > > #14 0x0000000000000000 in ?? ()

-- 
/"\  Best regards,                      | mlaier_at_freebsd.org
\ /  Max Laier                          | ICQ #67774661
 X   http://pf4freebsd.love2party.net/  | mlaier_at_EFnet
/ \  ASCII Ribbon Campaign              | Against HTML Mail and News