Re: somewhat reproducable vimage panic

Marko Zec wrote this message on Tue, Jul 21, 2020 at 11:31 +0200:
> On Tue, 21 Jul 2020 02:16:55 -0700
> John-Mark Gurney <jmg_at_funkthat.com> wrote:
> 
> > I'm running:
> > FreeBSD test 13.0-CURRENT FreeBSD 13.0-CURRENT #0 r362596: Thu Jun 25
> > 05:02:51 UTC 2020
> > root_at_releng1.nyi.freebsd.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
> >  amd64
> > 
> > and I'm working on improve the if_ure driver.  I've put together a
> > little script that I've attached that I'm using to test the driver..
> > It puts a couple ue interfaces each into their own jail, configures
> > them, and tries to pass traffic.  This assumes that the two interfaces
> > are connected together.
> > 
> > Pretty regularly when destroying the jails, I get the following
> > panic: CURVNET_SET at /usr/src/sys/netinet/in_mcast.c:626
> > inm_release() curvnet=0 vnet=0xfffff80154c82a80
> 
> Perhaps the attached patch could help? (disclaimer: not even
> compile-tested)

The patch compiled, but it just moved the panic earlier than before.

#4  0xffffffff80bc2123 in panic (fmt=<unavailable>)
    at ../../../kern/kern_shutdown.c:839
#5  0xffffffff80d61726 in inm_release_task (arg=<optimized out>, 
    pending=<optimized out>) at ../../../netinet/in_mcast.c:633
#6  0xffffffff80c2166a in taskqueue_run_locked (queue=0xfffff800033cfd00)
    at ../../../kern/subr_taskqueue.c:476
#7  0xffffffff80c226e4 in taskqueue_thread_loop (arg=<optimized out>)
    at ../../../kern/subr_taskqueue.c:793

Now it panics at the location of the new CURVNET_SET and not the
old one..

Ok, decided to dump the contents of the vnet, and it looks like
it's a use after free:
(kgdb) print/x *(struct vnet *)0xfffff8012a283140
$2 = {vnet_le = {le_next = 0xdeadc0dedeadc0de, le_prev = 0xdeadc0dedeadc0de}, vnet_magic_n = 0xdeadc0de, 
  vnet_ifcnt = 0xdeadc0de, vnet_sockcnt = 0xdeadc0de, vnet_state = 0xdeadc0de, vnet_data_mem = 0xdeadc0dedeadc0de, 
  vnet_data_base = 0xdeadc0dedeadc0de, vnet_shutdown = 0xde}

The patch did seem to make it happen quicker, or maybe I was just more
lucky this morning...

> > (kgdb) #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:55
> > #1  doadump (textdump=1) at /usr/src/sys/kern/kern_shutdown.c:394
> > #2  0xffffffff80bc6250 in kern_reboot (howto=260)
> >     at /usr/src/sys/kern/kern_shutdown.c:481
> > #3  0xffffffff80bc66aa in vpanic (fmt=<optimized out>, ap=<optimized
> > out>) at /usr/src/sys/kern/kern_shutdown.c:913
> > #4  0xffffffff80bc6403 in panic (fmt=<unavailable>)
> >     at /usr/src/sys/kern/kern_shutdown.c:839
> > #5  0xffffffff80d6553b in inm_release (inm=0xfffff80029043700)
> >     at /usr/src/sys/netinet/in_mcast.c:630
> > #6  inm_release_task (arg=<optimized out>, pending=<optimized out>)
> >     at /usr/src/sys/netinet/in_mcast.c:312
> > #7  0xffffffff80c2521a in taskqueue_run_locked
> > (queue=0xfffff80003116b00) at /usr/src/sys/kern/subr_taskqueue.c:476
> > #8  0xffffffff80c26294 in taskqueue_thread_loop (arg=<optimized out>)
> >     at /usr/src/sys/kern/subr_taskqueue.c:793
> > #9  0xffffffff80b830f0 in fork_exit (
> >     callout=0xffffffff80c26200 <taskqueue_thread_loop>, 
> >     arg=0xffffffff81cf4f70 <taskqueue_thread>,
> > frame=0xfffffe0049e99b80) at /usr/src/sys/kern/kern_fork.c:1052
> > #10 <signal handler called>
> > (kgdb) 
> > 
> > I have the core files so I can get additional information.
> > 
> > Let me know if you need any additional information.
> > 
> 

> Index: sys/netinet/in_mcast.c
> ===================================================================
> --- sys/netinet/in_mcast.c	(revision 363386)
> +++ sys/netinet/in_mcast.c	(working copy)
> _at__at_ -309,8 +309,10 _at__at_
>  	IN_MULTI_LOCK();
>  	SLIST_FOREACH_SAFE(inm, &inm_free_tmp, inm_nrele, tinm) {
>  		SLIST_REMOVE_HEAD(&inm_free_tmp, inm_nrele);
> +		CURVNET_SET(inm->inm_ifp->if_vnet);
>  		MPASS(inm);
>  		inm_release(inm);
> +		CURVNET_RESTORE();
>  	}
>  	IN_MULTI_UNLOCK();
>  }


-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."