Re: r315684 panic: sleepq_add: td 0xfffff80003c01a40 to sleep on wchan 0xfffff80006f0873c with sleeping prohibited

On Sat, Mar 25, 2017 at 11:45:29AM +0200, Konstantin Belousov wrote:
K> On Fri, Mar 24, 2017 at 08:31:42PM -0700, Gleb Smirnoff wrote:
K> >   Darren,
K> > 
K> > On Sat, Mar 25, 2017 at 03:03:14AM +0200, Konstantin Belousov wrote:
K> > K> On Fri, Mar 24, 2017 at 05:40:15PM +0000, Darren wrote:
K> > K> > I am getting this panic every hour to every couple of hours.
K> > K> > 
K> > K> > FreeBSD asrock 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r315684: Thu Mar 23 14:56:45 EDT 2017     darren_at_asrock:/usr/obj/usr/src/sys/GENERIC  amd64
K> > K> > I manually typed out the following, apologize for any typos. 
K> > K> > 
K> > K> > 
K> > K> > panic: sleepq_add: td 0xfffff80003c01a40 to sleep on wchan 0xfffff80006f0873c with sleeping prohibited
K> > K> > cpuid = 0
K> > K> > time = 1490372797
K> > K> > KDB: stack backtrace:
K> > K> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0072e33690
K> > K> > vpanic() at vpanic+0x19c/frame 0xfffffe0072e33710
K> > K> > kassert_panic() at kassert_panic+0x126/frame 0xfffffe0072e33780
K> > K> > sleepq_add() at sleepq_add+0x34f/frame 0xfffffe0072337d0
K> > K> > _sleep() at _sleep+0x28d/frame 0xfffffe0072e33870
K> > K> > soclose() at soclose+0xda/frame 0xfffffe0072e338b0
K> > K> > _fdrop() at _fdrop+0x1a/frame 0xfffffe0072e338d0
K> > K> > sendfile_iodone() at sendfile_iodone+0x19d/frame 0xfffffe0072e33910
K> > K> > vnode_pager_generic_getpages_done_async() at vnode_pager_generic_getpages_done_async+037/frame 0xfffffe0072e33930
K> > K> > bufdone() at bufdone+0x64/frame 0xfffffe0072e33960
K> > K> > g_io_deliver() at g_io_deliver+0x276/frame 0xfffffe0072e339b0
K> > K> > g_io_deliver() at g_io_deliver+0x276/frame 0xfffffe0072e33a00
K> > K> > g_disk_done() at g_disk_done+0x104/frame 0xfffffe0072e33a40
K> > K> > xpt_done_process() at xpt_done_process+0x35f/frame 0xfffffe0072e33a80
K> > K> > xpt_done_direct() at ahci_ch_intr_direct+0xd5/frame 0xfffffe0072e33af0
K> > K> > ahci_itr() at ahci_intr+0x102/frame 0xfffffe0072e33b20
K> > K> > intr_event_execute_handlers() at intr_event_execute_handlers+0x99/frame 0xfffffe0072e33b60
K> > K> > ithread_loop() at ithread_loop+0xb6/frame 0xfffffe0072e33bb0
K> > K> > fork_exit() at fork_exit+0x84/frame 0xfffffe0072e33bf0
K> > K> > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0072e33bf0
K> > K> > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
K> > K> > KDB: enter: panic
K> > K> > [ thread pid 12 tid 100038 ]
K> > K> > Stopped at      kdb_enter+0x3b: movq    $0,kdb_why
K> > K> > db>
K> > K> 
K> > K> Indeed, the context where sendfile_iodone() is executed, cannot call fdrop().
K> > 
K> > Can you please test the attached patch?
K> > 
K> > -- 
K> > Totus tuus, Glebius.
K> 
K> > Index: sys/kern/kern_sendfile.c
K> > ===================================================================
K> > --- sys/kern/kern_sendfile.c	(revision 315926)
K> > +++ sys/kern/kern_sendfile.c	(working copy)
K> > _at__at_ -296,8 +296,9 _at__at_ sendfile_iodone(void *arg, vm_page_t *pg, int coun
K> >  		CURVNET_RESTORE();
K> >  	}
K> >  
K> > -	/* XXXGL: curthread */
K> > -	fdrop(sfio->sock_fp, curthread);
K> > +	ACCEPT_LOCK();
K> > +	SOCK_LOCK(so);
K> > +	sorele(so);
K> >  	free(sfio, M_TEMP);
K> >  }
K> >  
K> > _at__at_ -860,7 +861,9 _at__at_ prepend_header:
K> >  		} else {
K> >  			sfio->sock_fp = sock_fp;
K> >  			sfio->npages = npages;
K> > -			fhold(sock_fp);
K> > +			SOCK_LOCK(so);
K> > +			soref(so);
K> > +			SOCK_UNLOCK(so);
K> >  			error = (*so->so_proto->pr_usrreqs->pru_send)
K> >  			    (so, PRUS_NOTREADY, m, NULL, NULL, td);
K> >  			sendfile_iodone(sfio, NULL, 0, 0);
K> 
K> With this patch, what prevents a close of the sfio->sock_fp file, which is
K> needed to get the pointer to socket ?

You are right, patch is unfinished. Here is better one.

-- 
Totus tuus, Glebius.