Re: ZFS panic: System call unlink returning with 1 locks held

On Sat, Jul 28, 2007 at 08:31:32AM +0200, Thierry Herbelot wrote:
> Le Friday 27 July 2007, Kris Kennaway a ?crit :
> > On Fri, Jul 27, 2007 at 03:08:26PM -0500, Dan Nelson wrote:
> > > In the last episode (Jul 21), Thierry Herbelot said:
> > > > with a recent -current -built yesterday), I just got a panic while
> > > > rebuilding -j4 the world and portupgrading firefox.
> > > >
> > > > the machine is pretty much memory limited (only 320 MB of RAM), with
> > > > two CPUs, running a straight GENERIC kernel, including WITNESS and
> > > > INVARIANTS.
> > >
> > > [..]
> > >
> > > > the panic message is :
> > > >
> > > > panic: System call unlink returning with 1 locks held
> > > > cpuid = 0
> > > > KDB: enter: panic
> > > > [thread pid 42789 tid 100102 ]
> > > > Stopped at      kdb_enter+0x32: leave
> > > > db> where
> > > > Tracing pid 42789 tid 100102 td 0xc2ce3200
> > > > kdb_enter(c0a92bc5,0,c0ac0a31,d5457c8c,0,...) at kdb_enter+0x32
> > > > panic(c0ac0a31,c0a98f5c,1,c0a98f5c,c0b3f030,...) at panic+0x124
> > > > syscall(d5457d38) at syscall+0x46e
> > > > Xint0x80_syscall() at Xint0x80_syscall+0x20
> > >
> > > I've been seeing this, as late as on a Jul 24 kernel.  Happened once
> > > during the cleandir stage of a buildworld, and few more times when the
> > > system was relatively idle (although it is an mrtg server so lots of
> > > files are constantly created and rm'd).  My system is i386 with 1GB of
> > > RAM, has a ZFS root, and is SMP.  I've also gotten a similar "System
> > > call rename returning with 1 locks held" panic.  Is there any way to
> > > find out what lock is being held?  I've got a couple crashdumps.
> >
> > It appears to be a leak in the lock counters somewhere, perhaps
> > related to recursively acquired rwlocks (e.g. double increment, single
> > free).  I eventually disabled the check because even adding extensive
> > extra debugging there was no evidence of an actual lock being leaked
> > anywhere.
> >
> > Kris
> 
> Hello,
> 
> I saw the same panic once or twice since the first report (same trace, 
> different phases of make buildworld).
> 
> I had vfs.zfs.zil_disable="1" in /boot/loader.conf, which I just removed. 
> we'll see if the stability improves.

AFAIK, disabling zil remains necessary to avoid low memory deadlocks.
Just remove the test or turn off INVARIANTS to disable it.

Kris