Re: Deadlock in nullfs/zfs somewhere

On 16 July 2013 12:32, Andriy Gapon <avg_at_freebsd.org> wrote:

> vmcore.0 was useless for some reason - an interesting address was not accessible.

Eek.

> vmcore.1 seems to be very similar and is actually useful.

Oh good.

> This problem looks like an interesting deadlock involving ZFS and VFS and vnode
> shortage.
> The most obvious things are that many threads could not allocate a new vnode and
> are waiting in getnewvnode_reserve and also many threads are stuck waiting on
> vnode locks held by the former threads.
> In effect, they all wait for vnlru, which in turn is stuck in
> zfs_freebsd_reclaim on z_teardown_lock.
> That lock is held by a thread doing a rollback ioctl.
> And that thread waits for zfs sync thread to actually perform the rollback.
> The sync thread waits on zfs quiesce thread to declare the current transaction
> group as quiesced.
> The quiesce thread, obviously, waits for all operations running in the current
> transaction group to complete.
> Some of those operations are e.g. VOP_CREATE -> zfs_create.  They already
> started a zfs transaction (as a part of the current transaction group) and they
> execute zfs_mknode which needs a new vnode.  So these threads are waiting for a
> new vnode and do not let the current transaction group become quiesced.
> GOTO beginning.
>
> Compressing the above description to the extreme, it boils down to: ZFS needs a
> new vnode from vnlru and is waiting on it, while vnlru has to wait on ZFS.

:(  So it's a deadlock. Ok, so what's next?



-adrian