Re: [ANN] unionfs patchset-11 release

On Tue, 25 Apr 2006, Daichi GOTO wrote:

>   Changes in unionfs-p11.diff
>     - Changed a few implementations around the lock/unlock
>       mechanism. Because of this, you can use both the unionfs
>       and the nullfs together without LK_CANRECURSE.
>     - Fixed a bug that sometimes does not unlock if it cannot
>       create shadow file.

First off, thanks again for working on this!

>  If someone knows the details of vnode's lock status via
>  VOP_GETWRITEMOUNT, Please teach us (daichi, ozawa). We
>  want to know the details.

Basically, file systems supporting full file system snapshots (UFS) provide a 
mechanism to "lock out" writers before they enter VFS so that they don't end 
up holding write locks for long periods, leading to deadlock. 
vn_start_write() is called to notify the file system that a thread is about to 
enter the file system for a write, and vn_write_finished() is called to notify 
the file system it is done.  In effect, it's a giant reader-writer lock, which 
allows multiple readers and multipler writers, except during snapshot 
generation, when it blocks new writers until the snapshot is generated.

In general, you'll notice two sorts of logic around calls to vn_start_write(): 
a first set, where vn_start_write() is called once holding a vnode reference, 
is acquired, and then things continue as normal, with a final 
vn_finished_write() call at the end.  In this situation, vnode locks are 
acquired after the vn_start_write() call, but vnode references are held before 
(since vn_start_write() takes a vnode so that it can find the file system).

The other circumstance is where vnode locks may already be held, in which case 
a non-sleeping acquire is performed, since in effect this is a violation of 
lock order.  If it fails, the vnode lock is released, the reference is 
acquired, and then the whole operation is restarted so that we can try again 
to acquire the vnode lock under circumstances where file system snapshot lock 
can be safely acquired.  So basically, it has deadlock detection and recovery 
logic.  The V_XSLEEP lock basically says "Sleep until the snapshot lock would 
be available, then return", which loops back so we can re-try the acquires.

So according to the above, the file system snapshot lock is *before* the vnode 
locks in the lock order, although in practice we acquire in any order as long 
as it won't lead to deadlock (in which case we recover).  The logic here is a 
little shaky in practice -- among other things, it looks like potentially the 
mount point could go away during the call to vn_start_write() once the vnode 
is released in the deadlock detection code, but in practice this probably 
never happens.

Notice that the above is all couched in terms of a single file system, not 
stacking.  This is probably because it was all written with UFS and not 
stacking in mind.

Robert N M Watson