Re: MPSAFE VFS -- List of upcoming actions

On Mon, Jul 02, 2012 at 11:03:40AM -0400, Alexander Kabaev wrote:
> On Mon, Jul 02, 2012 at 02:12:20AM -0400, Christoph Hellwig wrote:
> > On Sun, Jul 01, 2012 at 03:52:05PM +0200, Attilio Rao wrote:
> > > anything by SoC involved people about NTFS and certainly I don't see a
> > > plan to get XFS locked.
> > 
> > Stupid question, but what amount of locking does XFS in FreeBSD still
> > need?  I'm one of the maintainer of XFS on Linux, and while I know
> > FreeBSD imported a really old version compared to the current one the
> > codebases on IRIX and later Linux never relied on any global Giant-style
> > locking.  So if there is anything to fix it would be the in the small
> > bits of FreeBSD-specific code.
> > 
> 
> When I stopped being interested in XFS, I left is marked as non-MPSAFE
> entirely because of the lack of proper testing and because VFS locking
> was still evolving, there was no officially proper way of locking the
> FS and no other FS in the tree was MPSAFE. At that time the only
> problematic area was around inode instantiation, but sereval other
> lockingi changes have made it into the tree since then, namely ones that
> deal with insmntque and also VOP_LOOKUP changes. To mark XFS MPSAFE, one
> needs to simply audit the code and make sure it still makse sense for today's
> VFS, which is not a huge amount of work. One step further woule be to take
> most of the XFS from under the exclusive vnode locking to improve the
> performance.

If filesystem uses some global internal locks, that locks usually are
placed after the vnode locks in global lock order, because VOP methods
call into fs with vnode locked. Then, VOP_LOOKUP() usual sequence of
events, when method is called with lookup directory locked, causes LOR.
It appears because you lock global lock upon entry into VOP_LOOKUP(),
and then need to lock the returned vnode.
Dropping global lock inside VOP_LOOKUP() usually exposes races which
were the reason to introduce the global lock. Having filesystem
non-MPSAFE makes the LOR go away without the need to drop global lock.

Example of FreeBSD native filesystem which suffered from this issue
and required quite non-trivial handling is devfs. Devfs uses per-mount
global lock. See devfs_allocv() and devfs_allocv_drop_refs() for
the gory details.