Re: fsid change of ZFS?

Hiroki Sato wrote:
> Hiroki Sato <hrs_at_freebsd.org> wrote
> in <20110819.002046.908756241495481148.hrs_at_allbsd.org>:
> 
> hr> Hi,
> hr>
> hr> I have experienced "Stale NFS file handle" issue when switching
> hr> between oldnfs and newnfs on a CURRENT box (NFS server exporting
> ZFS
> hr> mountpoints). The cause was that fsid was changed in the following
> hr> conditions and not in the NFS subsystem itself, but I am wondering
> if
> hr> these are expected behavior...
> hr>
> hr> First, I tried the following configurations of NFS and ZFS, and
> saw
> hr> if fsid of the same mountpoint (a mounted ZFS dataset) changed or
> hr> not by using statfs(2):
> hr>
> hr> compile opts kld module fsid[0:1] kld loaded by
> hr>
> ----------------------------------------------------------------------------
> hr> NFSSERVER+NFSCLIENT zfs 865798fa:8346ef02 loader
> hr>
> hr> NFSSERVER+NFSCLIENT zfs 865798fa:8346ef07 kldload(8)
> hr>
> hr> NFSSERVER+NFSCLIENT+
> hr> NFSD+NFSCL zfs 865798fa:8346ef03 loader
> hr>
> hr> NFSSERVER+NFSCLIENT+
> hr> NFSD+NFSCL zfs 865798fa:8346ef08 kldload(8)
> hr>
> hr> NFSSERVER+NFSCLIENT nfsd+nfscl+zfs 865798fa:8346ef08 loader
> hr>
> ----------------------------------------------------------------------------
> 
> Ah, I found why this happened:
> 
> /*
> * The fsid is 64 bits, composed of an 8-bit fs type, which
> * separates our fsid from any other filesystem types, and a
> * 56-bit objset unique ID. The objset unique ID is unique to
> * all objsets open on this system, provided by unique_create().
> * The 8-bit fs type must be put in the low bits of fsid[1]
> * because that's where other Solaris filesystems put it.
> */
> fsid_guid = dmu_objset_fsid_guid(zfsvfs->z_os);
> ASSERT((fsid_guid & ~((1ULL<<56)-1)) == 0);
> vfsp->vfs_fsid.val[0] = fsid_guid;
> vfsp->vfs_fsid.val[1] = ((fsid_guid>>32) << 8) |
> vfsp->mnt_vfc->vfc_typenum & 0xFF;
> 
> Since the vfc_typenum variable is incremented every time a new vfs is
> installed, loading order of modules that call vfs_register() affects
> ZFS's fsid.
> 
> Anyway, possibility of fsid change is troublesome especially for an
> NFS server with a lot of clients running. Can zeroing or setting a
> fixed value to the lowest 8-bit of vfs_fsid.val[1] be harmful?
> 
> -- Hiroki
Well, the problem is that the fsid needs to be unique among all mounts.
The vfs_typenum field is used to try and ensure that it does not end up
the same value as a non-ZFS file system.

(A) I think making that field a fixed constant should be ok, if the function
checks for a conflict by calling vfs_getvfs() to check for one.
See vfs_getnewfsid() for how this is done. (There is a mutex lock that
needs to be held while doing it.) Alternately, if ZFS can call vfs_getnewfsid()
instead of doing its own, that might be nicer?

(B) Another way to fix this would be to modify vfs_register() to look up
file systems in a table (by vfc_name) and used a fixed, assigned value
from the table for vfc_typenum for entries found in the table. Only do
the "maxvfsconf++" when there isn't an entry for the fstype in the table.
(VFS_GENERIC can be set to the size of the table. That's what happened
 in the bad old days when vfsconf was a table built at kernel config time.)

If you guys think (B) is preferred, I could come up with a patch. I don't
know enough about ZFS to do (A).

rick