Re: Occassional "permission denied" in the middle of a large transfer over NFS

Hello again,

On Sun, Jul 08, 2012 at 06:35:50PM +0100, Vincent Hoffman wrote:
> Replying to myself just as a record, I have tried nfse and I didnt get
> the permission denied at all.
> The only issue I had with it is that it strictly adheres to the syntax
> in exports(5) while mountd is a little more flexible.
> 
> I had
> /usr/local/export -alldirs -maproot=root 85.xx.xx.xx
> 
> /usr is the root of that filesystem
> 
> mountd - allowed this but actually silently exports /usr (and all dirs
> below)
> 
> nfse - didnt allow this, it needed to be the correct
> /usr  -alldirs -maproot=root 85.xx.xx.xx
> 
> which is I guess a POLA violation but follows the documentation.
> this was using nfse in mountd compatibility mode. I havent played with
> its more advanced features.

The -alldirs option in the exports(5) file does not mean only
"all directories".

Please read these lines from exports(5) DESCRIPTION:

     The second is to specify the pathname of the root of the file system fol-
     lowed by the -alldirs flag; this form allows the host(s) to mount at any
     point within the file system, including regular files if the -r option is
     used on mountd(8).

The -alldirs option specifies that this pathname must be the root (mount
point) of some file system.

And read these lines from EXAMPLES:

     The file system rooted at /cdrom will be exported read-only to the entire
     network 192.168.33.0/24, including all its subdirectories.  Since /cdrom
     is the conventional mountpoint for a CD-ROM device, this export will fail
     if no CD-ROM medium is currently mounted there since that line would then
     attempt to export a subdirectory of the root file system with the
     -alldirs option which is not allowed.  The -quiet option will then sup-
     press the error message for this condition that would normally be sys-
     logged.  As soon as an actual CD-ROM is going to be mounted, mount(8)
     will notify mountd(8) about this situation, and the /cdrom file system
     will be exported as intended.  Note that without using the -alldirs
     option, the export would always succeed.  While there is no CD-ROM medium
     mounted under /cdrom, it would export the (normally empty) directory
     /cdrom of the root file system instead.

Here -alldirs means that /cdrom should be a file system.  As I remember
this even worked before revision 1.85 of mountd/mountd.c, then mountd.c
began to use nmount(2).  Now if one puts /cdrom in /etc/exports and
there is no file system mounted on /cdrom, then the entire / will be
exported.

Just create /etc/exports with one line "/cdrom -alldirs" and try to
mount <server>:/ on another system, you will get access to the / of
the <server>.

The current version of mountd is not "little more flexible", it does
not follow traditional -alldirs option's logic and does not follow
description of the -alldirs option in the exports(5) manual page, I guess
this is a POLA violation.

Now let's back to your example where /usr is the root of the file system:

/usr/local/export -alldirs -maproot=root 1.2.3.4

1. mountd exports /usr and all subdirectories under it for NFSv2/3 clients.
   Actually it has to export /usr/local/export only and all subdirectories
   under it only if /usr/local/export is or will be the root of some
   file system.

2. nfse in the compatibility mode sees that there is the -alldirs option
   and will export /usr/local/export and all subdirectories under it
   only if /usr/local/exports is or will be the root of some file system.

If one runs "nfse -Ct ..." for this file then its output will be:

configure: reading file /etc/exports
configure: converting configuration to native format

Pathname /usr/local/export
    Export specifications:
        -rw -sec sys -maproot 0:0:5 -host 1.2.3.4
    Subdirectories for NFSv2/3:
        /usr/local/export
            -alldirs -host 1.2.3.4

Warning: subdirectories exports are insecure

This is output in nfse native format.  Using NFSE part in the kernel,
nfse verifies whether /usr/local/export is a mount point, whether file
system supports NFS, and only if everything is correct, it starts to
export it.  When file system is unmounted or when another file system
is mounted, then NFSE part in the kernel is informed by EVENTHANDLERs
vfs_mount_event/vfs_unmount_event and NFS server verifies whether it
still can export this file system (it can be unmounted or it can be
shadowed).  The userland utility nfse got information of VFS events
via kevents VQ_MOUNT/VQ_UNMOUNT and only the verifies using NFSE
part in the kernel whether some file system is still exported or can
become exported.

There is one implementation mistake when "nfse -C ..." is used.  The
nfse utility follows implementation mistake from mountd and verifies
what is a file system and what is not a file system using the f_mntonname
field in the struct mount{}.  This is wrong since parts of mount point
directory can be renamed and this is not reflected in the f_mntonname
value.  That's why I do not recommend compatibility mode.  But this
implementation mistake is just a compatibility mode rule.  When
nfse is used in native mode, then it never checks f_mntonname values
and mount points can be renamed.

3. The corresponding configuration in the nfse native nfs.exports(5)
   format should be:

/usr/local/export -maproot root 1.2.3.4
/usr/local/export -subdir -alldirs

And the output of "nfse -t ..." will be:

configure: reading file /etc/nfs.exports

Pathname /usr/local/export
    Export specifications:
        -rw -sec sys -maproot 0:0:5 -host 1.2.3.4
    Subdirectories for NFSv2/3:
        /usr/local/export
            -alldirs *

Warning: subdirectories exports are insecure

Well, one can duplicate 1.2.3.4 in the -subdir line, this is just a style
decision in this example.

Let's consider the same example but without the -alldirs options:

/usr/local/export -maproot=root 1.2.3.4

1. mountd will export /usr and will administratively export /usr/local/export
   for NFSv2/3 clients.  A client can guess a filehandle and try to access
   any part of the /usr file system.  If /usr is not a mount point, then
   mountd will export /.

2. nfse in the compatibility mode will do exactly the same as mountd does
   (suppose that /usr is mount point when nfse was started):

configure: reading file /etc/exports
configure: converting configuration to native format

Pathname /usr
    Export specifications:
        -rw -sec sys -maproot 0:0:5 -host 1.2.3.4
    Subdirectories for NFSv2/3:
        /usr/local/export
            -host 1.2.3.4

Warning: subdirectories exports are insecure

3. nfse in the native mode will export /usr if it is or will be a mount point:

configure: reading file /etc/nfs.exports

Pathname /usr/local/export
    Export specifications:
        -rw -sec sys -maproot 0:0:5 -host 1.2.3.4

Let's back again to your example.  I guess you wanted administratively
export subdirectory /usr/local/exports and all subdirectories under
it from the /usr file system.  It is impossible to write such configuration
using the exports(5) file syntax.  But it is possible to do this with
the nfs.exports(5) syntax:

/usr -maproot=root 1.2.3.4
/usr/local/export -subdir -alldirs

And "nfse -t ..." output:

configure: reading file /etc/nfs.exports

Pathname /usr
    Export specifications:
        -rw -sec sys -maproot 0:0:5 -host 1.2.3.4
    Subdirectories for NFSv2/3:
        /usr/local/export
            -alldirs *

Warning: subdirectories exports are insecure

Am I correct?  Did I correctly understand your example?