Re: Occassional "permission denied" in the middle of a large transfer over NFS

Vincent Hoffman wrote:
> On 07/07/2012 13:26, Vincent Hoffman wrote:
> > On 01/07/2012 12:18, Rick Macklem wrote:
> >> Vincent Hoffman wrote:
> >>> On 01/07/2012 01:53, Rick Macklem wrote:
> >>>> To modify mountd to use the kernel changes is more work than I
> >>>> have
> >>>> time for, in part because mountd.c is a very ugly old piece of C
> >>>> code, imho.
> >>>>
> >>>> I do have a patch that suspends/resumes execution of the nfsd
> >>>> threads
> >>>> (new, experimental for FreeBSD8.n only) when mountd reloads
> >>>> /etc/exports.
> >>>> This approach has certain disadvantages vs Andrey's transactional
> >>>> load/commit
> >>>> model, but it is simple and might be sufficient for your problem.
> >>>> If you want to try this patch, it is at:
> >>>>    http://people.freebsd.org/~rmacklem/atomic-export.patch
> >>>> (The patch affects both the kernel and mountd.c.)
> >>> Happy to try it, to be certain I have understood, this is for the
> >>> newnfs
> >>> server (experimental in 8.x default for 8
> >>> 9+)
> >> Yes, that is correct. For the old (default on 8.x) it will have no
> >> effect.
> >>
> >>> I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm
> >>> when
> >>> i
> >>> get a minute.
> >> Also, to enable it, you must add a "-S" flag to mountd_flags, after
> >> you've
> >> replaced the mountd executable.
> >>
> >> The patch is pretty straightforward, but has not seen much testing.
> >> Hopefully, it might be useful for you, rick
> > Hi Rick,
> >
> > I'm afraid this didnt make any real difference for me.
> > Since I couldnt test it on the live system I tried it on a test vm.
> > on the vm (nfs server) I set a looping mount/umount
> > while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp
> > ; done
> >
> > and on the client I set a loop of tars of large directorys to the
> > nfs
> > mount running under time to see how well it survived. Then
> > replicated
> > the test with the patch and without.
> >
> >
> > [root_at_seaurchin ~]# ministat nopatch.txt atomicpatch.txt
> > x nopatch.txt
> > + atomicpatch.txt
> > +--------------------------------------------------------------------------------------------------+
> > |
> > *
> > |
> > |
> > *
> > |
> > |
> > x*
> > |
> > |   xx*
> > x
> > |
> > |  +x**
> > xx
> > |
> > |  **** xxx
> > x
> > |
> > |  **** xxx +x+
> > +
> > |
> > |  ****+*xx +x+ x
> > +
> > |
> > |  ****+*x****++++x + +
> > x |
> > |  *************+*xx+ +++x * x
> > x |
> > |  ****************x**++*x+***x+ x*+ x ++*+ + x+ +x +
> > + +|
> > |||_______M_M_A__A_______|______|
> > |
> > +--------------------------------------------------------------------------------------------------+
> >     N Min Max Median Avg Stddev
> > x 101 1.25 106.8 14.08 21.892178 22.196005
> > + 101 1.21 186.93 18.46 27.995842 30.523218
> > No difference proven at 95.0% confidence
> >
> >
> > (excuse wrapped ascii art)
> >
> > I think I'll have a look at the nfse patch set and see how that
> > performs.
> Replying to myself just as a record, I have tried nfse and I didnt get
> the permission denied at all.
> The only issue I had with it is that it strictly adheres to the syntax
> in exports(5) while mountd is a little more flexible.
> 
> I had
> /usr/local/export -alldirs -maproot=root 85.xx.xx.xx
> 
> /usr is the root of that filesystem
> 
> mountd - allowed this but actually silently exports /usr (and all dirs
> below)
> 
Not exactly correct. mountd exports the entire file system in the kernel
for the NFS server, since exports can only be attached to the mount points
in the kernel. However, when the client's mount protocol requests a mount
file handle for anything other than /usr/local/export, it will refuse that.
(Which means that to mount anything other than /usr/local/export, the client
 must maliciously "guess" the file handle for mounting.)

Put another way, a "non-malicious" NFSv3 client will only be able to mount
/usr/local/export. Robert Watson calls this an "administrative control" and
feels that it is necessary.
Until this is resolved by "the collective", I do not see how mountd can be
replaced by "nfse", although I'd like to see mountd replaced because of the
above problem + mountd.c is very old, ugly (implies difficult to change) code.

> nfse - didnt allow this, it needed to be the correct
> /usr -alldirs -maproot=root 85.xx.xx.xx
> 
> which is I guess a POLA violation but follows the documentation.
> this was using nfse in mountd compatibility mode. I havent played with
> its more advanced features.
> 
> Vince
> > Thanks for all your work on NFS on FreeBSD.
> >
> > Vince
> >
> 
> 
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to
> "freebsd-current-unsubscribe_at_freebsd.org"