Re: Occassional "permission denied" in the middle of a large transfer over NFS

From: Vincent Hoffman <vince_at_unsane.co.uk>
Date: Sun, 01 Jul 2012 12:13:30 +0100
On 01/07/2012 01:53, Rick Macklem wrote:
> Vincent Hoffman wrote:
>> Just a note to say I have tested this on -CURRENT with the new nfs
>> server and it is still the case.
>>
>> On the client (FreeBSD seaurchin 8.3-RELEASE-p3 FreeBSD 8.3-RELEASE-p3
>> #0: Tue Jun 12 00:39:29 UTC 2012
>> root_at_amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64)
>>
>> [root_at_seaurchin /mnt/nfs/vm]# date ; tar -cf foo.tar
>> /var/nfsen/profiles-data/ ; date
>> Thu Jun 28 20:12:36 BST 2012
>> tar: Removing leading '/' from member names
>> tar: Write error
>> Thu Jun 28 20:12:38 BST 2012
>> [root_at_seaurchin /mnt/nfs/vm]#
>>
>>
>>
>> on the Server
>> [root_at_fbsd /var/tmp]# uname -a ; date ; while true ; do mount /dev/md0
>> /mnt/tmp/ ; umount /mnt/tmp ; done
>> FreeBSD fbsd 10.0-CURRENT FreeBSD 10.0-CURRENT #22 r237646: Wed Jun 27
>> 19:13:26 BST 2012
>> toor_at_fbsd.bmk.namesco.net:/usr/obj/usr/src/sys/unsane-vm amd64
>> Thu Jun 28 20:12:38 BST 2012
>> ^C
>> [root_at_fbsd /var/tmp]#
>>
>> any suggestions welcome.
>>
>> Vince
>>
>>
>> On 27/06/2012 16:27, Vincent Hoffman wrote:
>>> Hi,
>>> After only one off-list reply from the author of kern/136865 (see
>>> below)
>>> after asking -questions, I thought it worth asking -CURRENT.
>>> Basically:
>>>
>>> I seem to have run into the problems described in this old thread.
>>> http://lists.freebsd.org/pipermail/freebsd-questions/2004-April/044927.html
>>>
>>> tl:dr mountd may give incorrect permission denied errors when it is
>>> refreshing the exports list due to non-atomic operations,
>>> /sbin/mount has code that sends SIGHUP to
>>> mountd on any mount operation, which implies that any manual mount
>>> request would cause the problem.
>>>
>>> Currently I have still only tested on 8.3-RELEASE but the svn log
>>> doesnt
>>> seem to mention a fix since then. I'm currently taking a VM up to
>>> -CURRENT to test.
>>>
>>> Looking though old PRs I see the following related.
>>> kern/131342
>>> kern/136865 (with patch for 7.2 and links to
>>> http://nfse.sourceforge.net/ for -CURRENT )
>>>
>>> Does anyone who is qualified (sadly not me) feel like looking at the
>>> code to see if its suitable for inclusion in part/whole as not
>>> having
>>> NFS transfers interrupted by local mount operations on the nfs
>>> server
>>> would be very handy :)
>>>
> I haven't looked at Andrey's patch, but conceptually it sounds like
> the best approach. As I understand it, the problem with replacing
> mountd with nfse (at least in the FreeBSD source tree) is that nfse
> is not 100% backwards compatible with /etc/exports and, as such, is
> a POLA violation.
Understood. Its far from a simple drop in replacement.
>
> To modify mountd to use the kernel changes is more work than I have
> time for, in part because mountd.c is a very ugly old piece of C code, imho.
>
> I do have a patch that suspends/resumes execution of the nfsd threads
> (new, experimental for FreeBSD8.n only) when mountd reloads /etc/exports.
> This approach has certain disadvantages vs Andrey's transactional load/commit
> model, but it is simple and might be sufficient for your problem.
> If you want to try this patch, it is at:
>    http://people.freebsd.org/~rmacklem/atomic-export.patch
> (The patch affects both the kernel and mountd.c.)
Happy to try it, to be certain I have understood, this is for the newnfs
server (experimental in 8.x default for 8
9+)
I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when i
get a minute.
>
> Also, you could easily hack mount.c so that it doesn't send a SIGHUP
> to mountd (which causes the exports to be reloaded) every time a local
> fs is mounted.
True and I may have to do that for the production NAS for the time being.
Thanks for looking at this.

Vince
>
> rick
>
>>> thanks, Vince
>>>
>>>
>>> _______________________________________________
>>> freebsd-current_at_freebsd.org mailing list
>>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to
>>> "freebsd-current-unsubscribe_at_freebsd.org"
>>
>> _______________________________________________
>> freebsd-current_at_freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to
>> "freebsd-current-unsubscribe_at_freebsd.org"
> _______________________________________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_freebsd.org"
Received on Sun Jul 01 2012 - 09:13:36 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:28 UTC