Re: Occassional "permission denied" in the middle of a large transfer over NFS

On 01/07/2012 12:18, Rick Macklem wrote:
> Vincent Hoffman wrote:
>> On 01/07/2012 01:53, Rick Macklem wrote:
>>> To modify mountd to use the kernel changes is more work than I have
>>> time for, in part because mountd.c is a very ugly old piece of C
>>> code, imho.
>>>
>>> I do have a patch that suspends/resumes execution of the nfsd
>>> threads
>>> (new, experimental for FreeBSD8.n only) when mountd reloads
>>> /etc/exports.
>>> This approach has certain disadvantages vs Andrey's transactional
>>> load/commit
>>> model, but it is simple and might be sufficient for your problem.
>>> If you want to try this patch, it is at:
>>>    http://people.freebsd.org/~rmacklem/atomic-export.patch
>>> (The patch affects both the kernel and mountd.c.)
>> Happy to try it, to be certain I have understood, this is for the
>> newnfs
>> server (experimental in 8.x default for 8
>> 9+)
> Yes, that is correct. For the old (default on 8.x) it will have no effect.
>
>> I'll apply to my -CUREENT test VM for now. Will fire up a 8.3 vm when
>> i
>> get a minute.
> Also, to enable it, you must add a "-S" flag to mountd_flags, after you've
> replaced the mountd executable.
>
> The patch is pretty straightforward, but has not seen much testing.
> Hopefully, it might be useful for you, rick

Hi Rick,

I'm afraid this didnt make any real difference for me.
Since I couldnt test it on the live system I tried it on a test vm.
on the vm (nfs server) I set a looping mount/umount
while true ; do mount /dev/md0 /mnt/tmp ; sleep 1 ; umount /mnt/tmp ;  done

and on the client I set a loop of tars of large directorys to the nfs
mount running under time to see how well it survived. Then replicated
the test with the patch and without.


[root_at_seaurchin ~]# ministat nopatch.txt atomicpatch.txt
x nopatch.txt
+ atomicpatch.txt
+--------------------------------------------------------------------------------------------------+
|    
*                                                                                           
|
|    
*                                                                                           
|
|   
x*                                                                                           
|
|   xx* 
x                                                                                        
|
|  +x** 
xx                                                                                       
|
|  **** xxx 
x                                                                                    
|
|  **** xxx +x+      
+                                                                           
|
|  ****+*xx +x+   x  
+                                                                           
|
|  ****+*x****++++x + +                        
x                                                  |
|  *************+*xx+ +++x *  x                
x                                                  |
|  ****************x**++*x+***x+ x*+ x  ++*+  + x+      +x +      
+                              +|
|||_______M_M_A__A_______|______|                                                                 
|
+--------------------------------------------------------------------------------------------------+
    N           Min           Max        Median           Avg        Stddev
x 101          1.25         106.8         14.08     21.892178     22.196005
+ 101          1.21        186.93         18.46     27.995842     30.523218
No difference proven at 95.0% confidence


(excuse wrapped ascii art)

I think I'll have a look at the nfse patch set and see how that performs.

Thanks for all your work on NFS on FreeBSD.

Vince

>
>>> Also, you could easily hack mount.c so that it doesn't send a SIGHUP
>>> to mountd (which causes the exports to be reloaded) every time a
>>> local
>>> fs is mounted.
>> True and I may have to do that for the production NAS for the time
>> being.
>> Thanks for looking at this.
>>
>> Vince
>>> rick
>>>
>>>>> thanks, Vince