NFS write() calls lead to read() calls?

Hi,

I observe a strange effect, when using the following setup: Three
FreeBSD 6.2[1] machines on Gigabit Ethernet using em(4) interfaces.

HostC is the NFS server, HostB has /net/share mounted from HostC. I
will use HostA and HostB to demonstrate the issue. Picture this:

hostA # scp 500MB hostB:/net/share/

Iff the file "500MB" does not yet exist on the NFS share, I can see X
MB/s going out of HostA, X MB/s coming in on HostB, X MB/s going out
on hostB again and finally X MB/s coming in on HostC.

If I run the scp again, I can see X MB/s going out from HostA, 2*X
MB/s coming in on HostB and X MB/s out plus X MB/s in on HostC. What's
happening is, that HostB issues one NFS READ call for every WRITE
call. The traffic flows like this:

  ----->   ----->
A        B        C
           <-----

If I rm(1) the file on the NFS share, then the first scp(1) will not
show this behaviour. It is only when overwritting files, that this
happens.

The real weirdness comes into play, when I simply cp(1) from HostB
itself like this:

hostB # cp 500MB /net/share/

I can do this over and over again, and _never_ get any noteworthy
amount of NFS READ calls, only WRITE. The network traffic is also, as
you would expect.

Then I tested using ssh(1) instead of scp(1), like this:

hostA # cat 500MB | ssh hostB "cat >/net/share/500MB"

This works, too. Probably, because sh(1) is truncating the file?

So, can someone please explain to me, what is happening and if/how it
can be avoided?

Cheers,
Uli

[1] I know this is current, but on stable_at_ nobody picked up the
thread, so I'll try my luck here.