Re: HPN and None options in OpenSSH

On 01/23/16 09:15, Kevin Oberman wrote:

> Are you sure of this? I have not looked at the code, but my former
> colleagues at the high performance research network ESnet claim at
> http://fasterdata.es.net/data-transfer-tools/say-no-to-scp/  that the
> internal buffers and effective window size have recently been increased
> from 64KB to 1MB an allow for transfer rates of up to 140 Mbps over a link
> with 53 ms. latency. With the HPN patches, they report 1.2 Gbps, making HPN
> patches still significant over high latency paths.

DES wrote:

>  The buffer code in 7.1
> supports dynamically-sized buffers with a hard limit of 128 MB.  The
> default window size for client sessions is 2 MB, or 1 MB if associated
> with a tty.  I'm not sure what the maximum size is. 

I'll try to do some cross-country or trans-Atlantic testing this weekend
or next week, using a mix of ssh versions and HPN-patched versus not
(and CentOS vs. FreeBSD vs. possibly Debian unstable with the 4.2+
kernel as yet another degree of freedom).  I'll see what basic results I
can get and we can update fasterdata.es.net as necessary.

> That said, scp still performed poorly when compared to other technologies
> (i.e. GridFTP) for bulk data transfer over high-latency high-bandwidth
> links. (ESnet provides links of up to 400 Gbps between the US and Europe as
> well as within the US, so this sort of thing is quite important to them.)

That it is!

> scp is a horrible protocol, use sftp or (preferably) rsync over ssh.

I still think <anything> over ssh transport is lousy for bulk-data
transfers, but it is the one thing that's generally installed by default
on every OS and and is allowed by many firewalls.  And, of course, it
encrypts in flight.  Certainly gridFTP, aspera (if you can afford it!)
and other packages optimized for bulk data transfer will work better.

michael
ESnet