Re: slow USB 3.0 on -current

From: John-Mark Gurney <jmg_at_funkthat.com>
Date: Sun, 12 Jul 2020 21:51:49 -0700
Mark Millard wrote this message on Sun, Jul 12, 2020 at 18:26 -0700:
> John-Mark Gurney jmg at funkthat.com wrote on
> Sat Jul 11 22:44:36 UTC 2020 :
> 
> > I'm having issues getting good ethernet performance from a USB ethernet
> > adapter (ure) under FreeBSD on an HP EliteDesk 705 G2 Mini[1].  It's an
> > AMD PRO A10-8700B based system using the AMD A78 FCH chipset.
> > 
> > Under FreeBSD -current (r362596), 12.1-R and 11.4-R, the RealTek USB
> > adapter only gets around 10MB/sec performance.  During the transfer,
> > the CPU usage is only around 3-5%, so it's definitely not CPU bound.
> > 
> > I have tested Windows 10 and NetBSD 9.0 performance, and both provide
> > 100MB/sec+ w/o troubles.
> > 
> > I have attached dmesg from both FreeBSD -current and NetBSD 9.0.
> > 
> > Any hints on how to fix this?
> > 
> > This may be related, but I'm also having issues w/ booting when I have
> > both a SD USB 2.0 card reader AND the ure plugged into USB 3.0 ports.
> > 
> > If I move the SD card reader to USB 2.0, the umass device will attach
> > and work.  I have also attached a clip of the dmesg from that
> > happening.
> > 
> > Has anyone else seen this issue?  Ideas or thoughts on how to resolve
> > the performance issues?
> 
> It might prove useful to use iperf3 with
> 
> # iperf3 -s
> 
> on one machine and doing
> 
> # iperf3 -c ADDR
> . . .
> # iperf3 -R -c ADDR
> . . .
> 
> on the other. (That last swaps the
> sender/receiver status.)
> 
> All 3 commands will have output. The
> -s one will produce output for each of
> the -c ones.
> 
> The outputs for the sender(s) will include Cwnd
> (congestion window size) information that may
> be relevant. It will report bit rate and
> retry count sampling (and overall figures).

Here is the results for FreeBSD w/ USB3 ure.  .80 is the USB3
adapter side:
gold,pts,/home/jmg,502$iperf3 -c 192.168.0.80
Connecting to host 192.168.0.80, port 5201
[  5] local 192.168.0.2 port 50042 connected to 192.168.0.80 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  8.94 MBytes  75.0 Mbits/sec  931   15.5 KBytes
[  5]   1.00-2.00   sec  9.98 MBytes  83.7 Mbits/sec  919   27.3 KBytes
[  5]   2.00-3.00   sec  9.95 MBytes  83.5 Mbits/sec  954   5.71 KBytes
[  5]   3.00-4.00   sec  9.97 MBytes  83.7 Mbits/sec  939   28.7 KBytes
[  5]   4.00-5.00   sec  9.97 MBytes  83.6 Mbits/sec  951   17.3 KBytes
[  5]   5.00-6.00   sec  9.99 MBytes  83.8 Mbits/sec  913   31.5 KBytes
[  5]   6.00-7.00   sec  9.96 MBytes  83.5 Mbits/sec  956   20.1 KBytes
[  5]   7.00-8.00   sec  10.0 MBytes  83.9 Mbits/sec  913   33.0 KBytes
[  5]   8.00-9.00   sec  9.97 MBytes  83.6 Mbits/sec  945   24.4 KBytes
[  5]   9.00-10.00  sec  9.99 MBytes  83.8 Mbits/sec  916   34.4 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  98.7 MBytes  82.8 Mbits/sec  9337             sender
[  5]   0.00-10.25  sec  98.7 MBytes  80.8 Mbits/sec                  receiver

iperf Done.
gold,pts,/home/jmg,503$iperf3 -R -c 192.168.0.80
Connecting to host 192.168.0.80, port 5201
Reverse mode, remote host 192.168.0.80 is sending
[  5] local 192.168.0.2 port 51024 connected to 192.168.0.80 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  9.69 MBytes  81.3 Mbits/sec
[  5]   1.00-2.00   sec  10.7 MBytes  89.8 Mbits/sec
[  5]   2.00-3.00   sec  10.7 MBytes  89.8 Mbits/sec
[  5]   3.00-4.00   sec  10.7 MBytes  89.8 Mbits/sec
[  5]   4.00-5.00   sec  10.7 MBytes  89.8 Mbits/sec
[  5]   5.00-6.00   sec  10.7 MBytes  89.8 Mbits/sec
[  5]   6.00-7.00   sec  10.7 MBytes  89.8 Mbits/sec
[  5]   7.00-8.00   sec  10.4 MBytes  87.6 Mbits/sec
[  5]   8.00-9.00   sec  10.7 MBytes  89.9 Mbits/sec
[  5]   9.00-10.00  sec  10.7 MBytes  89.8 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   106 MBytes  88.9 Mbits/sec  1381             sender
[  5]   0.00-10.00  sec   106 MBytes  88.7 Mbits/sec                  receiver

iperf Done.

As you can see, it matches what I measured earlier.

And just to prove that the machine CAN move 100MB/sec, I've run iperf3
using the onboard wired ethernet...  I need multiple interfaces, which is
why I'm bothering trying to get USB3 ethernet working.

This is using the onboard bge interface.  It's IP is .79:
gold,pts,/home/jmg,507$iperf3 -c 192.168.0.79
Connecting to host 192.168.0.79, port 5201
[  5] local 192.168.0.2 port 61500 connected to 192.168.0.79 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   101 MBytes   850 Mbits/sec    0    488 KBytes
[  5]   1.00-2.00   sec   112 MBytes   940 Mbits/sec    0    488 KBytes
[  5]   2.00-3.00   sec   112 MBytes   943 Mbits/sec    0    731 KBytes
[  5]   3.00-4.00   sec   112 MBytes   940 Mbits/sec    0    731 KBytes
[  5]   4.00-5.00   sec   112 MBytes   941 Mbits/sec    0    731 KBytes
[  5]   5.00-6.00   sec   112 MBytes   941 Mbits/sec    0    731 KBytes
[  5]   6.00-7.00   sec   112 MBytes   940 Mbits/sec    0    731 KBytes
[  5]   7.00-8.00   sec   112 MBytes   941 Mbits/sec    0    731 KBytes
[  5]   8.00-9.00   sec   112 MBytes   940 Mbits/sec    0    731 KBytes
[  5]   9.00-10.00  sec   112 MBytes   940 Mbits/sec    0    731 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   932 Mbits/sec    0             sender
[  5]   0.00-10.01  sec  1.08 GBytes   931 Mbits/sec                  receiver

iperf Done.
gold,pts,/home/jmg,508$iperf3 -R -c 192.168.0.79
Connecting to host 192.168.0.79, port 5201
Reverse mode, remote host 192.168.0.79 is sending
[  5] local 192.168.0.2 port 61726 connected to 192.168.0.79 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  97.0 MBytes   814 Mbits/sec
[  5]   1.00-2.00   sec   109 MBytes   917 Mbits/sec
[  5]   2.00-3.00   sec   110 MBytes   920 Mbits/sec
[  5]   3.00-4.00   sec   109 MBytes   917 Mbits/sec
[  5]   4.00-5.00   sec   110 MBytes   920 Mbits/sec
[  5]   5.00-6.00   sec   109 MBytes   916 Mbits/sec
[  5]   6.00-7.00   sec   110 MBytes   919 Mbits/sec
[  5]   7.00-8.00   sec   109 MBytes   917 Mbits/sec
[  5]   8.00-9.00   sec   110 MBytes   919 Mbits/sec
[  5]   9.00-10.00  sec   109 MBytes   916 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.03  sec  1.06 GBytes   905 Mbits/sec  2866             sender
[  5]   0.00-10.00  sec  1.06 GBytes   907 Mbits/sec                  receiver

iperf Done.


> Comparing the output of using iperf3 under
> NetBSD 9.0 or Windows 10 could be instructive.

And NetBSD 9.0.  Here, NetBSD got assigned .50, but it's also using
the USB3 adapter.

gold,pts,/home/jmg,505$iperf3 -c 192.168.0.50
Connecting to host 192.168.0.50, port 5201
[  5] local 192.168.0.2 port 55535 connected to 192.168.0.50 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  96.7 MBytes   811 Mbits/sec    0   50.9 KBytes
[  5]   1.00-2.00   sec   112 MBytes   938 Mbits/sec    0   82.1 KBytes
[  5]   2.00-3.00   sec   112 MBytes   938 Mbits/sec   14    114 KBytes
[  5]   3.00-4.00   sec   110 MBytes   924 Mbits/sec   60    146 KBytes
[  5]   4.00-5.00   sec   112 MBytes   938 Mbits/sec  107    178 KBytes
[  5]   5.00-6.00   sec   112 MBytes   939 Mbits/sec  122    193 KBytes
[  5]   6.00-7.00   sec   112 MBytes   938 Mbits/sec  144    193 KBytes
[  5]   7.00-8.00   sec   112 MBytes   938 Mbits/sec  165    193 KBytes
[  5]   8.00-9.00   sec   112 MBytes   939 Mbits/sec  144    193 KBytes
[  5]   9.00-10.00  sec   112 MBytes   938 Mbits/sec  165    193 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   924 Mbits/sec  921             sender
[  5]   0.00-10.00  sec  1.08 GBytes   924 Mbits/sec                  receiver

iperf Done.
gold,pts,/home/jmg,506$iperf3 -R -c 192.168.0.50
Connecting to host 192.168.0.50, port 5201
Reverse mode, remote host 192.168.0.50 is sending
[  5] local 192.168.0.2 port 55691 connected to 192.168.0.50 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  78.3 MBytes   657 Mbits/sec
[  5]   1.00-2.00   sec  88.4 MBytes   742 Mbits/sec
[  5]   2.00-3.00   sec  88.5 MBytes   742 Mbits/sec
[  5]   3.00-4.00   sec  88.4 MBytes   741 Mbits/sec
[  5]   4.00-5.00   sec  88.7 MBytes   744 Mbits/sec
[  5]   5.00-6.00   sec  88.2 MBytes   740 Mbits/sec
[  5]   6.00-7.00   sec  87.8 MBytes   737 Mbits/sec
[  5]   7.00-8.00   sec  88.5 MBytes   742 Mbits/sec
[  5]   8.00-9.00   sec  89.0 MBytes   746 Mbits/sec
[  5]   9.00-10.00  sec  88.8 MBytes   745 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   875 MBytes   734 Mbits/sec    0             sender
[  5]   0.00-10.00  sec   874 MBytes   734 Mbits/sec                  receiver

iperf Done.

As you can see, both match approximately what I measured other methods,
so, it's definitely not the way I measured performance.

> My observation would be that neither type
> of USB3 Ethernet adapter that I've tried

What is the chipset that you tried?  One of the earlier ones that I
tried was an axe iirc, and was  limited to around 500Mbps or so...

> (different chipsets) get anywhere near
> 100 MByte/s when ifconfig reports
> 1000baseT <full-duplex>. The Cwnd figures
> are smaller than for the built-in Ethernets
> that manage much faster overall transfer
> rates.

[...]

> I'll note that between machines with built-in EtherNet
> that can sustain fast transfers overall, the Cwnd figures
> tend to vary but can reach 1 MBytes+. The Retr counts
> tend to still exist.
> 
> By contrast, when the USB3 EtherNet is receiving above,
> the maximum Cwnd reported above for the sender at the
> time was: 29.8 KBytes.
> 
> I have not tried NetBSD, Windows 10, or Linux comparisons.

As you can see above, NetBSD easily achieves around 8-10x the
speed using the exact same USB3 device as FreeBSD does, so the
hardware CAN push the speeds, just FreeBSD cannot.

Hence, my original post, what can I do to possibly get FreeBSD's
performance up to what the hardware can achieve?

-- 
  John-Mark Gurney				Voice: +1 415 225 5579

     "All that I will do, has been done, All that I have, has not."
Received on Mon Jul 13 2020 - 02:51:54 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:24 UTC