Re: [CFT] VMware vmxnet3 ethernet driver

 Bezüglich Bryan Venteicher's Nachricht vom 05.08.2013 02:12 (localtime):
> Hi,
>
> I've ported the OpenBSD vmxnet3 ethernet driver to FreeBSD. I did a
> lot of cleanup, bug fixes, new features, etc (+2000 new lines) along
> the way so there is not much of a resemblance left.
>
> The driver is in good enough shape I'd like additional testers. A patch
> against -CURRENT is at [1]. Alternatively, the driver and a Makefile is
> at [2]; this should compile at least as far back as 9.1. I can look at
> 8-STABLE if there is interest.
>
> Obviously, besides reports of 'it works', I'm interested performance vs
> the emulated e1000, and (for those using it) the VMware tools vmxnet3
> driver. Hopefully it is no worse :)

Hello Bryan,

thanks a lot for your hard work!

It seems if_vmx doesn't support jumbo frames. If I set mtu 9000, I get
»vmx0: cannot populate Rx queue 0«, I have no problems using jumbo
frames with vmxnet3.

I took a oldish host (4x2,8GHz Core2[LGA775]) with recent software: ESXi
5.1U1 and FreeBSD-9.2-RC2
Two guests are connected to one MTU9000 "VMware Software Switch".

Simple iperf (standard TCP) results:

vmxnet3jumbo <-> vmxnet3jumbo
    5.3Gbits/sec, load: 40-60%Sys 0.5-2%Intr

vmxnet3 <-> vmxnet3
    1.85 GBits/sec, load: 60-80%Sys 0-0.8%Intr


if_vmx <-> if_vmx
    1.51 GBits/sec, load: 10-45%Sys 40-48%Intr
                                                                !!!
if_vmxjumbo <-> if_vmxjumbo not possible


if_em(e1000) <-> if_em(e1000)
    1.23 GBits/sec, load: 80-60%Sys 0.5-8%Intr

if_em(e1000)jumbo <-> if_em(e1000)jumbo
    2.27Gbits/sec, load: 40-30%Sys 0.5-5%Intr


if_igb(e1000e)junmbo <-> if_igb(e1000e)jumbo
    5.03 Gbits/s, load: 70-60%Sys 0.5%Intr

if_igb(e1000e) <-> if_igb(e1000e)
    1.39 Gbits/s, load: 60-80%Sys 0.5%Intr


f_igb(e1000e) <-> if_igb(e1000e), both hw.em.[rt]xd=4096
    1.66 Gbits/s, load: 65-90%Sys 0.5%Intr

if_igb(e1000e)junmbo <-> if_igb(e1000e)jumbo, both hw.em.[rt]xd=4096
    4.81 Gbits/s, load: 65%Sys 0.5%Intr

Conclusion:
if_vmx performs well compared to the regular emulated nics and standard
MTU, but it's behind tuned e1000e nic emulation and can't reach vmxnet3
performance with regular mtu. If one needs throughput, the missing jumbo
frame support in if_vmx  is a show stopper.

e1000e is preferable over e1000, even if not officially choosable with
"FreeBSD"-selection as guest (edit .vmx and alter ethernet0.virtualDev =
"e1000e", and dont forget to set hw.em.enable_msix=0 in loader.conf,
although the driver e1000e attaches is if_igb!)

Thanks,

-Harry