Re: Problems with realtek NIC

Am 02.05.21 um 01:37 schrieb Greg Rivers:
> On Saturday, 1 May 2021 16:45:03 CDT Stefan Esser wrote:
>> Am 01.05.21 um 21:48 schrieb Greg Rivers via freebsd-current:
>>> On Saturday, 1 May 2021 14:09:46 CDT Nilton Jose Rizzo wrote:
>>>> I using a FreeBSD 14-Current and get random error with my NIC. The watchdog timer send a timeout message and I loose connection temporaly. In 
logs show only this message:
>>>>
>>> Switch to the official Realtek driver in ports: net/realtek-re-kmod
>>
>> The "official" RealTek driver is based on a very old version of "our"
>> driver that was written by Bill Paul.
>>
>> It lacks many features that have been introduced in FreeBSD in the
>> last decade (or even earlier) like NETMAP-Support.
>>
>> The RealTek-driver has special cases for some 50 variants of RealTek
>> Ethernet chips and contains individual firmware patches for nearly all
>> of them.
>>
>> I had started to merge chip specific changes from the official driver
>> to the FreeBSD driver in the hope to get it to support the RTL8125A/B
>> chips. But I have stopped that project for lack of RTL8125 documentation,
>> especially regarding the PHY, which has its own driver module in our
>> version but not the RealTek code. (And somebody claimed to know that
>> another FreeBSD developer was working on RTL8125 support but did not
>> tell who that might be and whether he had documentation.)
>>
>> Anyway, there are changes regarding the initialization and error recovery
>> of different RealTek chips in the official driver that could be merged
>> into our version. But I do not know whether these changes require the
>> firmware changes provided by the RealTek driver to correctly work.
>>
> Thanks for the information Stefan, and for your work on FreeBSD. My use
> of the term "official" was apparently inaccurate. I was not aware of the
> deficiencies in the RealTek driver. I would prefer to use the FreeBSD
> driver, but I don't for purely pragmatic reasons: the FreeBSD driver
> continually locks up and resets under load (as described by the OP),
> while the RealTek driver does not.

Hi Greg,

the RealTek driver is "official" in the sense that it is provided by the
vendor and written with knowledge about all the (many!) deficiencies of
the RealTek Ethernet chips. And yes, the FreeBSD drived definitely needs
work to fully support all variants of the RealTek chip. I guess that due
to uncovered chip specifics or hardware issues, the FreeBSD driver will
often lack the special code (or firmware patches) and will have to recover
chip operations be going through a hard reset.

If you look at the "official" driver sources, you'll find #ifdefs for
FreeBSD versions  before 4.9, but that is not the reason the main driver
source file is more than 30000 lines long.

The driver distinguishes between more than 70 different chip versions
(identified by MACFG_3 to MACFG_84 with some IDs missing). And each one
has specific requirements regarding firmware patches, initialization and
reset behavior, error handling, ...

I have analyzed these differences (see the attached file) but for lack
of RTL8125 documentation not preceded with this project at this time.
(The column lx_fw identifies firmware patches used by the Linux driver,
while rt_fw identifies those embedded into the RealTek driver for
FreeBSD - and those differ somewhat, and I have no idea why ...).

I have local modifications of the re driver in my sources, but have
one other project that I really want to get ready in the next few
months (after working on it for nearly 2 years) and I do not want to
become responsible for issues of the RealTek driver in base (after
committing fixes that also might cause regressions, if they need to
be accompanied by firmware patches ...)

> FWIW, here are the particulars on the RealTek chip-set that I've got:
> 
> re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xe000-0xe0ff mem 0xb0804000-0xb0804fff,0xb0800000-0xb0803fff irq 16 at device 0.0 on pci1
> re0: Using 1 MSI-X message
> re0: Chip rev. 0x2c800000
> re0: MAC rev. 0x00100000

The Chip rev. indicates that you have got a RTL8168E_VL, which does not
need a firmware patch according to RealTek driver, but gets one in Linux.

It is identified by MACFG_38 in the RealTek driver, BTW, and there are
only a few chip specific code fragments relevant to that chip. It seems,
it needs special handling when the MAC address is programmed, but I did
not spot any other special code for that particular chip in the "official"
driver.

> re0_at_pci0:1:0:0: class=0x020000 rev=0x06 hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x1458 subdevice=0xe000
>     vendor     = 'Realtek Semiconductor Co., Ltd.'
>     device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
>     class      = network
>     subclass   = ethernet
> 
> Would the FreeBSD Foundation be able to help with getting documentation 
from RealTek?

I have no idea - there is a RTL8125 driver in OpenBSD (which is separate
from the RTL8111/8168 drivers, there) and that driver is actually relatively
short, but it consists just of sequences of register reads and writes without
any comments that could help understand the purpose of these operations.
And there are not so many differences between the RTL8125 and prior chips
(except that the RTL8125 expects 32 bit wide register writes, while the
RTL8111 was limited to 16 bit reads/writes - but this is trivially solved
by having 2 sets of low-level routines without any impact on the higher
level code).

There is also driver code in Linux that can be used to reverse engineer
device specific setup and operation procedures. But that driver is ugly
(IMHO at least) and I'm afraid that there might be legal issues if I use
initialization sequences, firmware patches or other information that can
be found in that driver. Therefore I have only taken a cursory look and
identified the internal chip IDs in the Linux driver and the firmware
patches used for each supported chip.

If there was detailed documentation (detailed register set specification
and operation details like initialization sequence, delays/timeouts that
are required, etc.) this would greatly help getting the RTL8125 into our
version of the "re" driver (with NETMAP and other features that we have
that are not present in the driver from RealTek).

I might try to get support from the author of the OpenBSD driver, but he
might have received documentation under an NDA that does not allow to share
it with us.

Anyway, since the RealTek driver packages works quite well, I can use my
RTL8125 chip (on an AMD B550 mainboard) and it was not that urgent (for me)
to get that chip supported in base.

But if I get documentation, I might be able to integrate the RTL8125 (and
at the same time apply all the special handling and firmware patching the
RealTek driver does - it is still BSD licensed, but in the 4-clause form)
during summer.

Best regards, STefan