Re: nfe/PXE problem

On Tue, Mar 13, 2007 at 09:10:38AM +0200, Danny Braniss wrote:
 > 
 > > well, I can PXE boot this box if I use an fxp NIC, with the nfe I tracked the 
 > > problem
 > > to sys/nfsclient/nfs_diskless, where in nfs_setup_diskless(void)
 > > it does not find the nfe interface, ie, something does not match, but the 
 > > interface
 > > was detected!.
 > > i'll try and do some more debugging.
 > > 
 > ok, I found the problem, in nfs_diskless.c nfs_setup_diskless(),
 > there is a loop to search for the interface that was used to boot from,
 > and no match is found because the hadrware ethernet address
 > in the nfe is in the wrong byte order! and so the bcmp(...) fails.

Good catch!

 > Interestingly, if booting NOT via PXE the Ethernet address is OK!
 > 
 > nfe0: <NVIDIA nForce 430 MCP13 Networking Adapter> port 0xdc00-0xdc07 mem 
 > 0xfe02c000-0xfe02cfff irq 22 at device 20.0 on pci0
 > 	nfe0: Ethernet address: 00:18:f3:a9:6c:57
 > and via PXE:
 > 	nfe0: Ethernet address: 57:6c:a9:f3:18:00
 > 
 > can someone with the right knowledge fix this?
 > 

AFAIK there is no known way to get an ethernet address via EEPROM on
NVIDIA NIC so nfe(4) reads specific address registers to get ethernet
address as a workaround. The address registers are filled
automatically by hardware after hardware reset. This type of
acquisition of ethernet address is *NOT* correct way as it could get
a fake etherent address since adiministor can program other ethernet
address into the NIC.(e.g. ifconfig(8) can change ethernet address.)
Therefore nfe(4) is very careful to save original ethernet address
in device attach phase and restores the ethernet address at device
detach time.
To make matters worse, it seems that ethernet address in NVIDIA NIC
is loaded backwards into registers so nfe(4) corrects it in
nfe_init_locked(). However, it seems PXE code does not do necessary
swapping for ethernet address and does not restore original MAC
address in the end of PXE phase so the NIC will have bogusly
programmed MAC address. When kernel boots and nfe(4) is attached it
will load the bogus ethernet address which was already programmed by
PXE. If /etc/start_if.nfe0 is honored in diskless boot you can override
the ethernet address to use the same ethernet address as PXE did.

Even if there is a way to get an ethernet address from EEPROM, PXE
should be fixed first since it uses bogus ethernet address in booting
stage. If I encounter the same situation I would contact vendor for
updated PXE image for the NVIDIA NIC. If that is not available I think
the only remaining option would be adding a new tunable to swap the
ethernet address.

I'm not familiar with PXE and I din't use PXE for a long time so I
could be completely wrong.

 > thanks,
 > 	danny
 > 
 > 
-- 
Regards,
Pyun YongHyeon