Re: nfe/PXE problem

> On Tue, Mar 13, 2007 at 09:10:38AM +0200, Danny Braniss wrote:
>  > 
>  > > well, I can PXE boot this box if I use an fxp NIC, with the nfe I tracked the 
>  > > problem
>  > > to sys/nfsclient/nfs_diskless, where in nfs_setup_diskless(void)
>  > > it does not find the nfe interface, ie, something does not match, but the 
>  > > interface
>  > > was detected!.
>  > > i'll try and do some more debugging.
>  > > 
>  > ok, I found the problem, in nfs_diskless.c nfs_setup_diskless(),
>  > there is a loop to search for the interface that was used to boot from,
>  > and no match is found because the hadrware ethernet address
>  > in the nfe is in the wrong byte order! and so the bcmp(...) fails.
> 
> Good catch!
> 
>  > Interestingly, if booting NOT via PXE the Ethernet address is OK!
>  > 
>  > nfe0: <NVIDIA nForce 430 MCP13 Networking Adapter> port 0xdc00-0xdc07 mem 
>  > 0xfe02c000-0xfe02cfff irq 22 at device 20.0 on pci0
>  > 	nfe0: Ethernet address: 00:18:f3:a9:6c:57
>  > and via PXE:
>  > 	nfe0: Ethernet address: 57:6c:a9:f3:18:00
>  > 
>  > can someone with the right knowledge fix this?
>  > 
> 
> AFAIK there is no known way to get an ethernet address via EEPROM on
> NVIDIA NIC so nfe(4) reads specific address registers to get ethernet
> address as a workaround. The address registers are filled
> automatically by hardware after hardware reset. This type of
> acquisition of ethernet address is *NOT* correct way as it could get
> a fake etherent address since adiministor can program other ethernet
> address into the NIC.(e.g. ifconfig(8) can change ethernet address.)
> Therefore nfe(4) is very careful to save original ethernet address
> in device attach phase and restores the ethernet address at device
> detach time.
> To make matters worse, it seems that ethernet address in NVIDIA NIC
> is loaded backwards into registers so nfe(4) corrects it in
> nfe_init_locked(). However, it seems PXE code does not do necessary
> swapping for ethernet address and does not restore original MAC
> address in the end of PXE phase so the NIC will have bogusly
> programmed MAC address. When kernel boots and nfe(4) is attached it
> will load the bogus ethernet address which was already programmed by
> PXE. If /etc/start_if.nfe0 is honored in diskless boot you can override
> the ethernet address to use the same ethernet address as PXE did.
> 
> Even if there is a way to get an ethernet address from EEPROM, PXE
> should be fixed first since it uses bogus ethernet address in booting
> stage. If I encounter the same situation I would contact vendor for
> updated PXE image for the NVIDIA NIC. If that is not available I think
> the only remaining option would be adding a new tunable to swap the
> ethernet address.
> 
> I'm not familiar with PXE and I din't use PXE for a long time so I
> could be completely wrong.

not completely :-)
the DHCP/PXE boot works with the 'correct ethernet address'.
this is because I use the static/registered mac, and so a bogus mac address
can't get past the DHCP/PXE stage.
so, the PXE is using the correct mac address.
I can't use /etc/... because of the horse-cart problem, it's booting
diskless, and the 'disk' is nfs readable via the NIC, but the NIC is not 
working

the question now, is to see if the nfe driver can check if it was
used by the PXE, and flip the address. or have NVIDIA comeup with
a patch ...

thanks,

	danny