kernel pointer polka, possibly by mount_nfs

I have a 100% reproducible case here where it looks like mount_nfs
tramples on the softc of a led(4) device.

Stock -current kernel, HZ=1000, I've added a couple of sanity-checks
in the timeout routine of led(4) and they trigger reliably on a
byte which should not have been zero.

In all cases so far, the currently running program is mount_nfs run
from /etc/rc.mumble somewhere.

The machine is a Soekris 4501 booting diskless.

I have also seen a reproducible page fault panic in in_pcbremlist()
if I put "set -x" as the second line in /etc/rc on the same machine,
it smells the same to me.

This problem likely affects 5.2-WHATEVER as well, and could be
responsible for other Heisenbugs, and could be considered a
showstopper.

If anybody is interested in working on this, I'll be happy to test
patches or provide tracebacks etc from DDB (sorry, no GDB spoken).

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk_at_FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.