Re: dev/em: Link is not up until 2 seconds after "ifconfig up"

Jun Kuriyama <kuriyama_at_imgsrc.co.jp> writes:

> My experimental 1U box which has em0 and em1 shows following result.
> Doing "ifconfig em1 up" after "ifconfig em1 down", link is down 1 or
> more seconds.

   I've had this problem too.  You don't really notice it until you're
trying to do virtual hosting and bring up in excess of 100 addresses on a
box.  At 3-4 seconds each, it can make reboots an interminable wait.
   I've hacked around the wait for adding aliases, but it's still there for
the original setup, where a few seconds doesn't bother me.  What's happening
is every time you add an address, the driver's own ioctl handler calls
ether_ioctl, which in turn calls the driver's if_init.  I didn't fix the
wait in em's if_init, but I did avoid calling it.
   The hack doesn't break anything in 4.7, but I don't know if that's just
luck and something is waiting in the wings.  I also don't know what it would
do to Current.  Anyway, in ether_ioctl, I only call if_init if the interface
isn't already set up (as tested by the IFF_RUNNING flag, since ether_ioctl
has already set IFF_UP):

  		switch (ifa->ifa_addr->sa_family) {
  #ifdef INET
  		case AF_INET:
+ 		    /* Only initialize the interface when it first comes up.
+ 		     * This works around a seconds-long freeze on the em(4)
+ 		     * interface when adding virtual addresses.  It might also
+ 		     * do something bad, but it doesn't appear to.
+ 		     */
+ 		    if (!(ifp->if_flags & IFF_RUNNING))
  			ifp->if_init(ifp->if_softc);	/* before arpwhohas */
  			arp_ifinit(ifp, ifa);
  			break;
  #endif

Yes, this is only for AF_INET addresses, since that's all I care about.  And
yes, this still leaves the question as to why em hangs in the first place.
I kind of have an answer to that, but not a fix.  Em has a watchdog timer
that runs every two seconds, and sets some current state from the hardware.
Apparently, there's no way for the hardware to immediately know what state
it's in: I tried shortening the initial invocation of this timer, and if I
get much below a second it doesn't work until the next interval.  My
"doesn't work", I mean I get the same wait as before.

- Jamie