Re: mbuf LOR

On Wed, Apr 02, 2003 at 12:39:30PM -0800, Nate Lawson wrote:
> I was testing some changes to make fxp MPSAFE and got a LOR in allocating
> the mbuf cluster and then finally a panic when trying to dereference the
> cluster header.  Is the mbuf system MPSAFE?  Is it ok to call m_getcl
> with a device lock held (but not Giant)?

  Not necessarily if you're doing a TRYWAIT mbuf allocation.  Notably,
  the TRYWAIT allocation may end up in a call to the reclaim routine
  which calls the protocol drain routines which may end up in an LOR if
  you're holding some lock that can also be touched from the drain
  routines.  So, in other words, if you do do that you'd have to be very
  careful.  But there is another problem.  kmem_malloc() still requires
  Giant for blockable allocations (TRYWAIT in this case) and so Giant
  must be held going in there if you're doing a TRYWAIT allocation.

  For what concerns DONTWAIT, you should theoretically be allowed to
  hold a driver lock.  But again, there may be a problem.  Specifically,
  I see that UMA code has some explicit Giant acquires/frees in certain
  places.  When the UMA code gets called from the malloc() code while
  the bucket is being internally allocated in mb_alloc, you may be
  hitting one of those paths.  In fact, unless you have a specific Giant
  acquire in the fxp_* routines you list in your trace below, that is
  undoubtedly what is happening because there are no explicit Giant
  acquires in the code path from m_getcl() to malloc(), so they must be
  happening higher up in the call stack.

> The lock reversal was: 1. fxp softc lock, 2. Giant.
> 
> Traceback:
> zalloc...
> malloc()
> mb_pop_cont()
> mb_alloc()
> m_getcl()
> fxp_add_rfabuf()
> fxp_intr_body()
> fxp_intr() -- locks fxp softc
> 
> -Nate

-- 
Bosko Milekic
bmilekic_at_unixdaemons.com
bmilekic_at_FreeBSD.org