Re: mbuf LOR

On Wed, 2 Apr 2003, Bosko Milekic wrote:

BM>  For what concerns DONTWAIT, you should theoretically be allowed to
BM>  hold a driver lock.  But again, there may be a problem.  Specifically,
BM>  I see that UMA code has some explicit Giant acquires/frees in certain
BM>  places.  When the UMA code gets called from the malloc() code while
BM>  the bucket is being internally allocated in mb_alloc, you may be
BM>  hitting one of those paths.  In fact, unless you have a specific Giant
BM>  acquire in the fxp_* routines you list in your trace below, that is
BM>  undoubtedly what is happening because there are no explicit Giant
BM>  acquires in the code path from m_getcl() to malloc(), so they must be
BM>  happening higher up in the call stack.

Well, that's interesting.

A month ago or so I sent a patch to smp_at_ with an update to the malloc page
documenting the locking requirements. I got only one response. I wonder,
how people are expected to correctly use an API, when that API is poorly
documented (malloc(9) in this case, but mbuf(9)  does not mention 'lock'
either). I would ask people knowing the topic to comment on this. As soon
as this EC proposal nightmare is over, I may try to write a corresponding
section for mbuf(9).

Regards,
harti

Index: malloc.9
===================================================================
RCS file: /home/ncvs/src/share/man/man9/malloc.9,v
retrieving revision 1.30
diff -u -r1.30 malloc.9
--- malloc.9	24 Feb 2003 05:53:27 -0000	1.30
+++ malloc.9	17 Mar 2003 15:06:14 -0000
_at__at_ -147,44 +147,22 _at__at_
 to return
 .Dv NULL
 if the request cannot be immediately fulfilled due to resource shortage.
-Otherwise, the current process may be put to sleep to wait for
-resources to be released by other processes.
-If this flag is set,
-.Fn malloc
-will return
-.Dv NULL
-rather than block.
 Note that
 .Dv M_NOWAIT
-is defined to be 0, meaning that blocking operation is the default.
-Also note that
-.Dv M_NOWAIT
 is required when running in an interrupt context.
-.Pp
-Programmers should be careful not to confuse
-.Dv M_NOWAIT ,
-the
-.Fn malloc
-flag, with
-.Dv M_DONTWAIT ,
-an
-.Xr mbuf 9
-allocation flag, which is not a valid argument to
-.Fn malloc .
 .It Dv M_WAITOK
-Indicates that it is Ok to wait for resources.  It is unconveniently
-defined as 0 so care should be taken never to compare against this value
-directly or try to AND it as a flag.  The default operation is to block
-until the memory allocation succeeds.
+Indicates that it is ok to wait for resources.
+If the request cannot be immediately fulfilled the current process is put
+to sleep to wait for resources to be released by other processes.
 The
 .Fn malloc ,
 .Fn realloc ,
 and
 .Fn reallocf
-functions can only return
+functions cannot return
 .Dv NULL
 if
-.Dv M_NOWAIT
+.Dv M_WAITOK
 is specified.
 .It Dv M_USE_RESERVE
 Indicates that the system can dig into its reserve in order to obtain the
_at__at_ -194,6 +172,12 _at__at_
 programming.
 .El
 .Pp
+Exactly one of either
+.Dv M_WAITOK
+or
+.Dv M_NOWAIT
+must be specified.
+.Pp
 The
 .Fa type
 argument is used to perform statistics on memory usage, and for
_at__at_ -244,11 +228,37 _at__at_
 While it should not be relied upon, this information may be useful for
 optimizing the efficiency of memory use.
 .Pp
-Malloc flags documented above should
-.Em NOT
-be used with
+Programmers should be careful not to confuse the malloc flags
+.Dv M_NOWAIT
+and
+.Dv M_WAITOK
+with the
 .Xr mbuf 9
-routines as it will cause undesired results.
+flags
+.Dv M_DONTWAIT
+and
+.Dv M_TRYWAIT .
+.Sh LOCKING CONSIDERATIONS
+.Fn malloc ,
+.Fn realloc
+and
+.Fn reallocf
+may not be called from fast interrupts handlers.
+When called from threaded interrupts
+.Ar flag
+must contain
+.Dv M_NOWAIT .
+.Pp
+.Fn malloc ,
+.Fn realloc
+and
+.Fn reallocf
+must not be called with
+.Dv M_WAITOK
+while a mutex other than Giant is held.
+Giant may or may not be held when
+.Fn free
+is called.
 .Pp
 Any calls to
 .Fn malloc
--
harti brandt,
http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private
brandt_at_fokus.fraunhofer.de, harti_at_freebsd.org