Re: SMP problem with uma_zalloc

On Mon, Jul 21, 2003 at 03:47:54PM +0200, Harti Brandt wrote:
> On Mon, 21 Jul 2003, Bosko Milekic wrote:
[...]
> BM>  It sounds to me like your example is really not the general-case one.
> BM>  Basically, you're using a zone capped off at 1 page.  Currently in
> BM>  UMA, this is the size of the slab.  So, basically, you have this whole
> BM>  zone (with all associated overhead) so as to serve a maximum of only
> BM>  one slab.  This defeats most of the assumptions made when the zone is
> BM>  created with PCPU caches.  The zone maximum exists to prevent more
> BM>  than the specified amount of resources to be allocated toward the
> BM>  given zone; I don't think that the intention was "to ensure that if
> BM>  the maximum items aren't allocated, there will always be one
> BM>  available," despite the fact that that is the effective behavior on
> BM>  UP.
> BM>
> BM>  The solution to your really small zone problem is to either make the
> BM>  zone bigger, or to hack at UMA to export the UMA_ZONE_INTERNAL API
> BM>  properly so that you can skip the pcpu caches for all allocations and
> BM>  go straight to the zone.  I'd suggest that you make the zone bigger,
> BM>  unless there's a Really Good reason not to.
> 
> I think I take two paths: for stuffs like VCC where there may be a large
> number I will just remove the limit. The limits were a leftover when the
> ATM code had its own memory pool code. For stuff where there is a high
> probability that only a handful (usually 1 or 2) of them will be allocated
> (network interfaces) I will try to make it to use malloc().

  Ahhhh.  Given the explanation, the small size of the limits makes a
  lot more sense now.  Previously, the limit probably enforced the
  actual number of cached (pre-allocated) items in the pool.  So, it was
  more than just a "limit," it was a cache size parameter.  That is
  probably why its size was kept relatively small.  In the zone setting,
  the limit can easily be made larger or removed altogether (if there is
  no danger of that structure consuming all of kernel memory).

> How do you think about adding a paragraph for uma_zone_set_max to the man
> page?:
> 
> An upper limit of items in the zone can be specified with a call to
> uma_zone_set_max. This limits the total number of items which includes:
> allocated items, free items and free items in the per-cpu caches. On
> systems with more than one CPU it may not be possible to allocate the
> specified number of items, because all of the remaining free items may
> be in the caches of the other CPUs when the limit is hit.

  Given that it has obviously led to confusion, this sort of change to
  the man page would be encouraging.

  Perhaps you would also ammend to it the purpose of uma_zone_set_max(),
  as it currently stands:

  "The purpose of uma_zone_set_max() is to limit the maximum
  amount of memory that the system can dedicate toward the zone
  specified by the 'zone' argument." <insert your text from above here>

  Would you like to commit the change?

> Regards,
> harti
>
> -- 
> harti brandt,
> http://www.fokus.fraunhofer.de/research/cc/cats/employees/hartmut.brandt/private
> brandt_at_fokus.fraunhofer.de, harti_at_freebsd.org

Cheers,
-- 
Bosko Milekic  *  bmilekic_at_technokratis.com  *  bmilekic_at_FreeBSD.org
TECHNOkRATIS Consulting Services  *  http://www.technokratis.com/