Re: kmem_map auto-sizing and size dependencies

On Fri, Jan 18, 2013 at 7:29 AM, Andre Oppermann <andre_at_freebsd.org> wrote:
> The (inital?) size of the kmem_map is determined by some voodoo magic,
> a sprinkle of nmbclusters * PAGE_SIZE incrementor and lots of tunables.
> However it seems to work out to an effective kmem_map_size of about 58MB
> on my 16GB AMD64 dev machine:
>
> vm.kvm_size: 549755809792
> vm.kvm_free: 530233421824
> vm.kmem_size: 16,594,300,928
> vm.kmem_size_min: 0
> vm.kmem_size_max: 329,853,485,875
> vm.kmem_size_scale: 1
> vm.kmem_map_size: 59,518,976
> vm.kmem_map_free: 16,534,777,856
>
> The kmem_map serves kernel malloc (via UMA), contigmalloc and everthing
> else that uses UMA for memory allocation.
>
> Mbuf memory too is managed by UMA which obtains the backing kernel memory
> from the kmem_map.  The limits of the various mbuf memory types have
> been considerably raised recently and may make use of 50-75% of all
> physically
> present memory, or available KVM space, whichever is smaller.
>
> Now my questions/comments are:
>
>  Does the kmem_map automatically extend itself if more memory is requested?

Not that I recall.

>  Should it be set to a larger initial value based on min(physical,KVM) space
>  available?

It needs to be smaller than the physical space, because the only limit
on the kernel's use of (pinned) memory is the size of the map.  So if
it is too large there is nothing to stop the kernel from consuming all
available memory.  The lowmem handler is called when running out of
virtual space only (i.e. a failure to allocate a range in the map).

>  The naming and output of the various vm.kmem_* and vm.kvm_* sysctls is
>  confusing and not easy to reconcile.  Either we need some more detailing
>  more aspects or less.  Plus perhaps sysctl subtrees to better describe the
>  hierarchy of the maps.
>
>  Why are separate kmem submaps being used?  Is it to limit memory usage of
>  certain subsystems?  Are those limits actually enforced?

I mostly know about memguard, since I added memguard_fudge().  IIRC
some of the submaps are used.  The memguard_map specifically is used
to know whether an allocation is guarded or not, so at free(9) it can
be handled as normal malloc() or as memguard.

Cheers,
matthew