Re: kmem_map auto-sizing and size dependencies

I'll follow up with detailed answers to your questions over the weekend.
For now, I will, however, point out that you've misinterpreted the
tunables.  In fact, they say that your kmem map can hold up to 16GB and the
current used space is about 58MB.  Like other things, the kmem map is
auto-sized based on the available physical memory and capped so as not to
consume too much of the overall kernel address space.

Regards,
Alan

On Fri, Jan 18, 2013 at 9:29 AM, Andre Oppermann <andre_at_freebsd.org> wrote:

> The autotuning work is reaching into many places of the kernel and
> while trying to tie up all lose ends I've got stuck in the kmem_map
> and how it works or what its limitations are.
>
> During startup the VM is initialized and an initial kernel virtual
> memory map is setup in kmem_init() covering the entire KVM address
> range.  Only the kernel itself is actually allocated within that
> map.  A bit later on a number of other submaps are allocated (clean_map,
> buffer_map, pager_map, exec_map).  Also in kmeminit() (in kern_malloc.c,
> different from kmem_init) the kmem_map is allocated.
>
> The (inital?) size of the kmem_map is determined by some voodoo magic,
> a sprinkle of nmbclusters * PAGE_SIZE incrementor and lots of tunables.
> However it seems to work out to an effective kmem_map_size of about 58MB
> on my 16GB AMD64 dev machine:
>
> vm.kvm_size: 549755809792
> vm.kvm_free: 530233421824
> vm.kmem_size: 16,594,300,928
> vm.kmem_size_min: 0
> vm.kmem_size_max: 329,853,485,875
> vm.kmem_size_scale: 1
> vm.kmem_map_size: 59,518,976
> vm.kmem_map_free: 16,534,777,856
>
> The kmem_map serves kernel malloc (via UMA), contigmalloc and everthing
> else that uses UMA for memory allocation.
>
> Mbuf memory too is managed by UMA which obtains the backing kernel memory
> from the kmem_map.  The limits of the various mbuf memory types have
> been considerably raised recently and may make use of 50-75% of all
> physically
> present memory, or available KVM space, whichever is smaller.
>
> Now my questions/comments are:
>
>  Does the kmem_map automatically extend itself if more memory is requested?
>
>  Should it be set to a larger initial value based on min(physical,KVM)
> space
>  available?
>
>  The use of nmbclusters for the initial kmem_map size calculation isn't
>  appropriate anymore due to it being set up later and nmbclusters isn't the
>  only mbuf relevant mbuf type.  We make significant use of page sized mbuf
>  clusters too.
>
>  The naming and output of the various vm.kmem_* and vm.kvm_* sysctls is
>  confusing and not easy to reconcile.  Either we need some more detailing
>  more aspects or less.  Plus perhaps sysctl subtrees to better describe the
>  hierarchy of the maps.
>
>  Why are separate kmem submaps being used?  Is it to limit memory usage of
>  certain subsystems?  Are those limits actually enforced?
>
> --
> Andre
> ______________________________**_________________
> freebsd-current_at_freebsd.org mailing list
> http://lists.freebsd.org/**mailman/listinfo/freebsd-**current<http://lists.freebsd.org/mailman/listinfo/freebsd-current>
> To unsubscribe, send any mail to "freebsd-current-unsubscribe_at_**
> freebsd.org <freebsd-current-unsubscribe_at_freebsd.org>"
>