Re: Trouble with recent auto-tuning changes

On 01/28/2013 08:22, Ian Lepore wrote:
> On Mon, 2013-01-28 at 00:09 -0600, Alan Cox wrote:
>> On Sun, Jan 27, 2013 at 12:11 PM, Ian Lepore <ian_at_freebsd.org> wrote:
>>
>>> I ran into a panic while attempting to un-tar a large file on a
>>> DreamPlug (arm-based system) running -current.  The source and dest of
>>> the un-tar is the root filesystem on sdcard, and I get this:
>>>
>>> panic: kmem_malloc(4096): kmem_map too small: 12582912 total allocated
>>>
>>> Just before the panic I see the tar process get hung in a "nokva" wait.
>>> 12582912 is the value of VM_KMEM_SIZE from arm/include/vmparam.h.
>>>
>>> In r245575 the init order for mbuf limits was changed from
>>> SI_SUB_TUNABLES to SI_SUB_KMEM so that mbuf limits could be based on the
>>> results of sizing kernel memory.  Unfortunately, the process of sizing
>>> kernel memory relies on the mbuf limits; in kmeminit():
>>>
>>>         vm_kmem_size = VM_KMEM_SIZE + nmbclusters * PAGE_SIZE;
>>>
>>> Since r245575, nmbclusters is zero when this line of code runs.  If I
>>> manually plugin "32768" (the number tunable_mbinit() comes up with for
>>> this platform) in that line, the panic stops happening.
>>>
>>> So we've got two problems here... one is the circular dependency in
>>> calculating the mbuf limits.  The other is the fact that some
>>> non-trivial amount of kernel memory we're allowing for mbufs is actually
>>> being used for other things.  That is, if my system was actually using
>>> all the mbufs that tunable_mbinit() allowed for, then this panic while
>>> untarring a huge file would still have happened.
>>>
>>>
>> All of this is factually correct.  However, it's a red herring.  The real
>> problem is that arm, unlike every other architecture in the tree, does not
>> enable auto-sizing of the kmem map based on the physical memory size.
>> Specifically, you'll find VM_KMEM_SIZE_SCALE defined in
>> "arch"/include/vmparam.h on every other architecture, just not on arm.
>> This auto-sizing overrides the value of VM_KMEM_SIZE.
>>
> Aha.  I'll investigate what other architectures do with that and try to
> get the same thing going for arm.
>

i386 or (32-bit) MIPS would be the most similar.  Also, I would
encourage you to look for other definitions that those architectures
have that arm doesn't.  As physical memory sizes continue to grow on
arm-based systems, they may require other changes in vmparam.h and the
machine-dependent param.h that were made on those other architectures
year ago.

If you have any questions about any of the definitions, feel free to
e-mail me.

Alan

P.S. When I get a little more free time, I intend to get in touch with
Andre to address the apparent circular dependency.  For now just know
that unless that circular dependency is combined with a lack of kmem map
auto-sizing, like arm, it's basically harmless.