Re: vm_page_max_wired and gpg...

(This bounced first time around due to DNS problems)

Hi,
>For some time now, gpg has been having issues on my -CURRENT system 
>with being unable to mlock() the ~8k buffer it uses to hold the 
>decrypted secret key. A ktrace shows that it's being returned an 
>EAGAIN from mlock(), which my peeking around has shown could be:
>
>1) I hit the RLIMIT_MLOCK limit on memory locking, or
>2) I hit the system-wide "wired pages" limit.

I ran into this when trying to get "memcheck" working, so I suppose
I should share my results.

The kernel has a "max wired pages" limit, that's set when the swapper
starts up to be one third of physical memory. You can see this in
src/sys/vm_pageout.c, on about line 1414:

> if (vm_page_max_wired == 0)
>   vm_page_max_wired = cnt.v_free_count / 3;

This is pretty much a third of what you see at boot time (and in
/var/log/messages or dmesg) for "avail memory = "

Now, unfortunately, pretty much the only thing that tries to obey
this limit is "mlock". So, just creating dirty buffers can be enough
to stop any user process locking any memory at all.

On my 128M laptop, There's about 14M wired at boot time. However,
creating dirty filesystem buffers will rapidly raise this. For
example, "dd if=/dev/random of=/tmp/dirtyMeSomeBuffers" will rapidly
bring me up to about 34 megs.

There's obviously some more checks and balances to ensure the number
of buffers doesn't consume all available memory, but there's other
places in the kernel that pages get wired down, and it's pretty easy
to hit that one-third limit.

I really don't know the VM well enough to say how to fix it, but
it seems that mlock() is at the mercy of the rest of the kernel,
and the rest of the kernel doesn't really care. Maybe either
a better heuristic to decide wheather to allow mlock() to complete,
or even allow mlock() to launder itself a few new pages when things
are tight might be an idea. (Or, maybe the I/O system shouldn't
be robbing a share of "wired pages", and have its own category for
that data.)

For your purpose, making vm_page_max_wired a sysctl would probably
fix the problem in the short term.

Anyone got any ideas for the long term? :-)
-- 
Peter Edwards