Re: mlock(2), unprivileged users, and RLIMIT_MEMLOCK

On Sat, 2008-04-12 at 15:09 -0400, Coleman Kane wrote:
> Hello,
> 
> Recently we've been having a discussion on the GNOME list about fixing
> the seahorse breakage introduced with the latest GNOME 2.22, rooted in
> the fact that FreeBSD's mlock(2) implementation is only usable if you
> have superuser privileges. Due to bugs in seahorse, the lack of mlock(2)
> causes many seahorse applications to die. I've posted a suggested patch
> to 
> 
> From my understanding, a significant reasoning for this is because if
> unprivileged users could mlock(2), then they could incur a DoS attack on
> a system by spawning off at most RLIMIT_NPROC processes, each
> wiring-down RLIMIT_MEMLOCK bytes of memory in an effort to steal away
> all real system RAM from the rest of the system, and bring usage to a
> screeching halt.
> 
> I've posted up a short page about it on my site here:
> http://www.cokane.org/dokuwiki/freebsd/mlock-support
> 
> I'd like to know if there are any other patches that are floating around
> for the same thing, or even if there are some good alternatives to
> mlock(2) that yield similar results (secure memory accessible by the
> user). I'd also welcome any comments that others have on the topic, as I
> am looking for approaches to implement the support under FreeBSD without
> compromising the security of the OS.

As mezz pointed out, Peter Jeremy commented on this a while ago:

http://lists.freebsd.org/pipermail/freebsd-arch/2006-July/005496.html

> 
> An idea that came to mind, but I am less familiar with, is to also
> support some sort of MAC policy checks that can be enforced by the
> administrator on the system to provide some users with secure access
> support, while preventing others from using it.
> 
> A second idea might be to turn RLIMIT_MEMLOCK into a per-user (or even
> system-wide) resource limit, rather than a per-process limit.
> 
> As a third idea, we could leave the per-process limit (to abide by
> historical documentation), but also add a sysctl that enforces a
> system-wide "max mlock pages" which can be tested by the mlock(2)
> syscall, refusing to mlock(2) more memory if the limit is hit.

I think this already exists in -CURRENT: vm.max_wired ("System-wide
limit to wired page count").  This is tested by mlock(2) in addition to
RLIMIT_MEMLOCK.

I also looked through the kernel for instances where RLIMIT_MEMLOCK is
checked, and the only other place is in the vslock() function.  The only
consumer of this function I could find is sysctl_wire_old_buffer() which
is used by quite a few sysctl handlers.  If the rlimit is changed from
infinity, users might have problems getting results from certain
sysctls.

Joe

-- 
Joe Marcus Clarke
FreeBSD GNOME Team      ::      gnome_at_FreeBSD.org
FreeNode / #freebsd-gnome
http://www.FreeBSD.org/gnome