malloc(3) ignores RLIMIT_DATA

malloc(3) in FreeBSD 7 uses mmap() (then sbrk() on 32-bit platform)
and ignores RLIMIT_DATA. FreeBSD 8's malloc() can be configured
to use sbrk() only ("Dm"), but default setting is "DM".

Instead of gracefully handling ENOMEM condition processes grows and
swaps out, causing livelock.

Using RLIMIT_AS (aka RLIMIT_VMEM) as suggested interacts badly
with stack growth in process low memory condition.

As sbrk() is less preferable because of framentation and race conditions,
why not to create mmap() flag MMAP_DSS to check RLIMIT_DATA and to use it
in malloc(3) ?


-- 
Igor Sysoev
http://sysoev.ru/en/