Re: Virtual memory consumption (both user and kernel) in modern CURRENT

On Fri, Feb 17, 2006 at 08:28:43AM -0800, Jason Evans wrote:
> Yes, it is possible to avoid using large amounts of virtual memory.  See 
> the malloc(3) man page for info on all of the ways that the allocator 
> can be tuned.  In this case, you are looking for the 'k' option.  For 
> instance, if you  set /etc/malloc.conf to 'kkkkkk', the allocator will 
> use a chunk size of 256kB rather than 16MB.
> 
> I would like to change the default chunk size to something in the 
> 256kB-1MB range, but Peter Wemm has some concerns about how that will 
> interact with the VM subsystem.  The basic problem is that the kernel is 
> particularly naive about how it coalesces adjacent mappings, so it's 
> possible to end up with a horribly fragmented virtual memory map.  I 
> don't have the time to fix this problem right now, and I'm not going to 
> change the default chunk size unless Peter Wemm and/or Alan Cox say it's 
> okay to do so.

Would it be possible to have some sort of "slow start" algorithm? For
example, first block allocated is 1MB, next block allocated is 4MB,
subsequent blocks are 16MB?

I don't know anything about the internal data structures of jemalloc, but if
you treat "4MB" as 4 x 1MB and "16MB" as 16 x 1MB, then you'd have
effectively a pool of 1MB blocks, but apart from the first few, they'd be
allocated contiguously.

Regards,

Brian.