Re: sbrk(2) broken

On Fri, 4 Jan 2008, Igor Mozolevsky wrote:

> Of course, if you're afraid of memory overcommit and you know in advance
>> how much memory you need, you can simply allocate a sufficient amount of 
>> address space at startup and touch it all.  This way, you will either be 
>> killed right away, or be guaranteed to have sufficient memory for the rest 
>> of your (process) lifetime.  Alternatively, do what Varnish does: create a 
>> large file, mmap it, and allocate everything you need from that area, so 
>> you have your own private swap space.  Just make sure to actually allocate 
>> the disk space you need (by filling the file with zeroes, or at the minimum 
>> writing a zero to the file every sb.st_blksize bytes, preferably 
>> sequentially to avoid excessive fragmentation)
>
> Surely you can just fseek() on the file at the correct lenght?

That will create a sparse file without file system blocks to back it, and is 
effectively also over-commit.  When the file system runs out of room, you will 
get SIGSEGV when the vnode pager discovers it can't write a page to disk.  If 
you zero-fill it, the blocks are pre-allocated.  In a more ideal world, we 
might support an ioctl or system call to pre-allocate but not hook up the 
blocks until they were written to, in order to avoid writing lots of zeros to 
disk, but we don't live in that ideal world yet.

Allowing malloc to support alternative sources of pages for memory mapping, 
such as specific files, would be very neat indeed.

Robert N M Watson
Computer Laboratory
University of Cambridge