Re: perl malloc slow?

On Wed, Jan 07, 2004 at 02:11:09PM +0100, Poul-Henning Kamp wrote:
>One of the assumptions I made ten years ago, was that we would expose
>more of the possible VM gymnastics to userland and in particular it
>was my expectation that it would be cheap for a process to do some
>sort of page-flipping or page-exchange.
>
>This has not materialized in the meantime, and VMwizards have
>generally been a lot less than enthusiastic about it when I have
>tried to coax them into providing this sort of thing.

Maybe we need a mremap(2) implementation.  This would allow cheap
realloc(), though phkmalloc would need to be modified to use mmap()
rather than [s]brk() for heap allocation.  (We need mremap() for the
Linux emulation anyway).  I don't know how easy this would be to
implement - I suspect not very, and excessive use of mremap() could
severely fragment a processes address space (which could wreak havoc
with the page tables).

>VM systems on the other hand, operates on a page level, and modern
>code would be much better off like this:
>
>
>	l = PAGE_SIZE;
>	p = malloc(l);
>	for (;;) {
>		[...]
>		/* Damn */
>		l *= 16;
>		p = realloc(p, l);
>		[...]
>	}
>	/* Now trim */
>	p = realloc(p, strlen(p));
>
>(For some value of 16.)

Does this behaviour belong in the application or the malloc
implementation?  I believe most other realloc implementations use a
exponentially increasing bucket size so the application effectively
gets this behaviour (with '16' typically being either 2 or roughly
sqrt(2)) whether the application uses linear or exponential size
growth.

Peter