Re: core dumps are HUGE...

Hi, Jason--

First, thanks for your work on the new jemalloc.

Jason Evans wrote:
> On Mar 21, 2006, at 2:20 PM, John-Mark Gurney wrote:
[ ... ]
>> And yes, I'm pretty sure that I have a world from before your reduction
>> in chunk size...  Having a minimum of a 6meg core doesn't sound like a
>> good idea...  It's definately not going to play nice with small systems..
> 
> I don't think that a 6 MB core file is the big deal you are making it
> out to be.  A 50 MB core file is a pain, mainly because it takes a long
> time to dump core.

The smallest processes that people run often are likely going to be /bin/sh and
the typical fodder of shell scripts (grep, awk, and so forth).  Under 4.x,
/bin/sh tended to have VSIZE of 640K and RSS of ~300K; under 5.x and later,
/bin/sh has VSIZE of 1.6MB and RSS of ~1MB.

Creating a 50MB corefile from a 2MB VSIZE process is unreasonable.  Creating a
6MB corefile from a 2MB VSIZE process would be reasonable but probably could be
improved by a factor of two.

On the other hand, the staticly linked "t" program is:

chuck  62964  0.0  0.1   232   128  p0  TX    8:09AM   0:00.03 /tmp/t

128K RSS, 232 VSIZE.  A 6MB corefile is a factor of 25 larger, which strikes me
as something that could be improved by more than a factor of ten.

What else is being dumped that's not counted by VSIZE, aside from a header and
maybe some per-thread state, (although aren't the per-thread stacks already part
of VSIZE)...?

Can jemalloc only create per-CPU arenas only for processes which are themselves
multithreaded, when it's running on a multi-CPU system?  Would that help reduce
the amount of allocated but unreferenced memory that is involved for the common
case of /bin/sh and friends?

> Even so, consider that core dumps on OS X are in
> excess of 90 MB, and that this hasn't caused the world to stop revolving
> (though we should blame OS X for global warming).

Dear Cthulhu, you're right.

I was sure this was FUD, and I would have been wrong. :)  The one-line test
program from earlier in the thread creates a 147MB coredump under 10.4.5...I
need to file a radar or two with Apple.

-- 
-Chuck