Re: malloc(3) (hopefully) set for 7.0

On Thu, Mar 29, 2007 at 10:40:53PM +0200, Attilio Rao wrote:
> 2007/3/29, Kris Kennaway <kris_at_obsecurity.org>:
> >On Thu, Mar 29, 2007 at 07:51:56PM +0200, Ivan Voras wrote:
> >> Jason Evans wrote:
> >>
> >> > I have developed some novel algorithms for essentially eliminating
> >> > thread contention on SMP systems, but it is too late in the development
> >> > cycle to introduce such changes (not to mention that I lack the 
> >hardware
> >> >  to evaluate the algorithms).  Thanks again for your patience and
> >> > support.  Please let me know if I can be of help in diagnosing 
> >suspected
> >> > malloc issues.
> >>
> >> First, thanks :)
> >>
> >> Second, as a user, I'd really like if you could manage to implement
> >> those ideas before 7.0, and here's why:
> >>
> >> - The standard for new servers here is 4 cores (in various socket
> >> arrangements), and we're not at all high-tech. This is likely to go up.
> >> - If you include hyperthreading, even all *desktops* are SMPs! In short,
> >> even including desktops, I haven't installed a UP kernel in about a year.
> >> - It's too long to wait for 8.0 for something as important as this. As
> >> far as I can see, 7.0 will be one of the "break as many things as you
> >> need" releases (in the "good" sense, of course), so why not go for it.
> >> Judging from past releases, "even" releases (4.x, 6.x) have been the
> >> ones people trusted the most, so if you do get a glitch in 7.0 it won't
> >> be as bad :) (of course, you can fix it in 7.1 :) )
> >>
> >> Maybe you could borrow the 8CPU machine used for MySQL / filedesc tuning
> >> jeffr and others have been using (of course, once they've finished...)?
> >
> >I will be happy to (continue to) work with Jason on testing his
> >changes, but there appears to be no urgent need for this: the mysql
> >benchmark specifically shows that jemalloc scales well on 8 CPUs.  In
> >fact, the scalability problem seen on Linux turned out to be precisely
> >because of poor scaling of glibc malloc
> >
> > http://ozlabs.org/~anton/linux/sysbench/
> 
> Well, I'm not sure, since this test refers to core=4 while your tests
> were using a lot of more threads...

Well, it at least fixed *a* serious problem and it looks like it was
probably the main one.  Jeff hasn't been available to retest it on his
linux machine to see where we stand now though.

Kris