Re: MySQL crashes on amd64

Stefan Cars wrote:
> Greg 'groggy' Lehey wrote:
> 
>> [Sequence recovered--see http://www.lemis.com/email/email-format.html]
>>
>> On Thursday, 13 October 2005 at 17:56:11 +0200, Stefan Cars wrote:
>>
>>> Greg 'groggy' Lehey wrote:
>>>
>>>> On Wednesday, 12 October 2005 at  9:27:17 +0200, Stefan Cars wrote:
>>>>
>>>>
>>>>> Hi!
>>>>>
>>>>> We are having troubles with MySQL 4.1 on a amd64 (it's crashing
>>>>> randomly with Seg fault, signal 11. gdb bt says: Cannot access
>>>>> memory at address 0x800000000000). We have got information saying
>>>>> this is a 64bit related issue and should be fixed by using the i386
>>>>> version instead of amd64 (this is an Intel Xeon).
>>>>
>>>>
>>>> Where did you get that information from?
>>
>>
>>
>> I'd still like to know an answer to this question.
> 
> 
> I got information from a guy working on the floor below us, although his 
> thoughts were only that "amd64 is a bit unstable so that should be the 
> problem"
> 
>>
>>
>>>>> What is the way to go when moving from amd64 to i386 ?
>>>>
>>>>
>>>> If you mean "how do I install an i386 kernel on this machine", I can't
>>>> think of any way except to start from scratch.  It would be a good
>>>> idea to install a separate disk, so you can access the configuration
>>>> files and the database on the old disk.  But before doing this, I'd be
>>>> very interested in knowing what the problem is.  Is the backtrace
>>>> always the same?  Where does it crash?
>>>
>>>
>>> After doing some testing this is what I found out:
>>>
>>> 1) Exchanging memory on the machine did not work. Same error.
>>> 2) Trying it on another 64 bit machine with same FreeBSD (RC1) creates
>>> EXACT same problem
>>> 3) Installing the i386 version of RC1 instead of amd64 on the same
>>> machines and it works terrific. No crash.
>>
>>
>>
>> Hmm.  That's interesting.  This is obviously not a hardware issue.
>>
>>
>>> The bt is always the same and it always crash the same, look here:
>>>
>>> #784 0x0000000000000000 in ?? ()
>>> #785 0x247c8d48002454ff in ?? ()
>>> #786 0x01a1c0c748006a10 in ?? ()
>>> #787 0x66fdebf4050f0000 in ?? ()
>>> #788 0x9066669066669066 in ?? ()
>>> #789 0x00007fffffffe778 in ?? ()
>>> #790 0x0000000000000006 in ?? ()
>>> #791 0x00007fffffffe7b0 in ?? ()
>>> #792 0x0000000000000017 in ?? ()
>>> Cannot access memory at address 0x800000000000
>>
>>
>>
>> Without function names instead of ??, it's impossible to say what's
>> happening here.  You'd need to build with debug symbols.
>>
>> Since you've been told that this is an issue, it would be good to know
>> more.  As we've mentioned on other threads, there are reasons to
>> believe that there are problems with the threading libraries, but
>> currently we don't have enough information to investigate them.  Note
>> that the other recent thread refers to problems running in the
>> configuration you have just installed: see
>> http://bugs.mysql.com/bug.php?id=12251 for more details.  If you see
>> anything similar, please let us know.
>>
> 
> 
> After doing alot of testing and trying this is some more new information:
> 
> It actually DO crash on i386 FreeBSD 6.0-RC1 aswell, but very seldom 
> (around once every 4-5th hour, this time we get a Signal 10 instead, see 
> below). The errors are reproducable for MySQL 4.1.15 and MySQL 5.0.15 so 
> the problem still exists on 5.0.15. Of course we have tried on different 
>  machines aswell to make sure it's not hardware related. On a FreeBSD 
> 5.3 machine Mysql 4.1.15 works fine.
> 
> 
> mysqld got signal 10;
> This could be because you hit a bug. It is also possible that this binary
> or one of the libraries it was linked against is corrupt, improperly built,
> or misconfigured. This error can also be caused by malfunctioning hardware.
> We will try our best to scrape up some info that will hopefully help 
> diagnose
> the problem, but since we have already crashed, something is definitely 
> wrong
> and this may fail.
> 
> key_buffer_size=1048576
> read_buffer_size=131072
> max_used_connections=99
> max_connections=400
> threads_connected=44
> It is possible that mysqld could use up to
> key_buffer_size + (read_buffer_size + sort_buffer_size)*max_connections 
> = 461820 K
> bytes of memory
> Hope that's ok; if not, decrease some variables in the equation.




Adding to this, using the following in libmap.conf fixes the problems:

libpthread.so   libthr.so
libpthread.so.1 libthr.so.2
libpthread.so.2 libthr.so.2