From: Rick James Date: March 27 2012 9:24pm Subject: Re: mysqld got signal 6 (problem why bigger than I initially realised) List-Archive: http://lists.mysql.com/mysql/227062 Message-Id: <4F723010.9070703@yahoo-inc.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit MaxClients 1000 could overwhelm max_connections = 41 . Strongly recommend you decrease MaxClients to less than max_connections (not the other way around). Uptime | 18492 Not very long to see stuff. | innodb_buffer_pool_size | 8388608 | | key_buffer_size | 8384512 | Much too small for a 12GB machine. See http://mysql.rjweb.org/doc.php/memory for tuning advice. Back to signal 6 (Abort) -- was anything useful in mysqld.err ? On 3/27/12 2:06 PM, Brent Clark wrote: > Hi Rick > > Thank you for replying. > > Please see my answers to your questions. > > On Tue, Mar 27, 2012 at 7:58 PM, Rick James wrote: >> Do you have 12GB of RAM? > total used free shared buffers cached > Mem: 12038 11959 78 0 139 11234 > -/+ buffers/cache: 586 11451 > Swap: 2047 0 2047 > >> Is this a 64-bit mysqld? > Linux hostname 2.6.32-5-amd64 #1 SMP Mon Jan 16 16:22:28 UTC 2012 > x86_64 GNU/Linux > >> Let's see >> SHOW VARIABLES LIKE '%buffer%'; > +-------------------------+---------+ > | Variable_name | Value | > +-------------------------+---------+ > | bulk_insert_buffer_size | 8388608 | > | innodb_buffer_pool_size | 8388608 | > | innodb_log_buffer_size | 1048576 | > | join_buffer_size | 131072 | > | key_buffer_size | 8384512 | > | myisam_sort_buffer_size | 8388608 | > | net_buffer_length | 16384 | > | preload_buffer_size | 32768 | > | read_buffer_size | 131072 | > | read_rnd_buffer_size | 262144 | > | sort_buffer_size | 2097144 | > | sql_buffer_result | OFF | > +-------------------------+---------+ > >> SHOW GLOBAL STATUS LIKE 'Max%'; > +----------------------+-------+ > | Variable_name | Value | > +----------------------+-------+ > | Max_used_connections | 3 | > +----------------------+-------+ > >> SHOW GLOBAL STATUS LIKE 'Up%'; > +---------------------------+-------+ > | Variable_name | Value | > +---------------------------+-------+ > | Uptime | 18492 | > | Uptime_since_flush_status | 18492 | > +---------------------------+-------+ > >> What is Apache's MaxClients? > MaxClients 1000 > > Thank you again. > Brent > >> >> On 3/27/12 6:25 AM, Brent Clark wrote: >>> Good day KarthiK.P.R >>> >>> Other than the replication settings in '/etc/mysql/conf.d/replication.cnf' >>> and our /etc/mysql/conf.d/custom.cnf >>> >>> xyz-web02:/data# cat /etc/mysql/conf.d/custom.cnf >>> [mysqld] >>> innodb_file_per_table >>> bind-address = 0.0.0.0 >>> datadir = /data >>> binlog_format=mixed >>> key_buffer_size=8384512 >>> max_connections=41 >>> >>> The original /etc/mysql/my.cnf is untouched. >>> >>> The only other service we have running is DRBD (active / active) and >>> apache, nothing is hitting apache as this is the standby node. >>> And there is no load, or anything consuming resources. >>> >>> Thanks >>> Brent >>> >>> >>> On 27/03/2012 14:04, P.R.Karthik wrote: >>>> HI Brent, >>>> >>>> Can you please paste your configuration file settings (my.cnf) ? It will >>>> help to identify where things went wrong. >>>> >>>> Is there any other memory consuming application running on the server >>>> beyond mysql ? >>>> >>>> Regards, >>>> KarthiK.P.R >>>> >>>> >>>> >>>> On Tue, Mar 27, 2012 at 4:17 PM, Brent >>>> Clark> wrote: >>>> >>>> Hey Guys >>>> >>>> Yesterday I sent an email, about '1 client is using or hasn't closed >>>> the table properly', but the problem is actually bigger than I realised. >>>> >>>> We run Mysql replication, and on the second node, Mysql is crashing >>>> with 'mysqld got signal 6' every so often. >>>> >>>> Other than the drives, we have replaced the hardware, and still the >>>> problem persists. >>>> >>>> Googling, I thought it was a memory exhaustion issue, so I started >>>> playing with the following variables and values. >>>> >>>> key_buffer_size=8384512 >>>> max_connections=41 >>>> >>>> But it still crashes >>>> >>>> Mar 27 12:36:53 xyz-web02 mysqld: key_buffer_size=8384512 >>>> Mar 27 12:36:53 xyz-web02 mysqld: read_buffer_size=131072 >>>> Mar 27 12:36:53 xyz-web02 mysqld: max_used_connections=3 >>>> Mar 27 12:36:53 xyz-web02 mysqld: max_threads=41 >>>> Mar 27 12:36:53 xyz-web02 mysqld: thread_count=1 >>>> Mar 27 12:36:53 xyz-web02 mysqld: connection_count=1 >>>> Mar 27 12:36:53 xyz-web02 mysqld: It is possible that mysqld could >>>> use up to >>>> Mar 27 12:36:53 xyz-web02 mysqld: key_buffer_size + (read_buffer_size >>>> + sort_buffer_size)*max_threads = 97828 K bytes of memory >>>> >>>> xyz-web02:# free -m >>>> total used free shared buffers >>>> cached >>>> Mem: 12038 11966 71 0 117 >>>> 11378 >>>> -/+ buffers/cache: 469 11568 >>>> Swap: 2047 0 2047 >>>> >>>> We running Debian Squeeze 64bit. >>>> >>>> I have attached the full crash message. >>>> >>>> If anyone can help, I would be *most* grateful (If you are in South >>>> Africa, I will buy you a meal, let alone a beer :) >>>> >>>> Kindest Regards >>>> Brent Clark >>>> >>>> >>>> -- >>>> MySQL General Mailing List >>>> For list archives: http://lists.mysql.com/mysql >>>> To unsubscribe: http://lists.mysql.com/mysql >>>> >>>> >> -- >> Rick James - MySQL Geek >> >> >> >> -- >> MySQL General Mailing List >> For list archives: http://lists.mysql.com/mysql >> To unsubscribe: http://lists.mysql.com/mysql >> -- Rick James - MySQL Geek