List:Cluster« Previous MessageNext Message »
From:Jaya Date:August 1 2011 5:52pm
Subject:MySQL CLuster Crashes
View as plain text  
Hi ,
MySQL cluster crashes everyfew days. I dont know what is the actual cause
for this. I am looking into the logs <sys_name>.err , and cant really
pinpoint what could be the issue. I have basic setup. My Setup has 2
machines only. M/C 1 has mgmt,data node,sql node configured. and M/C 2 has
data node and SQL node configured.  Due to the recurrent crashing, I brought
down one SQL/Data node. Only M/C 1 is up and running. But still MYSQL
crashes. The configs are as below.
my.cnf:
********
[mysqld]
ndbcluster
ndb-connectstring=sys1
datadir=d:\\mySQL_cluster\\mysqld_data
basedir=c:\\mysqlc
port=5000
********
config.ini:
[ndb_mgmd]
hostname=sys1
datadir=D:\mySQL_cluster\ndb_data
id=1
[ndbd]
hostname=sys1
datadir=D:\mySQL_cluster\ndb_data
id=2
[ndbd]
hostname=sys2
datadir=D:\mySQL_cluster\ndb_data
id=3
[mysqld]
Id=50
hostname=sys1
[mysqld]
Id=51
hostname=sys2

***
ndb_2_out.log:
2011-07-25 13:38:06 [ndbd] INFO     -- Angel shutting down ndbd with pid
7968
2011-07-25 13:38:10 [ndbd] INFO     -- Angel shutting down ndbd with pid
7968
2011-07-25 13:38:10 [ndbd] INFO     -- Angel shutting down ndbd with pid
7968
2011-07-25 13:38:10 [ndbd] INFO     -- Angel shutting down
2011-07-25 13:38:10 [ndbd] INFO     -- Node 2: Node shutdown completed.
2011-07-25 13:41:38 [ndbd] INFO     -- Angel pid: 6280 started child: 9140
****
ndb_1_out.log:
Failed to flush buffer to socket, errno: 34
==INITIAL==
Warning: could not add log destination
'FILE:filename=D:\mySQL_cluster\ndb_data\ndb_1_cluster.log,maxsize=1000000,maxfiles=6'.
Reason: File exists
==CONFIRMED==
Node 2 failed
Node 2 failed
Node 2 failed
Node 2 failed
Node 2 failed
Node 2 failed
Node 2 failed
Node 2 failed
*********
ndb_1_cluster.log:
2011-07-25 13:37:39 [MgmtSrvr] WARNING  -- Failed to convert connection from
'10.243.238.45:51281' to transporter
2011-07-25 13:37:54 [MgmtSrvr] WARNING  -- Failed to convert connection from
'10.243.238.45:51294' to transporter
2011-07-25 13:38:06 [MgmtSrvr] INFO     -- Shutting down server...
2011-07-25 13:38:15 [MgmtSrvr] INFO     -- Node 2: Node shutdown completed.
*******
sys1.err
110725 13:35:59 [Warning] NDB: Could not acquire global schema lock
(4009)Cluster Failure
2011-07-25 13:36:04 [NdbApi] INFO     -- Management server closed connection
early. It is probably being shut down (or has problems). We will retry the
connection. 145 Time out talking to management server Error line: 454
110725 13:36:04 [ERROR] C:\mysqlc\bin\mysqld: Incorrect information in file:
'.\mirest\jbm_msg_ref.frm'
110725 13:36:08 [Note] Restarting Cluster Binlog
110725 13:36:10 [ERROR] C:\mysqlc\bin\mysqld: Incorrect information in file:
'.\mirest\jbm_msg_ref.frm'
2011-07-25 13:36:21 [NdbApi] INFO     -- Management server closed connection
early. It is probably being shut down (or has problems). We will retry the
connection. 145 Time out talking to management server Error line: 454
110725 13:36:21 [ERROR] C:\mysqlc\bin\mysqld: Incorrect information in file:
'.\mirest\jbm_msg_ref.frm'
110725 13:36:36 [ERROR] C:\mysqlc\bin\mysqld: Incorrect information in file:
'.\mirest\jbm_msg_ref.frm'
2011-07-25 13:36:41 [NdbApi] INFO     -- Management server closed connection
early. It is probably being shut down (or has problems). We will retry the
connection. 145 Time out talking to management server Error line: 454
110725 13:36:42 [ERROR] C:\mysqlc\bin\mysqld: Incorrect information in file:
'.\mirest\jbm_msg_ref.frm'
2011-07-25 13:36:52 [NdbApi] INFO     -- Management server closed connection
early. It is probably being shut down (or has problems). We will retry the
connection. 145 Time out talking to management server Error line: 454
110725 13:37:23 [Note] table '.\mirest\jbm_msg_ref' opened read only
110725 13:37:23 [Note] NDB_SHARE: trailing share
.\mysql\ndb_schema(connect_count: 188) released after NSS_DROPPED check at
connect_count: 191
110725 13:37:23 [Note] NDB Binlog: DISCOVER TABLE Event:
REPL$mysql/ndb_schema
2011-07-25 13:37:28 [NdbApi] INFO     -- Management server closed connection
early. It is probably being shut down (or has problems). We will retry the
connection. 145 Time out talking to management server Error line: 454
110725 13:37:30 [ERROR] NDB Binlog: ndbevent->execute failed for
REPL$mysql/ndb_schema; 4009 Cluster Failure
110725 13:37:30 [ERROR] NDB Binlog:FAILED CREATE (DISCOVER) EVENT OPERATIONS
Event: REPL$mysql/ndb_schema
110725 13:37:30 [Warning] NDB: Releasing global schema lock (4025)Node
failure caused abort of transaction
110725 13:37:41 [Note] Restarting Cluster Binlog
110725 13:37:54 [Warning] NDB: Could not acquire global schema lock
(4009)Cluster Failure
110725 13:37:59 [Warning] NDB: Could not acquire global schema lock
(4009)Cluster Failure
110725 13:38:06 [Warning] NDB: Could not acquire global schema lock
(4009)Cluster Failure
2011-07-25 13:38:09 [NdbApi] INFO     -- Management server closed connection
early. It is probably being shut down (or has problems). We will retry the
connection. 145 Time out talking to management server Error line: 454
110725 13:38:09 [Note] C:\mysqlc\bin\mysqld: Normal shutdown

110725 13:38:09 [Note] Event Scheduler: Purging the queue. 0 events
110725 13:38:12 [Warning] C:\mysqlc\bin\mysqld: Forcing close of thread 96 
user: 'jboss'

110725 13:38:12 [Warning] C:\mysqlc\bin\mysqld: Forcing close of thread 95 
user: 'jboss'

110725 13:38:12 [Warning] C:\mysqlc\bin\mysqld: Forcing close of thread 92 
user: 'root'

110725 13:38:12 [Warning] C:\mysqlc\bin\mysqld: Forcing close of thread 45 
user: 'root'

110725 13:38:12 [Note] Stopping Cluster Utility thread
110725 13:38:12 [Note] Stopping Cluster Binlog
110725 13:38:45  InnoDB: Starting shutdown...
110725 13:38:50  InnoDB: Shutdown completed; log sequence number 0 693706
110725 13:38:50 [Note] C:\mysqlc\bin\mysqld: Shutdown complete


I could't figure out, how this crash is happenening and how to rectify it.
Any help apreciated.
Thanks
J.

--
View this message in context:
http://mysql-cluster.1050023.n4.nabble.com/MySQL-CLuster-Crashes-tp3710377p3710377.html
Sent from the MySQL - Cluster mailing list archive at Nabble.com.
Thread
MySQL CLuster CrashesJaya1 Aug
  • Re: MySQL CLuster CrashesRavi Kiran PENTYALA2 Aug