(DESPERATE) ERROR 1005 (HY000): Can't create table 'test.test3' (errno: 157)

[Daniel Corbe]

Dear MySQL Cluster List,

I cannot figure out how to get my cluster running.  I've followed what
installation instructions I could find in the handbook and the advice of
several people and I am struggling to get my cluster going.  I have tested
many versions of MySQL all the way back to 5.0 with no luck.  Can someone
here please point me in the right direction?  My current iteration of
"trying to get this going" revolves around the newer versions of the NDB
engine for the 5.1.x branch.

The environment is made up of all FreeBSD 6.2-RELEASE servers.

mysql> create table test3 (i int) engine=ndbcluster;
ERROR 1005 (HY000): Can't create table 'test.test3' (errno: 157)
mysql> show warnings;
+-------+------+----------------------------------------------+
| Level | Code | Message                                      |
+-------+------+----------------------------------------------+
| Error | 1005 | Can't create table 'test.test3' (errno: 157) |
+-------+------+----------------------------------------------+
1 row in set (0.00 sec)

mysql> show engines
    -> ;
+------------+---------+-----------------------------------------------------------+--------------+----+------------+
| Engine     | Support | Comment
      | Transactions | XA | Savepoints |
+------------+---------+-----------------------------------------------------------+--------------+----+------------+
| ndbcluster | YES     | Clustered, fault-tolerant tables
       | YES          | NO | NO         |
| MRG_MYISAM | YES     | Collection of identical MyISAM tables
      | NO           | NO | NO         |
| CSV        | YES     | CSV storage engine
       | NO           | NO | NO         |
| MyISAM     | DEFAULT | Default engine as of MySQL 3.23 with great
performance    | NO           | NO | NO         |
| MEMORY     | YES     | Hash based, stored in memory, useful for temporary
tables | NO           | NO | NO         |
+------------+---------+-----------------------------------------------------------+--------------+----+------------+
5 rows in set (0.00 sec)

mysql>

ndb_mgm> show
Cluster Configuration
---------------------
[ndbd(NDB)]     2 node(s)
id=2    @216.151.143.116  (mysql-5.1.23 ndb-6.3.10, Nodegroup: 0, Master)
id=3    @216.151.143.83  (mysql-5.1.23 ndb-6.3.10, Nodegroup: 0)

[ndb_mgmd(MGM)] 1 node(s)
id=1    @216.152.248.236  (mysql-5.1.23 ndb-6.3.10)

[mysqld(API)]   4 node(s)
id=4    @216.151.143.110  (mysql-5.1.23 ndb-6.3.10)
id=5    @216.151.143.111  (mysql-5.1.23 ndb-6.3.10)
id=6 (not connected, accepting connect from any host)
id=7 (not connected, accepting connect from any host)

config.ini:

# Options affecting ndbd processes on all data nodes:
[ndbd default]
NoOfReplicas=2    # Number of replicas
DataMemory=96M    # How much memory to allocate for data storage
IndexMemory=64M   # How much memory to allocate for index storage
                  # For DataMemory and IndexMemory, we have used the
                  # default values. Since the "world" database takes up
                  # only about 500KB, this should be more than enough for
                  # this example Cluster setup.

# Management process options:
[ndb_mgmd]
hostname=216.152.248.236           # Hostname or IP address of MGM node
datadir=/var/lib/mysql-cluster  # Directory for MGM node log files

# Options for data node "A":
[ndbd]
                                # (one [ndbd] section per data node)
hostname=216.151.143.116        # Hostname or IP address
datadir=/var/mysql/data   # Directory for this data node's data files

# Options for data node "B":
[ndbd]
hostname=216.151.143.83         # Hostname or IP address
datadir=/var/mysql/data   # Directory for this data node's data files

# SQL node options:
[mysqld]
hostname=216.151.143.110        # Hostname or IP address
                                # (additional mysqld connections can be
                                # specified for this node for various
                                # purposes such as running ndb_restore)

[mysqld]
hostname=216.151.143.111        # Hostname or IP address
                                # (additional mysqld connections can be
                                # specified for this node for various
                                # purposes such as running ndb_restore)

[mysqld]

[mysqld]


my.cnf on each server:

# Options for mysqld process:
[mysqld]
ndbcluster                      # run NDB storage engine
ndb-connectstring=216.152.248.236  # location of management server

# Options for ndbd process:
[mysql_cluster]
ndb-connectstring=216.152.248.236  # location of management server

When I look at the error log on the API nodes I see the following message
over and over again:
080302 22:13:27 [ERROR] NDB: CREATE TABLE IF NOT EXISTS
mysql.ndb_apply_status ( server_id INT UNSIGNED NOT NULL, epoch BIGINT
UNSIGNED NOT NULL,  log_name VARCHAR(255) BINARY NOT NULL,  start_pos BIGINT
UNSIGNED NOT NULL,  end_pos BIGINT UNSIGNED NOT NULL,  PRIMARY KEY USING
HASH (server_id) ) ENGINE=NDB: error Can't create table '
mysql.ndb_apply_status' (errno: 157) 1005(ndb: 0) 1 1

Please help!