I have had a MySQL replication pair going for a while now, and we
recently had some hardware issues on the slave. We've fixed the
hardware issues on the slave, and now I went to re-start replication.
I've done this probably 100 times, but for some reason I just cannot
get this one to go, and I suspect that something may be wrong with my
InnoDB databases and tables. Here's the steps I've taken:
On the functioning master server, I did:
flush tables with read lock;
show master status;
And I copied and pasted the master status to a text file for later.
Then, in a separate window, I created a ZFS snapshot, like this:
zfs snapshot tank/root/var@2012-10-16
And then once that was complete, back on the master server I did:
Then I rsync'd the data from the snapshot to the slave server:
rsync --progress --recursive --delete --times --perms --owner --group
But, when I started up MySQL, I get the following in my error log:
121016 10:40:20 mysqld_safe Starting mysqld daemon with databases from
121016 10:40:20 InnoDB: The InnoDB memory heap is disabled
121016 10:40:20 InnoDB: Mutexes and rw_locks use GCC atomic builtins
121016 10:40:20 InnoDB: Compressed tables use zlib 1.2.5
121016 10:40:20 InnoDB: Initializing buffer pool, size = 256.0M
121016 10:40:20 InnoDB: Completed initialization of buffer pool
121016 10:40:20 InnoDB: highest supported file format is Barracuda.
InnoDB: The log sequence number in ibdata files does not match
InnoDB: the log sequence number in the ib_logfiles!
121016 10:40:20 InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Error: trying to add tablespace 1182 of name './foo/TECH.ibd'
InnoDB: to the tablespace memory cache, but tablespace
InnoDB: 1182 of name './bar/patches.ibd' already exists in the tablespace
InnoDB: memory cache!
121016 10:40:38 mysqld_safe mysqld from pid file
So, I went back to the master server, backed up the "foo" database and
dropped and re-created it, and then restored the data, and repeated
the whole process, but then I just get the same error for another pair
of database names. I did this three times before giving up. No data
appears to be corrupted at all on the master server.
The master is running 5.5.25 on FreeBSD 8.2p4, and the slave is
running 5.5.27 on FreeBSD 9.0p4.
So, am I doing something wrong here? It seems like something is
broken with InnoDB, but I can't be sure. I'm doing an "optimize
table" on all my tables across all my databases now to make sure
there's no weird corruption going on, but I'm not holding my breath
with regard to if that will fix anything.
Baskin Engineering, Room 313A