| List: | Commits | « Previous MessageNext Message » | |
| From: | ingo | Date: | May 21 2007 5:48pm |
| Subject: | bk commit into 4.1 tree (istruewing:1.2630) BUG#26379 | ||
| View as plain text | |||
Below is the list of changes that have just been committed into a local 4.1 repository of istruewing. When istruewing does a push these changes will be propagated to the main repository and, within 24 hours after the push, to the public repository. For information on how to access the public repository see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html ChangeSet@stripped, 2007-05-21 17:48:22+02:00, istruewing@stripped +23 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table This bug report revealed three problems: 1. A thread trying to lock a MERGE table performs busy waiting while REPAIR TABLE or a similar table administration task is ongoing on one or more of its MyISAM tables. 2. A thread trying to lock a MERGE table performs busy waiting until all threads that did REPAIR TABLE or similar table administration tasks on one or more of its MyISAM tables in LOCK TABLES segments do UNLOCK TABLES. The difference against problem #1 is that the busy waiting takes place *after* the administration task. It is terminated by UNLOCK TABLES only. 3. Two FLUSH TABLES within a LOCK TABLES segment can invalidate the lock. This does *not* require a MERGE table. The first FLUSH TABLES can be replaced by any statement that requires other threads to reopen the table. In 5.0 and 5.1 a single FLUSH TABLES can provoke the problem. Problem #1 and #2 are fixed by opening the MERGE child tables through the table cache. So they are visible to other threads and even for certain statements in the same thread. The child tables are chained to the parent table, but do not appear in thd->open_tables nor thd->locked_tables. This does also fix the bugs 8306, 25038, 25700, 26377, 26867, and 27660. I reverted the preliminary fix for 8306. Problem #3 is fixed by setting THD::some_tables_deleted for all threads with open tables before releasing LOCK_open when waiting for all threads to close their tables. The misbehaviour for problems #1 and #3 can only be repeated reliably with some sleep()-injection in the code. Problem #2 can be watched by enlarging the sleep time in the test file. Please see the bug report for more information and tests for manual testing. See bug 25700 for a stress test. include/myisammrg.h@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +6 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added callback elements to MYRG_INFO. Added callback arguments to myrg_open(). include/raid.h@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +2 -2 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed compiler warnings. myisam/mi_create.c@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +0 -15 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Reverted fix for bug 8306. It is implicitly and better fixed with this patch. myisam/mi_open.c@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +1 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Reverted fix for bug 8306. It is implicitly and better fixed with this patch. myisam/myisamdef.h@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +0 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Reverted fix for bug 8306. It is implicitly and better fixed with this patch. myisammrg/myrg_close.c@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +16 -3 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Using callback for closing child tables if set. Otherwise close MyISAM tables directly. myisammrg/myrg_open.c@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +82 -16 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Using callback for opening child tables if set. Otherwise open MyISAM tables directly. Store close_callback and callback_param in MYRG_INFO for use with myrg_close(). Fixed error handling for failed allocation. Fixed indentation. mysql-test/r/merge.result@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +215 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed results. Moved test result for bug 8306 from myisam.result to here. Added test results from other fixed bugs. mysql-test/r/myisam.result@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +0 -18 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Reverted fix for bug 8306. It is implicitly and better fixed with this patch. Moved test result from here to merge.result. mysql-test/t/merge.test@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +273 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed error number. Moved test for bug 8306 from myisam.test to here. Added tests for other fixed bugs. mysql-test/t/myisam.test@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +0 -26 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Reverted fix for bug 8306. It is implicitly and better fixed with this patch. Moved test from here to merge.test. mysys/thr_lock.c@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +35 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added DBUG calls. sql/ha_myisam.cc@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +21 -2 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added new function myisam_engine_handle() to extract a MI_INFO pointer from a handler object. sql/ha_myisam.h@stripped, 2007-05-21 17:48:19+02:00, istruewing@stripped +1 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Declared new function myisam_engine_handle() as friend of ha_myisam class. sql/ha_myisammrg.cc@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +239 -11 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added callback functions for open and close. Allowed lock_count() and store_lock() to work on a closed table. Fixed/added DBUG calls. sql/ha_myisammrg.h@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +9 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added struct st_myrg_callback_param to use with callback functions. Added table_ptr() method to use with callback functions. sql/lock.cc@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +30 -2 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added DBUG calls. sql/mysql_priv.h@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +6 -3 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'is_child' parameter to open_table() for MERGE table childs. Added 'is_temporary' parameter to open_temporary_table() to tell the difference between a real temporary table and intermediate tables, which are used during ALTER TABLE. Added declaration for myisam_engine_handle() to use with MERGE children. sql/sql_base.cc@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +430 -100 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table On FLUSH TABLES, set 'some_tables_deleted' for all threads with open tables. Added awareness of MERGE parent or child tables to several functions. Enabled close_thread_table() to accept thd == NULL. Changed close_thread_table() to not add MERGE child tables to unused_tables. Added new parameter 'is_child' to open_tables(). If true, don't lock/unlock LOCK_open as it is done for the whole MERGE table (when handling the parent). When picking the table from unused_tables, set in_use for children too. Do not link children in open_tables chain. When reopening a table, fix the mrg_parent links of all children. Extended table_in_use() to check parent and children too. Changed reopen_table() to handle MERGE tables properly. Added parameter 'is_temporary' to open_temporary_table(). MERGE tables need to know if we open "real" temporary tables as the path names of their temporary children cannot be used to produce names for open_table(). Luckily there is no need for locking of temporary tables. So it is fine for temporary MERGE tables to open their children directly. When removing a child table from cache, remove the whole MERGE table. When aborting locks, abort them for the parent table, if exists. sql/sql_delete.cc@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +1 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added argument to new 'is_temporary' parameter of open_temporary_table(). sql/sql_table.cc@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +13 -7 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added argument to new 'is_temporary' parameter of open_temporary_table(). Added argument to new 'is_child' parameter of open_table(). Moved locking of LOCK_open before intern_close_table() in mysql_alter_table(). If old table was MERGE, we may need to close child tables. This needs LOCK_open. sql/table.cc@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +40 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added closing of MERGE table children to closefrm(). sql/table.h@stripped, 2007-05-21 17:48:20+02:00, istruewing@stripped +3 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added mrg_parent and mrg_child to TABLE for use with MERGE tables. # This is a BitKeeper patch. What follows are the unified diffs for the # set of deltas contained in the patch. The rest of the patch, the part # that BitKeeper cares about, is below these diffs. # User: istruewing # Host: chilla.local # Root: /home/mydev/mysql-4.1-bug26379 --- 1.15/include/myisammrg.h 2007-05-21 17:48:27 +02:00 +++ 1.16/include/myisammrg.h 2007-05-21 17:48:27 +02:00 @@ -73,6 +73,8 @@ typedef struct st_myrg_info LIST open_list; QUEUE by_key; ulong *rec_per_key_part; /* for sql optimizing */ + void *callback_param; /* open/close callback parameter */ + void (*close_callback)(); /* Set if open_callback has been used. */ } MYRG_INFO; @@ -80,7 +82,10 @@ typedef struct st_myrg_info extern int myrg_close(MYRG_INFO *file); extern int myrg_delete(MYRG_INFO *file,const byte *buff); -extern MYRG_INFO *myrg_open(const char *name,int mode,int wait_if_locked); +extern MYRG_INFO *myrg_open(const char *name,int mode,int wait_if_locked, + MI_INFO *(*open_callback)(void*, char*), + void (*close_callback)(void*), + void *callback_param); extern int myrg_panic(enum ha_panic_function function); extern int myrg_rfirst(MYRG_INFO *file,byte *buf,int inx); extern int myrg_rlast(MYRG_INFO *file,byte *buf,int inx); --- 1.19/include/raid.h 2007-05-21 17:48:27 +02:00 +++ 1.20/include/raid.h 2007-05-21 17:48:27 +02:00 @@ -141,7 +141,7 @@ class RaidFd { inline void Calculate() { DBUG_ENTER("RaidFd::_Calculate"); - DBUG_PRINT("info",("_position: %lu _raid_chunksize: %d, _size: %lu", + DBUG_PRINT("info",("_position: %lu _raid_chunksize: %lu _size: %lu", (ulong) _position, _raid_chunksize, (ulong) _size)); _total_block = (ulong) (_position / _raid_chunksize); @@ -149,7 +149,7 @@ class RaidFd { _remaining_bytes = (uint) (_raid_chunksize - (_position - _total_block * _raid_chunksize)); DBUG_PRINT("info", - ("_total_block: %d this_block: %d _remaining_bytes:%d", + ("_total_block: %lu this_block: %u _remaining_bytes: %u", _total_block, _this_block, _remaining_bytes)); DBUG_VOID_RETURN; } --- 1.51/myisam/mi_create.c 2007-05-21 17:48:27 +02:00 +++ 1.52/myisam/mi_create.c 2007-05-21 17:48:27 +02:00 @@ -564,21 +564,6 @@ int mi_create(const char *name,uint keys create_flag=MY_DELETE_OLD; } - /* - If a MRG_MyISAM table is in use, the mapped MyISAM tables are open, - but no entry is made in the table cache for them. - A TRUNCATE command checks for the table in the cache only and could - be fooled to believe, the table is not open. - Pull the emergency brake in this situation. (Bug #8306) - */ - if (test_if_reopen(filename)) - { - my_printf_error(0, "MyISAM table '%s' is in use " - "(most likely by a MERGE table). Try FLUSH TABLES.", - MYF(0), name + dirname_length(name)); - goto err; - } - if ((file= my_create_with_symlink(linkname_ptr, filename, 0, create_mode, MYF(MY_WME | create_flag))) < 0) goto err; --- 1.90/myisam/mi_open.c 2007-05-21 17:48:27 +02:00 +++ 1.91/myisam/mi_open.c 2007-05-21 17:48:27 +02:00 @@ -50,7 +50,7 @@ if (pos > end_pos) \ ** In MySQL the server will handle version issues. ******************************************************************************/ -MI_INFO *test_if_reopen(char *filename) +static MI_INFO *test_if_reopen(char *filename) { LIST *pos; --- 1.82/myisam/myisamdef.h 2007-05-21 17:48:27 +02:00 +++ 1.83/myisam/myisamdef.h 2007-05-21 17:48:27 +02:00 @@ -728,7 +728,6 @@ void mi_copy_status(void* to,void *from) my_bool mi_check_status(void* param); void mi_disable_non_unique_index(MI_INFO *info, ha_rows rows); -extern MI_INFO *test_if_reopen(char *filename); my_bool check_table_is_closed(const char *name, const char *where); int mi_open_datafile(MI_INFO *info, MYISAM_SHARE *share, File file_to_dup); int mi_open_keyfile(MYISAM_SHARE *share); --- 1.6/myisammrg/myrg_close.c 2007-05-21 17:48:27 +02:00 +++ 1.7/myisammrg/myrg_close.c 2007-05-21 17:48:27 +02:00 @@ -23,10 +23,23 @@ int myrg_close(MYRG_INFO *info) int error=0,new_error; MYRG_TABLE *file; DBUG_ENTER("myrg_close"); + DBUG_PRINT("info", ("myrg_info: 0x%lx close_cb: 0x%lx cb_param: 0x%lx", + (long) info, (long) info->close_callback, + (long) info->callback_param)); - for (file=info->open_tables ; file != info->end_table ; file++) - if ((new_error=mi_close(file->table))) - error=new_error; + /* + If there was a non-NULL close_callback argument given with the + corresponding myrg_open(), use this function for close instead + of closing the MyISAM tables directly. + */ + if (info->close_callback) + (*info->close_callback)(info->callback_param); + else + { + for (file=info->open_tables ; file != info->end_table ; file++) + if ((new_error=mi_close(file->table))) + error=new_error; + } delete_queue(&info->by_key); pthread_mutex_lock(&THR_LOCK_open); myrg_open_list=list_delete(myrg_open_list,&info->open_list); --- 1.32/myisammrg/myrg_open.c 2007-05-21 17:48:27 +02:00 +++ 1.33/myisammrg/myrg_open.c 2007-05-21 17:48:27 +02:00 @@ -23,14 +23,37 @@ #include "mrg_static.c" #endif -/* - open a MyISAM MERGE table - if handle_locking is 0 then exit with error if some table is locked - if handle_locking is 1 then wait if table is locked -*/ +/** + @brief Open a MyISAM MERGE table + + @detail If open_callback and close_callback functions are given, + they are used for opening and closing the child tables. + open_callback() is called for each child table. It opens one table. + close_callback() is called once for all child tables. It closes all + tables. + + The reason for having close_callback here is that we can make the + decision, if to use callbacks or not, just once before opening the + table. MySQL uses callbacks for non-temporary tables and does not use + them for temporary tables. So the decision if we have a temporary + table is required only on open. + + @param[in] name db/table_name + @param[in] mode O_RDONLY or O_RDWR + @param[in] handle_locking 0: error if some table is locked + 1: wait if table is locked + @param[in] open_callback function to use for opening a child table + @param[in] close_callback function to use for closing all child tables + @param[in] callback_param data pointer to give to the callbacks + @return pointer to opened MERGE table structure + @retval NULL on error +*/ -MYRG_INFO *myrg_open(const char *name, int mode, int handle_locking) +MYRG_INFO *myrg_open(const char *name, int mode, int handle_locking, + MI_INFO *(*open_callback)(void*, char*), + void (*close_callback)(void*), + void *callback_param) { int save_errno,errpos=0; uint files= 0, i, dir_length, length, key_parts, min_keys= 0; @@ -42,18 +65,23 @@ MYRG_INFO *myrg_open(const char *name, i MI_INFO *isam=0; uint found_merge_insert_method= 0; DBUG_ENTER("myrg_open"); + DBUG_PRINT("enter", ("name: '%s' mode: %d flags: %d", + name, mode, handle_locking)); + DBUG_PRINT("enter", ("open_cb: 0x%lx close_cb: 0x%lx cb_param: 0x%lx", + (long) open_callback, (long) close_callback, + (long) callback_param)); LINT_INIT(key_parts); bzero((char*) &file,sizeof(file)); if ((fd=my_open(fn_format(name_buff,name,"",MYRG_NAME_EXT,4), O_RDONLY | O_SHARE,MYF(0))) < 0) - goto err; - errpos=1; - if (init_io_cache(&file, fd, 4*IO_SIZE, READ_CACHE, 0, 0, + goto err; + errpos=1; + if (init_io_cache(&file, fd, 4*IO_SIZE, READ_CACHE, 0, 0, MYF(MY_WME | MY_NABP))) - goto err; - errpos=2; + goto err; + errpos=2; dir_length=dirname_part(name_buff,name); while ((length=my_b_gets(&file,buff,FN_REFLEN-1))) { @@ -88,10 +116,27 @@ MYRG_INFO *myrg_open(const char *name, i } else fn_format(buff, buff, "", "", 0); - if (!(isam=mi_open(buff,mode,(handle_locking?HA_OPEN_WAIT_IF_LOCKED:0)))) + DBUG_PRINT("info", ("child: '%s'", buff)); + /* + If a non-NULL open_callback argument is present, use it for + opening the MyISAM table instead of opening it directly. + */ + if (open_callback) { - my_errno= HA_ERR_WRONG_MRG_TABLE_DEF; - goto err; + if (!(isam= (*open_callback)(callback_param, buff))) + { + DBUG_PRINT("myrg", ("open_callback failed, myerrno: %d", my_errno)); + goto err; + } + } + else + { + if (!(isam= mi_open(buff, mode, (handle_locking ? + HA_OPEN_WAIT_IF_LOCKED : 0)))) + { + my_errno= HA_ERR_WRONG_MRG_TABLE_DEF; + goto err; + } } if (!m_info) /* First file */ { @@ -100,7 +145,18 @@ MYRG_INFO *myrg_open(const char *name, i files*sizeof(MYRG_TABLE) + key_parts*sizeof(long), MYF(MY_WME|MY_ZEROFILL)))) + { + /* + Close first file. errpos is still == 2. + If a non-NULL close_callback argument is present, use it + instead of closing the MyISAM table directly. + */ + if (close_callback) + (*close_callback)(callback_param); + else + mi_close(isam); goto err; + } if (files) { m_info->open_tables=(MYRG_TABLE *) (m_info+1); @@ -109,6 +165,9 @@ MYRG_INFO *myrg_open(const char *name, i files= 0; } m_info->reclength=isam->s->base.reclength; + /* Store callback information for use with myrg_close(). */ + m_info->close_callback= close_callback; + m_info->callback_param= callback_param; min_keys= isam->s->base.keys; errpos=3; } @@ -163,8 +222,15 @@ err: save_errno=my_errno; switch (errpos) { case 3: - while (files) - mi_close(m_info->open_tables[--files].table); + /* + If a non-NULL close_callback argument is present, use it + instead of closing the MyISAM tables directly. + */ + if (close_callback) + (*close_callback)(callback_param); + else + while (files) + mi_close(m_info->open_tables[--files].table); my_free((char*) m_info,MYF(0)); /* Fall through */ case 2: --- 1.42/mysys/thr_lock.c 2007-05-21 17:48:27 +02:00 +++ 1.43/mysys/thr_lock.c 2007-05-21 17:48:27 +02:00 @@ -137,6 +137,9 @@ static int check_lock(struct st_lock_lis fprintf(stderr, "Warning: prev link %d didn't point at previous lock at %s: %s\n", count, lock_type, where); + DBUG_PRINT("warning",( + "Warning: prev link %d didn't point at previous lock at %s: %s\n", + count, lock_type, where)); return 1; } if (same_thread && ! pthread_equal(data->thread,first_thread) && @@ -145,6 +148,9 @@ static int check_lock(struct st_lock_lis fprintf(stderr, "Warning: Found locks from different threads in %s: %s\n", lock_type,where); + DBUG_PRINT("warning",( + "Warning: Found locks from different threads in %s: %s\n", + lock_type,where)); return 1; } if (no_cond && data->cond) @@ -152,6 +158,9 @@ static int check_lock(struct st_lock_lis fprintf(stderr, "Warning: Found active lock with not reset cond %s: %s\n", lock_type,where); + DBUG_PRINT("warning",( + "Warning: Found active lock with not reset cond %s: %s\n", + lock_type,where)); return 1; } prev= &data->next; @@ -160,6 +169,8 @@ static int check_lock(struct st_lock_lis { fprintf(stderr,"Warning: found too many locks at %s: %s\n", lock_type,where); + DBUG_PRINT("warning",("Warning: found too many locks at %s: %s\n", + lock_type,where)); return 1; } } @@ -167,6 +178,8 @@ static int check_lock(struct st_lock_lis { fprintf(stderr,"Warning: last didn't point at last lock at %s: %s\n", lock_type, where); + DBUG_PRINT("warning",("Warning: last didn't point at last lock at %s: %s\n", + lock_type, where)); return 1; } return 0; @@ -201,6 +214,8 @@ static void check_locks(THR_LOCK *lock, found_errors++; fprintf(stderr, "Warning at '%s': Locks read_no_write_count was %u when it should have been %u\n", where, lock->read_no_write_count,count); + DBUG_PRINT("warning",( + "Warning at '%s': Locks read_no_write_count was %u when it should have been %u\n", where, lock->read_no_write_count,count)); } if (!lock->write.data) @@ -212,6 +227,9 @@ static void check_locks(THR_LOCK *lock, fprintf(stderr, "Warning at '%s': No locks in use but locks are in wait queue\n", where); + DBUG_PRINT("warning",( + "Warning at '%s': No locks in use but locks are in wait queue\n", + where)); } if (!lock->write_wait.data) { @@ -221,6 +239,9 @@ static void check_locks(THR_LOCK *lock, fprintf(stderr, "Warning at '%s': No write locks and waiting read locks\n", where); + DBUG_PRINT("warning",( + "Warning at '%s': No write locks and waiting read locks\n", + where)); } } else @@ -236,6 +257,8 @@ static void check_locks(THR_LOCK *lock, found_errors++; fprintf(stderr, "Warning at '%s': Write lock %d waiting while no exclusive read locks\n",where,(int) lock->write_wait.data->type); + DBUG_PRINT("warning",( + "Warning at '%s': Write lock %d waiting while no exclusive read locks\n",where,(int) lock->write_wait.data->type)); } } } @@ -251,6 +274,9 @@ static void check_locks(THR_LOCK *lock, fprintf(stderr, "Warning at '%s': Found WRITE_ALLOW_WRITE lock waiting for WRITE_ALLOW_WRITE lock\n", where); + DBUG_PRINT("warning",( + "Warning at '%s': Found WRITE_ALLOW_WRITE lock waiting for WRITE_ALLOW_WRITE lock\n", + where)); } } if (lock->read.data) @@ -283,6 +309,11 @@ static void check_locks(THR_LOCK *lock, where, (int) lock->read_wait.data->type, (int) lock->write.data->type); + DBUG_PRINT("warning",( + "Warning at '%s': Found read lock of type %d waiting for write lock of type %d\n", + where, + (int) lock->read_wait.data->type, + (int) lock->write.data->type)); } } } @@ -371,6 +402,7 @@ static my_bool wait_for_lock(struct st_l pthread_cond_t *cond=get_cond(); struct st_my_thread_var *thread_var=my_thread_var; int result; + DBUG_ENTER("wait_for_lock"); if (!in_wait_list) { @@ -390,6 +422,8 @@ static my_bool wait_for_lock(struct st_l if (data->cond != cond) break; } + DBUG_PRINT("lock", ("aborted: %d changed cond: %d", + thread_var->abort, (data->cond != cond))); if (data->cond || data->type == TL_UNLOCK) { @@ -419,7 +453,7 @@ static my_bool wait_for_lock(struct st_l thread_var->current_mutex= 0; thread_var->current_cond= 0; pthread_mutex_unlock(&thread_var->mutex); - return result; + DBUG_RETURN(result); } --- 1.167/sql/ha_myisam.cc 2007-05-21 17:48:27 +02:00 +++ 1.168/sql/ha_myisam.cc 2007-05-21 17:48:27 +02:00 @@ -572,9 +572,11 @@ int ha_myisam::open(const char *name, in int ha_myisam::close(void) { - MI_INFO *tmp=file; + MI_INFO *myisam= file; + DBUG_ENTER("ha_myisam::close"); + file=0; - return mi_close(tmp); + DBUG_RETURN(mi_close(myisam)); } int ha_myisam::write_row(byte * buf) @@ -1819,3 +1821,20 @@ uint ha_myisam::checksum() const return (uint)file->s->state.checksum; } + +/** + @brief Extract the MyISAM table structure pointer from a handler object. + + @param[in] handler_handle pointer to handler object + + @return MyISAM table structure pointer + @retval NULL if handler has not a MyISAM table open +*/ + +MI_INFO *myisam_engine_handle(handler *handler_handle) +{ + DBUG_ENTER("myisam_engine_handle"); + if (strcmp(handler_handle->table_type(), "MyISAM")) + DBUG_RETURN(NULL); + DBUG_RETURN(((ha_myisam*) handler_handle)->file); +} --- 1.65/sql/ha_myisam.h 2007-05-21 17:48:27 +02:00 +++ 1.66/sql/ha_myisam.h 2007-05-21 17:48:27 +02:00 @@ -133,4 +133,5 @@ class ha_myisam: public handler int dump(THD* thd, int fd); int net_read_dump(NET* net); #endif + friend MI_INFO *myisam_engine_handle(handler *handler_handle); }; --- 1.65/sql/ha_myisammrg.cc 2007-05-21 17:48:27 +02:00 +++ 1.66/sql/ha_myisammrg.cc 2007-05-21 17:48:27 +02:00 @@ -14,6 +14,41 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +/* + Initially MERGE table children were opened by the parent via the + MyISAM storage engine interface directly. The problem was that the + children did not show up in the table cache. So many functions did + not notice the MyISAM tables that were in use through the MERGE table. + Also opening a MERGE table bypassed important table cache checks when + opening its children. + + A good fix would be to include the UNION list of the MERGE table in + its .frm file and open/close the children through the table cache. + However this would raise a bunch of migration problems. + + Now we use callbacks from the engine into its handler, which in turn + uses table cache functions to open/close the children. This way we + keep the old .MRG file and avoid migration problems. The .MRG file + contains the paths to the child tables. Instead of opening these paths + directly by mi_open(), we call a callback function, which extracts + database and table name from the path and calls open_table(). + + However, we open temporary tables the old way, directly with mi_open(). + We cannot reliably derive database and table name from the path name + of a temporary table. Since temporary tables are local to a thread + and are not locked, this is no problem. + + When we opened the children via a callback function, we must also + close them via a callback function. We use table cache functions to + close them. There are two forms of closing. One is a full close. We + do close the children and delete them from the table cahe in this case. + The other is a close for reopen. The storage engine tables are closed, + but the TABLE object remain in the table cache. This is like name + locking. No other thread can take the children until we finally + release them. If the MERGE table is to be deleted from the table cache + while it is in this state, the child TABLEs must be deleted outside + of the table handler (in closefrm()). +*/ #ifdef USE_PRAGMA_IMPLEMENTATION #pragma implementation // gcc: Class implementation @@ -54,6 +89,156 @@ const char *ha_myisammrg::index_type(uin } +/** + @brief Callback function for opening a MERGE child table. + + @detail When using this function instead of opening the MyISAM table + directly, the MyISAM table is opened through the table cache. In a + couple of situations the MERGE child table can be found through the + table cache as in use. This solves a couple of MERGE problems. + + This is called for each child table. It opens one table. + + @param[in] callback_param data pointer as given to myrg_open() + @param[in] filename file name of MyISAM table + without extension. + + @return pointer to opened MyISAM table structure + @retval NULL on error +*/ + +MI_INFO *myisammrg_open_callback(void *callback_param, char *filename) +{ + struct st_myrg_callback_param *open_param; + TABLE *parent; + TABLE *child; + MI_INFO *myisam; + char *table_name; + char *db; + uint dirlen; + char dir_path[FN_REFLEN]; + bool refresh; + DBUG_ENTER("myisammrg_open_callback"); + + /* Extract child table name and database name from filename. */ + dirlen= dirname_length(filename); + if (dirlen >= FN_REFLEN) + { + my_errno= ENAMETOOLONG; + DBUG_RETURN(NULL); + } + table_name= filename + dirlen; + dirlen--; /* Strip trailing '/'. */ + memcpy(dir_path, filename, dirlen); + dir_path[dirlen]= '\0'; + db= base_name(dir_path); + DBUG_PRINT("file", ("open db: '%s' table_name: '%s'", db, table_name)); + + /* Extract the MERGE table parent pointer from callback_param. */ + open_param= (struct st_myrg_callback_param*) callback_param; + parent= open_param->myisammrg->table_ptr(); + + /* + Open child table through table cache. If we find an "old" TABLE in + the cache, we must retry (see also open_ltable()). However, if a + FLUSH TABLES happened while we are opening the children, we need to + restart from the beginning. + */ + do + { + child= open_table(open_param->thd, db, table_name, + table_name, &refresh, 1); + } while (!child && refresh && (open_param->thd->version == refresh_version)); + + if (!child) + { + DBUG_PRINT("file", ("open table '%s'.'%s' failed", db, table_name)); + my_errno= HA_ERR_LOCK_DEADLOCK; + DBUG_RETURN(NULL); + } + DBUG_PRINT("file", ("opened table: '%s' 0x%lx", + child->table_name, (long) child)); + + /* Extract the MyISAM table structure pointer from the handler object. */ + if (!(myisam= myisam_engine_handle(child->file))) + { + DBUG_PRINT("file", ("no MyISAM handle for table: '%s' 0x%lx", + child->table_name, (long) child)); + child->version= 0; /* Force deletion from cache. */ + VOID(close_thread_table(open_param->thd, &child)); + my_errno= HA_ERR_WRONG_MRG_TABLE_DEF; + } + else + { + DBUG_ASSERT(!child->mrg_parent && !child->mrg_child); + /* Link parent table to child. */ + child->mrg_parent= parent; + /* Link child table into child chain of parent. */ + child->mrg_child= parent->mrg_child; + parent->mrg_child= child; + } + DBUG_PRINT("file", ("MyISAM handle: 0x%lx my_errno: %d", + (long) myisam, my_errno)); + DBUG_RETURN(myisam); +} + + +/** + @brief Callback function for closing a MERGE child table. + + @detail If the MERGE child tables were opened through + myisammrg_open_callback(), they must be closed through + myisammrg_close_callback(). Both function pointers as well as the data + pointer 'callback_param' are given to myrg_open(). + + This is called once only and closes all child tables. + + @param[in] callback_param data pointer as given to myrg_open() +*/ + +void myisammrg_close_callback(void *callback_param) +{ + struct st_myrg_callback_param *open_param; + TABLE *parent; + TABLE *child; + TABLE *next; + DBUG_ENTER("myisammrg_close_callback"); + + /* Extract the MERGE table parent pointer from callback_param. */ + open_param= (struct st_myrg_callback_param*) callback_param; + parent= open_param->myisammrg->table_ptr(); + DBUG_PRINT("file", ("closing_for_reopen: %d table: '%s' 0x%lx", + parent->closing_for_reopen, parent->table_name, + (long) parent)); + + /* Close all tables from the child chain. */ + for (child= parent->mrg_child; child; child= next) + { + /* Remember next child before mangling the TABLE object. */ + next= child->mrg_child; + + /* Force close or deletion from cache respectively. */ + child->version= 0; + if (parent->closing_for_reopen) + { + /* Keep the parent-child relationship. */ + close_old_data_files(open_param->thd, child, 0, 0); + } + else + { + /* Detach child from parent. */ + child->mrg_parent= NULL; + child->mrg_child= NULL; + parent->mrg_child= NULL; + /* Close and delete child through table cache. */ + VOID(close_thread_table(open_param->thd, &child)); + } + } + + DBUG_VOID_RETURN; +}; + + int ha_myisammrg::open(const char *name, int mode, uint test_if_locked) { MI_KEYDEF *keyinfo; @@ -63,15 +248,31 @@ int ha_myisammrg::open(const char *name, uint keys= table->keys; int error; char name_buff[FN_REFLEN]; - - DBUG_PRINT("info", ("ha_myisammrg::open")); - if (!(file=myrg_open(fn_format(name_buff,name,"","",2 | 4), mode, - test_if_locked))) + DBUG_ENTER("ha_myisammrg::open"); + DBUG_PRINT("enter", ("name: '%s' mode: %d test_if_locked: %u", + name, mode, test_if_locked)); + + /* callback_param is part of the ha_myisammrg object. */ + callback_param.thd= current_thd; + callback_param.myisammrg= this; + /* + Temporary tables are not opened through the table cache. They are + local to the thread only. So there is no need to let the table cache + know about them. Supply callback information only to non-temporary + tables. Include close callback here so that we do not need to make + the decision about temporary in close handling again. + */ + if (!(file= (test_if_locked & HA_OPEN_TMP_TABLE) ? + myrg_open(fn_format(name_buff,name,"","",2 | 4), mode, test_if_locked, + NULL, NULL, NULL) : + myrg_open(fn_format(name_buff,name,"","",2 | 4), mode, test_if_locked, + myisammrg_open_callback, myisammrg_close_callback, + &callback_param))) { - DBUG_PRINT("info", ("ha_myisammrg::open exit %d", my_errno)); - return (my_errno ? my_errno : -1); + DBUG_PRINT("exit", ("my_errno %d", my_errno)); + DBUG_RETURN(my_errno ? my_errno : -1); } - DBUG_PRINT("info", ("ha_myisammrg::open myrg_extrafunc...")) + DBUG_PRINT("info", ("calling myrg_extrafunc...")) myrg_extrafunc(file, query_cache_invalidate_by_MyISAM_filename_ref); if (!(test_if_locked == HA_OPEN_WAIT_IF_LOCKED || test_if_locked == HA_OPEN_ABORT_IF_LOCKED)) @@ -116,16 +317,32 @@ int ha_myisammrg::open(const char *name, goto err; } #endif - return (0); + DBUG_RETURN(0); err: myrg_close(file); file=0; - return (my_errno= error); + DBUG_RETURN(my_errno= error); } int ha_myisammrg::close(void) { - return myrg_close(file); + int error; + DBUG_ENTER("ha_myisammrg::close"); + + DBUG_ASSERT(callback_param.myisammrg == this); + /* + We store the information about the close callback in the storage + engine table object. So we do not need to decide here if we have + a temporary table. + */ + error= myrg_close(file); + /* + It is possible that the table is closed, but ha_myisammrg kept. + Avoid any attempt to access the closed engine table. + See lock_count() and store_locks(). + */ + file= NULL; + DBUG_RETURN(error); } int ha_myisammrg::write_row(byte * buf) @@ -335,7 +552,11 @@ int ha_myisammrg::external_lock(THD *thd uint ha_myisammrg::lock_count(void) const { - return file->tables; + /* + It is possible that the table is closed, but ha_myisammrg kept. + See close(). + */ + return file ? file->tables : 0; } @@ -344,6 +565,13 @@ THR_LOCK_DATA **ha_myisammrg::store_lock enum thr_lock_type lock_type) { MYRG_TABLE *open_table; + + /* + It is possible that the table is closed, but ha_myisammrg kept. + See close(). + */ + if (!file) + return to; for (open_table=file->open_tables ; open_table != file->end_table ; --- 1.40/sql/ha_myisammrg.h 2007-05-21 17:48:27 +02:00 +++ 1.41/sql/ha_myisammrg.h 2007-05-21 17:48:27 +02:00 @@ -19,6 +19,13 @@ #pragma interface /* gcc class implementation */ #endif +/* Data to be passed to open_callback() and close_callback(). */ +struct st_myrg_callback_param +{ + THD *thd; + class ha_myisammrg *myisammrg; +}; + /* class for the the myisam merge handler */ #include <myisammrg.h> @@ -26,6 +33,7 @@ class ha_myisammrg: public handler { MYRG_INFO *file; + struct st_myrg_callback_param callback_param; /* For open/close callbacks */ public: ha_myisammrg(TABLE *table): handler(table), file(0) {} @@ -82,4 +90,5 @@ class ha_myisammrg: public handler void update_create_info(HA_CREATE_INFO *create_info); void append_create_info(String *packet); MYRG_INFO *myrg_info() { return file; } + TABLE *table_ptr() { return table; } }; --- 1.68/sql/lock.cc 2007-05-21 17:48:27 +02:00 +++ 1.69/sql/lock.cc 2007-05-21 17:48:27 +02:00 @@ -111,6 +111,8 @@ MYSQL_LOCK *mysql_lock_tables(THD *thd, if (! (sql_lock= get_lock_data(thd, tables, count, GET_LOCK_STORE_LOCKS, &write_lock_used))) break; + DBUG_PRINT("lock", ("lock sql_lock: 0x%lx count: %u", + (long) sql_lock, sql_lock->lock_count)); if (global_read_lock && write_lock_used && ! (flags & MYSQL_LOCK_IGNORE_GLOBAL_READ_LOCK)) @@ -173,6 +175,8 @@ MYSQL_LOCK *mysql_lock_tables(THD *thd, thd->proc_info=0; /* some table was altered or deleted. reopen tables marked deleted */ + DBUG_PRINT("lock", ("unlock sql_lock: 0x%lx count: %u", + (long) sql_lock, sql_lock->lock_count)); mysql_unlock_tables(thd,sql_lock); thd->locked=0; retry: @@ -234,6 +238,8 @@ static int lock_external(THD *thd, TABLE void mysql_unlock_tables(THD *thd, MYSQL_LOCK *sql_lock) { DBUG_ENTER("mysql_unlock_tables"); + DBUG_PRINT("lock", ("unlock sql_lock: 0x%lx count: %u", + (long) sql_lock, sql_lock->lock_count)); if (sql_lock->lock_count) thr_multi_unlock(sql_lock->locks,sql_lock->lock_count); if (sql_lock->table_count) @@ -251,9 +257,12 @@ void mysql_unlock_some_tables(THD *thd, { MYSQL_LOCK *sql_lock; TABLE *write_lock_used; + DBUG_ENTER("mysql_unlock_some_tables"); + if ((sql_lock= get_lock_data(thd, table, count, GET_LOCK_UNLOCK, &write_lock_used))) mysql_unlock_tables(thd, sql_lock); + DBUG_VOID_RETURN; } @@ -321,6 +330,7 @@ void mysql_unlock_read_tables(THD *thd, void mysql_lock_remove(THD *thd, MYSQL_LOCK *locked,TABLE *table) { + DBUG_ENTER("mysql_lock_remove"); mysql_unlock_some_tables(thd, &table,1); if (locked) { @@ -375,6 +385,7 @@ void mysql_lock_remove(THD *thd, MYSQL_L } } } + DBUG_VOID_RETURN; } /* abort all other threads waiting to get lock in table */ @@ -383,6 +394,8 @@ void mysql_lock_abort(THD *thd, TABLE *t { MYSQL_LOCK *locked; TABLE *write_lock_used; + DBUG_ENTER("mysql_lock_abort"); + if ((locked= get_lock_data(thd, &table, 1, GET_LOCK_UNLOCK, &write_lock_used))) { @@ -390,6 +403,7 @@ void mysql_lock_abort(THD *thd, TABLE *t thr_abort_locks(locked->locks[i]->lock); my_free((gptr) locked,MYF(0)); } + DBUG_VOID_RETURN; } @@ -667,7 +681,12 @@ static MYSQL_LOCK *get_lock_data(THD *th *to++= table; if (locks) for ( ; org_locks != locks ; org_locks++) + { (*org_locks)->debug_print_param= (void *) table; + DBUG_PRINT("lock", ("table: 0x%lx '%s' lock_count: %u data: 0x%lx", + (long) table, table->table_name, + table->lock_count, (long) *org_locks)); + } } DBUG_RETURN(sql_lock); } @@ -830,6 +849,8 @@ int lock_table_name(THD *thd, TABLE_LIST my_free((gptr) table,MYF(0)); DBUG_RETURN(-1); } + DBUG_PRINT("exit", ("locked by name table: '%s' 0x%lx", + table->table_name, (long) table)); if (remove_table_from_cache(thd, db, table_list->real_name, RTFC_NO_FLAG)) { @@ -851,12 +872,19 @@ void unlock_table_name(THD *thd, TABLE_L static bool locked_named_table(THD *thd, TABLE_LIST *table_list) { + DBUG_ENTER("locked_named_table"); for (; table_list ; table_list=table_list->next) { + DBUG_PRINT("lock", ("check locked_named_table: '%s'", + table_list->real_name)); if (table_list->table && table_is_used(table_list->table,0)) - return 1; + { + DBUG_PRINT("lock", ("yes")); + DBUG_RETURN(1); + } } - return 0; // All tables are locked + DBUG_PRINT("lock", ("no locked_named_table")); + DBUG_RETURN(0); // All tables are locked } --- 1.388/sql/mysql_priv.h 2007-05-21 17:48:27 +02:00 +++ 1.389/sql/mysql_priv.h 2007-05-21 17:48:27 +02:00 @@ -621,8 +621,8 @@ int mysql_delete(THD *thd, TABLE_LIST *t ha_rows rows, ulong options); int mysql_truncate(THD *thd, TABLE_LIST *table_list, bool dont_send_ok); TABLE *open_ltable(THD *thd, TABLE_LIST *table_list, thr_lock_type update); -TABLE *open_table(THD *thd,const char *db,const char *table,const char *alias, - bool *refresh); +TABLE *open_table(THD *thd, const char *db, const char *table, + const char *alias, bool *refresh, bool is_child); TABLE *reopen_name_locked_table(THD* thd, TABLE_LIST* table); TABLE *find_locked_table(THD *thd, const char *db,const char *table_name); bool reopen_table(TABLE *table,bool locked=0); @@ -773,7 +773,8 @@ int open_normal_and_derived_tables(THD * void relink_tables_for_derived(THD *thd); int lock_tables(THD *thd, TABLE_LIST *tables, uint counter); TABLE *open_temporary_table(THD *thd, const char *path, const char *db, - const char *table_name, bool link_in_list); + const char *table_name, bool link_in_list, + bool is_temporary); bool rm_temporary_table(enum db_type base, char *path); void free_io_cache(TABLE *entry); void intern_close_table(TABLE *entry); @@ -1331,3 +1332,5 @@ bool check_stack_overrun(THD *thd,char * inline void kill_delayed_threads(void) {} #define check_stack_overrun(A, B) 0 #endif + +MI_INFO *myisam_engine_handle(handler *handler_handle); --- 1.276/sql/sql_base.cc 2007-05-21 17:48:27 +02:00 +++ 1.277/sql/sql_base.cc 2007-05-21 17:48:27 +02:00 @@ -17,6 +17,36 @@ /* Basic functions needed by many modules */ +/* + MERGE table handling. + + MERGE table children are now opened through the table cache. When the + parent is opened, the storage engine calls back for every child so that + it can be opened by open_table(). The children are put in a chain + [parent]TABLE::mrg_child -> [child]TABLE::mrg_child -> [child]TABLE. + Each child table references the parent table [child]TABLE::mrg_parent -> + [parent]TABLE. The children are *not* added to thd->open_tables. + + At statement end the MERGE parent table is put to the unused_tables + chain like other tables, but the children are not put in that chain. + They remain in their parents mrg_child chain. TABLE::in_use is cleared + for parent and children. + + In some comments below I call MERGE children that are not in use, but + also not in unused_tables "floating" tables. They can be found by a + hash search on the table cache, but not by following the + thd->open_tables or unused_tables chains. + + When the MERGE table is reused, TABLE::in_use is set for it and its + children. + + When the MERGE table is deleted from the table cache, the children + are also deleted. + + When a child is to be deleted, the parent and all of its children are + deleted. +*/ + #include "mysql_priv.h" #include "sql_select.h" #include <m_ctype.h> @@ -87,10 +117,12 @@ static void check_unused(void) DBUG_PRINT("error",("Unused_links aren't connected")); /* purecov: inspected */ } } + /* Count open tables in the table cache. */ for (idx=0 ; idx < open_cache.records ; idx++) { TABLE *entry=(TABLE*) hash_element(&open_cache,idx); - if (!entry->in_use) + /* Skip unused tables and MERGE child tables. */ + if (!entry->in_use && !entry->mrg_parent) count--; } if (count != 0) @@ -162,7 +194,8 @@ OPEN_TABLE_LIST *list_open_tables(THD *t if (!strcmp(table->table,entry->real_name) && !strcmp(table->db,entry->table_cache_key)) { - if (entry->in_use) + /* Treat MERGE child tables as used. */ + if (entry->in_use || entry->mrg_parent) table->in_use++; if (entry->locked_by_name) table->locked++; @@ -181,7 +214,8 @@ OPEN_TABLE_LIST *list_open_tables(THD *t strmov(((*start_list)->db= (char*) ((*start_list)+1)), entry->table_cache_key)+1, entry->real_name); - (*start_list)->in_use= entry->in_use ? 1 : 0; + /* Treat MERGE child tables as used. */ + (*start_list)->in_use= (entry->in_use || entry->mrg_parent) ? 1 : 0; (*start_list)->locked= entry->locked_by_name ? 1 : 0; start_list= &(*start_list)->next; *start_list=0; @@ -194,12 +228,15 @@ OPEN_TABLE_LIST *list_open_tables(THD *t * Functions to free open table cache ****************************************************************************/ - void intern_close_table(TABLE *table) { // Free all structures + DBUG_ENTER("intern_close_table"); + DBUG_PRINT("table", ("table: '%s' 0x%lx", table->table_name, (long) table)); + free_io_cache(table); if (table->file) VOID(closefrm(table)); // close file + DBUG_VOID_RETURN; } /* @@ -216,10 +253,11 @@ void intern_close_table(TABLE *table) static void free_cache_entry(TABLE *table) { DBUG_ENTER("free_cache_entry"); + DBUG_PRINT("table", ("table: '%s' 0x%lx", table->table_name, (long) table)); safe_mutex_assert_owner(&LOCK_open); intern_close_table(table); - if (!table->in_use) + if (!table->in_use && table->next) /* MERGE children are in no chain. */ { table->next->prev=table->prev; /* remove from used chain */ table->prev->next=table->next; @@ -235,6 +273,7 @@ static void free_cache_entry(TABLE *tabl DBUG_VOID_RETURN; } + /* Free resources allocated by filesort() and read_record() */ void free_io_cache(TABLE *table) @@ -276,6 +315,42 @@ bool close_cached_tables(THD *thd, bool #endif } refresh_version++; // Force close of open tables + DBUG_PRINT("table", ("incremented global refresh_version to: %lu", + refresh_version)); + if (if_wait_for_refresh) + { + /* + Other threads could wait in a loop in mysql_lock_table() trying + to lock one or more of our tables. + + If they wait for the locks in thr_multi_lock(), their lock + request is aborted. If they wait for the table to become unused, + they see table->locked_by_flush=1. But if they passed the loop + in wait_for_tables(), released LOCK_open, we (the FLUSH TABLES + thread) are scheduled at this moment and enter + close_cached_tables(), they could awake while we sleep below, + waiting for others threads to close their open tables. If this + happens, the other threads would find the tables unlocked. They + would get the locks, one after the other, and could do their + destructive work. This is an issue if we have LOCK TABLES in + effect. + + The fix for this problem is to set some_tables_deleted for all + threads with open tables. These threads can still get their locks, + but will immediately release them again after checking this + variable. They will then reenter wait_for_tables(). There they + will wait until we update all tables version below. + + Setting some_tables_deleted is done by remove_table_from_cache() + in the other branch. + */ + for (uint idx=0 ; idx < open_cache.records ; idx++) + { + TABLE *table=(TABLE*) hash_element(&open_cache,idx); + if (table->in_use) + table->in_use->some_tables_deleted= 1; + } + } } else { @@ -400,11 +475,12 @@ void close_thread_tables(THD *thd, bool VOID(pthread_mutex_lock(&LOCK_open)); safe_mutex_assert_owner(&LOCK_open); - DBUG_PRINT("info", ("thd->open_tables=%p", thd->open_tables)); - - found_old_table= 0; + found_old_table= 0; while (thd->open_tables) + { + DBUG_PRINT("info", ("close open_tables: 0x%lx", (long) thd->open_tables)); found_old_table|=close_thread_table(thd, &thd->open_tables); + } thd->some_tables_deleted=0; /* Free tables to hold down open files */ @@ -422,22 +498,43 @@ void close_thread_tables(THD *thd, bool DBUG_VOID_RETURN; } -/* move one table to free list */ + +/* + @brief Move one table to free list or delete from table cache. + + @detail If table or thread is "old" (version != refresh_version) or + table is not open, then delete table from cache. Otherwise move it to + the unused_tables chain. + + If thd == NULL, assume that table needs to be deleted from table + cache. + + @param[in] thd Thread handle, may be NULL. + @param[in,out] table_ptr Pointer to table pointer. Will be + replaced by next TABLE in list. + + @return if table was "old" + @retval TRUE Table was "old". It is now deleted. + @retval FALSE Table was up-to-date. It is now in unused_tables. +*/ bool close_thread_table(THD *thd, TABLE **table_ptr) { - DBUG_ENTER("close_thread_table"); - bool found_old_table= 0; TABLE *table= *table_ptr; + DBUG_ENTER("close_thread_table"); + DBUG_PRINT("info", ("closing table: '%s' 0x%lx version: %lu", + table->table_name, (long) table, table->version)); + DBUG_ASSERT(table->key_read == 0); *table_ptr=table->next; if (table->version != refresh_version || - thd->version != refresh_version || !table->db_stat) + !thd || thd->version != refresh_version || !table->db_stat) { VOID(hash_delete(&open_cache,(byte*) table)); found_old_table=1; + DBUG_PRINT("info", ("deleted from cache")); } else { @@ -452,15 +549,40 @@ bool close_thread_table(THD *thd, TABLE table->file->reset(); } table->in_use=0; - if (unused_tables) + /* + Do not add MERGE child tables to unused_tables. Unused tables may + be deleted at any time. As long as a MERGE parent table is not + deleted, it will need its children when it is reopened. Since we + make the children kind of "floating" tables, we need to delete + them when the parent table is deleted. + */ + if (table->mrg_parent) { - table->next=unused_tables; /* Link in last */ - table->prev=unused_tables->prev; - unused_tables->prev=table; - table->prev->next=table; + DBUG_PRINT("info", ("set floating merge child")); + table->next= table->prev= table; } else - unused_tables=table->next=table->prev=table; + { + DBUG_PRINT("info", ("set unused")); + if (unused_tables) + { + table->next=unused_tables; /* Link in last */ + table->prev=unused_tables->prev; + unused_tables->prev=table; + table->prev->next=table; + } + else + unused_tables=table->next=table->prev=table; + } + /* If the table has children, clear their in_use too.*/ + for (TABLE *child= table->mrg_child; child; child= child->mrg_child) + { + DBUG_PRINT("info", ("set floating merge child: '%s' 0x%lx", + child->table_name, (long) child)); + DBUG_ASSERT(!thd || child->in_use == thd); + DBUG_ASSERT(child->mrg_parent == table); + child->in_use= NULL; + } } DBUG_RETURN(found_old_table); } @@ -758,6 +880,8 @@ TABLE *unlink_open_table(THD *thd, TABLE char key[MAX_DBKEY_LENGTH]; uint key_length=find->key_length; TABLE *start=list,**prev,*next; + DBUG_ENTER("unlink_open_table"); + prev= &start; memcpy(key,find->table_cache_key,key_length); for (; list ; list=next) @@ -779,7 +903,7 @@ TABLE *unlink_open_table(THD *thd, TABLE *prev=0; // Notify any 'refresh' threads pthread_cond_broadcast(&COND_refresh); - return start; + DBUG_RETURN(start); } @@ -790,6 +914,7 @@ TABLE *unlink_open_table(THD *thd, TABLE void wait_for_refresh(THD *thd) { + DBUG_ENTER("wait_for_refresh"); safe_mutex_assert_owner(&LOCK_open); /* Wait until the current table is up to date */ @@ -799,14 +924,18 @@ void wait_for_refresh(THD *thd) proc_info=thd->proc_info; thd->proc_info="Waiting for table"; if (!thd->killed) + { + DBUG_PRINT("pthreads", ("waiting for refresh")); (void) pthread_cond_wait(&COND_refresh,&LOCK_open); - + DBUG_PRINT("pthreads", ("awoke from refresh")); + } pthread_mutex_unlock(&LOCK_open); // Must be unlocked first pthread_mutex_lock(&thd->mysys_var->mutex); thd->mysys_var->current_mutex= 0; thd->mysys_var->current_cond= 0; thd->proc_info= proc_info; pthread_mutex_unlock(&thd->mysys_var->mutex); + DBUG_VOID_RETURN; } @@ -843,6 +972,8 @@ TABLE *reopen_name_locked_table(THD* thd pthread_mutex_unlock(&LOCK_open); table->next = thd->open_tables; thd->open_tables = table; + DBUG_PRINT("table", ("put into open_tables: '%s' 0x%lx", + table->table_name, (long) table)); table->tablenr=thd->current_tablenr++; table->used_fields=0; table->const_table=0; @@ -854,24 +985,37 @@ TABLE *reopen_name_locked_table(THD* thd } -/****************************************************************************** -** open a table -** Uses a cache of open tables to find a table not in use. -** If refresh is a NULL pointer, then the is no version number checking and -** the table is not put in the thread-open-list -** If the return value is NULL and refresh is set then one must close -** all tables and retry the open -******************************************************************************/ +/** + @brief open a table + @detail Uses a cache of open tables to find a table not in use. If + refresh is a NULL pointer, then there is no version number checking + and the table is not put in the thread-open-list. + + If the return value is NULL and refresh is set then one must close + all tables and retry the open. + + @param[in] thd thread handle + @param[in] db database name + @param[in] table_name table name + @param[in] alias table alias + @param[in,out] refresh pointer to refresh flag, may be NULL + @param[in] is_child if opening a MERGE child table + if TRUE, LOCK_open must be locked before + + @return pointer to the opened TABLE object + @retval NULL on error +*/ -TABLE *open_table(THD *thd,const char *db,const char *table_name, - const char *alias,bool *refresh) +TABLE *open_table(THD *thd, const char *db, const char *table_name, + const char *alias, bool *refresh, bool is_child) { reg1 TABLE *table; char key[MAX_DBKEY_LENGTH]; uint key_length; HASH_SEARCH_STATE state; DBUG_ENTER("open_table"); + DBUG_PRINT("enter", ("db: '%s' table_name: '%s'", db, table_name)); /* find a unused table in the open table cache */ if (refresh) @@ -882,45 +1026,52 @@ TABLE *open_table(THD *thd,const char *d int4store(key + key_length, thd->server_id); int4store(key + key_length + 4, thd->variables.pseudo_thread_id); - for (table=thd->temporary_tables; table ; table=table->next) + if (!is_child) { - if (table->key_length == key_length + TMP_TABLE_KEY_EXTRA && - !memcmp(table->table_cache_key, key, - key_length + TMP_TABLE_KEY_EXTRA)) - { - if (table->query_id == thd->query_id) - { - my_printf_error(ER_CANT_REOPEN_TABLE, - ER(ER_CANT_REOPEN_TABLE),MYF(0),table->table_name); - DBUG_RETURN(0); - } - table->query_id=thd->query_id; - table->clear_query_id=1; - thd->tmp_table_used= 1; - DBUG_PRINT("info",("Using temporary table")); - goto reset; + for (table=thd->temporary_tables; table ; table=table->next) + { + if (table->key_length == key_length + TMP_TABLE_KEY_EXTRA && + !memcmp(table->table_cache_key, key, + key_length + TMP_TABLE_KEY_EXTRA)) + { + if (table->query_id == thd->query_id) + { + my_printf_error(ER_CANT_REOPEN_TABLE, + ER(ER_CANT_REOPEN_TABLE),MYF(0),table->table_name); + DBUG_RETURN(0); + } + table->query_id=thd->query_id; + table->clear_query_id=1; + thd->tmp_table_used= 1; + DBUG_PRINT("info", ("Using temporary table: 0x%lx", (long) table)); + goto reset; + } } - } - if (thd->locked_tables) - { // Using table locks - for (table=thd->open_tables; table ; table=table->next) - { - if (table->key_length == key_length && - !memcmp(table->table_cache_key,key,key_length) && - !my_strcasecmp(system_charset_info, table->table_name, alias) && - table->query_id != thd->query_id) + if (thd->locked_tables) + { // Using table locks + for (table=thd->open_tables; table ; table=table->next) { - table->query_id=thd->query_id; - DBUG_PRINT("info",("Using locked table")); - goto reset; + if (table->key_length == key_length && + !memcmp(table->table_cache_key,key,key_length) && + !my_strcasecmp(system_charset_info, table->table_name, alias) && + table->query_id != thd->query_id) + { + table->query_id=thd->query_id; + DBUG_PRINT("info", ("Using locked table: 0x%lx", (long) table)); + goto reset; + } } + my_printf_error(ER_TABLE_NOT_LOCKED,ER(ER_TABLE_NOT_LOCKED),MYF(0),alias); + DBUG_RETURN(0); } - my_printf_error(ER_TABLE_NOT_LOCKED,ER(ER_TABLE_NOT_LOCKED),MYF(0),alias); - DBUG_RETURN(0); - } - VOID(pthread_mutex_lock(&LOCK_open)); + VOID(pthread_mutex_lock(&LOCK_open)); + } + /* + A MERGE child of an intermediate table (e.g. ALTER TABLE) is opened + without LOCK_open. So don't safe_mutex_assert_owner(&LOCK_open). + */ if (!thd->open_tables) thd->version=refresh_version; @@ -928,21 +1079,32 @@ TABLE *open_table(THD *thd,const char *d { /* Someone did a refresh while thread was opening tables */ *refresh=1; - VOID(pthread_mutex_unlock(&LOCK_open)); + if (!is_child) + VOID(pthread_mutex_unlock(&LOCK_open)); + DBUG_PRINT("info", ("Thread needs refresh")); DBUG_RETURN(0); } /* close handler tables which are marked for flush */ mysql_ha_flush(thd, (TABLE_LIST*) NULL, MYSQL_HA_REOPEN_ON_USAGE, TRUE); + /* + Find an unused table. But do not use a MERGE child. + While stepping through the cache, refresh "old" tables. But stop this + when an unused table is found. + */ for (table= (TABLE*) hash_first(&open_cache, (byte*) key, key_length, &state); - table && table->in_use ; + table && (table->in_use || table->mrg_parent); table= (TABLE*) hash_next(&open_cache, (byte*) key, key_length, &state)) { - if (table->version != refresh_version) + /* Do not refresh MERGE children. They are refreshed with their parent. */ + if ((table->version != refresh_version) && !table->mrg_parent) { + DBUG_PRINT("info", ("Version mismatch, table: 0x%lx version: %lu " + "refresh_version: %lu", (long) table, + table->version, refresh_version)); if (! refresh) { /* Ignore flush for now, but force close after usage. */ @@ -956,9 +1118,13 @@ TABLE *open_table(THD *thd,const char *d */ close_old_data_files(thd,thd->open_tables,0,0); if (table->in_use != thd) + { wait_for_refresh(thd); - else - VOID(pthread_mutex_unlock(&LOCK_open)); + if (is_child) + VOID(pthread_mutex_lock(&LOCK_open)); + } + else if (!is_child) + VOID(pthread_mutex_unlock(&LOCK_open)); if (refresh) *refresh=1; DBUG_RETURN(0); @@ -975,6 +1141,7 @@ TABLE *open_table(THD *thd,const char *d table->prev->next=table->next; /* Remove from unused list */ table->next->prev=table->prev; + DBUG_PRINT("info", ("Unused table: 0x%lx", (long) table)); } else { @@ -985,7 +1152,8 @@ TABLE *open_table(THD *thd,const char *d /* make a new table */ if (!(table=(TABLE*) my_malloc(sizeof(*table),MYF(MY_WME)))) { - VOID(pthread_mutex_unlock(&LOCK_open)); + if (!is_child) + VOID(pthread_mutex_unlock(&LOCK_open)); DBUG_RETURN(NULL); } if (open_unireg_entry(thd, table,db,table_name,alias) || @@ -994,24 +1162,45 @@ TABLE *open_table(THD *thd,const char *d { table->next=table->prev=table; free_cache_entry(table); - VOID(pthread_mutex_unlock(&LOCK_open)); + if (!is_child) + VOID(pthread_mutex_unlock(&LOCK_open)); DBUG_RETURN(NULL); } table->key_length=key_length; table->version=refresh_version; table->flush_version=flush_version; - DBUG_PRINT("info", ("inserting table %p into the cache", table)); + DBUG_PRINT("info", ("New table 0x%lx in cache", (long) table)); VOID(my_hash_insert(&open_cache,(byte*) table)); } table->in_use=thd; + + /* If the table has children, set their in_use too. */ + for (TABLE *child= table->mrg_child; child; child= child->mrg_child) + { + /* + In case of new opened MERGE table, children are already set up + through open_unireg_entry(). + */ + DBUG_ASSERT(!child->in_use || (child->in_use == thd)); + DBUG_ASSERT(child->mrg_parent == table); + child->in_use= thd; + } check_unused(); // Debugging call - - VOID(pthread_mutex_unlock(&LOCK_open)); - if (refresh) + + /* Do not add MERGE children to open_tables. */ + if (is_child) + table->next= NULL; + else { - table->next=thd->open_tables; /* Link into simple list */ - thd->open_tables=table; + VOID(pthread_mutex_unlock(&LOCK_open)); + if (refresh) + { + table->next=thd->open_tables; /* Link into simple list */ + thd->open_tables=table; + DBUG_PRINT("table", ("put into open_tables: '%s' 0x%lx", + table->table_name, (long) table)); + } } table->reginfo.lock_type=TL_READ; /* Assume read */ @@ -1065,6 +1254,8 @@ TABLE *open_table(THD *thd,const char *d table->timestamp_field_type= table->timestamp_field->get_auto_set_type(); DBUG_ASSERT(table->key_read == 0); DBUG_ASSERT(table->insert_values == 0); + DBUG_PRINT("exit", ("Opened table: '%s' 0x%lx version: %lu", + table->table_name, (long) table, table->version)); DBUG_RETURN(table); } @@ -1120,6 +1311,10 @@ bool reopen_table(TABLE *table,bool lock VOID(pthread_mutex_lock(&LOCK_open)); safe_mutex_assert_owner(&LOCK_open); + /* + Open a new table in the automatic (stack) variable'tmp'. In case of + a MERGE table, it creates a new set of child tables. + */ if (open_unireg_entry(current_thd,&tmp,db,table_name,table->table_name)) goto end; free_io_cache(table); @@ -1155,11 +1350,23 @@ bool reopen_table(TABLE *table,bool lock tmp.next= table->next; tmp.prev= table->prev; + /* + Close old table. In case of a MERGE table, close (and delete) old + children too. + */ if (table->file) VOID(closefrm(table)); // close file, free everything - *table=tmp; + /* + Copy new TABLE structure to old TABLE structure, reusing the old + (non-automatic) storage object. + */ + *table= tmp; + /* Fix table pointer in handler object. */ table->file->change_table_ptr(table); + /* Fix parent pointers in the new child tables. */ + for (TABLE *child= table->mrg_child; child; child= child->mrg_child) + child->mrg_parent= table; DBUG_ASSERT(table->table_name); for (field=table->field ; *field ; field++) @@ -1172,6 +1379,8 @@ bool reopen_table(TABLE *table,bool lock for (part=0 ; part < table->key_info[key].usable_key_parts ; part++) table->key_info[key].key_part[part].field->table= table; } + DBUG_PRINT("exit", ("Reopened table: '%s' 0x%lx version: %lu", + table->table_name, (long) table, table->version)); VOID(pthread_cond_broadcast(&COND_refresh)); error=0; @@ -1197,7 +1406,12 @@ bool close_data_tables(THD *thd,const ch !strcmp(table->table_cache_key,db)) { mysql_lock_remove(thd, thd->locked_tables,table); + /* Close storage engine tables, but keep TABLE object. */ + DBUG_PRINT("table", ("closing for reopen table: '%s' 0x%lx", + table->table_name, (long) table)); + table->closing_for_reopen= TRUE; table->file->close(); + table->closing_for_reopen= FALSE; table->db_stat=0; } } @@ -1212,15 +1426,15 @@ bool close_data_tables(THD *thd,const ch bool reopen_tables(THD *thd,bool get_locks,bool in_refresh) { + TABLE *table, *next, **prev; + TABLE **tables, **tables_ptr; // For locks + bool error= 0; DBUG_ENTER("reopen_tables"); safe_mutex_assert_owner(&LOCK_open); if (!thd->open_tables) DBUG_RETURN(0); - TABLE *table,*next,**prev; - TABLE **tables,**tables_ptr; // For locks - bool error=0; if (get_locks) { /* The ptr is checked later */ @@ -1230,7 +1444,9 @@ bool reopen_tables(THD *thd,bool get_loc } else tables= &thd->open_tables; - tables_ptr =tables; + tables_ptr= tables; + /* As we are going to reopen all tables, we declare thread as up-to-date. */ + thd->version= refresh_version; prev= &thd->open_tables; for (table=thd->open_tables; table ; table=next) @@ -1251,7 +1467,20 @@ bool reopen_tables(THD *thd,bool get_loc *tables_ptr++= table; // need new lock on this if (in_refresh) { - table->version=0; + /* + Formerly we set table->version=0; here. I do not know what the + sense was to clear the freshly assigned version number of a + freshly reopened table. I just can say that it made a crash + when reopening the same table twice in a thread. For example + as a MyISAM table and as a MERGE child table. The table that + was reopend as the second one closed the previously reopend + table because of its zero version. table->file became NULL + again. But there is no means to reopen an already reopened + table again. The code after reopen_tables() assumed that all + tables are open and happily called table->file->lock_count()... + Removing table->version=0; fixed the problem and does not seem + to have a negative impact. + */ table->locked_by_flush=0; } } @@ -1290,6 +1519,9 @@ void close_old_data_files(THD *thd, TABL bool found=send_refresh; for (; table ; table=table->next) { + DBUG_PRINT("table", ("table: '%s' 0x%lx version: %lu db_stat: %u", + table->table_name, (long) table, table->version, + table->db_stat)); if (table->version != refresh_version) { found=1; @@ -1303,8 +1535,13 @@ void close_old_data_files(THD *thd, TABL mysql_lock_remove(thd, thd->locked_tables,table); table->locked_by_flush=1; // Will be reopened with locks } - table->file->close(); - table->db_stat=0; + /* Close storage engine tables, but keep TABLE object. */ + DBUG_PRINT("table", ("closing for reopen table: '%s' 0x%lx", + table->table_name, (long) table)); + table->closing_for_reopen= TRUE; + table->file->close(); + table->closing_for_reopen= FALSE; + table->db_stat=0; } } } @@ -1314,6 +1551,11 @@ void close_old_data_files(THD *thd, TABL } +#define TABLE_IN_USE(_t_) ((_t_)->locked_by_flush || \ + (_t_)->locked_by_name && wait_for_name_lock || \ + (_t_)->db_stat && (_t_)->version < refresh_version) + + /* Wait until all threads has closed the tables in the list We have also to wait if there is thread that has a lock on this table even @@ -1322,6 +1564,9 @@ void close_old_data_files(THD *thd, TABL bool table_is_used(TABLE *table, bool wait_for_name_lock) { + DBUG_ENTER("table_is_used"); + DBUG_PRINT("table", ("table: '%s' 0x%lx wait_for_name_lock: %d", + table->table_name, (long) table, wait_for_name_lock)); do { HASH_SEARCH_STATE state; @@ -1333,13 +1578,24 @@ bool table_is_used(TABLE *table, bool wa search= (TABLE*) hash_next(&open_cache, (byte*) key, key_length, &state)) { - if (search->locked_by_flush || - search->locked_by_name && wait_for_name_lock || - search->db_stat && search->version < refresh_version) - return 1; // Table is used + if (TABLE_IN_USE(search)) + DBUG_RETURN(1); + /* For a MERGE table check its children too. */ + for (TABLE *child= search->mrg_child; child; child= child->mrg_child) + { + if (TABLE_IN_USE(child)) + DBUG_RETURN(1); + } + /* For a MERGE child, check its parent too. */ + if (search->mrg_parent) + { + if (TABLE_IN_USE(search->mrg_parent)) + DBUG_RETURN(1); + } } } while ((table=table->next)); - return 0; + DBUG_PRINT("table", ("no table is used")); + DBUG_RETURN(0); } @@ -1359,7 +1615,9 @@ bool wait_for_tables(THD *thd) mysql_ha_flush(thd, (TABLE_LIST*) NULL, MYSQL_HA_REOPEN_ON_USAGE, TRUE); if (!table_is_used(thd->open_tables,1)) break; + DBUG_PRINT("pthreads", ("waiting for refresh")); (void) pthread_cond_wait(&COND_refresh,&LOCK_open); + DBUG_PRINT("pthreads", ("awoke from refresh")); } if (thd->killed) result= 1; // aborted @@ -1367,7 +1625,6 @@ bool wait_for_tables(THD *thd) { /* Now we can open all tables without any interference */ thd->proc_info="Reopen tables"; - thd->version= refresh_version; result=reopen_tables(thd,0,0); } pthread_mutex_unlock(&LOCK_open); @@ -1618,7 +1875,7 @@ int open_tables(THD *thd, TABLE_LIST *st !(tables->table= open_table(thd, tables->db, tables->real_name, - tables->alias, &refresh))) + tables->alias, &refresh, 0))) { if (refresh) // Refresh in progress { @@ -1731,7 +1988,7 @@ TABLE *open_ltable(THD *thd, TABLE_LIST thd->current_tablenr= 0; while (!(table=open_table(thd,table_list->db, table_list->real_name,table_list->alias, - &refresh)) && refresh) ; + &refresh, 0)) && refresh) ; if (table) { @@ -1924,16 +2181,46 @@ int lock_tables(THD *thd, TABLE_LIST *ta /* - Open a single table without table caching and don't set it in open_list - Used by alter_table to open a temporary table and when creating + @brief Open a single table without table caching and don't set it in open_list + + @detail Used by alter_table to open a intermediate table and when creating a temporary table with CREATE TEMPORARY ... + + Temporary table can be created anywhere in the file system. The path + name is unrelated to the internally used db+table_name. For example + CREATE TEMPORARY TABLE db1.tt1 ... will create table files like + TMPDIR/#sql-1234-56.* but internally the table has the name 'tt1' and + belongs to the database 'db1'. + + Intermediate tables as used by ALTER TABLE have also a "temporary" name + like #sql-1234-56, but are located in the same directory as the related + permanent table. They are normal tables in every aspect but the name. + + For a MERGE table it is important to know if the new table is a real + temporary table. The path names of their temporary children cannot be + used to produce names for open_table(). In that case the child tables + are not opened through the table cache. + + @param[in] thd thread handle + @param[in] path path name for table files (external use) + @param[in] db database for internal use + @param[in] table_name table name for internal use + @param[in] link_in_list if to link in thd->temporary_tables + @param[in] is_temporary if it is a temporary table, not an + intermediate table as used by ALTER TABLE. + + @return pointer to opened TABLE object + @retval NULL on error */ TABLE *open_temporary_table(THD *thd, const char *path, const char *db, - const char *table_name, bool link_in_list) + const char *table_name, bool link_in_list, + bool is_temporary) { TABLE *tmp_table; DBUG_ENTER("open_temporary_table"); + DBUG_PRINT("enter", ("path: '%s' db: '%s' name: '%s' link: %d temp: %d", + path, db, table_name, link_in_list, is_temporary)); /* The extra size in my_malloc() is for table_cache_key @@ -1948,7 +2235,8 @@ TABLE *open_temporary_table(THD *thd, co DBUG_RETURN(0); /* purecov: inspected */ if (openfrm(path, table_name, - (uint) (HA_OPEN_KEYFILE | HA_OPEN_RNDFILE | HA_GET_INDEX), + (uint) (HA_OPEN_KEYFILE | HA_OPEN_RNDFILE | HA_GET_INDEX | + (is_temporary ? HA_OPEN_TEMPORARY : 0)), READ_KEYINFO | COMPUTE_TYPES | EXTRA_RECORD, ha_open_options, tmp_table)) @@ -1980,6 +2268,8 @@ TABLE *open_temporary_table(THD *thd, co if (thd->slave_thread) slave_open_temp_tables++; } + DBUG_PRINT("info", ("opened temporary table: '%s' 0x%lx", + tmp_table->table_name, (long) tmp_table)); DBUG_RETURN(tmp_table); } @@ -3064,7 +3354,12 @@ void remove_db_from_cache(const char *db if (!strcmp(table->table_cache_key,db)) { table->version=0L; /* Free when thread is ready */ - if (!table->in_use) + /* + If table is unused, move it first in unused link. + Do not do that for MERGE children. They are not in the unused + chain. They will be deleted with their parent. + */ + if (!table->in_use && !table->mrg_parent) relink_unused(table); } } @@ -3109,7 +3404,8 @@ bool remove_table_from_cache(THD *thd, c TABLE *table; bool result=0, signalled= 0; DBUG_ENTER("remove_table_from_cache"); - + DBUG_PRINT("enter", ("db: '%s' table_name: '%s' flags: %u", + db, table_name, flags)); key_length=(uint) (strmov(strmov(key,db)+1,table_name)-key)+1; for (;;) @@ -3117,6 +3413,7 @@ bool remove_table_from_cache(THD *thd, c HASH_SEARCH_STATE state; result= signalled= 0; + /* Remove all TABLE objects for this table from cache. */ for (table= (TABLE*) hash_first(&open_cache, (byte*) key, key_length, &state); table; @@ -3124,14 +3421,28 @@ bool remove_table_from_cache(THD *thd, c &state)) { THD *in_use; + DBUG_PRINT("table", ("found in cache table: 0x%lx", (long) table)); + + /* + If a MERGE child is found, switch to the parent. This removes + the parent and all of its children, including the found one. + */ + if (table->mrg_parent) + { + table= table->mrg_parent; + DBUG_PRINT("table", ("parent table: 0x%lx", (long) table)); + } + table->version=0L; /* Free when thread is ready */ if (!(in_use=table->in_use)) { - DBUG_PRINT("info",("Table was not in use")); + DBUG_PRINT("table", ("unused table: 0x%lx", (long) table)); relink_unused(table); } else if (in_use != thd) { + DBUG_PRINT("table", ("in use table: 0x%lx by thread: %ld", + (long) table, in_use->dbug_thread_id)); in_use->some_tables_deleted=1; if (table->db_stat) result=1; @@ -3158,22 +3469,39 @@ bool remove_table_from_cache(THD *thd, c thd_table ; thd_table= thd_table->next) { - if (thd_table->db_stat) // If table is open - signalled|= mysql_lock_abort_for_thread(thd, thd_table); + /* + For a MERGE child abort the locks of the parent and thus + implicitly all children, including this one. + */ + if (thd_table->db_stat) // If table is open + signalled|= mysql_lock_abort_for_thread(thd, + thd_table->mrg_parent ? + thd_table->mrg_parent : + thd_table); } } else + { result= result || (flags & RTFC_OWNED_BY_THD_FLAG); + DBUG_PRINT("table", ("in use table: 0x%lx by self", (long) table)); + } } + + DBUG_PRINT("table", ("deleting unused tables")); while (unused_tables && !unused_tables->version) VOID(hash_delete(&open_cache,(byte*) unused_tables)); + if (result && (flags & RTFC_WAIT_OTHER_THREAD_FLAG)) { if (!(flags & RTFC_CHECK_KILLED_FLAG) || !thd->killed) { dropping_tables++; if (likely(signalled)) + { + DBUG_PRINT("pthreads", ("waiting for refresh")); (void) pthread_cond_wait(&COND_refresh, &LOCK_open); + DBUG_PRINT("pthreads", ("awoke from refresh")); + } else { struct timespec abstime; @@ -3188,7 +3516,9 @@ bool remove_table_from_cache(THD *thd, c remove_table_from_cache routine. */ set_timespec(abstime, 10); + DBUG_PRINT("pthreads", ("waiting for refresh 10 msec")); pthread_cond_timedwait(&COND_refresh, &LOCK_open, &abstime); + DBUG_PRINT("pthreads", ("awoke from refresh")); } dropping_tables--; continue; --- 1.143/sql/sql_delete.cc 2007-05-21 17:48:27 +02:00 +++ 1.144/sql/sql_delete.cc 2007-05-21 17:48:27 +02:00 @@ -661,7 +661,7 @@ int mysql_truncate(THD *thd, TABLE_LIST ha_create_table(path, &create_info,1); // We don't need to call invalidate() because this table is not in cache if ((error= (int) !(open_temporary_table(thd, path, table_list->db, - table_list->real_name, 1)))) + table_list->real_name, 1, 1)))) (void) rm_temporary_table(table_type, path); /* If we return here we will not have logged the truncation to the bin log --- 1.312/sql/sql_table.cc 2007-05-21 17:48:27 +02:00 +++ 1.313/sql/sql_table.cc 2007-05-21 17:48:27 +02:00 @@ -1514,7 +1514,7 @@ int mysql_create_table(THD *thd,const ch if (create_info->options & HA_LEX_CREATE_TMP_TABLE) { /* Open table and put in temporary table list */ - if (!(open_temporary_table(thd, path, db, table_name, 1))) + if (!(open_temporary_table(thd, path, db, table_name, 1, 1))) { (void) rm_temporary_table(create_info->db_type, path); goto end; @@ -1648,7 +1648,7 @@ TABLE *create_table_from_items(THD *thd, if (!mysql_create_table(thd, db, name, create_info, alter_info, 0, select_field_count)) { - if (!(table=open_table(thd,db,name,name,(bool*) 0))) + if (!(table= open_table(thd, db, name, name, (bool*) 0, 0))) quick_rm_table(create_info->db_type,db,table_case_name(create_info,name)); } reenable_binlog(thd); @@ -1780,6 +1780,7 @@ static void wait_while_table_is_used(THD static bool close_cached_table(THD *thd, TABLE *table) { DBUG_ENTER("close_cached_table"); + DBUG_PRINT("enter", ("table: '%s'", table->table_name)); wait_while_table_is_used(thd, table, HA_EXTRA_PREPARE_FOR_DELETE); /* Close lock if this is not got with LOCK TABLES */ @@ -2490,7 +2491,7 @@ int mysql_create_like_table(THD* thd, TA if (create_info->options & HA_LEX_CREATE_TMP_TABLE) { - if (err || !open_temporary_table(thd, dst_path, db, table_name, 1)) + if (err || !open_temporary_table(thd, dst_path, db, table_name, 1, 1)) { (void) rm_temporary_table(create_info->db_type, dst_path); /* purecov: inspected */ @@ -3223,12 +3224,12 @@ int mysql_alter_table(THD *thd,char *new DBUG_RETURN(error); } if (table->tmp_table) - new_table=open_table(thd,new_db,tmp_name,tmp_name,0); + new_table= open_table(thd, new_db, tmp_name, tmp_name, 0, 0); else { char path[FN_REFLEN]; build_table_path(path, sizeof(path), new_db, tmp_name, ""); - new_table=open_temporary_table(thd, path, new_db, tmp_name,0); + new_table=open_temporary_table(thd, path, new_db, tmp_name, 0, 0); } if (!new_table) { @@ -3296,9 +3297,13 @@ int mysql_alter_table(THD *thd,char *new goto end_temporary; } + /* + If old table was MERGE, we may need to close child tables. This + needs LOCK_open. + */ + VOID(pthread_mutex_lock(&LOCK_open)); intern_close_table(new_table); /* close temporary table */ my_free((gptr) new_table,MYF(0)); - VOID(pthread_mutex_lock(&LOCK_open)); if (error) { VOID(quick_rm_table(new_db_type,new_db,tmp_name)); @@ -3442,7 +3447,8 @@ int mysql_alter_table(THD *thd,char *new */ char path[FN_REFLEN]; build_table_path(path, sizeof(path), new_db, table_name, ""); - table=open_temporary_table(thd, path, new_db, tmp_name,0); + table=open_temporary_table(thd, path, new_db, tmp_name, 0, + create_info->options & HA_LEX_CREATE_TMP_TABLE); if (table) { intern_close_table(table); --- 1.142/sql/table.cc 2007-05-21 17:48:27 +02:00 +++ 1.143/sql/table.cc 2007-05-21 17:48:27 +02:00 @@ -87,7 +87,7 @@ int openfrm(const char *name, const char SQL_CRYPT *crypted=0; MEM_ROOT **root_ptr, *old_root; DBUG_ENTER("openfrm"); - DBUG_PRINT("enter",("name: '%s' form: %lx",name,outparam)); + DBUG_PRINT("enter", ("name: '%s' form: 0x%lx", name, (long) outparam)); bzero((char*) outparam,sizeof(*outparam)); outparam->blob_ptr_size=sizeof(char*); @@ -841,6 +841,45 @@ int closefrm(register TABLE *table) { int error=0; DBUG_ENTER("closefrm"); + DBUG_PRINT("table", ("table: '%s' 0x%lx db_stat: %u mrg_child: 0x%lx", + table->table_name, (long) table, + table->db_stat, (long) table->mrg_child)); + + /* + The special handling of MERGE table children is required in + situations when the table has been closed for reopen already. + In this case the engine tables are closed, but the TABLE objects + are still in the table cache. We cannot remove them through the + handler. A second close is not possible. Hence we have to care + for the TABLEs here. This is the lowest layer in table closing, + where this can be done. So we should catch all possible cases. + + It is no problem to close the children here even if the engine + tables are still open. The MERGE engine deals with the closed + children properly when the parent is closed. + */ + if (table->mrg_child) + { + TABLE *child; + TABLE *next; + + /* Close all tables from the child chain. */ + for (child= table->mrg_child; child; child= next) + { + /* Remember next child before mangling the TABLE object. */ + next= child->mrg_child; + + /* Force deletion from cache. */ + child->version= 0; + /* Detach child from parent. */ + child->mrg_parent= NULL; + child->mrg_child= NULL; + table->mrg_child= NULL; + /* Close child through table cache. */ + VOID(close_thread_table(NULL, &child)); + } + } + if (table->db_stat) error=table->file->close(); if (table->table_name) --- 1.76/sql/table.h 2007-05-21 17:48:27 +02:00 +++ 1.77/sql/table.h 2007-05-21 17:48:27 +02:00 @@ -165,6 +165,7 @@ struct st_table { my_bool no_keyread, no_cache; my_bool clear_query_id; /* To reset query_id for tables and cols */ my_bool auto_increment_field_not_null; + my_bool closing_for_reopen; /* Used by MERGE children. */ Field *next_number_field, /* Set if next_number is activated */ *found_next_number_field, /* Set on open */ *rowid_field; @@ -208,6 +209,8 @@ struct st_table { uint derived_select_number; THD *in_use; /* Which thread uses this */ struct st_table *next,*prev; + struct st_table *mrg_parent; /* Parent MERGE table. */ + struct st_table *mrg_child; /* List of MERGE child tables. */ }; --- 1.46/mysql-test/r/merge.result 2007-05-21 17:48:27 +02:00 +++ 1.47/mysql-test/r/merge.result 2007-05-21 17:48:27 +02:00 @@ -779,7 +779,7 @@ ERROR HY000: Unable to open underlying t DROP TABLE t1, t2; CREATE TABLE t2(a INT) ENGINE=MERGE UNION=(t3); SELECT * FROM t2; -ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +ERROR 42S02: Table 'test.t3' doesn't exist DROP TABLE t2; CREATE TABLE t1(a INT, b TEXT); CREATE TABLE tm1(a TEXT, b INT) ENGINE=MERGE UNION=(t1); @@ -819,3 +819,217 @@ ALTER TABLE m1 ENGINE=MERGE UNION=(t1); SELECT * FROM m1; c1 c2 c3 c4 c5 c6 c7 c8 c9 DROP TABLE t1, m1; +create table t1 (c1 int, index(c1)); +create table t2 (c1 int, index(c1)) engine=merge union=(t1); +insert into t1 values (1); +flush tables; +select * from t2; +c1 +1 +flush tables; +truncate table t1; +insert into t1 values (1); +flush tables; +select * from t2; +c1 +1 +truncate table t1; +insert into t1 values (1); +drop table t1,t2; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT) ENGINE= MRG_MyISAM UNION= (t1, t2); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (2); +SELECT * FROM t3; +c1 +1 +2 +TRUNCATE TABLE t1; +SELECT * FROM t3; +c1 +2 +DROP TABLE t1, t2, t3; +CREATE TABLE t1 (id INTEGER, grp TINYINT, id_rev INTEGER); +SET @rnd_max= 2147483647; +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(128.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +set @@read_buffer_size=2*1024*1024; +CREATE TABLE t2 SELECT * FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +CREATE TABLE t3 (id INTEGER, grp TINYINT, id_rev INTEGER) +ENGINE= MRG_MyISAM UNION= (t1, t2); +SELECT COUNT(*) FROM t1; +COUNT(*) +130 +SELECT COUNT(*) FROM t2; +COUNT(*) +80 +SELECT COUNT(*) FROM t3; +COUNT(*) +210 +SELECT COUNT(DISTINCT a1.id) FROM t3 AS a1, t3 AS a2 +WHERE a1.id = a2.id GROUP BY a2.grp; +TRUNCATE TABLE t1; +SELECT COUNT(*) FROM t1; +COUNT(*) +0 +SELECT COUNT(*) FROM t2; +COUNT(*) +80 +SELECT COUNT(*) FROM t3; +COUNT(*) +80 +DROP TABLE t1, t2, t3; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MyISAM UNION=(t1) INSERT_METHOD= LAST; +INSERT INTO t2 VALUES (1); +SELECT * FROM t2; +c1 +1 +LOCK TABLES t2 WRITE, t1 WRITE; +FLUSH TABLES; +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +CHECK TABLE t1; +Table Op Msg_type Msg_text +test.t1 check status OK +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +Table Op Msg_type Msg_text +test.t1 check status OK +LOCK TABLES t2 WRITE, t1 WRITE; +SELECT * FROM t2; +c1 +1 +LOCK TABLES t2 WRITE, t1 WRITE; +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +CHECK TABLE t1; +Table Op Msg_type Msg_text +test.t1 check status OK +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +Table Op Msg_type Msg_text +test.t1 check status OK +DROP TABLE t1, t2; +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MERGE UNION=(t1); +LOCK TABLES t1 WRITE, m1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MERGE UNION=(t1); +LOCK TABLES m1 WRITE, t1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +REPAIR TABLE t1; +INSERT INTO t2 VALUES (1); +Table Op Msg_type Msg_text +test.t1 repair status OK +DROP TABLE t1, t2; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; +INSERT INTO t2 VALUES (1); +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +DROP TABLE t1, t2; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +LOCK TABLE t1 WRITE; +INSERT INTO t1 VALUES (1); +FLUSH TABLES; +FLUSH TABLES; +SELECT * FROM t1; +c1 +UNLOCK TABLES; +DROP TABLE t1; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; +INSERT INTO t2 VALUES (1); +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +DROP TABLE t1, t2; +CREATE TABLE t1 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT, c2 INT) ENGINE= MRG_MyISAM UNION(t1, t2); +INSERT INTO t1 VALUES (1, 1); +INSERT INTO t2 VALUES (2, 2); +SELECT * FROM t3; +c1 c2 +1 1 +2 2 +ALTER TABLE t1 ENGINE= MEMORY; +INSERT INTO t1 VALUES (0, 0); +SELECT * FROM t3; +ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +DROP TABLE t1, t2, t3; --- 1.43/mysql-test/t/merge.test 2007-05-21 17:48:27 +02:00 +++ 1.44/mysql-test/t/merge.test 2007-05-21 17:48:27 +02:00 @@ -397,7 +397,7 @@ CREATE TABLE t2(a INT) ENGINE=MERGE UNIO SELECT * FROM t2; DROP TABLE t1, t2; CREATE TABLE t2(a INT) ENGINE=MERGE UNION=(t3); ---error 1168 +--error 1146 SELECT * FROM t2; DROP TABLE t2; @@ -453,5 +453,277 @@ CREATE TABLE m1 LIKE t1; ALTER TABLE m1 ENGINE=MERGE UNION=(t1); SELECT * FROM m1; DROP TABLE t1, m1; + +# +# Bug #8306: TRUNCATE leads to index corruption +# Obsoleted by fix for Bug#26379 (Combination of FLUSH TABLE and +# REPAIR TABLE corrupts a MERGE table). +# +create table t1 (c1 int, index(c1)); +create table t2 (c1 int, index(c1)) engine=merge union=(t1); +insert into t1 values (1); +# Close all tables. +flush tables; +# Open t2 and (implicitly) t1. +select * from t2; +# Truncate after flush works (unless another threads reopens t2 in between). +flush tables; +truncate table t1; +insert into t1 values (1); +# Close all tables. +flush tables; +# Open t2 and (implicitly) t1. +select * from t2; +# Truncate t1, wich was not recognized as open without the bugfix. +# After fix for Bug#8306 and before fix for Bug#26379, +# it should fail with a table-in-use error message, otherwise succeed. +truncate table t1; +# The insert used to fail on the crashed table. +insert into t1 values (1); +drop table t1,t2; + +# +# Bug#25038 - Waiting TRUNCATE +# +# Show that truncate of child table after use of parent table works. +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT) ENGINE= MRG_MyISAM UNION= (t1, t2); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (2); +SELECT * FROM t3; +TRUNCATE TABLE t1; +SELECT * FROM t3; +DROP TABLE t1, t2, t3; +# +# Show that truncate of child table waits while parent table is used. +# (test partly borrowed from count_distinct3.) +CREATE TABLE t1 (id INTEGER, grp TINYINT, id_rev INTEGER); +SET @rnd_max= 2147483647; +let $1 = 10; +while ($1) +{ + SET @rnd= RAND(); + SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); + SET @id_rev= @rnd_max - @id; + SET @grp= CAST(128.0 * @rnd AS UNSIGNED); + INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); + dec $1; +} +set @@read_buffer_size=2*1024*1024; +CREATE TABLE t2 SELECT * FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +CREATE TABLE t3 (id INTEGER, grp TINYINT, id_rev INTEGER) + ENGINE= MRG_MyISAM UNION= (t1, t2); +SELECT COUNT(*) FROM t1; +SELECT COUNT(*) FROM t2; +SELECT COUNT(*) FROM t3; +connect (con1,localhost,root,,); + # As t3 contains random numbers, results are different from test to test. + # That's okay, because we test only that select doesn't yield an + # error. Note, that --disable_result_log doesn't suppress error output. + --disable_result_log + send SELECT COUNT(DISTINCT a1.id) FROM t3 AS a1, t3 AS a2 + WHERE a1.id = a2.id GROUP BY a2.grp; +connection default; +sleep 1; +TRUNCATE TABLE t1; + connection con1; + reap; + --enable_result_log +connection default; +SELECT COUNT(*) FROM t1; +SELECT COUNT(*) FROM t2; +SELECT COUNT(*) FROM t3; +DROP TABLE t1, t2, t3; +disconnect con1; + +# +# Bug#25700 - merge base tables get corrupted by optimize/analyze/repair table +# +# Using FLUSH TABLES before REPAIR. +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MyISAM UNION=(t1) INSERT_METHOD= LAST; +INSERT INTO t2 VALUES (1); +SELECT * FROM t2; +LOCK TABLES t2 WRITE, t1 WRITE; +FLUSH TABLES; +REPAIR TABLE t1; +CHECK TABLE t1; +REPAIR TABLE t1; +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +# +# Not using FLUSH TABLES before REPAIR. +LOCK TABLES t2 WRITE, t1 WRITE; +SELECT * FROM t2; +LOCK TABLES t2 WRITE, t1 WRITE; +REPAIR TABLE t1; +CHECK TABLE t1; +REPAIR TABLE t1; +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +DROP TABLE t1, t2; + +# +# Bug#26377 - Deadlock with MERGE and FLUSH TABLE +# +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MERGE UNION=(t1); +# Lock t1 first. This did always work. +LOCK TABLES t1 WRITE, m1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; +# +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MERGE UNION=(t1); +# Lock m1 first. This did deadlock. +LOCK TABLES m1 WRITE, t1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; + +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Preparation +connect (con1,localhost,root,,); +connection default; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Problem #1 +# A thread trying to lock a MERGE table performed busy waiting while +# REPAIR TABLE or a similar table administration task was ongoing on one or +# more of its MyISAM tables. +# To allow for observability it was necessary to enter a multi-second sleep +# in mysql_admin_table() after remove_table_from_cache(), which comes after +# mysql_abort_lock(). The sleep faked a long running operation. One could +# watch a high CPU load during the sleep time. +# The problem was that mysql_abort_lock() upgrades the write lock to +# TL_WRITE_ONLY. This lock type persisted until the final unlock at the end +# of the administration task. The effect of TL_WRITE_ONLY is to reject any +# attempt to lock the table. The trying thread must close the table and wait +# until it is no longer used. Unfortunately there is no way to detect that +# one of the MyISAM tables of a MERGE table is in use. When trying to lock +# the MERGE table, all MyISAM tables are locked. If one fails on +# TL_WRITE_ONLY, all locks are aborted and wait_for_tables() is entered. +# But this doesn't see the MERGE table as used, so it seems appropriate to +# retry a lock... +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +send REPAIR TABLE t1; + connection con1; + sleep 1; # let repair run into its sleep + INSERT INTO t2 VALUES (1); +connection default; +reap; +DROP TABLE t1, t2; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Problem #2 +# A thread trying to lock a MERGE table performed busy waiting until all +# threads that did REPAIR TABLE or similar table administration tasks on +# one or more of its MyISAM tables in LOCK TABLES segments did +# UNLOCK TABLES. +# The difference against problem #1 is that the busy waiting took place +# *after* the administration task. It was terminated by UNLOCK TABLES only. +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; + connection con1; + send INSERT INTO t2 VALUES (1); +connection default; +sleep 1; # Let INSERT go into thr_multi_lock(). +REPAIR TABLE t1; +sleep 2; # con1 performs busy waiting during this sleep. +UNLOCK TABLES; + connection con1; + reap; +connection default; +DROP TABLE t1, t2; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Problem #3 +# Two FLUSH TABLES within a LOCK TABLES segment could invalidate the lock. +# This did *not* require a MERGE table. +# To increase reproducibility it was necessary to enter a sleep of 2 seconds +# at the end of wait_for_tables() after unlock of LOCK_open. In 5.0 and 5.1 +# the sleep must be inserted in open_and_lock_tables() after open_tables() +# instead. wait_for_tables() is not used in this case. +# The problem was that FLUSH TABLES releases LOCK_open while having unlocked +# and closed all tables. When this happened while a thread was in the loop in +# mysql_lock_tables() right after wait_for_tables() and before retrying to +# lock, the thread got the lock. (Translate to similar code places in 5.0 +# and 5.1). And it did not notice that the table needed a refresh. So it +# executed its statement on the table. +# The first FLUSH TABLES kicked the INSERT out of thr_multi_lock() and let +# it wait in wait_for_tables(). (open_table() in 5.0 and 5.1). The second +# FLUSH TABLES must happen while the INSERT was on its way from +# wait_for_tables() to the next call of thr_multi_lock(). This needed to be +# supported by a sleep to make it repeatable. +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +LOCK TABLE t1 WRITE; + connection con1; + send INSERT INTO t1 VALUES (1); +connection default; +sleep 1; # Let INSERT go into thr_multi_lock(). +FLUSH TABLES; +sleep 1; # Let INSERT go through wait_for_tables() where it sleeps. +FLUSH TABLES; +# This should give no result. But it will with sleep(2) at the right place. +SELECT * FROM t1; +UNLOCK TABLES; + connection con1; + reap; +connection default; +DROP TABLE t1; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Cleanup +disconnect con1; + +# +# Bug#26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; + connect (con1,localhost,root,,); + send INSERT INTO t2 VALUES (1); +connection default; +sleep 1; # Let INSERT go into thr_multi_lock(). +REPAIR TABLE t1; +sleep 2; # con1 performs busy waiting during this sleep. +UNLOCK TABLES; + connection con1; + reap; + disconnect con1; +connection default; +DROP TABLE t1, t2; + +# +# Bug#27660 - Falcon: merge table possible +# +# Normal MyISAM MERGE operation. +CREATE TABLE t1 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT, c2 INT) ENGINE= MRG_MyISAM UNION(t1, t2); +INSERT INTO t1 VALUES (1, 1); +INSERT INTO t2 VALUES (2, 2); +SELECT * FROM t3; +# Try an unsupported engine. +ALTER TABLE t1 ENGINE= MEMORY; +INSERT INTO t1 VALUES (0, 0); +# Before fixing, this succeeded, but (0, 0) was missing. +--error 1168 +SELECT * FROM t3; +DROP TABLE t1, t2, t3; # End of 4.1 tests --- 1.73/mysql-test/r/myisam.result 2007-05-21 17:48:27 +02:00 +++ 1.74/mysql-test/r/myisam.result 2007-05-21 17:48:27 +02:00 @@ -593,24 +593,6 @@ select count(*) from t1 where a is null; count(*) 2 drop table t1; -create table t1 (c1 int, index(c1)); -create table t2 (c1 int, index(c1)) engine=merge union=(t1); -insert into t1 values (1); -flush tables; -select * from t2; -c1 -1 -flush tables; -truncate table t1; -insert into t1 values (1); -flush tables; -select * from t2; -c1 -1 -truncate table t1; -ERROR HY000: MyISAM table 't1' is in use (most likely by a MERGE table). Try FLUSH TABLES. -insert into t1 values (1); -drop table t1,t2; create table t1 (c1 int, c2 varchar(4) not null default '', key(c2(3))) default charset=utf8; insert into t1 values (1,'A'), (2, 'B'), (3, 'A'); --- 1.59/mysql-test/t/myisam.test 2007-05-21 17:48:27 +02:00 +++ 1.60/mysql-test/t/myisam.test 2007-05-21 17:48:27 +02:00 @@ -573,32 +573,6 @@ select count(*) from t1 where a is null; drop table t1; # -# Bug #8306: TRUNCATE leads to index corruption -# -create table t1 (c1 int, index(c1)); -create table t2 (c1 int, index(c1)) engine=merge union=(t1); -insert into t1 values (1); -# Close all tables. -flush tables; -# Open t2 and (implicitly) t1. -select * from t2; -# Truncate after flush works (unless another threads reopens t2 in between). -flush tables; -truncate table t1; -insert into t1 values (1); -# Close all tables. -flush tables; -# Open t2 and (implicitly) t1. -select * from t2; -# Truncate t1, wich was not recognized as open without the bugfix. -# Now, it should fail with a table-in-use error message. ---error 1105 -truncate table t1; -# The insert used to fail on the crashed table. -insert into t1 values (1); -drop table t1,t2; - -# # bug9188 - Corruption Can't open file: 'table.MYI' (errno: 145) # create table t1 (c1 int, c2 varchar(4) not null default '',
| Thread | ||
|---|---|---|
| • bk commit into 4.1 tree (istruewing:1.2630) BUG#26379 | ingo | 21 May |
© 1995-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
Page generated in 1.294 sec. using MySQL 5.1.14-beta-log