| List: | Commits | « Previous MessageNext Message » | |
| From: | Ingo Struewing | Date: | October 15 2007 4:02pm |
| Subject: | bk commit into 5.1 tree (istruewing:1.2576) BUG#26379 | ||
| View as plain text | |||
Below is the list of changes that have just been committed into a local 5.1 repository of istruewing. When istruewing does a push these changes will be propagated to the main repository and, within 24 hours after the push, to the public repository. For information on how to access the public repository see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html ChangeSet@stripped, 2007-10-15 16:02:51+02:00, istruewing@stripped +28 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Bug 26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging Bug 26377 - Deadlock with MERGE and FLUSH TABLE Bug 25038 - Waiting TRUNCATE Bug 25700 - merge base tables get corrupted by optimize/analyze/repair table Bug 30275 - Merge tables: flush tables or unlock tables causes server to crash Bug 19627 - temporary merge table locking Bug 27660 - Falcon: merge table possible Bug 30273 - merge tables: Can't lock file (errno: 155) Not to be pushed. For intermediate review only. This is an implementation of a new MERGE table open approach. Still to be done: - Work on review comments regarding the tests. - Test if tampering with a child from another thread can require to detach (and re-attach later in reopen_tables()) the children in close_handle_and_leave_table_as_lock(). - Test if another thread can break into CREATE...MERGE...SELECT as I need to unlock the children in create_table_from_items(). - Test ALTER TABLE (fast + copy) within LOCK TABLES to see if I need to re-attach the children after reopen_table(). - Decide what value ::reset() should return if children are detached. General patch description: When opening a MERGE table in open_tables() we do now add the child tables to the list of tables to be opened by open_tables() (the "query_list"). The children are not opened in the handler at this stage. After opening the parent, open_tables() opens each child from the now extended query_list. When the last child is opened, we remove the children from the query_list again and attach the children to the parent. This behaves similar to the old open. However it does not open the MyISAM tables directly, but grabs them from the already open children. When closing a MERGE table in close_thread_table() we detach the children only. Closing of the children is done implicitly because they are in thd->open_tables. For more detail see the comment at the top of ha_myisammrg.cc. Changed from open_ltable() to open_and_lock_tables() in all places that can be relevant for MERGE tables. The latter can handle tables added to the list on the fly. When open_ltable() was used in a loop over a list of tables, the list must be temporarily terminated after every table for open_and_lock_tables(). table_list->required_type is set to FRMTYPE_TABLE to avoid open of special tables. Handling of derived tables is suppressed. These details are handled by the new function open_n_lock_single_table(), which has nearly the same signature as open_ltable() and can replace it in most cases. In reopen_tables() some of the tables open by a thread can be closed and reopened. When a MERGE child is affected, the parent must be closed and reopened too. Closing of the parent is forced before the first child is closed. Reopen happens in the order of thd->open_tables. MERGE parents do not attach their children automatically at open. This is done after all tables are reopened. So all children are open when attaching them. Special lock handling like mysql_lock_abort() or mysql_lock_remove() needs to be suppressed for MERGE children or forwarded to the parent. This depends on the situation. In loops over all open tables one suppresses child lock handling. When a single table is touched, forwarding is done. This patch changes the behavior of temporary MERGE tables. Temporary MERGE must have temporary children. The old behavior was wrong. A temporary table is not locked. Hence even non-temporary children were not locked. See Bug 19627 - temporary merge table locking. include/my_base.h@stripped, 2007-10-15 16:02:48+02:00, istruewing@stripped +7 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added HA_EXTRA_ATTACH_CHILDREN and HA_EXTRA_DETACH_CHILDREN. include/myisammrg.h@stripped, 2007-10-15 16:02:48+02:00, istruewing@stripped +9 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added element 'children_attached' to MYRG_INFO. Added declarations for myrg_parent_open(), myrg_attach_children() and myrg_detach_children() for the new MERGE table open approach. mysql-test/extra/binlog_tests/blackhole.test@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +9 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Preliminarily added new error message with a comment. mysql-test/r/merge.result@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +336 -16 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added test results. mysql-test/r/myisam.result@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +0 -18 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test result for bug 8306 from here to merge.result. mysql-test/suite/binlog/r/binlog_stm_blackhole.result@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +1 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test result. mysql-test/t/delayed.test@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +1 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Exchanged error number by symbolic code. mysql-test/t/merge.test@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +387 -3 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test for new temporary MERGE table behavior. Exchanged error numbers by symbolic codes. Added tests. Included are tests for bugs 8306 (moved from myisam.test), 26379, 19627, 25038, 25700, 26377, 26867, 27660, 30275, and 30273. mysql-test/t/myisam.test@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +0 -26 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test for bug 8306 from here to merge.test. sql/ha_ndbcluster_binlog.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +1 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/handler.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +6 -5 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/mysql_priv.h@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +19 -3 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added declaration for fix_merge_after_open(). Renamed open_and_lock_tables() to open_and_lock_tables_derived() with additional parameter 'derived'. Added inline functions simple_open_n_lock_tables() and open_and_lock_tables(), which call open_and_lock_tables_derived() and add the argument for 'derived'. Added new function open_n_lock_single_table(), which can be used as an replacement for open_ltable() in most situations. Internally it calls simple_open_n_lock_tables() so hat it is appropriate for MERGE tables. Added 'thd' argument to init_tmp_table_share(). sql/slave.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +2 -2 ug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Changed from open_ltable() to open_n_lock_single_table(). sql/sql_base.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +582 -74 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Defined new functions add_merge_table_list(), attach_merge_children(), detach_merge_children(), and fix_merge_after_open() for the new MERGE table open approach. Added calls of the new functions to close_handle_and_leave_table_as_lock(), close_thread_tables(), close_thread_table(), reopen_table(), drop_locked_tables(), and open_tables() respectively. Prevented special lock handling of merge children (like mysql_lock_remove, mysql_lock_merge or mysql_lock_abort) at many places. Some of these calls are forwarded to the parent table instead. Added MERGE children to the list of unusable tables in open_table(). Added MERGE table handling to reopen_table(). Added code for re-attaching children in reopen_tables(). Added code for removing the children list from the statement list to prepare for a repetition in open_tables(). Added new function open_n_lock_single_table(), which can be used as an replacement for open_ltable() in most situations. Internally it calls simple_open_n_lock_tables() so hat it is appropriate for MERGE tables. Disabled use of open_ltable() for MERGE tables. Removed function simple_open_n_lock_tables(). It is now inline declared in mysql_priv.h. Renamed open_and_lock_tables() to open_and_lock_tables_derived() with additional parameter 'derived'. open_and_lock_tables() is now inline declared in mysql_priv.h. Added a check for end-of-list in two loops in lock_tables(). Added 'thd' argument to init_tmp_table_share(). sql/sql_insert.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +50 -2 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Changed from open_ltable() to open_n_lock_single_table() in handle_delayed_insert(). Added 'thd' argument to init_tmp_table_share(). Added MERGE handling to create_table_from_items(). Asserted that the branch for non-temporary MERGE table in create_table_from_items() (CREATE...SELECT) cannot be used under LOCK TABLES. Added fix_merge_after_open() to the branch for temporary MERGE tables in create_table_from_items(). sql/sql_parse.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +12 -13 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Changed from open_ltable() to open_n_lock_single_table() in mysql_table_dump(). Removed condition for calling close_thread_tables() from dispatch_command() (in COM_QUERY after every mysql_parse() and after the big switch). Set lock_type to TL_WRITE in check_merge_table_access(). sql/sql_partition.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +1 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed comment typo. sql/sql_select.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +1 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/sql_table.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +24 -2 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Changed from open_ltable() to open_n_lock_single_table() in mysql_alter_table() and mysql_checksum_table(). Initialized table_list->table in mysql_recreate_table(). sql/table.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +8 -2 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). Setting table_map_id from query_id in init_tmp_table_share(). sql/table.h@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +11 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added elements for MERGE tables to TABLE and TABLE_LIST. storage/myisam/ha_myisam.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +27 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added new function myisam_engine_handle() to extract a MI_INFO pointer from a handler object. Added an unrelated comment to the function comment of table2myisam(). storage/myisam/ha_myisam.h@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +4 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Declared new function myisam_engine_handle() as friend of class ha_myisam. storage/myisammrg/ha_myisammrg.cc@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +555 -90 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added callback functions to support parent open and children attach of MERGE tables. Changed ha_myisammrg::open() to initialize storage engine structures and create a list of child tables only. Child tables are not opened. Added ha_myisammrg::attach_children(), which does now the main part of MERGE open. Added ha_myisammrg::detach_children(). Added calls to ::attach_children() and ::detach_children() to ::extra() on HA_EXTRA_ATTACH_CHILDREN and HA_EXTRA_DETACH_CHILDREN respectively. Added a check for matching TEMPORARY type for children against parent. Added support for thd->open_options to attach_children(). Changed child path name generation for temporary tables so that it does nothing special for temporary tables. storage/myisammrg/ha_myisammrg.h@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +8 -1 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added elements to class ha_myisammrg to support the new open approach. Changed empty destructor definition to a declaration. Implemented in ha_myisammrg.cc. Added declaration for methods attach_children() and detach_children(). Added definition for method table_ptr() for use with callback functions. storage/myisammrg/myrg_close.c@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +27 -3 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added a check to avoid closing of MyISAM tables when the child tables are not attached. Added freeing of rec_per_key_part when the child tables are not attached. storage/myisammrg/myrg_extra.c@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +4 -0 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Some ::extra() functions and ::reset() can be called when children are detached. storage/myisammrg/myrg_open.c@stripped, 2007-10-15 16:02:49+02:00, istruewing@stripped +308 -7 Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Kept old myrg_open() for MERGE use independent from MySQL. Removed an always true condition in myrg_open(). Set children_attached for independent MERGE use in myrg_open(). Added myrg_parent_open(), myrg_attach_children, and myrg_detach_children() for the new MERGE table open approach. diff -Nrup a/include/my_base.h b/include/my_base.h --- a/include/my_base.h 2007-07-25 00:58:05 +02:00 +++ b/include/my_base.h 2007-10-15 16:02:48 +02:00 @@ -187,7 +187,13 @@ enum ha_extra_function { Inform handler that an "INSERT...ON DUPLICATE KEY UPDATE" will be executed. This condition is unset by HA_EXTRA_NO_IGNORE_DUP_KEY. */ - HA_EXTRA_INSERT_WITH_UPDATE + HA_EXTRA_INSERT_WITH_UPDATE, + /* + Orders MERGE handler to attach or detach its child tables. Used at + begin and end of a statement. + */ + HA_EXTRA_ATTACH_CHILDREN, + HA_EXTRA_DETACH_CHILDREN }; /* The following is parameter to ha_panic() */ diff -Nrup a/include/myisammrg.h b/include/myisammrg.h --- a/include/myisammrg.h 2007-05-10 11:59:24 +02:00 +++ b/include/myisammrg.h 2007-10-15 16:02:48 +02:00 @@ -69,6 +69,8 @@ typedef struct st_myrg_info uint merge_insert_method; uint tables,options,reclength,keys; my_bool cache_in_use; + /* If MERGE children attached to parent. See top comment in ha_myisammrg.cc */ + my_bool children_attached; LIST open_list; QUEUE by_key; ulong *rec_per_key_part; /* for sql optimizing */ @@ -80,6 +82,13 @@ typedef struct st_myrg_info extern int myrg_close(MYRG_INFO *file); extern int myrg_delete(MYRG_INFO *file,const uchar *buff); extern MYRG_INFO *myrg_open(const char *name,int mode,int wait_if_locked); +extern MYRG_INFO *myrg_parent_open(const char *parent_name, + int (*callback)(void*, const char*), + void *callback_param); +extern int myrg_attach_children(MYRG_INFO *m_info, int handle_locking, + MI_INFO *(*callback)(void*), + void *callback_param); +extern int myrg_detach_children(MYRG_INFO *m_info); extern int myrg_panic(enum ha_panic_function function); extern int myrg_rfirst(MYRG_INFO *file,uchar *buf,int inx); extern int myrg_rlast(MYRG_INFO *file,uchar *buf,int inx); diff -Nrup a/mysql-test/extra/binlog_tests/blackhole.test b/mysql-test/extra/binlog_tests/blackhole.test --- a/mysql-test/extra/binlog_tests/blackhole.test 2007-06-15 18:35:09 +02:00 +++ b/mysql-test/extra/binlog_tests/blackhole.test 2007-10-15 16:02:49 +02:00 @@ -134,6 +134,15 @@ drop table t1,t2,t3; # table # CREATE TABLE t1(a INT) ENGINE=BLACKHOLE; +# NOTE: After exchanging open_ltable() by open_and_lock_tables() in +# handle_delayed_insert() to fix problems with MERGE tables (Bug#26379), +# problems with INSERT DELAYED and BLACKHOLE popped up. open_ltable() +# does not check if the binlogging capabilities of the statement and the +# table match. So the below used to succeed. But since INSERT DELAYED +# switches to row-based logging in mixed-mode and BLACKHOLE cannot do +# row-based logging, it could not really work. Until this problem is +# correctly fixed, we have that error here. +--error ER_BINLOG_LOGGING_IMPOSSIBLE INSERT DELAYED INTO t1 VALUES(1); DROP TABLE t1; diff -Nrup a/mysql-test/r/merge.result b/mysql-test/r/merge.result --- a/mysql-test/r/merge.result 2007-10-09 14:18:07 +02:00 +++ b/mysql-test/r/merge.result 2007-10-15 16:02:49 +02:00 @@ -584,9 +584,7 @@ insert into t1 values (1); insert into t2 values (2); create temporary table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); select * from t3; -a -1 -2 +ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist create temporary table t4 (a int not null); create temporary table t5 (a int not null); insert into t4 values (1); @@ -597,6 +595,39 @@ a 1 2 drop table t6, t3, t1, t2, t4, t5; +create temporary table t1 (a int not null); +create temporary table t2 (a int not null); +insert into t1 values (1); +insert into t2 values (2); +create table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +select * from t3; +ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +drop table t3, t2, t1; +create table t1 (a int not null); +create temporary table t2 (a int not null); +insert into t1 values (1); +insert into t2 values (2); +create table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +select * from t3; +ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +drop table t3; +create temporary table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +select * from t3; +ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +drop table t3, t2, t1; +CREATE TEMPORARY TABLE t1 (c1 INT NOT NULL); +CREATE TEMPORARY TABLE t2 (c1 INT NOT NULL); +CREATE TABLE t3 (c1 INT NOT NULL); +INSERT INTO t3 VALUES (1), (2); +LOCK TABLES t3 READ; +CREATE TEMPORARY TABLE t4 (c1 INT NOT NULL) ENGINE=MERGE UNION=(t1,t2) +INSERT_METHOD=LAST SELECT * FROM t3; +SELECT * FROM t4; +c1 +1 +2 +UNLOCK TABLES; +DROP TABLE t4, t3, t2, t1; CREATE TABLE t1 ( fileset_id tinyint(3) unsigned NOT NULL default '0', file_code varchar(32) NOT NULL default '', @@ -621,7 +652,7 @@ id select_type table type possible_keys EXPLAIN SELECT * FROM t2 WHERE fileset_id = 2 AND file_code BETWEEN '0000000115' AND '0000000120' LIMIT 1; id select_type table type possible_keys key key_len ref rows Extra -1 SIMPLE t2 range PRIMARY,files PRIMARY 35 NULL 5 Using where +1 SIMPLE t2 ref PRIMARY,files files 1 const 9 Using where EXPLAIN SELECT * FROM t1 WHERE fileset_id = 2 AND file_code BETWEEN '0000000115' AND '0000000120' LIMIT 1; id select_type table type possible_keys key key_len ref rows Extra @@ -708,7 +739,7 @@ a b c 1 1 0 show index from t3; Table Non_unique Key_name Seq_in_index Column_name Collation Cardinality Sub_part Packed Null Index_type Comment -t3 1 a 1 a A NULL NULL NULL YES BTREE +t3 1 a 1 a A 0 NULL NULL YES BTREE t3 1 a 2 b A NULL NULL NULL YES BTREE t3 1 a 3 c A NULL NULL NULL YES BTREE drop table t1, t2, t3; @@ -784,7 +815,7 @@ ERROR HY000: Unable to open underlying t DROP TABLE t1, t2; CREATE TABLE t2(a INT) ENGINE=MERGE UNION=(t3); SELECT * FROM t2; -ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +ERROR 42S02: Table 'test.t3' doesn't exist DROP TABLE t2; CREATE TABLE t1(a INT, b TEXT); CREATE TABLE tm1(a TEXT, b INT) ENGINE=MERGE UNION=(t1); @@ -849,21 +880,18 @@ drop table t2; drop table t1; CREATE TABLE tm1(a INT) ENGINE=MERGE UNION=(t1, t2); SELECT * FROM tm1; -ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +ERROR 42S02: Table 'test.t1' doesn't exist CHECK TABLE tm1; Table Op Msg_type Msg_text -test.tm1 check Error Table 'test.t1' is differently defined or of non-MyISAM type or doesn't exist -test.tm1 check Error Table 'test.t2' is differently defined or of non-MyISAM type or doesn't exist -test.tm1 check Error Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist -test.tm1 check error Corrupt +test.tm1 check Error Table 'test.t1' doesn't exist +test.tm1 check status OK CREATE TABLE t1(a INT); SELECT * FROM tm1; -ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +ERROR 42S02: Table 'test.t2' doesn't exist CHECK TABLE tm1; Table Op Msg_type Msg_text -test.tm1 check Error Table 'test.t2' is differently defined or of non-MyISAM type or doesn't exist -test.tm1 check Error Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist -test.tm1 check error Corrupt +test.tm1 check Error Table 'test.t2' doesn't exist +test.tm1 check status OK CREATE TABLE t2(a BLOB); SELECT * FROM tm1; ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist @@ -871,7 +899,7 @@ CHECK TABLE tm1; Table Op Msg_type Msg_text test.tm1 check Error Table 'test.t2' is differently defined or of non-MyISAM type or doesn't exist test.tm1 check Error Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist -test.tm1 check error Corrupt +test.tm1 check status OK ALTER TABLE t2 MODIFY a INT; SELECT * FROM tm1; a @@ -885,3 +913,295 @@ CREATE TABLE IF NOT EXISTS t1 SELECT * F ERROR HY000: You can't specify target table 't1' for update in FROM clause DROP TABLE t1, t2; End of 5.0 tests +create table t1 (c1 int, index(c1)); +create table t2 (c1 int, index(c1)) engine=merge union=(t1); +insert into t1 values (1); +flush tables; +select * from t2; +c1 +1 +flush tables; +truncate table t1; +insert into t1 values (1); +flush tables; +select * from t2; +c1 +1 +truncate table t1; +insert into t1 values (1); +drop table t1,t2; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +REPAIR TABLE t1; +INSERT INTO t2 VALUES (1); +Table Op Msg_type Msg_text +test.t1 repair status OK +DROP TABLE t1, t2; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; +INSERT INTO t2 VALUES (1); +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +DROP TABLE t1, t2; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +LOCK TABLE t1 WRITE; +INSERT INTO t1 VALUES (1); +FLUSH TABLES; +FLUSH TABLES; +SELECT * FROM t1; +c1 +UNLOCK TABLES; +DROP TABLE t1; +CREATE TABLE t1 (c1 INT); +CREATE TABLE t2 (c1 INT); +CREATE TABLE t3 (c1 INT); +INSERT INTO t3 VALUES (1), (2); +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2) +INSERT_METHOD=LAST SELECT * FROM t3; +SHOW CREATE TABLE t4; +Table Create Table +t4 CREATE TABLE `t4` ( + `c1` int(11) DEFAULT NULL +) ENGINE=MRG_MyISAM DEFAULT CHARSET=latin1 INSERT_METHOD=LAST UNION=(`t1`,`t2`) +SELECT * FROM t4; +c1 +1 +2 +DROP TABLE t4; +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2) +INSERT_METHOD=FIRST SELECT * FROM t3; +SHOW CREATE TABLE t4; +Table Create Table +t4 CREATE TABLE `t4` ( + `c1` int(11) DEFAULT NULL +) ENGINE=MRG_MyISAM DEFAULT CHARSET=latin1 INSERT_METHOD=FIRST UNION=(`t1`,`t2`) +SELECT * FROM t4; +c1 +1 +2 +1 +2 +DROP TABLE t4; +LOCK TABLES t1 WRITE, t2 WRITE, t3 READ; +CREATE TABLE t4 (c1 INT); +DROP TABLE t4; +CREATE TABLE t4 (c1 INT) SELECT * FROM t3; +ERROR HY000: Table 't4' was not locked with LOCK TABLES +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2) +INSERT_METHOD=LAST SELECT * FROM t3; +ERROR HY000: Table 't4' was not locked with LOCK TABLES +UNLOCK TABLES; +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2); +LOCK TABLES t4 READ; +SELECT * from t1, t4; +ERROR HY000: Table 't1' was not locked with LOCK TABLES +DROP TABLE t4; +UNLOCK TABLES; +DROP TABLE t1, t2, t3; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1, t2); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (2); +SELECT * FROM t3; +c1 +1 +2 +TRUNCATE TABLE t1; +SELECT * FROM t3; +c1 +2 +DROP TABLE t1, t2, t3; +CREATE TABLE t1 (id INTEGER, grp TINYINT, id_rev INTEGER); +SET @rnd_max= 2147483647; +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +SET @rnd= RAND(); +SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); +SET @id_rev= @rnd_max - @id; +SET @grp= CAST(127.0 * @rnd AS UNSIGNED); +INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); +set @@read_buffer_size=2*1024*1024; +CREATE TABLE t2 SELECT * FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +CREATE TABLE t3 (id INTEGER, grp TINYINT, id_rev INTEGER) +ENGINE= MRG_MYISAM UNION= (t1, t2); +SELECT COUNT(*) FROM t1; +COUNT(*) +130 +SELECT COUNT(*) FROM t2; +COUNT(*) +80 +SELECT COUNT(*) FROM t3; +COUNT(*) +210 +SELECT COUNT(DISTINCT a1.id) FROM t3 AS a1, t3 AS a2 +WHERE a1.id = a2.id GROUP BY a2.grp; +TRUNCATE TABLE t1; +SELECT COUNT(*) FROM t1; +COUNT(*) +0 +SELECT COUNT(*) FROM t2; +COUNT(*) +80 +SELECT COUNT(*) FROM t3; +COUNT(*) +80 +DROP TABLE t1, t2, t3; +CREATE TABLE t1 (c1 INT) ENGINE=MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1) INSERT_METHOD=LAST; +INSERT INTO t2 VALUES (1); +SELECT * FROM t2; +c1 +1 +LOCK TABLES t2 WRITE, t1 WRITE; +FLUSH TABLES; +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +CHECK TABLE t1; +Table Op Msg_type Msg_text +test.t1 check status OK +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +Table Op Msg_type Msg_text +test.t1 check status OK +LOCK TABLES t2 WRITE, t1 WRITE; +SELECT * FROM t2; +c1 +1 +LOCK TABLES t2 WRITE, t1 WRITE; +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +CHECK TABLE t1; +Table Op Msg_type Msg_text +test.t1 check status OK +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +Table Op Msg_type Msg_text +test.t1 check status OK +DROP TABLE t1, t2; +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MRG_MYISAM UNION=(t1); +LOCK TABLES t1 WRITE, m1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MRG_MYISAM UNION=(t1); +LOCK TABLES m1 WRITE, t1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; +INSERT INTO t2 VALUES (1); +REPAIR TABLE t1; +Table Op Msg_type Msg_text +test.t1 repair status OK +UNLOCK TABLES; +DROP TABLE t1, t2; +CREATE TABLE t1 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT, c2 INT) ENGINE= MRG_MYISAM UNION(t1, t2); +INSERT INTO t1 VALUES (1, 1); +INSERT INTO t2 VALUES (2, 2); +SELECT * FROM t3; +c1 c2 +1 1 +2 2 +ALTER TABLE t1 ENGINE= MEMORY; +INSERT INTO t1 VALUES (0, 0); +SELECT * FROM t3; +ERROR HY000: Unable to open underlying table which is differently defined or of non-MyISAM type or doesn't exist +DROP TABLE t1, t2, t3; +CREATE TABLE t1 (c1 INT, KEY(c1)); +CREATE TABLE t2 (c1 INT, KEY(c1)) ENGINE=MRG_MYISAM UNION=(t1) +INSERT_METHOD=FIRST; +LOCK TABLE t1 WRITE, t2 WRITE; +FLUSH TABLES t2, t1; +OPTIMIZE TABLE t1; +Table Op Msg_type Msg_text +test.t1 optimize status Table is already up to date +FLUSH TABLES t1; +UNLOCK TABLES; +FLUSH TABLES; +INSERT INTO t1 VALUES (1); +LOCK TABLE t1 WRITE, t2 WRITE; +FLUSH TABLES t2, t1; +OPTIMIZE TABLE t1; +Table Op Msg_type Msg_text +test.t1 optimize status OK +FLUSH TABLES t1; +UNLOCK TABLES; +DROP TABLE t1, t2; +CREATE TABLE t1 (ID INT) ENGINE=MYISAM; +CREATE TABLE m1 (ID INT) ENGINE=MRG_MYISAM UNION=(t1) INSERT_METHOD=FIRST; +INSERT INTO t1 VALUES (); +INSERT INTO m1 VALUES (); +LOCK TABLE t1 WRITE, m1 WRITE; +FLUSH TABLES m1, t1; +OPTIMIZE TABLE t1; +Table Op Msg_type Msg_text +test.t1 optimize status OK +FLUSH TABLES m1, t1; +UNLOCK TABLES; +DROP TABLE t1, m1; diff -Nrup a/mysql-test/r/myisam.result b/mysql-test/r/myisam.result --- a/mysql-test/r/myisam.result 2007-05-29 14:57:33 +02:00 +++ b/mysql-test/r/myisam.result 2007-10-15 16:02:49 +02:00 @@ -606,24 +606,6 @@ select count(*) from t1 where a is null; count(*) 2 drop table t1; -create table t1 (c1 int, index(c1)); -create table t2 (c1 int, index(c1)) engine=merge union=(t1); -insert into t1 values (1); -flush tables; -select * from t2; -c1 -1 -flush tables; -truncate table t1; -insert into t1 values (1); -flush tables; -select * from t2; -c1 -1 -truncate table t1; -ERROR HY000: MyISAM table 't1' is in use (most likely by a MERGE table). Try FLUSH TABLES. -insert into t1 values (1); -drop table t1,t2; create table t1 (c1 int, c2 varchar(4) not null default '', key(c2(3))) default charset=utf8; insert into t1 values (1,'A'), (2, 'B'), (3, 'A'); diff -Nrup a/mysql-test/suite/binlog/r/binlog_stm_blackhole.result b/mysql-test/suite/binlog/r/binlog_stm_blackhole.result --- a/mysql-test/suite/binlog/r/binlog_stm_blackhole.result 2007-06-27 14:27:27 +02:00 +++ b/mysql-test/suite/binlog/r/binlog_stm_blackhole.result 2007-10-15 16:02:49 +02:00 @@ -124,6 +124,7 @@ master-bin.000001 # Query # # use `test` drop table t1,t2,t3; CREATE TABLE t1(a INT) ENGINE=BLACKHOLE; INSERT DELAYED INTO t1 VALUES(1); +ERROR HY000: Binary logging not possible. Message: Row-based format required for this statement, but not allowed by this combination of engines DROP TABLE t1; CREATE TABLE t1(a INT, b INT) ENGINE=BLACKHOLE; DELETE FROM t1 WHERE a=10; diff -Nrup a/mysql-test/t/delayed.test b/mysql-test/t/delayed.test --- a/mysql-test/t/delayed.test 2007-06-06 19:48:06 +02:00 +++ b/mysql-test/t/delayed.test 2007-10-15 16:02:49 +02:00 @@ -248,7 +248,7 @@ DROP TABLE t1; # CREATE TABLE t1(c1 INT) ENGINE=MyISAM; CREATE TABLE t2(c1 INT) ENGINE=MERGE UNION=(t1); ---error 1031 +--error ER_ILLEGAL_HA INSERT DELAYED INTO t2 VALUES(1); DROP TABLE t1, t2; diff -Nrup a/mysql-test/t/merge.test b/mysql-test/t/merge.test --- a/mysql-test/t/merge.test 2007-10-09 14:18:07 +02:00 +++ b/mysql-test/t/merge.test 2007-10-15 16:02:49 +02:00 @@ -221,6 +221,7 @@ create table t2 (a int not null); insert into t1 values (1); insert into t2 values (2); create temporary table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +--error ER_WRONG_MRG_TABLE select * from t3; create temporary table t4 (a int not null); create temporary table t5 (a int not null); @@ -229,6 +230,43 @@ insert into t5 values (2); create temporary table t6 (a int not null) ENGINE=MERGE UNION=(t4,t5); select * from t6; drop table t6, t3, t1, t2, t4, t5; +# +# Bug#19627 - temporary merge table locking +# MERGE table and its children must match in temporary type. +# Forbid temporary merge on non-temporary children: shown above. +# Forbid non-temporary merge on temporary children: +create temporary table t1 (a int not null); +create temporary table t2 (a int not null); +insert into t1 values (1); +insert into t2 values (2); +create table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +--error ER_WRONG_MRG_TABLE +select * from t3; +drop table t3, t2, t1; +# Forbid children mismatch in temporary: +create table t1 (a int not null); +create temporary table t2 (a int not null); +insert into t1 values (1); +insert into t2 values (2); +create table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +--error ER_WRONG_MRG_TABLE +select * from t3; +drop table t3; +create temporary table t3 (a int not null) ENGINE=MERGE UNION=(t1,t2); +--error ER_WRONG_MRG_TABLE +select * from t3; +drop table t3, t2, t1; +# Show CREATE...SELECT under LOCK TABLES: +CREATE TEMPORARY TABLE t1 (c1 INT NOT NULL); +CREATE TEMPORARY TABLE t2 (c1 INT NOT NULL); +CREATE TABLE t3 (c1 INT NOT NULL); +INSERT INTO t3 VALUES (1), (2); +LOCK TABLES t3 READ; +CREATE TEMPORARY TABLE t4 (c1 INT NOT NULL) ENGINE=MERGE UNION=(t1,t2) + INSERT_METHOD=LAST SELECT * FROM t3; +SELECT * FROM t4; +UNLOCK TABLES; +DROP TABLE t4, t3, t2, t1; # # testing merge::records_in_range and optimizer @@ -403,7 +441,7 @@ CREATE TABLE t2(a INT) ENGINE=MERGE UNIO SELECT * FROM t2; DROP TABLE t1, t2; CREATE TABLE t2(a INT) ENGINE=MERGE UNION=(t3); ---error 1168 +--error ER_NO_SUCH_TABLE SELECT * FROM t2; DROP TABLE t2; @@ -495,11 +533,11 @@ drop table t1; # CREATE TABLE fails # CREATE TABLE tm1(a INT) ENGINE=MERGE UNION=(t1, t2); ---error 1168 +--error ER_NO_SUCH_TABLE SELECT * FROM tm1; CHECK TABLE tm1; CREATE TABLE t1(a INT); ---error 1168 +--error ER_NO_SUCH_TABLE SELECT * FROM tm1; CHECK TABLE tm1; CREATE TABLE t2(a BLOB); @@ -526,3 +564,349 @@ CREATE TABLE IF NOT EXISTS t1 SELECT * F DROP TABLE t1, t2; --echo End of 5.0 tests + +# +# Bug #8306: TRUNCATE leads to index corruption +# +create table t1 (c1 int, index(c1)); +create table t2 (c1 int, index(c1)) engine=merge union=(t1); +insert into t1 values (1); +# Close all tables. +flush tables; +# Open t2 and (implicitly) t1. +select * from t2; +# Truncate after flush works (unless another threads reopens t2 in between). +flush tables; +truncate table t1; +insert into t1 values (1); +# Close all tables. +flush tables; +# Open t2 and (implicitly) t1. +select * from t2; +# Truncate t1, wich was not recognized as open without the bugfix. +# After fix for Bug#8306 and before fix for Bug#26379, +# it should fail with a table-in-use error message, otherwise succeed. +truncate table t1; +# The insert used to fail on the crashed table. +insert into t1 values (1); +drop table t1,t2; + +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Preparation +connect (con1,localhost,root,,); +connect (con2,localhost,root,,); +connection default; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Problem #1 +# A thread trying to lock a MERGE table performed busy waiting while +# REPAIR TABLE or a similar table administration task was ongoing on one or +# more of its MyISAM tables. +# To allow for observability it was necessary to enter a multi-second sleep +# in mysql_admin_table() after remove_table_from_cache(), which comes after +# mysql_abort_lock(). The sleep faked a long running operation. One could +# watch a high CPU load during the sleep time. +# The problem was that mysql_abort_lock() upgrades the write lock to +# TL_WRITE_ONLY. This lock type persisted until the final unlock at the end +# of the administration task. The effect of TL_WRITE_ONLY is to reject any +# attempt to lock the table. The trying thread must close the table and wait +# until it is no longer used. Unfortunately there is no way to detect that +# one of the MyISAM tables of a MERGE table is in use. When trying to lock +# the MERGE table, all MyISAM tables are locked. If one fails on +# TL_WRITE_ONLY, all locks are aborted and wait_for_tables() is entered. +# But this doesn't see the MERGE table as used, so it seems appropriate to +# retry a lock... +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +send REPAIR TABLE t1; + connection con1; + sleep 1; # let repair run into its sleep + INSERT INTO t2 VALUES (1); +connection default; +reap; +DROP TABLE t1, t2; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Problem #2 +# A thread trying to lock a MERGE table performed busy waiting until all +# threads that did REPAIR TABLE or similar table administration tasks on +# one or more of its MyISAM tables in LOCK TABLES segments did +# UNLOCK TABLES. +# The difference against problem #1 is that the busy waiting took place +# *after* the administration task. It was terminated by UNLOCK TABLES only. +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; + connection con1; + send INSERT INTO t2 VALUES (1); +connection default; +sleep 1; # Let INSERT go into thr_multi_lock(). +REPAIR TABLE t1; +sleep 2; # con1 performs busy waiting during this sleep. +UNLOCK TABLES; + connection con1; + reap; +connection default; +DROP TABLE t1, t2; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Problem #3 +# Two FLUSH TABLES within a LOCK TABLES segment could invalidate the lock. +# This did *not* require a MERGE table. +# To increase reproducibility it was necessary to enter a sleep of 2 seconds +# at the end of wait_for_tables() after unlock of LOCK_open. In 5.0 and 5.1 +# the sleep must be inserted in open_and_lock_tables() after open_tables() +# instead. wait_for_tables() is not used in this case. +# The problem was that FLUSH TABLES releases LOCK_open while having unlocked +# and closed all tables. When this happened while a thread was in the loop in +# mysql_lock_tables() right after wait_for_tables() and before retrying to +# lock, the thread got the lock. (Translate to similar code places in 5.0 +# and 5.1). And it did not notice that the table needed a refresh. So it +# executed its statement on the table. +# The first FLUSH TABLES kicked the INSERT out of thr_multi_lock() and let +# it wait in wait_for_tables(). (open_table() in 5.0 and 5.1). The second +# FLUSH TABLES must happen while the INSERT was on its way from +# wait_for_tables() to the next call of thr_multi_lock(). This needed to be +# supported by a sleep to make it repeatable. +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +LOCK TABLE t1 WRITE; + connection con1; + send INSERT INTO t1 VALUES (1); +connection default; +sleep 1; # Let INSERT go into thr_multi_lock(). +FLUSH TABLES; +sleep 1; # Let INSERT go through wait_for_tables() where it sleeps. +FLUSH TABLES; +# This should give no result. But it will with sleep(2) at the right place. +SELECT * FROM t1; +UNLOCK TABLES; + connection con1; + reap; +connection default; +DROP TABLE t1; +# +# Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table +# Cleanup +disconnect con1; +disconnect con2; +# +# Extra test for Bug#26379. +# Test usage of reopen_name_locked_table() in CREATE ... SELECT +# +CREATE TABLE t1 (c1 INT); +CREATE TABLE t2 (c1 INT); +CREATE TABLE t3 (c1 INT); +INSERT INTO t3 VALUES (1), (2); +# Show that LAST table is opened. +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2) + INSERT_METHOD=LAST SELECT * FROM t3; +SHOW CREATE TABLE t4; +SELECT * FROM t4; +DROP TABLE t4; +# Show that FIRST table is opened. +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2) + INSERT_METHOD=FIRST SELECT * FROM t3; +SHOW CREATE TABLE t4; +SELECT * FROM t4; +DROP TABLE t4; +LOCK TABLES t1 WRITE, t2 WRITE, t3 READ; +CREATE TABLE t4 (c1 INT); +DROP TABLE t4; +--error ER_TABLE_NOT_LOCKED +CREATE TABLE t4 (c1 INT) SELECT * FROM t3; +--error ER_TABLE_NOT_LOCKED +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2) + INSERT_METHOD=LAST SELECT * FROM t3; +UNLOCK TABLES; +CREATE TABLE t4 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1,t2); +LOCK TABLES t4 READ; +--error ER_TABLE_NOT_LOCKED +SELECT * from t1, t4; +# Show that dropping a lcked MERGE table works. +DROP TABLE t4; +UNLOCK TABLES; +DROP TABLE t1, t2, t3; + +# +# Bug#25038 - Waiting TRUNCATE +# +# Show that truncate of child table after use of parent table works. +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1, t2); +INSERT INTO t1 VALUES (1); +INSERT INTO t2 VALUES (2); +SELECT * FROM t3; +TRUNCATE TABLE t1; +SELECT * FROM t3; +DROP TABLE t1, t2, t3; +# +# Show that truncate of child table waits while parent table is used. +# (test partly borrowed from count_distinct3.) +CREATE TABLE t1 (id INTEGER, grp TINYINT, id_rev INTEGER); +SET @rnd_max= 2147483647; +let $1 = 10; +while ($1) +{ + SET @rnd= RAND(); + SET @id = CAST(@rnd * @rnd_max AS UNSIGNED); + SET @id_rev= @rnd_max - @id; + SET @grp= CAST(127.0 * @rnd AS UNSIGNED); + INSERT INTO t1 (id, grp, id_rev) VALUES (@id, @grp, @id_rev); + dec $1; +} +set @@read_buffer_size=2*1024*1024; +CREATE TABLE t2 SELECT * FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +INSERT INTO t2 (id, grp, id_rev) SELECT id, grp, id_rev FROM t1; +INSERT INTO t1 (id, grp, id_rev) SELECT id, grp, id_rev FROM t2; +CREATE TABLE t3 (id INTEGER, grp TINYINT, id_rev INTEGER) + ENGINE= MRG_MYISAM UNION= (t1, t2); +SELECT COUNT(*) FROM t1; +SELECT COUNT(*) FROM t2; +SELECT COUNT(*) FROM t3; +connect (con1,localhost,root,,); + # As t3 contains random numbers, results are different from test to test. + # That's okay, because we test only that select doesn't yield an + # error. Note, that --disable_result_log doesn't suppress error output. + --disable_result_log + send SELECT COUNT(DISTINCT a1.id) FROM t3 AS a1, t3 AS a2 + WHERE a1.id = a2.id GROUP BY a2.grp; +connection default; +sleep 1; +TRUNCATE TABLE t1; + connection con1; + reap; + --enable_result_log + disconnect con1; +connection default; +SELECT COUNT(*) FROM t1; +SELECT COUNT(*) FROM t2; +SELECT COUNT(*) FROM t3; +DROP TABLE t1, t2, t3; + +# +# Bug#25700 - merge base tables get corrupted by optimize/analyze/repair table +# +# Using FLUSH TABLES before REPAIR. +CREATE TABLE t1 (c1 INT) ENGINE=MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE=MRG_MYISAM UNION=(t1) INSERT_METHOD=LAST; +INSERT INTO t2 VALUES (1); +SELECT * FROM t2; +LOCK TABLES t2 WRITE, t1 WRITE; +FLUSH TABLES; +REPAIR TABLE t1; +CHECK TABLE t1; +REPAIR TABLE t1; +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +# +# Not using FLUSH TABLES before REPAIR. +LOCK TABLES t2 WRITE, t1 WRITE; +SELECT * FROM t2; +LOCK TABLES t2 WRITE, t1 WRITE; +REPAIR TABLE t1; +CHECK TABLE t1; +REPAIR TABLE t1; +UNLOCK TABLES; +CHECK TABLE t1 EXTENDED; +DROP TABLE t1, t2; + +# +# Bug#26377 - Deadlock with MERGE and FLUSH TABLE +# +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MRG_MYISAM UNION=(t1); +# Lock t1 first. This did always work. +LOCK TABLES t1 WRITE, m1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; +# +CREATE TABLE t1 ( a INT ) ENGINE=MyISAM; +CREATE TABLE m1 ( a INT ) ENGINE=MRG_MYISAM UNION=(t1); +# Lock m1 first. This did deadlock. +LOCK TABLES m1 WRITE, t1 WRITE; +FLUSH TABLE t1; +UNLOCK TABLES; +DROP TABLE m1, t1; + +# +# Bug#26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging +# +CREATE TABLE t1 (c1 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT) ENGINE= MRG_MYISAM UNION= (t1) INSERT_METHOD= LAST; +LOCK TABLE t1 WRITE; + connect (con1,localhost,root,,); + send INSERT INTO t2 VALUES (1); +connection default; +sleep 1; # Let INSERT go into thr_multi_lock(). +REPAIR TABLE t1; +sleep 2; # con1 performs busy waiting during this sleep. +UNLOCK TABLES; + connection con1; + reap; + disconnect con1; +connection default; +DROP TABLE t1, t2; + +# +# Bug#27660 - Falcon: merge table possible +# +# Normal MyISAM MERGE operation. +CREATE TABLE t1 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t2 (c1 INT, c2 INT) ENGINE= MyISAM; +CREATE TABLE t3 (c1 INT, c2 INT) ENGINE= MRG_MYISAM UNION(t1, t2); +INSERT INTO t1 VALUES (1, 1); +INSERT INTO t2 VALUES (2, 2); +SELECT * FROM t3; +# Try an unsupported engine. +ALTER TABLE t1 ENGINE= MEMORY; +INSERT INTO t1 VALUES (0, 0); +# Before fixing, this succeeded, but (0, 0) was missing. +--error 1168 +SELECT * FROM t3; +DROP TABLE t1, t2, t3; + +# +# Bug#30275 - Merge tables: flush tables or unlock tables causes server to crash +# +CREATE TABLE t1 (c1 INT, KEY(c1)); +CREATE TABLE t2 (c1 INT, KEY(c1)) ENGINE=MRG_MYISAM UNION=(t1) + INSERT_METHOD=FIRST; +LOCK TABLE t1 WRITE, t2 WRITE; +FLUSH TABLES t2, t1; +OPTIMIZE TABLE t1; +FLUSH TABLES t1; +UNLOCK TABLES; +# +FLUSH TABLES; +INSERT INTO t1 VALUES (1); +LOCK TABLE t1 WRITE, t2 WRITE; +FLUSH TABLES t2, t1; +OPTIMIZE TABLE t1; +FLUSH TABLES t1; +UNLOCK TABLES; +DROP TABLE t1, t2; + +# +# Test derived from test program for +# Bug#30273 - merge tables: Can't lock file (errno: 155) +# +CREATE TABLE t1 (ID INT) ENGINE=MYISAM; +CREATE TABLE m1 (ID INT) ENGINE=MRG_MYISAM UNION=(t1) INSERT_METHOD=FIRST; +INSERT INTO t1 VALUES (); +INSERT INTO m1 VALUES (); +LOCK TABLE t1 WRITE, m1 WRITE; +FLUSH TABLES m1, t1; +OPTIMIZE TABLE t1; +FLUSH TABLES m1, t1; +UNLOCK TABLES; +DROP TABLE t1, m1; + diff -Nrup a/mysql-test/t/myisam.test b/mysql-test/t/myisam.test --- a/mysql-test/t/myisam.test 2007-08-29 14:46:27 +02:00 +++ b/mysql-test/t/myisam.test 2007-10-15 16:02:49 +02:00 @@ -576,32 +576,6 @@ select count(*) from t1 where a is null; drop table t1; # -# Bug #8306: TRUNCATE leads to index corruption -# -create table t1 (c1 int, index(c1)); -create table t2 (c1 int, index(c1)) engine=merge union=(t1); -insert into t1 values (1); -# Close all tables. -flush tables; -# Open t2 and (implicitly) t1. -select * from t2; -# Truncate after flush works (unless another threads reopens t2 in between). -flush tables; -truncate table t1; -insert into t1 values (1); -# Close all tables. -flush tables; -# Open t2 and (implicitly) t1. -select * from t2; -# Truncate t1, wich was not recognized as open without the bugfix. -# Now, it should fail with a table-in-use error message. ---error 1105 -truncate table t1; -# The insert used to fail on the crashed table. -insert into t1 values (1); -drop table t1,t2; - -# # bug9188 - Corruption Can't open file: 'table.MYI' (errno: 145) # create table t1 (c1 int, c2 varchar(4) not null default '', diff -Nrup a/sql/ha_ndbcluster_binlog.cc b/sql/ha_ndbcluster_binlog.cc --- a/sql/ha_ndbcluster_binlog.cc 2007-09-07 11:15:04 +02:00 +++ b/sql/ha_ndbcluster_binlog.cc 2007-10-15 16:02:49 +02:00 @@ -321,7 +321,7 @@ ndbcluster_binlog_open_table(THD *thd, N DBUG_ENTER("ndbcluster_binlog_open_table"); safe_mutex_assert_owner(&LOCK_open); - init_tmp_table_share(table_share, share->db, 0, share->table_name, + init_tmp_table_share(thd, table_share, share->db, 0, share->table_name, share->key); if ((error= open_table_def(thd, table_share, 0))) { diff -Nrup a/sql/handler.cc b/sql/handler.cc --- a/sql/handler.cc 2007-08-31 08:57:57 +02:00 +++ b/sql/handler.cc 2007-10-15 16:02:49 +02:00 @@ -2582,7 +2582,7 @@ int ha_create_table(THD *thd, const char TABLE_SHARE share; DBUG_ENTER("ha_create_table"); - init_tmp_table_share(&share, db, 0, table_name, path); + init_tmp_table_share(thd, &share, db, 0, table_name, path); if (open_table_def(thd, &share, 0) || open_table_from_share(thd, &share, "", 0, (uint) READ_ALL, 0, &table, TRUE)) @@ -2649,7 +2649,7 @@ int ha_create_table_from_engine(THD* thd if (error) DBUG_RETURN(2); - init_tmp_table_share(&share, db, 0, name, path); + init_tmp_table_share(thd, &share, db, 0, name, path); if (open_table_def(thd, &share, 0)) { DBUG_RETURN(3); @@ -3655,11 +3655,12 @@ int handler::ha_reset() int handler::ha_write_row(uchar *buf) { int error; + DBUG_ENTER("handler::ha_write_row"); if (unlikely(error= write_row(buf))) - return error; + DBUG_RETURN(error); if (unlikely(error= binlog_log_row<Write_rows_log_event>(table, 0, buf))) - return error; - return 0; + DBUG_RETURN(error); + DBUG_RETURN(0); } diff -Nrup a/sql/mysql_priv.h b/sql/mysql_priv.h --- a/sql/mysql_priv.h 2007-08-31 20:13:22 +02:00 +++ b/sql/mysql_priv.h 2007-10-15 16:02:49 +02:00 @@ -1142,6 +1142,8 @@ TABLE *table_cache_insert_placeholder(TH bool lock_table_name_if_not_cached(THD *thd, const char *db, const char *table_name, TABLE **table); TABLE *find_locked_table(THD *thd, const char *db,const char *table_name); +bool fix_merge_after_open(TABLE_LIST *old_child_list, TABLE_LIST **old_last, + TABLE_LIST *new_child_list, TABLE_LIST **new_last); bool reopen_table(TABLE *table); bool reopen_tables(THD *thd,bool get_locks,bool in_refresh); void close_data_files_and_morph_locks(THD *thd, const char *db, @@ -1392,8 +1394,21 @@ int init_ftfuncs(THD *thd, SELECT_LEX* s void wait_for_condition(THD *thd, pthread_mutex_t *mutex, pthread_cond_t *cond); int open_tables(THD *thd, TABLE_LIST **tables, uint *counter, uint flags); -int simple_open_n_lock_tables(THD *thd,TABLE_LIST *tables); -bool open_and_lock_tables(THD *thd,TABLE_LIST *tables); +/* open_and_lock_tables with optional derived handling */ +bool open_and_lock_tables_derived(THD *thd, TABLE_LIST *tables, bool derived); +/* simple open_and_lock_tables without derived handling */ +inline bool simple_open_n_lock_tables(THD *thd, TABLE_LIST *tables) +{ + return open_and_lock_tables_derived(thd, tables, FALSE); +} +/* open_and_lock_tables with derived handling */ +inline bool open_and_lock_tables(THD *thd, TABLE_LIST *tables) +{ + return open_and_lock_tables_derived(thd, tables, TRUE); +} +/* simple open_and_lock_tables without derived handling for single table */ +TABLE *open_n_lock_single_table(THD *thd, TABLE_LIST *table_l, + thr_lock_type lock_type); bool open_normal_and_derived_tables(THD *thd, TABLE_LIST *tables, uint flags); int lock_tables(THD *thd, TABLE_LIST *tables, uint counter, bool *need_reopen); int decide_logging_format(THD *thd, TABLE_LIST *tables); @@ -1969,7 +1984,8 @@ int format_number(uint inputflag,uint ma /* table.cc */ TABLE_SHARE *alloc_table_share(TABLE_LIST *table_list, char *key, uint key_length); -void init_tmp_table_share(TABLE_SHARE *share, const char *key, uint key_length, +void init_tmp_table_share(THD *thd, TABLE_SHARE *share, const char *key, + uint key_length, const char *table_name, const char *path); void free_table_share(TABLE_SHARE *share); int open_table_def(THD *thd, TABLE_SHARE *share, uint db_flags); diff -Nrup a/sql/slave.cc b/sql/slave.cc --- a/sql/slave.cc 2007-08-29 23:28:34 +02:00 +++ b/sql/slave.cc 2007-10-15 16:02:49 +02:00 @@ -1013,8 +1013,8 @@ static int create_table_from_dump(THD* t goto err; // mysql_parse took care of the error send thd->proc_info = "Opening master dump table"; - tables.lock_type = TL_WRITE; - if (!open_ltable(thd, &tables, TL_WRITE, 0)) + tables.table= NULL; /* was set by mysql_rm_table(). */ + if (!open_n_lock_single_table(thd, &tables, TL_WRITE)) { sql_print_error("create_table_from_dump: could not open created table"); goto err; diff -Nrup a/sql/sql_base.cc b/sql/sql_base.cc --- a/sql/sql_base.cc 2007-09-14 19:29:17 +02:00 +++ b/sql/sql_base.cc 2007-10-15 16:02:49 +02:00 @@ -671,6 +671,64 @@ TABLE_SHARE *get_cached_table_share(cons } +/** + @brief Detach MERGE children from the parent. + + @note When closing thread tables at end of statement, both parent and + children are in thd->open_tables and will be closed. In most cases + the children will be closed before the parent. They are opened after + the parent and thus stacked into thd->open_tables before it. + + Hence it is vital that detach does not touch the children in any + way. + + All references to the children should be removed. + + @param[in] table the TABLE object of the parent + @param[in] clear_refs if to clear TABLE references + this must be true when called from + close_thread_tables() to enable fresh + open in open_tables() + it must be false when called in preparation + for reopen_tables() +*/ + +static void detach_merge_children(TABLE *table, bool clear_refs) +{ + TABLE_LIST *child_l; + DBUG_ENTER("detach_merge_children"); + DBUG_PRINT("myrg", ("table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); + DBUG_PRINT("myrg", ("MERGE parent table, detach children")); + + /* + In a open_tables() loop it can happen that not all tables have their + children attached yet. + */ + if (table->children_attached) + { + VOID(table->file->extra(HA_EXTRA_DETACH_CHILDREN)); + table->children_attached= FALSE; + } + + if (clear_refs) + { + /* Clear TABLE references to force new assignment at next open.*/ + for (child_l= table->child_l; ; child_l= child_l->next_global) + { + /* Do not DBUG_ASSERT(child_l->table); open_tables might be incomplete. */ + if (child_l->table) + child_l->table->parent= NULL; + child_l->table= NULL; + /* Break when this was the last child. */ + if (&child_l->next_global == table->child_last_l) + break; + } + } + DBUG_VOID_RETURN; +} + + /* Close file handle, but leave the table in the table cache @@ -711,6 +769,12 @@ void close_handle_and_leave_table_as_loc share->tmp_table= INTERNAL_TMP_TABLE; // for intern_close_table() } + /* + When closing a MERGE parent table, detach its children first. + Do not clear child table references to allow for reopen. + */ + if (table->child_l) + detach_merge_children(table, FALSE); table->file->close(); table->db_stat= 0; // Mark file closed release_table_share(table->s, RELEASE_NORMAL); @@ -1116,11 +1180,21 @@ static void mark_used_tables_as_free_for void close_thread_tables(THD *thd, bool lock_in_use, bool skip_derived) { + TABLE *table; bool found_old_table; prelocked_mode_type prelocked_mode= thd->prelocked_mode; DBUG_ENTER("close_thread_tables"); /* + Detach MERGE children to allow new attach at next open. + */ + for (table= thd->temporary_tables; table; table= table->next) + { + if (table->child_l) + detach_merge_children(table, TRUE); + } + + /* We are assuming here that thd->derived_tables contains ONLY derived tables for this substatement. i.e. instead of approach which uses query_id matching for determining which of the derived tables belong @@ -1133,7 +1207,7 @@ void close_thread_tables(THD *thd, bool */ if (thd->derived_tables && !skip_derived) { - TABLE *table, *next; + TABLE *next; /* Close all derived tables generated in queries like SELECT * FROM (SELECT * FROM t1) @@ -1262,8 +1336,17 @@ bool close_thread_table(THD *thd, TABLE DBUG_ENTER("close_thread_table"); DBUG_ASSERT(table->key_read == 0); DBUG_ASSERT(!table->file || table->file->inited == handler::NONE); + DBUG_PRINT("tcache", ("table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); *table_ptr=table->next; + /* + When closing a MERGE parent table, detach its children first. + Clear child table references to force new assignment at next open. + */ + if (table->child_l) + detach_merge_children(table, TRUE); + if (table->needs_reopen_or_name_lock() || thd->version != refresh_version || !table->db_stat) { @@ -1696,16 +1779,19 @@ TABLE *find_temporary_table(THD *thd, TA bool close_temporary_table(THD *thd, TABLE_LIST *table_list) { TABLE *table; + DBUG_ENTER("close_temporary_table"); + DBUG_PRINT("tmptable", ("closing table: '%s'.'%s'", + table_list->db, table_list->table_name)); if (!(table= find_temporary_table(thd, table_list))) - return 1; + DBUG_RETURN(1); /* If LOCK TABLES list is not empty and contains this table, unlock the table and remove the table from this list. */ mysql_lock_remove(thd, thd->locked_tables, table, FALSE); close_temporary_table(thd, table, 1, 1); - return 0; + DBUG_RETURN(0); } /* @@ -1715,6 +1801,11 @@ bool close_temporary_table(THD *thd, TAB void close_temporary_table(THD *thd, TABLE *table, bool free_share, bool delete_table) { + DBUG_ENTER("close_temporary_table"); + DBUG_PRINT("tmptable", ("closing table: '%s'.'%s' 0x%lx alias: '%s'", + table->s->db.str, table->s->table_name.str, + (long) table, table->alias)); + if (table->prev) { table->prev->next= table->next; @@ -1741,6 +1832,7 @@ void close_temporary_table(THD *thd, TAB slave_open_temp_tables--; } close_temporary(table, free_share, delete_table); + DBUG_VOID_RETURN; } @@ -1756,6 +1848,8 @@ void close_temporary(TABLE *table, bool { handlerton *table_type= table->s->db_type(); DBUG_ENTER("close_temporary"); + DBUG_PRINT("tmptable", ("closing table: '%s'.'%s'", + table->s->db.str, table->s->table_name.str)); free_io_cache(table); closefrm(table, 0); @@ -1849,7 +1943,8 @@ void unlink_open_table(THD *thd, TABLE * if (list->s->table_cache_key.length == key_length && !memcmp(list->s->table_cache_key.str, key, key_length)) { - if (unlock && thd->locked_tables) + /* Do not handle locks of MERGE children. */ + if (unlock && thd->locked_tables && !list->parent) mysql_lock_remove(thd, thd->locked_tables, list, TRUE); VOID(hash_delete(&open_cache,(uchar*) list)); // Close table } @@ -2341,9 +2436,14 @@ TABLE *open_table(THD *thd, TABLE_LIST * table->s->table_name.str); DBUG_RETURN(0); } + /* + When looking for a usable TABLE, ignore MERGE children, as they + belong to their parent and cannot be used explicitly. + */ if (!my_strcasecmp(system_charset_info, table->alias, alias) && table->query_id != thd->query_id && /* skip tables already used */ - !(thd->prelocked_mode && table->query_id)) + !(thd->prelocked_mode && table->query_id) && + !table->parent) { int distance= ((int) table->reginfo.lock_type - (int) table_list->lock_type); @@ -2486,6 +2586,8 @@ TABLE *open_table(THD *thd, TABLE_LIST * table= (TABLE*) hash_next(&open_cache, (uchar*) key, key_length, &state)) { + DBUG_PRINT("tcache", ("in_use table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); /* Here we flush tables marked for flush. Normally, table->s->version contains the value of @@ -2570,6 +2672,8 @@ TABLE *open_table(THD *thd, TABLE_LIST * } if (table) { + DBUG_PRINT("tcache", ("unused table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); /* Unlink the table from "unused_tables" list. */ if (table == unused_tables) { // First unused @@ -2585,6 +2689,7 @@ TABLE *open_table(THD *thd, TABLE_LIST * { /* Insert a new TABLE instance into the open cache */ int error; + DBUG_PRINT("tcache", ("opening new table")); /* Free cache if too big */ while (open_cache.records > table_cache_size && unused_tables) VOID(hash_delete(&open_cache,(uchar*) unused_tables)); /* purecov: tested */ @@ -2651,7 +2756,9 @@ TABLE *open_table(THD *thd, TABLE_LIST * VOID(pthread_mutex_unlock(&LOCK_open)); DBUG_RETURN(0); // VIEW } - DBUG_PRINT("info", ("inserting table 0x%lx into the cache", (long) table)); + DBUG_PRINT("info", ("inserting table '%s'.'%s' 0x%lx into the cache", + table->s->db.str, table->s->table_name.str, + (long) table)); VOID(my_hash_insert(&open_cache,(uchar*) table)); } @@ -2716,6 +2823,101 @@ TABLE *find_locked_table(THD *thd, const } +/** + @brief Fix MERGE children after open. + + @param[in] old_child_list first list member from original table + @param[in] old_last pointer to &next_global of last list member + @param[in] new_child_list first list member from freshly opened table + @param[in] new_last pointer to &next_global of last list member + + @return mismatch + @retval FALSE OK, no mismatch + @retval TRUE Error, lists mismatch + + @detail Main action is to copy TABLE reference for each member of + original child list to new child list. After a fresh open these + references are NULL. Assign the old children to the new table. + Some of them might also be reopened or will be reopened soon. + + Other action is to verify that the table definition with respect to + the UNION list did not change. +*/ + +bool fix_merge_after_open(TABLE_LIST *old_child_list, TABLE_LIST **old_last, + TABLE_LIST *new_child_list, TABLE_LIST **new_last) +{ + bool mismatch= FALSE; + DBUG_ENTER("fix_merge_after_open"); + DBUG_PRINT("myrg", ("old last addr: 0x%lx new last addr: 0x%lx", + (long) old_last, (long) new_last)); + + for (;;) + { + DBUG_PRINT("myrg", ("old list item: 0x%lx new list item: 0x%lx", + (long) old_child_list, (long) new_child_list)); + /* Break if one of the list is at its end. */ + if (!old_child_list || !new_child_list) + break; + /* Old table has references to child TABLEs. */ + DBUG_ASSERT(old_child_list->table); + /* New table does not yet have references to child TABLEs. */ + DBUG_ASSERT(!new_child_list->table); + DBUG_PRINT("myrg", ("old table: '%s'.'%s' new table: '%s'.'%s'", + old_child_list->db, old_child_list->table_name, + new_child_list->db, new_child_list->table_name)); + /* Child db.table names must match. */ + if (strcmp(old_child_list->table_name, new_child_list->table_name) || + strcmp(old_child_list->db, new_child_list->db)) + { + mismatch= TRUE; + break; + } + /* + Copy TABLE reference. Child TABLE objects are still in place + though not necessarily open yet. + */ + DBUG_PRINT("myrg", ("old table ref: 0x%lx replaces new table ref: 0x%lx", + (long) old_child_list->table, + (long) new_child_list->table)); + new_child_list->table= old_child_list->table; + /* + Break if one of the lists is at its last child. When the child + list is part of the statement list, the list may continue after + the last child. The test for NULL is insufficient. + */ + DBUG_PRINT("myrg", ("old next addr: 0x%lx new next addr: 0x%lx", + (long) &old_child_list->next_global, + (long) &new_child_list->next_global)); + if ((old_last && (&old_child_list->next_global == old_last)) || + (new_last && (&new_child_list->next_global == new_last))) + break; + /* Step both lists. */ + old_child_list= old_child_list->next_global; + new_child_list= new_child_list->next_global; + } + DBUG_PRINT("myrg", ("end of list, mismatch: %d", mismatch)); + if (!mismatch && + (/* old list at end */ + (!old_child_list || (!old_last && !old_child_list->next_global) || + (old_last && (&old_child_list->next_global == old_last))) && + /* and new list not at end */ + (new_child_list && new_child_list->next_global && + (!new_last || (&new_child_list->next_global != new_last)))) || + (/* new list at end */ + (!new_child_list || (!new_last && !new_child_list->next_global) || + (new_last && (&new_child_list->next_global == new_last))) && + /* and old list not at end */ + (old_child_list && old_child_list->next_global && + (!old_last || (&old_child_list->next_global != old_last))))) + mismatch= TRUE; + if (mismatch) + my_error(ER_TABLE_DEF_CHANGED, MYF(0)); + + DBUG_RETURN(mismatch); +} + + /* Reopen an table because the definition has changed. @@ -2741,9 +2943,12 @@ bool reopen_table(TABLE *table) TABLE_LIST table_list; THD *thd= table->in_use; DBUG_ENTER("reopen_table"); + DBUG_PRINT("tcache", ("table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); DBUG_ASSERT(table->s->ref_count == 0); DBUG_ASSERT(!table->sort.io_cache); + DBUG_ASSERT(!table->children_attached); #ifdef EXTRA_DEBUG if (table->db_stat) @@ -2784,6 +2989,17 @@ bool reopen_table(TABLE *table) tmp.next= table->next; tmp.prev= table->prev; + /* Preserve MERGE parent. */ + tmp.parent= table->parent; + /* Fix MERGE child list and check for unchanged union. */ + if ((table->child_l || tmp.child_l) && + fix_merge_after_open(table->child_l, table->child_last_l, + tmp.child_l, tmp.child_last_l)) + { + VOID(closefrm(&tmp, 1)); // close file, free everything + goto end; + } + delete table->triggers; if (table->file) VOID(closefrm(table, 1)); // close file, free everything @@ -2805,6 +3021,11 @@ bool reopen_table(TABLE *table) } if (table->triggers) table->triggers->set_table(table); + /* + Do not attach MERGE children here. The children might be reopened + after the parent. Do it after reopening all tables that require + reopen. See for example reopen_tables(). + */ broadcast_refresh(); error=0; @@ -2856,7 +3077,8 @@ void close_data_files_and_morph_locks(TH if (!strcmp(table->s->table_name.str, table_name) && !strcmp(table->s->db.str, db)) { - if (thd->locked_tables) + /* Do not handle locks of MERGE children. */ + if (thd->locked_tables && !table->parent) mysql_lock_remove(thd, thd->locked_tables, table, TRUE); table->open_placeholder= 1; close_handle_and_leave_table_as_lock(table); @@ -2891,6 +3113,7 @@ bool reopen_tables(THD *thd,bool get_loc TABLE *table,*next,**prev; TABLE **tables,**tables_ptr; // For locks bool error=0, not_used; + bool merge_table_found= FALSE; DBUG_ENTER("reopen_tables"); if (!thd->open_tables) @@ -2899,10 +3122,15 @@ bool reopen_tables(THD *thd,bool get_loc safe_mutex_assert_owner(&LOCK_open); if (get_locks) { - /* The ptr is checked later */ + /* + The ptr is checked later + Do not handle locks of MERGE children. + */ uint opens=0; for (table= thd->open_tables; table ; table=table->next) - opens++; + if (!table->parent) + opens++; + DBUG_PRINT("tcache", ("open tables to lock: %u", opens)); tables= (TABLE**) my_alloca(sizeof(TABLE*)*opens); } else @@ -2914,6 +3142,12 @@ bool reopen_tables(THD *thd,bool get_loc { uint db_stat=table->db_stat; next=table->next; + DBUG_PRINT("tcache", ("open table: '%s'.'%s' 0x%lx " + "parent: 0x%lx db_stat: %u", + table->s->db.str, table->s->table_name.str, + (long) table, (long) table->parent, db_stat)); + if (table->child_l && !db_stat) + merge_table_found= TRUE; if (!tables || (!db_stat && reopen_table(table))) { my_error(ER_CANT_REOPEN_TABLE, MYF(0), table->alias); @@ -2922,9 +3156,12 @@ bool reopen_tables(THD *thd,bool get_loc } else { + DBUG_PRINT("tcache", ("opened. need lock: %d", + get_locks && !db_stat && !table->parent)); *prev= table; prev= &table->next; - if (get_locks && !db_stat) + /* Do not handle locks of MERGE children. */ + if (get_locks && !db_stat && !table->parent) *tables_ptr++= table; // need new lock on this if (in_refresh) { @@ -2933,6 +3170,36 @@ bool reopen_tables(THD *thd,bool get_loc } } } + /* + When all tables are open again, we can re-attach MERGE children to + their parents. All TABLE objects are still present. + */ + DBUG_PRINT("tcache", ("re-attaching MERGE tables: %d", merge_table_found)); + if (!error && merge_table_found) + { + for (table= thd->open_tables; table; table= next) + { + next= table->next; + DBUG_PRINT("tcache", ("check table: '%s'.'%s' 0x%lx next: 0x%lx", + table->s->db.str, table->s->table_name.str, + (long) table, (long) next)); + /* Reattach children for MERGE tables with "closed data files" only. */ + if (table->child_l && !table->children_attached) + { + DBUG_PRINT("tcache", ("MERGE parent, attach children")); + if(table->file->extra(HA_EXTRA_ATTACH_CHILDREN)) + { + my_error(ER_CANT_REOPEN_TABLE, MYF(0), table->alias); + VOID(hash_delete(&open_cache,(uchar*) table)); + error=1; + } + else + table->children_attached= TRUE; + } + } + } + DBUG_PRINT("tcache", ("open tables to lock: %u", + (uint) (tables_ptr - tables))); if (tables != tables_ptr) // Should we get back old locks { MYSQL_LOCK *lock; @@ -2978,6 +3245,11 @@ void close_old_data_files(THD *thd, TABL for (; table ; table=table->next) { + DBUG_PRINT("tcache", ("checking table: '%s'.'%s' 0x%lx", + table->s->db.str, table->s->table_name.str, + (long) table)); + DBUG_PRINT("tcache", ("needs refresh: %d is open: %u", + table->needs_reopen_or_name_lock(), table->db_stat)); /* Reopen marked for flush. */ @@ -2989,13 +3261,22 @@ void close_old_data_files(THD *thd, TABL if (morph_locks) { /* - Wake up threads waiting for table-level lock on this table - so they won't sneak in when we will temporarily remove our - lock on it. This will also give them a chance to close their - instances of this table. + Forward lock handling to MERGE parent. But unlock parent + once only. */ - mysql_lock_abort(thd, table, TRUE); - mysql_lock_remove(thd, thd->locked_tables, table, TRUE); + TABLE *ulcktbl= table->parent ? table->parent : table; + if (ulcktbl->lock_count) + { + /* + Wake up threads waiting for table-level lock on this table + so they won't sneak in when we will temporarily remove our + lock on it. This will also give them a chance to close their + instances of this table. + */ + mysql_lock_abort(thd, ulcktbl, TRUE); + mysql_lock_remove(thd, thd->locked_tables, ulcktbl, TRUE); + ulcktbl->lock_count= 0; + } /* We want to protect the table from concurrent DDL operations (like RENAME TABLE) until we will re-open and re-lock it. @@ -3140,7 +3421,9 @@ TABLE *drop_locked_tables(THD *thd,const if (!strcmp(table->s->table_name.str, table_name) && !strcmp(table->s->db.str, db)) { - mysql_lock_remove(thd, thd->locked_tables, table, TRUE); + /* If MERGE child, forward lock handling to parent. */ + mysql_lock_remove(thd, thd->locked_tables, + table->parent ? table->parent : table, TRUE); if (!found) { found= table; @@ -3148,6 +3431,12 @@ TABLE *drop_locked_tables(THD *thd,const if (table->db_stat) { table->db_stat= 0; + /* + When closing a MERGE parent table, detach its children first. + Clear child table references in case this object is opened again. + */ + if (table->child_l) + detach_merge_children(table, TRUE); table->file->close(); } } @@ -3189,7 +3478,8 @@ void abort_locked_tables(THD *thd,const if (!strcmp(table->s->table_name.str, table_name) && !strcmp(table->s->db.str, db)) { - mysql_lock_abort(thd,table, TRUE); + /* If MERGE child, forward lock handling to parent. */ + mysql_lock_abort(thd, table->parent ? table->parent : table, TRUE); break; } } @@ -3220,7 +3510,8 @@ void abort_locked_tables(THD *thd,const share->table_map_id is given a value that with a high certainty is not used by any other table (the only case where a table id can be reused is on wrap-around, which means more than 4 billion table - shares open at the same time). + share opens have been executed while one table was open all the + time). share->table_map_id is not ~0UL. */ @@ -3459,6 +3750,129 @@ err: } +/** + @brief Add list of MERGE children to a TABLE_LIST list. + + @detail When a MERGE parent table has just been opened, insert the + TABLE_LIST chain from the MERGE handle into the table list used for + opening tables for this statement. This lets the children be opened + too. + + @param[in] tlist the TABLE_LIST object just opened + + @return status + @retval 0 OK + @retval != 0 Error +*/ + +static int add_merge_table_list(TABLE_LIST *tlist) +{ + TABLE *parent= tlist->table; + TABLE_LIST *child_l; + DBUG_ENTER("add_merge_table_list"); + DBUG_PRINT("myrg", ("table: '%s'.'%s' 0x%lx", parent->s->db.str, + parent->s->table_name.str, (long) parent)); + + /* Must not call this with attached children. */ + DBUG_ASSERT(!parent->children_attached); + /* Must not call this with children list in place. */ + DBUG_ASSERT(tlist->next_global != parent->child_l); + + /* Fix children.*/ + for (child_l= parent->child_l; ; child_l= child_l->next_global) + { + /* + Note: child_l->table may still be set if this parent was taken + from the unused_tables chain. Ignore this fact here. The + reference will be replaced by the handler in + ::extra(HA_EXTRA_ATTACH_CHILDREN). + */ + + /* Set lock type. */ + child_l->lock_type= tlist->lock_type; + + /* Set parent reference. */ + child_l->parent_l= tlist; + + /* Break when this was the last child. */ + if (&child_l->next_global == parent->child_last_l) + break; + } + + /* Insert children into the table list. */ + *parent->child_last_l= tlist->next_global; + tlist->next_global= parent->child_l; + + /* + Do not fix the prev_global pointers. We will remove the + chain soon anyway. + */ + + DBUG_RETURN(0); +} + + +/** + @brief Attach MERGE children to the parent. + + @detail When the last MERGE child has just been opened, let the + handler attach the MyISAM tables to the MERGE table. Remove MERGE + TABLE_LIST chain from the statement list so that it cannot be + changed or freed. + + @param[in] tlist the TABLE_LIST object just opened + + @return status + @retval 0 OK + @retval != 0 Error +*/ + +static int attach_merge_children(TABLE_LIST *tlist) +{ + TABLE *parent= tlist->parent_l->table; + int error; + DBUG_ENTER("attach_merge_children"); + DBUG_PRINT("myrg", ("table: '%s'.'%s' 0x%lx", parent->s->db.str, + parent->s->table_name.str, (long) parent)); + + /* Must not call this with attached children. */ + DBUG_ASSERT(!parent->children_attached); + /* Must call this with children list in place. */ + DBUG_ASSERT(tlist->parent_l->next_global == parent->child_l); + + /* Attach MyISAM tables to MERGE table. */ + error= parent->file->extra(HA_EXTRA_ATTACH_CHILDREN); + + /* + Remove children from the table list. Even in case of an error. + This should prevent tampering with them. + */ + tlist->parent_l->next_global= *parent->child_last_l; + + /* + Do not fix the last childs next_global pointer. It is needed for + stepping to the next table in the enclosing loop in open_tables(). + Do not fix prev_global pointers. We did not set them. + */ + + if (error) + { + DBUG_PRINT("error", ("attaching MERGE children failed: %d", my_errno)); + parent->file->print_error(error, MYF(0)); + DBUG_RETURN(1); + } + + parent->children_attached= TRUE; + + /* + Note that we have the cildren in the thd->open_tables list at this + point. + */ + + DBUG_RETURN(0); +} + + /* Open all tables in list @@ -3549,6 +3963,9 @@ int open_tables(THD *thd, TABLE_LIST **s */ for (tables= *start; tables ;tables= tables->next_global) { + DBUG_PRINT("tcache", ("opening table: '%s'.'%s' item: 0x%lx", + tables->db, tables->table_name, (long) tables)); + safe_to_ignore_table= FALSE; /* @@ -3600,6 +4017,10 @@ int open_tables(THD *thd, TABLE_LIST **s else tables->table= open_table(thd, tables, &new_frm_mem, &refresh, flags); } + else + DBUG_PRINT("tcache", ("referenced table: '%s'.'%s' 0x%lx", + tables->db, tables->table_name, + (long) tables->table)); if (!tables->table) { @@ -3631,6 +4052,19 @@ int open_tables(THD *thd, TABLE_LIST **s goto process_view_routines; } + /* + If in a MERGE table open, we need to remove the children list + from statement table list before restarting. Otherwise the list + will be inserted another time. + */ + if (tables->parent_l) + { + TABLE_LIST *parent_l= tables->parent_l; + /* The parent table should be correctly open at this point. */ + DBUG_ASSERT(parent_l->table); + parent_l->next_global= *parent_l->table->child_last_l; + } + if (refresh) // Refresh in progress { /* @@ -3699,6 +4133,29 @@ int open_tables(THD *thd, TABLE_LIST **s thd->update_lock_default : tables->lock_type; tables->table->grant= tables->grant; + /* Attach MERGE children if not locked already. */ + DBUG_PRINT("tcache", ("is parent: %d is child: %d", + test(tables->table->child_l), + test(tables->parent_l))); + DBUG_PRINT("tcache", ("in lock tables: %d in prelock mode: %d", + test(thd->locked_tables), test(thd->prelocked_mode))); + if (((!thd->locked_tables && !thd->prelocked_mode) || + tables->table->s->tmp_table) && + ((tables->table->child_l && + add_merge_table_list(tables)) || + (tables->parent_l && + (&tables->next_global == tables->parent_l->table->child_last_l) && + attach_merge_children(tables)))) + { + /* + Some functions determine success as (tables->table != NULL). + tables->table is in thd->open_tables. It won't go lost. + */ + tables->table= NULL; + result= -1; + goto err; + } + process_view_routines: /* Again we may need cache all routines used by this view and add @@ -3770,6 +4227,54 @@ static bool check_lock_and_start_stmt(TH } +/** + @brief Open and lock one table + + @param[in] thd thread handle + @param[in] table_l table to open is first table in this list + @param[in] lock_type lock to use for table + + @return table + @retval != NULL OK, opened table returned + @retval NULL Error + + @note + If ok, the following are also set: + table_list->lock_type lock_type + table_list->table table + + @note + If table_l is a list, not a single table, the list is temporarily + broken. +*/ + +TABLE *open_n_lock_single_table(THD *thd, TABLE_LIST *table_l, + thr_lock_type lock_type) +{ + TABLE_LIST *save_next_global; + DBUG_ENTER("open_n_lock_single_table"); + + /* Remember old 'next' pointer. */ + save_next_global= table_l->next_global; + /* Break list. */ + table_l->next_global= NULL; + + /* Set requested lock type. */ + table_l->lock_type= lock_type; + /* Allow to open real tables only. */ + table_l->required_type= FRMTYPE_TABLE; + + /* Open the table. */ + if (simple_open_n_lock_tables(thd, table_l)) + table_l->table= NULL; /* Just to be sure. */ + + /* Restore list. */ + table_l->next_global= save_next_global; + + DBUG_RETURN(table_l->table); +} + + /* Open and lock one table @@ -3811,6 +4316,15 @@ TABLE *open_ltable(THD *thd, TABLE_LIST if (table) { + if (table->child_l) + { + /* A MERGE table must not come here. */ + my_error(ER_WRONG_OBJECT, MYF(0), table->s->db.str, + table->s->table_name.str, "BASE TABLE"); + table= 0; + goto end; + } + table_list->lock_type= lock_type; table_list->table= table; table->grant= table_list->grant; @@ -3828,56 +4342,21 @@ TABLE *open_ltable(THD *thd, TABLE_LIST table= 0; } } + + end: thd->proc_info=0; DBUG_RETURN(table); } /* - Open all tables in list and locks them for read without derived - tables processing. - - SYNOPSIS - simple_open_n_lock_tables() - thd - thread handler - tables - list of tables for open&locking - - RETURN - 0 - ok - -1 - error - - NOTE - The lock will automaticaly be freed by close_thread_tables() -*/ - -int simple_open_n_lock_tables(THD *thd, TABLE_LIST *tables) -{ - uint counter; - bool need_reopen; - DBUG_ENTER("simple_open_n_lock_tables"); - - for ( ; ; ) - { - if (open_tables(thd, &tables, &counter, 0)) - DBUG_RETURN(-1); - if (!lock_tables(thd, tables, counter, &need_reopen)) - break; - if (!need_reopen) - DBUG_RETURN(-1); - close_tables_for_reopen(thd, &tables); - } - DBUG_RETURN(0); -} - - -/* - Open all tables in list, locks them and process derived tables - tables processing. + Open all tables in list, locks them and optionally process derived tables. SYNOPSIS - open_and_lock_tables() + open_and_lock_tables_derived() thd - thread handler tables - list of tables for open&locking + derived - if to handle derived tables RETURN FALSE - ok @@ -3885,13 +4364,20 @@ int simple_open_n_lock_tables(THD *thd, NOTE The lock will automaticaly be freed by close_thread_tables() + + NOTE + There are two convenience functions: + - simple_open_n_lock_tables(thd, tables) without derived handling + - open_and_lock_tables(thd, tables) with derived handling + Both inline functions call open_and_lock_tables_derived() with + the third argument set appropriately. */ -bool open_and_lock_tables(THD *thd, TABLE_LIST *tables) +bool open_and_lock_tables_derived(THD *thd, TABLE_LIST *tables, bool derived) { uint counter; bool need_reopen; - DBUG_ENTER("open_and_lock_tables"); + DBUG_ENTER("open_and_lock_tables_derived"); for ( ; ; ) { @@ -3903,9 +4389,10 @@ bool open_and_lock_tables(THD *thd, TABL DBUG_RETURN(-1); close_tables_for_reopen(thd, &tables); } - if (mysql_handle_derived(thd->lex, &mysql_derived_prepare) || - (thd->fill_derived_tables() && - mysql_handle_derived(thd->lex, &mysql_derived_filling))) + if (derived && + (mysql_handle_derived(thd->lex, &mysql_derived_prepare) || + (thd->fill_derived_tables() && + mysql_handle_derived(thd->lex, &mysql_derived_filling)))) DBUG_RETURN(TRUE); /* purecov: inspected */ DBUG_RETURN(0); } @@ -4219,7 +4706,14 @@ int lock_tables(THD *thd, TABLE_LIST *ta thd->lock= 0; thd->in_lock_tables=0; - for (table= tables; table != first_not_own; table= table->next_global) + /* + When open_and_lock_tables() is called for a single table out of + a table list, the 'next_global' chain is temporarily broken. We + may not find 'first_not_own' before the end of the "list". + */ + for (table= tables; + table && table != first_not_own; + table= table->next_global) { if (!table->placeholder()) { @@ -4246,7 +4740,9 @@ int lock_tables(THD *thd, TABLE_LIST *ta else { TABLE_LIST *first_not_own= thd->lex->first_not_own_table(); - for (table= tables; table != first_not_own; table= table->next_global) + for (table= tables; + table && table != first_not_own; + table= table->next_global) { if (!table->placeholder() && check_lock_and_start_stmt(thd, table->table, table->lock_type)) @@ -4350,7 +4846,7 @@ TABLE *open_temporary_table(THD *thd, co saved_cache_key= strmov(tmp_path, path)+1; memcpy(saved_cache_key, cache_key, key_length); - init_tmp_table_share(share, saved_cache_key, key_length, + init_tmp_table_share(thd, share, saved_cache_key, key_length, strend(saved_cache_key)+1, tmp_path); if (open_table_def(thd, share, 0) || @@ -4383,6 +4879,8 @@ TABLE *open_temporary_table(THD *thd, co slave_open_temp_tables++; } tmp_table->pos_in_table_list= 0; + DBUG_PRINT("tmptable", ("opened table: '%s'.'%s' 0x%lx", tmp_table->s->db.str, + tmp_table->s->table_name.str, (long) tmp_table)); DBUG_RETURN(tmp_table); } @@ -7064,7 +7562,7 @@ my_bool mysql_rm_tmp_tables(void) /* We should cut file extention before deleting of table */ memcpy(filePathCopy, filePath, filePath_len - ext_len); filePathCopy[filePath_len - ext_len]= 0; - init_tmp_table_share(&share, "", 0, "", filePathCopy); + init_tmp_table_share(thd, &share, "", 0, "", filePathCopy); if (!open_table_def(thd, &share, 0) && ((handler_file= get_new_handler(&share, thd->mem_root, share.db_type())))) @@ -7166,7 +7664,7 @@ bool remove_table_from_cache(THD *thd, c TABLE_SHARE *share; bool result= 0, signalled= 0; DBUG_ENTER("remove_table_from_cache"); - DBUG_PRINT("enter", ("Table: '%s.%s' flags: %u", db, table_name, flags)); + DBUG_PRINT("enter", ("table: '%s'.'%s' flags: %u", db, table_name, flags)); key_length=(uint) (strmov(strmov(key,db)+1,table_name)-key)+1; for (;;) @@ -7181,6 +7679,8 @@ bool remove_table_from_cache(THD *thd, c &state)) { THD *in_use; + DBUG_PRINT("tcache", ("found table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); table->s->version=0L; /* Free when thread is ready */ if (!(in_use=table->in_use)) @@ -7225,7 +7725,8 @@ bool remove_table_from_cache(THD *thd, c thd_table ; thd_table= thd_table->next) { - if (thd_table->db_stat) // If table is open + /* Do not handle locks of MERGE children. */ + if (thd_table->db_stat && !thd_table->parent) // If table is open signalled|= mysql_lock_abort_for_thread(thd, thd_table); } } @@ -7428,7 +7929,9 @@ int abort_and_upgrade_lock(ALTER_PARTITI lpt->old_lock_type= lpt->table->reginfo.lock_type; VOID(pthread_mutex_lock(&LOCK_open)); - mysql_lock_abort(lpt->thd, lpt->table, TRUE); + /* If MERGE child, forward lock handling to parent. */ + mysql_lock_abort(lpt->thd, lpt->table->parent ? lpt->table->parent : + lpt->table, TRUE); VOID(remove_table_from_cache(lpt->thd, lpt->db, lpt->table_name, flags)); VOID(pthread_mutex_unlock(&LOCK_open)); DBUG_RETURN(0); @@ -7455,7 +7958,9 @@ void close_open_tables_and_downgrade(ALT remove_table_from_cache(lpt->thd, lpt->db, lpt->table_name, RTFC_WAIT_OTHER_THREAD_FLAG); VOID(pthread_mutex_unlock(&LOCK_open)); - mysql_lock_downgrade_write(lpt->thd, lpt->table, lpt->old_lock_type); + /* If MERGE child, forward lock handling to parent. */ + mysql_lock_downgrade_write(lpt->thd, lpt->table->parent ? lpt->table->parent : + lpt->table, lpt->old_lock_type); } @@ -7528,7 +8033,8 @@ void mysql_wait_completed_table(ALTER_PA thd_table ; thd_table= thd_table->next) { - if (thd_table->db_stat) // If table is open + /* Do not handle locks of MERGE children. */ + if (thd_table->db_stat && !thd_table->parent) // If table is open mysql_lock_abort_for_thread(lpt->thd, thd_table); } } @@ -7538,8 +8044,10 @@ void mysql_wait_completed_table(ALTER_PA those in use for removal after completion. Now we also need to abort all that are locked and are not progressing due to being locked by our lock. We don't upgrade our lock here. + If MERGE child, forward lock handling to parent. */ - mysql_lock_abort(lpt->thd, my_table, FALSE); + mysql_lock_abort(lpt->thd, my_table->parent ? my_table->parent : my_table, + FALSE); VOID(pthread_mutex_unlock(&LOCK_open)); DBUG_VOID_RETURN; } diff -Nrup a/sql/sql_insert.cc b/sql/sql_insert.cc --- a/sql/sql_insert.cc 2007-08-31 01:05:45 +02:00 +++ b/sql/sql_insert.cc 2007-10-15 16:02:49 +02:00 @@ -2265,7 +2265,8 @@ pthread_handler_t handle_delayed_insert( } /* open table */ - if (!(di->table=open_ltable(thd, &di->table_list, TL_WRITE_DELAYED, 0))) + if (!(di->table= open_n_lock_single_table(thd, &di->table_list, + TL_WRITE_DELAYED))) { thd->fatal_error(); // Abort waiting inserts goto err; @@ -3293,7 +3294,7 @@ static TABLE *create_table_from_items(TH tmp_table.alias= 0; tmp_table.timestamp_field= 0; tmp_table.s= &share; - init_tmp_table_share(&share, "", 0, "", ""); + init_tmp_table_share(thd, &share, "", 0, "", ""); tmp_table.s->db_create_options=0; tmp_table.s->blob_ptr_size= portable_sizeof_char_ptr; @@ -3372,6 +3373,43 @@ static TABLE *create_table_from_items(TH } else table= create_table->table; + + /* + MERGE parent and children need be open and attached for + select. The children have been opened and locked at statement + begin through the UNION=() clause of the CREATE TABLE part of + the statement. A simple CREATE TABLE does not open any table. + But CREATE...SELECT opens all tables, including the MERGE + child tables. + */ + if (table->child_l) + { + TABLE_LIST *tlist; + + if (fix_merge_after_open((TABLE_LIST*) create_info->merge_list.first, + NULL, table->child_l, table->child_last_l) || + table->file->extra(HA_EXTRA_ATTACH_CHILDREN)) + DBUG_RETURN(NULL); + table->children_attached= TRUE; + /* + Need to unlock children for locking through parent. + We do never come here under LOCK TABLES. It is safe to unlock. + */ + DBUG_ASSERT(!thd->locked_tables); + *table->child_last_l= NULL; + for (tlist= table->child_l; tlist; tlist=tlist->next_global) + { + DBUG_ASSERT(tlist->table->reginfo.lock_type == TL_WRITE); + /* + Wake up threads waiting for table-level lock on this table + so they won't sneak in when we will temporarily remove our + lock on it. This will also give them a chance to close their + instances of this table. + */ + mysql_lock_abort(thd, tlist->table, TRUE); + mysql_lock_remove(thd, thd->lock, tlist->table, TRUE); + } + } VOID(pthread_mutex_unlock(&LOCK_open)); } else @@ -3386,6 +3424,16 @@ static TABLE *create_table_from_items(TH just in case. */ close_temporary_table(thd, create_table); + } + /* MERGE children need be open for select. */ + if (table->child_l) + { + if (fix_merge_after_open((TABLE_LIST*) create_info->merge_list.first, + NULL, table->child_l, table->child_last_l) || + table->file->extra(HA_EXTRA_ATTACH_CHILDREN)) + close_temporary_table(thd, create_table); + else + table->children_attached= TRUE; } } } diff -Nrup a/sql/sql_parse.cc b/sql/sql_parse.cc --- a/sql/sql_parse.cc 2007-09-12 21:44:47 +02:00 +++ b/sql/sql_parse.cc 2007-10-15 16:02:49 +02:00 @@ -542,7 +542,7 @@ int mysql_table_dump(THD *thd, LEX_STRIN if (lower_case_table_names) my_casedn_str(files_charset_info, tbl_name); - if (!(table=open_ltable(thd, table_list, TL_READ_NO_INSERT, 0))) + if (!(table= open_n_lock_single_table(thd, table_list, TL_READ_NO_INSERT))) DBUG_RETURN(1); if (check_one_table_access(thd, SELECT_ACL, table_list)) @@ -961,12 +961,7 @@ bool dispatch_command(enum enum_server_c { char *next_packet= (char*) found_semicolon; net->no_send_error= 0; - /* - Multiple queries exits, execute them individually - */ - if (thd->lock || thd->open_tables || thd->derived_tables || - thd->prelocked_mode) - close_thread_tables(thd); + close_thread_tables(thd); ulong length= (ulong)(packet_end - next_packet); log_slow_statement(thd); @@ -1303,12 +1298,10 @@ bool dispatch_command(enum enum_server_c my_message(ER_UNKNOWN_COM_ERROR, ER(ER_UNKNOWN_COM_ERROR), MYF(0)); break; } - if (thd->lock || thd->open_tables || thd->derived_tables || - thd->prelocked_mode) - { - thd->proc_info="closing tables"; - close_thread_tables(thd); /* Free tables */ - } + + thd->proc_info= "closing tables"; + close_thread_tables(thd); /* Free tables */ + /* assume handlers auto-commit (if some doesn't - transaction handling in MySQL should be redesigned to support it; it's a big change, @@ -5082,6 +5075,12 @@ bool check_merge_table_access(THD *thd, TABLE_LIST *tmp; for (tmp= table_list; tmp; tmp= tmp->next_local) { + /* + We create or alter a MERGE table. In both cases the MERGE child + tables must be locked in write mode. + */ + tmp->lock_type= TL_WRITE; + if (!tmp->db || !tmp->db[0]) tmp->db=db; } diff -Nrup a/sql/sql_partition.cc b/sql/sql_partition.cc --- a/sql/sql_partition.cc 2007-09-14 12:17:40 +02:00 +++ b/sql/sql_partition.cc 2007-10-15 16:02:49 +02:00 @@ -4931,7 +4931,7 @@ the generated partition syntax in a corr We use the old partitioning also for the new table. We do this by assigning the partition_info from the table loaded in - open_ltable to the partition_info struct used by mysql_create_table + open_table to the partition_info struct used by mysql_create_table later in this method. Case IIb: diff -Nrup a/sql/sql_select.cc b/sql/sql_select.cc --- a/sql/sql_select.cc 2007-09-28 20:31:52 +02:00 +++ b/sql/sql_select.cc 2007-10-15 16:02:49 +02:00 @@ -9578,7 +9578,7 @@ create_tmp_table(THD *thd,TMP_TABLE_PARA table->keys_in_use_for_query.init(); table->s= share; - init_tmp_table_share(share, "", 0, tmpname, tmpname); + init_tmp_table_share(thd, share, "", 0, tmpname, tmpname); share->blob_field= blob_field; share->blob_ptr_size= mi_portable_sizeof_char_ptr; share->db_low_byte_first=1; // True for HEAP and MyISAM diff -Nrup a/sql/sql_table.cc b/sql/sql_table.cc --- a/sql/sql_table.cc 2007-08-25 10:43:15 +02:00 +++ b/sql/sql_table.cc 2007-10-15 16:02:49 +02:00 @@ -4016,6 +4016,8 @@ static bool mysql_admin_table(THD* thd, char* db = table->db; bool fatal_error=0; + DBUG_PRINT("admin", ("table: '%s'.'%s'", table->db, table->table_name)); + DBUG_PRINT("admin", ("extra_open_options: %u", extra_open_options)); strxmov(table_name, db, ".", table->table_name, NullS); thd->open_options|= extra_open_options; table->lock_type= lock_type; @@ -4048,14 +4050,18 @@ static bool mysql_admin_table(THD* thd, } if (prepare_func) { + DBUG_PRINT("admin", ("calling prepare_func")); switch ((*prepare_func)(thd, table, check_opt)) { case 1: // error, message written to net ha_autocommit_or_rollback(thd, 1); close_thread_tables(thd); + DBUG_PRINT("admin", ("simple error, admin next table")); continue; case -1: // error, message could be written to net + DBUG_PRINT("admin", ("severe error, stop")); goto err; default: // should be 0 otherwise + DBUG_PRINT("admin", ("prepare_func succeeded")); ; } } @@ -4070,6 +4076,7 @@ static bool mysql_admin_table(THD* thd, */ if (!table->table) { + DBUG_PRINT("admin", ("open table failed")); if (!thd->warn_list.elements) push_warning(thd, MYSQL_ERROR::WARN_LEVEL_ERROR, ER_CHECK_NO_SUCH_TABLE, ER(ER_CHECK_NO_SUCH_TABLE)); @@ -4160,7 +4167,9 @@ static bool mysql_admin_table(THD* thd, } + DBUG_PRINT("admin", ("calling operator_func '%s'", operator_name)); result_code = (table->table->file->*operator_func)(thd, check_opt); + DBUG_PRINT("admin", ("operator_func returned: %d", result_code)); send_result: @@ -5816,7 +5825,8 @@ view_err: start_waiting_global_read_lock(thd); DBUG_RETURN(error); } - if (!(table=open_ltable(thd, table_list, TL_WRITE_ALLOW_READ, 0))) + + if (!(table= open_n_lock_single_table(thd, table_list, TL_WRITE_ALLOW_READ))) DBUG_RETURN(TRUE); table->use_all_columns(); @@ -6566,6 +6576,11 @@ view_err: if (reopen_table(table)) goto err_with_placeholders; t_table= table; + /* + No need to attach MERGE children here. Table will not be used + during this statement any more. + */ + DBUG_ASSERT(!thd->locked_tables); } /* Tell the handler that a new frm file is in place. */ if (t_table->file->create_handler_files(path, NULL, CHF_INDEX_FLAG, @@ -6944,6 +6959,12 @@ bool mysql_recreate_table(THD *thd, TABL Alter_info alter_info; DBUG_ENTER("mysql_recreate_table"); + DBUG_ASSERT(!table_list->next_global); + /* + table_list->table has been closed and freed. Do not reference + uninitialized data. open_tables() could fail. + */ + table_list->table= NULL; bzero((char*) &create_info, sizeof(create_info)); create_info.db_type= 0; @@ -6975,6 +6996,7 @@ bool mysql_checksum_table(THD *thd, TABL Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF)) DBUG_RETURN(TRUE); + /* Open one table after the other to keep lock time as short as possible. */ for (table= tables; table; table= table->next_local) { char table_name[NAME_LEN*2+2]; @@ -6982,7 +7004,7 @@ bool mysql_checksum_table(THD *thd, TABL strxmov(table_name, table->db ,".", table->table_name, NullS); - t= table->table= open_ltable(thd, table, TL_READ, 0); + t= table->table= open_n_lock_single_table(thd, table, TL_READ); thd->clear_error(); // these errors shouldn't get client protocol->prepare_for_resend(); diff -Nrup a/sql/table.cc b/sql/table.cc --- a/sql/table.cc 2007-08-31 11:55:53 +02:00 +++ b/sql/table.cc 2007-10-15 16:02:49 +02:00 @@ -329,6 +329,7 @@ TABLE_SHARE *alloc_table_share(TABLE_LIS SYNOPSIS init_tmp_table_share() + thd thread handle share Share to fill key Table_cache_key, as generated from create_table_def_key. must start with db name. @@ -346,7 +347,7 @@ TABLE_SHARE *alloc_table_share(TABLE_LIS use key_length= 0 as neither table_cache_key or key_length will be used). */ -void init_tmp_table_share(TABLE_SHARE *share, const char *key, +void init_tmp_table_share(THD *thd, TABLE_SHARE *share, const char *key, uint key_length, const char *table_name, const char *path) { @@ -373,8 +374,13 @@ void init_tmp_table_share(TABLE_SHARE *s anyway to be able to catch errors. */ share->table_map_version= ~(ulonglong)0; - share->table_map_id= ~0UL; share->cached_row_logging_check= -1; + + /* + table_map_id is also used for MERGE tables to suppress repeated + compatibility checks. + */ + share->table_map_id= (ulong) thd->query_id; DBUG_VOID_RETURN; } diff -Nrup a/sql/table.h b/sql/table.h --- a/sql/table.h 2007-08-25 10:43:15 +02:00 +++ b/sql/table.h 2007-10-15 16:02:49 +02:00 @@ -455,6 +455,11 @@ struct st_table { #endif struct st_table *next, *prev; + /* For the below MERGE related members see top comment in ha_myisammrg.cc */ + struct st_table *parent; /* Set in MERGE child. Ptr to parent */ + TABLE_LIST *child_l; /* Set in MERGE parent. List of children */ + TABLE_LIST **child_last_l; /* Set in MERGE parent. End of list */ + THD *in_use; /* Which thread uses this */ Field **field; /* Pointer to fields */ @@ -604,6 +609,8 @@ struct st_table { my_bool insert_or_update; /* Can be used by the handler */ my_bool alias_name_used; /* true if table_name is alias */ my_bool get_fields_in_item_tree; /* Signal to fix_field */ + /* If MERGE children attached to parent. See top comment in ha_myisammrg.cc */ + my_bool children_attached; REGINFO reginfo; /* field connections */ MEM_ROOT mem_root; @@ -978,6 +985,8 @@ struct TABLE_LIST (non-zero only for merged underlying tables of a view). */ TABLE_LIST *referencing_view; + /* Ptr to parent MERGE table list item. See top comment in ha_myisammrg.cc */ + TABLE_LIST *parent_l; /* Security context (non-zero only for tables which belong to view with SQL SECURITY DEFINER) @@ -1010,6 +1019,8 @@ struct TABLE_LIST ulonglong algorithm; /* 0 any, 1 tmp tables , 2 merging */ ulonglong view_suid; /* view is suid (TRUE dy default) */ ulonglong with_check; /* WITH CHECK OPTION */ + /* Remembered MERGE child table_map_id. See top comment in ha_myisammrg.cc */ + ulong child_table_map_id; /* effective value of WITH CHECK OPTION (differ for temporary table algorithm) diff -Nrup a/storage/myisam/ha_myisam.cc b/storage/myisam/ha_myisam.cc --- a/storage/myisam/ha_myisam.cc 2007-09-21 10:15:03 +02:00 +++ b/storage/myisam/ha_myisam.cc 2007-10-15 16:02:49 +02:00 @@ -119,6 +119,10 @@ static void mi_check_print_msg(MI_CHECK definition for further use in mi_create or for a check for underlying table conformance in merge engine. + The caller needs to free *recinfo_out after use. Since *recinfo_out + and *keydef_out are allocated with a my_multi_malloc, *keydef_out + is freed automatically when *recinfo_out is freed. + RETURN VALUE 0 OK !0 error code @@ -2119,3 +2123,26 @@ my_bool ha_myisam::register_query_cache_ return TRUE; } #endif + + +/** + @brief Extract the MyISAM table structure pointer from a handler object. + + @param[in] handler_handle pointer to handler object + + @return MyISAM table structure pointer + @retval !=NULL Ok, returning pointer + @retval NULL Error: handler has not a MyISAM table open +*/ + +MI_INFO *myisam_engine_handle(handler *handler_handle) +{ + DBUG_ENTER("myisam_engine_handle"); + if (handler_handle->ht->db_type != DB_TYPE_MYISAM) + { + DBUG_PRINT("error", ("not a MyISAM table")); + DBUG_RETURN(NULL); + } + DBUG_RETURN(((ha_myisam*) handler_handle)->file); +} + diff -Nrup a/storage/myisam/ha_myisam.h b/storage/myisam/ha_myisam.h --- a/storage/myisam/ha_myisam.h 2007-08-13 15:11:15 +02:00 +++ b/storage/myisam/ha_myisam.h 2007-10-15 16:02:49 +02:00 @@ -33,6 +33,8 @@ extern ulong myisam_sort_buffer_size; extern TYPELIB myisam_recover_typelib; extern ulong myisam_recover_options; +extern MI_INFO *myisam_engine_handle(handler *handler_handle); + class ha_myisam: public handler { MI_INFO *file; @@ -140,4 +142,6 @@ class ha_myisam: public handler *engine_callback, ulonglong *engine_data); #endif + + friend MI_INFO *myisam_engine_handle(handler *handler_handle); }; diff -Nrup a/storage/myisammrg/ha_myisammrg.cc b/storage/myisammrg/ha_myisammrg.cc --- a/storage/myisammrg/ha_myisammrg.cc 2007-08-13 15:11:17 +02:00 +++ b/storage/myisammrg/ha_myisammrg.cc 2007-10-15 16:02:49 +02:00 @@ -14,6 +14,67 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ +/* + MyISAM MERGE tables + + A MyISAM MERGE table is kind of a union of zero or more MyISAM tables. + + Besides the normal form file (.frm) a MERGE table has a meta file + (.MRG) with a list of tables. These are paths to the MyISAM table + files. The last two components of the path contain the database name + and the table name respectively. + + When a MERGE table is open, there exists an TABLE object for the MERGE + table itself and a TABLE object for each of the MyISAM tables. For + abbreviated writing, I call the MERGE table object "parent" and the + MyISAM table objects "children". + + A MERGE table is almost always opened through open_and_lock_tables() + and hence through open_tables(). When the parent appears in the list + of tables to open, the initial open of the handler does nothing but + read the meta file and collect a list of TABLE_LIST objects for the + children. This list is attached to the parent TABLE object as + TABLE::child_l. The end of the children list is saved in + TABLE::child_last_l. + + Back in open_tables(), add_merge_table_list() is called. It updates + each list member with the lock type and a back pointer to the parent + TABLE_LIST object TABLE_LIST::parent_l. The list is then inserted in + the list of tables to open, right behind the parent. Consequently, + open_tables() opens the children, one after the other. The TABLE + references of the TABLE_LIST objects are implicitly set to the open + tables. The children are opened as independent MyISAM tables, right as + if they are used by the SQL statement. + + When the last child is open, attach_merge_children() is called. It + removes the list of children from the open list. Then the children are + "attached" to the parent. All required references between parent and + children are set up. + + The MERGE storage engine sets up an array with references to the + low-level MyISAM table objects (MI_INFO). It remembers the state of + the table in MYRG_INFO::children_attached. + + Every child TABLE::parent references the parent TABLE object. That way + TABLE objects belonging to a MERGE table can be identified. + + If necessary, the compatibility of parent and children is checked. + This check is necessary when any of the objects are reopend. This is + detected by comparing the current table_map_id against the remembered + TABLE_LIST::child_table_map_id. On parent open, the list members are + initialized to an "impossible"/"undefined" map id value. So the check + is always executed on the first attach. + + Finally the parent TABLE::children_attached is set. + + --- + + On parent open the storage engine structures are allocated and initialized. + They stay with the open table until its final close. + + +*/ + #ifdef USE_PRAGMA_IMPLEMENTATION #pragma implementation // gcc: Class implementation #endif @@ -22,14 +83,11 @@ #include "mysql_priv.h" #include <mysql/plugin.h> #include <m_ctype.h> +#include "../myisam/ha_myisam.h" #include "ha_myisammrg.h" #include "myrg_def.h" -/***************************************************************************** -** MyISAM MERGE tables -*****************************************************************************/ - static handler *myisammrg_create_handler(handlerton *hton, TABLE_SHARE *table, MEM_ROOT *mem_root) @@ -38,10 +96,23 @@ static handler *myisammrg_create_handler } +/** + @brief Constructor +*/ + ha_myisammrg::ha_myisammrg(handlerton *hton, TABLE_SHARE *table_arg) :handler(hton, table_arg), file(0) {} + +/** + @brief Destructor +*/ + +ha_myisammrg::~ha_myisammrg(void) +{} + + static const char *ha_myisammrg_exts[] = { ".MRG", NullS @@ -89,25 +160,296 @@ const char *ha_myisammrg::index_type(uin } -int ha_myisammrg::open(const char *name, int mode, uint test_if_locked) +/** + @brief Callback function for open of a MERGE parent table. + + @detail This function adds a TABLE_LIST object for a MERGE child table + to a list of tables of the parent TABLE object. It is called for + each child table. + + The list of child TABLE_LIST objects is kept in the TABLE object of + the parent for the whole life time of the MERGE table. It is + inserted in the statement list behind the MERGE parent TABLE_LIST + object when the MERGE table is opened. It is removed from the + statement list after the last child is opened. + + All memeory used for the child TABLE_LIST objects and the strings + referred by it are taken from the parent TABLE::mem_root. Thus they + are all freed implicitly at the final close of the table. + + TABLE::child_l -> TABLE_LIST::next_global -> TABLE_LIST::next_global + # # ^ # ^ + # # | # | + # # +--------- TABLE_LIST::prev_global + # # | + # |<--- TABLE_LIST::prev_global | + # | + TABLE::child_last_l -----------------------------------------+ + + @param[in] callback_param data pointer as given to myrg_parent_open() + @param[in] filename file name of MyISAM table + without extension. + + @return status + @retval 0 OK + @retval != 0 Error +*/ + +static int myisammrg_parent_open_callback(void *callback_param, + const char *filename) +{ + ha_myisammrg *ha_myrg; + TABLE *parent; + TABLE_LIST *child_l; + const char *db; + const char *table_name; + uint dirlen; + char dir_path[FN_REFLEN]; + DBUG_ENTER("myisammrg_parent_open_callback"); + + /* Extract child table name and database name from filename. */ + dirlen= dirname_length(filename); + if (dirlen >= FN_REFLEN) + { + DBUG_PRINT("error", ("name too long: '%.64s'", filename)); + my_errno= ENAMETOOLONG; + DBUG_RETURN(1); + } + table_name= filename + dirlen; + dirlen--; /* Strip off trailing '/'. */ + memcpy(dir_path, filename, dirlen); + dir_path[dirlen]= '\0'; + db= base_name(dir_path); + dirlen-= db - dir_path; /* This is now the length of 'db'. */ + DBUG_PRINT("myrg", ("open: '%s'.'%s'", db, table_name)); + + ha_myrg= (ha_myisammrg*) callback_param; + parent= ha_myrg->table_ptr(); + + /* Get a TABLE_LIST object. */ + if (!(child_l= (TABLE_LIST*) alloc_root(&parent->mem_root, + sizeof(TABLE_LIST)))) + { + DBUG_PRINT("error", ("my_malloc error: %d", my_errno)); + DBUG_RETURN(1); + } + bzero((char*) child_l, sizeof(TABLE_LIST)); + + /* Set database (schema) name. */ + child_l->db_length= dirlen; + child_l->db= strmake_root(&parent->mem_root, db, dirlen); + /* Set table name. */ + child_l->table_name_length= strlen(table_name); + child_l->table_name= strmake_root(&parent->mem_root, table_name, + child_l->table_name_length); + /* Convert to lowercase if required. */ + if (lower_case_table_names && child_l->table_name_length) + child_l->table_name_length= my_casedn_str(files_charset_info, + child_l->table_name); + /* Set alias. */ + child_l->alias= child_l->table_name; + + /* Initialize table map to 'undefined'. */ + child_l->child_table_map_id= ~0UL; + + /* Link TABLE_LIST object into the parent list. */ + if (!parent->child_last_l) + { + /* Initialize parent->child_last_l when handling first child. */ + parent->child_last_l= &parent->child_l; + } + *parent->child_last_l= child_l; + child_l->prev_global= parent->child_last_l; + parent->child_last_l= &child_l->next_global; + + DBUG_RETURN(0); +} + + +/** + @brief Callback function for attaching a MERGE child table. + + @detail This function retrieves the MyISAM table handle from the + next child table. It is called for each child table. + + @param[in] callback_param data pointer as given to + myrg_attach_children() + + @return pointer to open MyISAM table structure + @retval !=NULL OK, returning pointer + @retval NULL, my_errno == 0 Ok, no more child tables + @retval NULL, my_errno != 0 error +*/ + +static MI_INFO *myisammrg_attach_children_callback(void *callback_param) +{ + ha_myisammrg *ha_myrg; + TABLE *parent; + TABLE *child; + TABLE_LIST *child_l; + MI_INFO *myisam; + DBUG_ENTER("myisammrg_attach_children_callback"); + + my_errno= 0; + ha_myrg= (ha_myisammrg*) callback_param; + parent= ha_myrg->table_ptr(); + + /* Get child list item. */ + child_l= ha_myrg->next_child_attach; + if (!child_l) + { + DBUG_PRINT("myrg", ("No more children to attach")); + DBUG_RETURN(NULL); + } + child= child_l->table; + DBUG_PRINT("myrg", ("child table: '%s'.'%s' 0x%lx", child->s->db.str, + child->s->table_name.str, (long) child)); + /* + Prepare for next child. Used as child_l in next call to this function. + We cannot rely on a NULL-terminated chain. + */ + if (&child_l->next_global == parent->child_last_l) + { + DBUG_PRINT("myrg", ("attaching last child")); + ha_myrg->next_child_attach= NULL; + } + else + ha_myrg->next_child_attach= child_l->next_global; + + /* Set parent reference. */ + child->parent= parent; + + /* + Do a quick compatibility check. TABLE::s::table_map_id is set when + the table share is created. TABLE_LIST::child_table_map_id is copied + from TABLE::s::table_map_id after a sucessful compatibility check. + We need to repeat the compatibility check only if a child is opened + from a different share than last time it was used with this MERGE + table. + */ + DBUG_PRINT("myrg", ("table_map_id last: %lu current: %lu", + (ulong) child_l->child_table_map_id, + (ulong) child->s->table_map_id)); + if (child_l->child_table_map_id != child->s->table_map_id) + ha_myrg->need_compat_check= TRUE; + + /* + If parent is temporary, children must be temporary too and vice + versa. This check must be done for every child on every open because + the table_map_id can overlap between temporary and non-temporary + tables. We need to detect the case where a non-temporary table has + been replaced with a temporary table of the same table_map_id. Or + vice versa. A very unlikely case, but it could happen. + */ + if (child->s->tmp_table != parent->s->tmp_table) + { + DBUG_PRINT("error", ("temporary table mismatch parent: %d child: %d", + parent->s->tmp_table, child->s->tmp_table)); + my_errno= HA_ERR_WRONG_MRG_TABLE_DEF; + goto err; + } + + /* Extract the MyISAM table structure pointer from the handler object. */ + if (!(myisam= myisam_engine_handle(child->file))) + { + DBUG_PRINT("error", ("no MyISAM handle for child table: '%s'.'%s' 0x%lx", + child->s->db.str, child->s->table_name.str, + (long) child)); + my_errno= HA_ERR_WRONG_MRG_TABLE_DEF; + } + DBUG_PRINT("myrg", ("MyISAM handle: 0x%lx my_errno: %d", + (long) myisam, my_errno)); + + err: + DBUG_RETURN(my_errno ? NULL : myisam); +} + + +/** + @brief Open a MERGE parent table, not its children. + + @detail This function initializes the MERGE storage engine structures + and adds a child list of TABLE_LIST to the parent TABLE. + + @param[in] name MERGE table path name + @param[in] mode read/write mode, unused + @param[in] test_if_locked open flags + + @return status + @retval 0 OK + @retval -1 Error, my_errno gives reason +*/ + +int ha_myisammrg::open(const char *name, int mode __attribute__((unused)), + uint test_if_locked) { - MI_KEYDEF *keyinfo; - MI_COLUMNDEF *recinfo; - MYRG_TABLE *u_table; - uint recs; - uint keys= table->s->keys; - int error; - char name_buff[FN_REFLEN]; - - DBUG_PRINT("info", ("ha_myisammrg::open")); - if (!(file=myrg_open(fn_format(name_buff,name,"","", - MY_UNPACK_FILENAME|MY_APPEND_EXT), - mode, test_if_locked))) + DBUG_ENTER("ha_myisammrg::open"); + DBUG_PRINT("myrg", ("name: '%s' table: 0x%lx", name, (long) table)); + DBUG_PRINT("myrg", ("test_if_locked: %u", test_if_locked)); + + /* Save for later use. */ + this->test_if_locked= test_if_locked; + + /* retrieve children table list. */ + my_errno= 0; + if (!(file= myrg_parent_open(name, myisammrg_parent_open_callback, this))) { - DBUG_PRINT("info", ("ha_myisammrg::open exit %d", my_errno)); - return (my_errno ? my_errno : -1); + DBUG_PRINT("error", ("my_errno %d", my_errno)); + DBUG_RETURN(my_errno ? my_errno : -1); } - DBUG_PRINT("info", ("ha_myisammrg::open myrg_extrafunc...")); + DBUG_PRINT("myrg", ("MYRG_INFO: 0x%lx", (long) file)); + DBUG_RETURN(0); +} + + +/** + @brief Attach children to a MERGE table. + + @detail Let the storage engine attach its children through a callback + function. Check table definitions for consistency. + + @note The required conversion table2myisam is done on the first + attach only. Its result (recs, recinfo, keyinfo) is stored in + ha_myisammrg for reuse with later attaches. + + @note Special thd->open_options may be in effect. We can make use of + them in attach. I.e. we use HA_OPEN_FOR_REPAIR to report the names + of mismatching child tables. We cannot transport these options in + ha_myisammrg::test_if_locked because they may change after the + parent is opened. The parent is kept open in the table cache over + multiple statements and can be used by other threads. Open options + can change over time. + + @return status + @retval 0 OK + @retval != 0 Error, my_errno gives reason +*/ + +int ha_myisammrg::attach_children(void) +{ + MYRG_TABLE *u_table; + MI_COLUMNDEF *recinfo; + MI_KEYDEF *keyinfo; + uint recs; + uint keys= table->s->keys; + int error; + DBUG_ENTER("ha_myisammrg::attach_children"); + DBUG_PRINT("myrg", ("table: '%s'.'%s' 0x%lx", table->s->db.str, + table->s->table_name.str, (long) table)); + DBUG_PRINT("myrg", ("test_if_locked: %u", this->test_if_locked)); + DBUG_ASSERT(!this->file->children_attached); + + next_child_attach= table->child_l; + need_compat_check= FALSE; + my_errno= 0; + if (myrg_attach_children(this->file, this->test_if_locked | + current_thd->open_options, + myisammrg_attach_children_callback, this)) + { + DBUG_PRINT("error", ("my_errno %d", my_errno)); + DBUG_RETURN(my_errno ? my_errno : -1); + } + DBUG_PRINT("myrg", ("calling myrg_extrafunc")); myrg_extrafunc(file, query_cache_invalidate_by_MyISAM_filename_ref); if (!(test_if_locked == HA_OPEN_WAIT_IF_LOCKED || test_if_locked == HA_OPEN_ABORT_IF_LOCKED)) @@ -116,69 +458,146 @@ int ha_myisammrg::open(const char *name, if (!(test_if_locked & HA_OPEN_WAIT_IF_LOCKED)) myrg_extra(file,HA_EXTRA_WAIT_LOCK,0); - if (table->s->reclength != stats.mean_rec_length && stats.mean_rec_length) - { - DBUG_PRINT("error",("reclength: %lu mean_rec_length: %lu", - table->s->reclength, stats.mean_rec_length)); - if (test_if_locked & HA_OPEN_FOR_REPAIR) - myrg_print_wrong_table(file->open_tables->table->filename); - error= HA_ERR_WRONG_MRG_TABLE_DEF; - goto err; - } - if ((error= table2myisam(table, &keyinfo, &recinfo, &recs))) - { - /* purecov: begin inspected */ - DBUG_PRINT("error", ("Failed to convert TABLE object to MyISAM " - "key and column definition")); - goto err; - /* purecov: end */ - } - for (u_table= file->open_tables; u_table < file->end_table; u_table++) + /* + The compatibility check is required only if one or more children + do not match their 'table_map_id' from the last check. This will + always happen at the first attach because the reference + 'child_table_map_id' is initialized to 'undefined' at open. + */ + DBUG_PRINT("myrg", ("need_compat_check: %d", need_compat_check)); + if (need_compat_check) { - if (check_definition(keyinfo, recinfo, keys, recs, - u_table->table->s->keyinfo, u_table->table->s->rec, - u_table->table->s->base.keys, - u_table->table->s->base.fields, false)) + TABLE_LIST *child_l; + + if (table->s->reclength != stats.mean_rec_length && stats.mean_rec_length) { - error= HA_ERR_WRONG_MRG_TABLE_DEF; + DBUG_PRINT("error",("reclength: %lu mean_rec_length: %lu", + table->s->reclength, stats.mean_rec_length)); if (test_if_locked & HA_OPEN_FOR_REPAIR) - myrg_print_wrong_table(u_table->table->filename); - else + myrg_print_wrong_table(file->open_tables->table->filename); + error= HA_ERR_WRONG_MRG_TABLE_DEF; + goto err; + } + /* + Both recinfo and keyinfo are allocated by my_multi_malloc(), thus + only recinfo must be freed. + */ + if ((error= table2myisam(table, &keyinfo, &recinfo, &recs))) + { + /* purecov: begin inspected */ + DBUG_PRINT("error", ("failed to convert TABLE object to MyISAM " + "key and column definition")); + goto err; + /* purecov: end */ + } + for (u_table= file->open_tables; u_table < file->end_table; u_table++) + { + if (check_definition(keyinfo, recinfo, keys, recs, + u_table->table->s->keyinfo, u_table->table->s->rec, + u_table->table->s->base.keys, + u_table->table->s->base.fields, false)) { - my_free((uchar*) recinfo, MYF(0)); - goto err; + DBUG_PRINT("error", ("table definition mismatch: '%s'", + u_table->table->filename)); + error= HA_ERR_WRONG_MRG_TABLE_DEF; + if (!(this->test_if_locked & HA_OPEN_FOR_REPAIR)) + { + my_free((uchar*) recinfo, MYF(0)); + goto err; + } + myrg_print_wrong_table(u_table->table->filename); } } + my_free((uchar*) recinfo, MYF(0)); + if (error == HA_ERR_WRONG_MRG_TABLE_DEF) + goto err; + + /* All checks passed so far. Now update child_table_map_id. */ + for (child_l= table->child_l; ; child_l= child_l->next_global) + { + child_l->child_table_map_id= child_l->table->s->table_map_id; + + if (&child_l->next_global == table->child_last_l) + break; + } } - my_free((uchar*) recinfo, MYF(0)); - if (error == HA_ERR_WRONG_MRG_TABLE_DEF) - goto err; #if !defined(BIG_TABLES) || SIZEOF_OFF_T == 4 /* Merge table has more than 2G rows */ if (table->s->crashed) { + DBUG_PRINT("error", ("MERGE table marked crashed")); error= HA_ERR_WRONG_MRG_TABLE_DEF; goto err; } #endif - return (0); + DBUG_RETURN(0); + err: - myrg_close(file); - file=0; - return (my_errno= error); + myrg_detach_children(file); + DBUG_RETURN(my_errno= error); } + +/** + @brief Detach all children from a MERGE table. + + @note Detach must not touch the children in any way. + They may have been closed at ths point already. + All references to the children should be removed. + + @return status + @retval 0 OK + @retval != 0 Error, my_errno gives reason +*/ + +int ha_myisammrg::detach_children(void) +{ + DBUG_ENTER("ha_myisammrg::detach_children"); + /* + Do not test children_attached. After a failed attach it may be FALSE. + In cases like ALTER TABLE the table could have been closed already. + */ + if (this->file && myrg_detach_children(this->file)) + { + DBUG_PRINT("error", ("my_errno %d", my_errno)); + DBUG_RETURN(my_errno ? my_errno : -1); + } + DBUG_RETURN(0); +} + + +/** + @brief Close a MERGE parent table, not its children. + + @note The children are expected to be closed separately by the caller. + + @return status + @retval 0 OK + @retval != 0 Error, my_errno gives reason +*/ + int ha_myisammrg::close(void) { - return myrg_close(file); + int rc; + DBUG_ENTER("ha_myisammrg::close"); + /* + Children must not be attached here. Unless the MERGE table has no + children. In this case children_attached is always true. + */ + DBUG_ASSERT(!this->file->children_attached || !this->file->tables); + rc= myrg_close(file); + file= 0; + DBUG_RETURN(rc); } int ha_myisammrg::write_row(uchar * buf) { + DBUG_ENTER("ha_myisammrg::write_row"); + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_write_count); if (file->merge_insert_method == MERGE_INSERT_DISABLED || !file->tables) - return (HA_ERR_TABLE_READONLY); + DBUG_RETURN(HA_ERR_TABLE_READONLY); if (table->timestamp_field_type & TIMESTAMP_AUTO_SET_ON_INSERT) table->timestamp_field->set_time(); @@ -186,13 +605,14 @@ int ha_myisammrg::write_row(uchar * buf) { int error; if ((error= update_auto_increment())) - return error; + DBUG_RETURN(error); } - return myrg_write(file,buf); + DBUG_RETURN(myrg_write(file,buf)); } int ha_myisammrg::update_row(const uchar * old_data, uchar * new_data) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_update_count); if (table->timestamp_field_type & TIMESTAMP_AUTO_SET_ON_UPDATE) table->timestamp_field->set_time(); @@ -201,6 +621,7 @@ int ha_myisammrg::update_row(const uchar int ha_myisammrg::delete_row(const uchar * buf) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_delete_count); return myrg_delete(file,buf); } @@ -209,6 +630,7 @@ int ha_myisammrg::index_read_map(uchar * key_part_map keypart_map, enum ha_rkey_function find_flag) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_key_count); int error=myrg_rkey(file,buf,active_index, key, keypart_map, find_flag); table->status=error ? STATUS_NOT_FOUND: 0; @@ -219,6 +641,7 @@ int ha_myisammrg::index_read_idx_map(uch key_part_map keypart_map, enum ha_rkey_function find_flag) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_key_count); int error=myrg_rkey(file,buf,index, key, keypart_map, find_flag); table->status=error ? STATUS_NOT_FOUND: 0; @@ -228,6 +651,7 @@ int ha_myisammrg::index_read_idx_map(uch int ha_myisammrg::index_read_last_map(uchar *buf, const uchar *key, key_part_map keypart_map) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_key_count); int error=myrg_rkey(file,buf,active_index, key, keypart_map, HA_READ_PREFIX_LAST); @@ -237,6 +661,7 @@ int ha_myisammrg::index_read_last_map(uc int ha_myisammrg::index_next(uchar * buf) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_next_count); int error=myrg_rnext(file,buf,active_index); table->status=error ? STATUS_NOT_FOUND: 0; @@ -245,6 +670,7 @@ int ha_myisammrg::index_next(uchar * buf int ha_myisammrg::index_prev(uchar * buf) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_prev_count); int error=myrg_rprev(file,buf, active_index); table->status=error ? STATUS_NOT_FOUND: 0; @@ -253,6 +679,7 @@ int ha_myisammrg::index_prev(uchar * buf int ha_myisammrg::index_first(uchar * buf) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_first_count); int error=myrg_rfirst(file, buf, active_index); table->status=error ? STATUS_NOT_FOUND: 0; @@ -261,6 +688,7 @@ int ha_myisammrg::index_first(uchar * bu int ha_myisammrg::index_last(uchar * buf) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_last_count); int error=myrg_rlast(file, buf, active_index); table->status=error ? STATUS_NOT_FOUND: 0; @@ -271,6 +699,7 @@ int ha_myisammrg::index_next_same(uchar const uchar *key __attribute__((unused)), uint length __attribute__((unused))) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_next_count); int error=myrg_rnext_same(file,buf); table->status=error ? STATUS_NOT_FOUND: 0; @@ -280,12 +709,14 @@ int ha_myisammrg::index_next_same(uchar int ha_myisammrg::rnd_init(bool scan) { + DBUG_ASSERT(this->file->children_attached); return myrg_reset(file); } int ha_myisammrg::rnd_next(uchar *buf) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_rnd_next_count); int error=myrg_rrnd(file, buf, HA_OFFSET_ERROR); table->status=error ? STATUS_NOT_FOUND: 0; @@ -295,6 +726,7 @@ int ha_myisammrg::rnd_next(uchar *buf) int ha_myisammrg::rnd_pos(uchar * buf, uchar *pos) { + DBUG_ASSERT(this->file->children_attached); ha_statistic_increment(&SSV::ha_read_rnd_count); int error=myrg_rrnd(file, buf, my_get_ptr(pos,ref_length)); table->status=error ? STATUS_NOT_FOUND: 0; @@ -303,6 +735,7 @@ int ha_myisammrg::rnd_pos(uchar * buf, u void ha_myisammrg::position(const uchar *record) { + DBUG_ASSERT(this->file->children_attached); ulonglong row_position= myrg_position(file); my_store_ptr(ref, ref_length, (my_off_t) row_position); } @@ -311,6 +744,7 @@ void ha_myisammrg::position(const uchar ha_rows ha_myisammrg::records_in_range(uint inx, key_range *min_key, key_range *max_key) { + DBUG_ASSERT(this->file->children_attached); return (ha_rows) myrg_records_in_range(file, (int) inx, min_key, max_key); } @@ -318,6 +752,7 @@ ha_rows ha_myisammrg::records_in_range(u int ha_myisammrg::info(uint flag) { MYMERGE_INFO mrg_info; + DBUG_ASSERT(this->file->children_attached); (void) myrg_status(file,&mrg_info,flag); /* The following fails if one has not compiled MySQL with -DBIG_TABLES @@ -387,6 +822,23 @@ int ha_myisammrg::info(uint flag) int ha_myisammrg::extra(enum ha_extra_function operation) { + if (operation == HA_EXTRA_ATTACH_CHILDREN) + { + int rc= attach_children(); + if (!rc) + (void) extra(HA_EXTRA_NO_READCHECK); // Not needed in SQL + return(rc); + } + else if (operation == HA_EXTRA_DETACH_CHILDREN) + { + /* + Note that detach must not touch the children in any way. + They may have been closed at ths point already. + */ + int rc= detach_children(); + return(rc); + } + /* As this is just a mapping, we don't have to force the underlying tables to be closed */ if (operation == HA_EXTRA_FORCE_REOPEN || @@ -404,6 +856,7 @@ int ha_myisammrg::reset(void) int ha_myisammrg::extra_opt(enum ha_extra_function operation, ulong cache_size) { + DBUG_ASSERT(this->file->children_attached); if ((specialflag & SPECIAL_SAFE_MODE) && operation == HA_EXTRA_WRITE_CACHE) return 0; return myrg_extra(file, operation, (void*) &cache_size); @@ -411,12 +864,17 @@ int ha_myisammrg::extra_opt(enum ha_extr int ha_myisammrg::external_lock(THD *thd, int lock_type) { + /* Unlocking can be tried with detached children, e.g close_old_data_files. */ + DBUG_ASSERT(this->file->children_attached || (lock_type == F_UNLCK)); + if (!file->children_attached) + return 0; return myrg_lock_database(file,lock_type); } uint ha_myisammrg::lock_count(void) const { - return file->tables; + /* This can be called with detached children, e.g close_old_data_files. */ + return file->children_attached ? file->tables : 0; } @@ -426,6 +884,10 @@ THR_LOCK_DATA **ha_myisammrg::store_lock { MYRG_TABLE *open_table; + /* This can be called with detached children, e.g close_old_data_files. */ + if (!file->children_attached) + return to; + for (open_table=file->open_tables ; open_table != file->end_table ; open_table++) @@ -519,47 +981,50 @@ int ha_myisammrg::create(const char *nam uint dirlgt= dirname_length(name); DBUG_ENTER("ha_myisammrg::create"); + /* Allocate a table_names array in thread mem_root. */ if (!(table_names= (const char**) thd->alloc((create_info->merge_list.elements+1) * sizeof(char*)))) DBUG_RETURN(HA_ERR_OUT_OF_MEM); + + /* Create child path names. */ for (pos= table_names; tables; tables= tables->next_local) { const char *table_name; - TABLE *tbl= 0; - if (create_info->options & HA_LEX_CREATE_TMP_TABLE) - tbl= find_temporary_table(thd, tables); - if (!tbl) - { - /* - Construct the path to the MyISAM table. Try to meet two conditions: - 1.) Allow to include MyISAM tables from different databases, and - 2.) allow for moving DATADIR around in the file system. - The first means that we need paths in the .MRG file. The second - means that we should not have absolute paths in the .MRG file. - The best, we can do, is to use 'mysql_data_home', which is '.' - in mysqld and may be an absolute path in an embedded server. - This means that it might not be possible to move the DATADIR of - an embedded server without changing the paths in the .MRG file. - */ - uint length= build_table_filename(buff, sizeof(buff), - tables->db, tables->table_name, "", 0); - /* - If a MyISAM table is in the same directory as the MERGE table, - we use the table name without a path. This means that the - DATADIR can easily be moved even for an embedded server as long - as the MyISAM tables are from the same database as the MERGE table. - */ - if ((dirname_length(buff) == dirlgt) && ! memcmp(buff, name, dirlgt)) - table_name= tables->table_name; - else - if (! (table_name= thd->strmake(buff, length))) - DBUG_RETURN(HA_ERR_OUT_OF_MEM); - } + + /* + Construct the path to the MyISAM table. Try to meet two conditions: + 1.) Allow to include MyISAM tables from different databases, and + 2.) allow for moving DATADIR around in the file system. + The first means that we need paths in the .MRG file. The second + means that we should not have absolute paths in the .MRG file. + The best, we can do, is to use 'mysql_data_home', which is '.' + in mysqld and may be an absolute path in an embedded server. + This means that it might not be possible to move the DATADIR of + an embedded server without changing the paths in the .MRG file. + + Do the same even for temporary tables. MERGE children are now + opened through the table cache. They are opened by db.table_name, + not by their path name. + */ + uint length= build_table_filename(buff, sizeof(buff), + tables->db, tables->table_name, "", 0); + /* + If a MyISAM table is in the same directory as the MERGE table, + we use the table name without a path. This means that the + DATADIR can easily be moved even for an embedded server as long + as the MyISAM tables are from the same database as the MERGE table. + */ + if ((dirname_length(buff) == dirlgt) && ! memcmp(buff, name, dirlgt)) + table_name= tables->table_name; else - table_name= tbl->s->path.str; + if (! (table_name= thd->strmake(buff, length))) + DBUG_RETURN(HA_ERR_OUT_OF_MEM); + *pos++= table_name; } *pos=0; + + /* Create a MERGE meta file from the table_names array. */ DBUG_RETURN(myrg_create(fn_format(buff,name,"","", MY_RESOLVE_SYMLINKS| MY_UNPACK_FILENAME|MY_APPEND_EXT), diff -Nrup a/storage/myisammrg/ha_myisammrg.h b/storage/myisammrg/ha_myisammrg.h --- a/storage/myisammrg/ha_myisammrg.h 2007-08-13 15:11:17 +02:00 +++ b/storage/myisammrg/ha_myisammrg.h 2007-10-15 16:02:49 +02:00 @@ -27,8 +27,12 @@ class ha_myisammrg: public handler MYRG_INFO *file; public: + TABLE_LIST *next_child_attach; /* next child to attach */ + uint test_if_locked; /* flags from ::open() */ + bool need_compat_check; /* if need compatibility check */ + ha_myisammrg(handlerton *hton, TABLE_SHARE *table_arg); - ~ha_myisammrg() {} + ~ha_myisammrg(); const char *table_type() const { return "MRG_MyISAM"; } const char **bas_ext() const; const char *index_type(uint key_number); @@ -53,6 +57,8 @@ class ha_myisammrg: public handler { return ulonglong2double(stats.data_file_length) / IO_SIZE + file->tables; } int open(const char *name, int mode, uint test_if_locked); + int attach_children(void); + int detach_children(void); int close(void); int write_row(uchar * buf); int update_row(const uchar * old_data, uchar * new_data); @@ -85,6 +91,7 @@ class ha_myisammrg: public handler void update_create_info(HA_CREATE_INFO *create_info); void append_create_info(String *packet); MYRG_INFO *myrg_info() { return file; } + TABLE *table_ptr() { return table; } bool check_if_incompatible_data(HA_CREATE_INFO *info, uint table_changes); int check(THD* thd, HA_CHECK_OPT* check_opt); }; diff -Nrup a/storage/myisammrg/myrg_close.c b/storage/myisammrg/myrg_close.c --- a/storage/myisammrg/myrg_close.c 2007-05-10 11:59:33 +02:00 +++ b/storage/myisammrg/myrg_close.c 2007-10-15 16:02:49 +02:00 @@ -23,9 +23,33 @@ int myrg_close(MYRG_INFO *info) MYRG_TABLE *file; DBUG_ENTER("myrg_close"); - for (file=info->open_tables ; file != info->end_table ; file++) - if ((new_error=mi_close(file->table))) - error=new_error; + /* + Assume that info->children_attached means that this is called from + direct use of MERGE, not from a MySQL server. In this case the + children must be closed and info->rec_per_key_part is part of the + 'info' multi_alloc. + If info->children_attached is false, this is called from a MySQL + server. Children are closed independently but info->rec_per_key_part + must be freed. + Just in case of a server panic (myrg_panic()) info->children_attached + might be true. We would close the children though they should be + closed independently and info->rec_per_key_part is not freed. + This should be acceptable for a panic. + In case of a MySQL server and no children, children_attached is + always true. In this case no rec_per_key_part has been allocated. + So it is correct to use the branch where an empty list of tables is + (not) closed. + */ + if (info->children_attached) + { + for (file=info->open_tables ; file != info->end_table ; file++) + if ((new_error=mi_close(file->table))) + error=new_error; + else + file->table= NULL; + } + else + my_free((uchar*) info->rec_per_key_part, MYF(MY_ALLOW_ZERO_PTR)); delete_queue(&info->by_key); pthread_mutex_lock(&THR_LOCK_open); myrg_open_list=list_delete(myrg_open_list,&info->open_list); diff -Nrup a/storage/myisammrg/myrg_extra.c b/storage/myisammrg/myrg_extra.c --- a/storage/myisammrg/myrg_extra.c 2007-03-28 20:05:08 +02:00 +++ b/storage/myisammrg/myrg_extra.c 2007-10-15 16:02:49 +02:00 @@ -29,6 +29,8 @@ int myrg_extra(MYRG_INFO *info,enum ha_e DBUG_ENTER("myrg_extra"); DBUG_PRINT("info",("function: %lu", (ulong) function)); + if (!info->children_attached) + DBUG_RETURN(1); if (function == HA_EXTRA_CACHE) { info->cache_in_use=1; @@ -73,6 +75,8 @@ int myrg_reset(MYRG_INFO *info) MYRG_TABLE *file; DBUG_ENTER("myrg_reset"); + if (!info->children_attached) + DBUG_RETURN(1); info->cache_in_use=0; info->current_table=0; info->last_used_table= info->open_tables; diff -Nrup a/storage/myisammrg/myrg_open.c b/storage/myisammrg/myrg_open.c --- a/storage/myisammrg/myrg_open.c 2007-06-07 10:39:11 +02:00 +++ b/storage/myisammrg/myrg_open.c 2007-10-15 16:02:49 +02:00 @@ -107,13 +107,11 @@ MYRG_INFO *myrg_open(const char *name, i key_parts*sizeof(long), MYF(MY_WME|MY_ZEROFILL)))) goto err; - if (files) - { - m_info->open_tables=(MYRG_TABLE *) (m_info+1); - m_info->rec_per_key_part=(ulong *) (m_info->open_tables+files); - m_info->tables= files; - files= 0; - } + DBUG_ASSERT(files); + m_info->open_tables=(MYRG_TABLE *) (m_info+1); + m_info->rec_per_key_part=(ulong *) (m_info->open_tables+files); + m_info->tables= files; + files= 0; m_info->reclength=isam->s->base.reclength; min_keys= isam->s->base.keys; errpos=3; @@ -163,6 +161,7 @@ MYRG_INFO *myrg_open(const char *name, i /* this works ok if the table list is empty */ m_info->end_table=m_info->open_tables+files; m_info->last_used_table=m_info->open_tables; + m_info->children_attached= TRUE; VOID(my_close(fd,MYF(0))); end_io_cache(&file); @@ -189,3 +188,305 @@ err: my_errno=save_errno; DBUG_RETURN (NULL); } + + +/** + @brief Open parent table of a MyISAM MERGE table. + + @detail Open MERGE meta file to get the table name paths for the child + tables. Count the children. Allocate and initialize MYRG_INFO + structure. Call a callback function for each child table. + + @param[in] parent_name merge table name path as "database/table" + @param[in] callback function to call for each child table + @param[in] callback_param data pointer to give to the callback + + @return MYRG_INFO pointer + @retval != NULL OK + @retval NULL Error +*/ + +MYRG_INFO *myrg_parent_open(const char *parent_name, + int (*callback)(void*, const char*), + void *callback_param) +{ + MYRG_INFO *m_info; + int rc; + int errpos; + int save_errno; + int insert_method; + uint length; + uint dir_length; + uint child_count; + size_t name_buff_length; + File fd; + IO_CACHE file_cache; + char parent_name_buff[FN_REFLEN * 2]; + char child_name_buff[FN_REFLEN]; + DBUG_ENTER("myrg_parent_open"); + + rc= 1; + errpos= 0; + bzero((char*) &file_cache, sizeof(file_cache)); + + /* Open MERGE meta file. */ + if ((fd= my_open(fn_format(parent_name_buff, parent_name, "", MYRG_NAME_EXT, + MY_UNPACK_FILENAME|MY_APPEND_EXT), + O_RDONLY | O_SHARE, MYF(0))) < 0) + goto err; + errpos= 1; + + if (init_io_cache(&file_cache, fd, 4 * IO_SIZE, READ_CACHE, 0, 0, + MYF(MY_WME | MY_NABP))) + goto err; + errpos= 2; + + /* Count children. Determine insert method. */ + child_count= 0; + insert_method= 0; + while ((length= my_b_gets(&file_cache, child_name_buff, FN_REFLEN - 1))) + { + /* Remove line terminator. */ + if (child_name_buff[length - 1] == '\n') + child_name_buff[--length]= '\0'; + + /* Skip empty lines. */ + if (!child_name_buff[0]) + continue; + + /* Skip comments, but evaluate insert method. */ + if (child_name_buff[0] == '#') + { + if (!strncmp(child_name_buff + 1, "INSERT_METHOD=", 14)) + { + /* Compare buffer with global methods list: merge_insert_method. */ + insert_method= find_type(child_name_buff + 15, + &merge_insert_method, 2); + } + continue; + } + + /* Count the child. */ + child_count++; + } + + /* Allocate MERGE parent table structure. */ + if (!(m_info= (MYRG_INFO*) my_malloc(sizeof(MYRG_INFO) + + child_count * sizeof(MYRG_TABLE), + MYF(MY_WME | MY_ZEROFILL)))) + goto err; + errpos= 3; + m_info->open_tables= (MYRG_TABLE*) (m_info + 1); + m_info->tables= child_count; + m_info->merge_insert_method= insert_method > 0 ? insert_method : 0; + /* This works even if the table list is empty. */ + m_info->end_table= m_info->open_tables + child_count; + if (!child_count) + { + /* Do not attach/detach an empty child list. */ + m_info->children_attached= TRUE; + } + + /* Call callback for each child. */ + dir_length= dirname_part(parent_name_buff, parent_name, &name_buff_length); + my_b_seek(&file_cache, 0); + while ((length= my_b_gets(&file_cache, child_name_buff, FN_REFLEN - 1))) + { + /* Remove line terminator. */ + if (child_name_buff[length - 1] == '\n') + child_name_buff[--length]= '\0'; + + /* Skip empty lines and comments. */ + if (!child_name_buff[0] || (child_name_buff[0] == '#')) + continue; + + if (!has_path(child_name_buff)) + { + VOID(strmake(parent_name_buff + dir_length, child_name_buff, + sizeof(parent_name_buff) - 1 - dir_length)); + VOID(cleanup_dirname(child_name_buff, parent_name_buff)); + } + else + fn_format(child_name_buff, child_name_buff, "", "", 0); + DBUG_PRINT("info", ("child: '%s'", child_name_buff)); + + /* Callback registers child with handler table. */ + if ((rc= (*callback)(callback_param, child_name_buff))) + goto err; + } + + end_io_cache(&file_cache); + VOID(my_close(fd, MYF(0))); + + m_info->open_list.data= (void*) m_info; + pthread_mutex_lock(&THR_LOCK_open); + myrg_open_list= list_add(myrg_open_list, &m_info->open_list); + pthread_mutex_unlock(&THR_LOCK_open); + + DBUG_RETURN(m_info); + +err: + save_errno= my_errno; + switch (errpos) { + case 3: + my_free((char*) m_info, MYF(0)); + /* Fall through */ + case 2: + end_io_cache(&file_cache); + /* Fall through */ + case 1: + VOID(my_close(fd, MYF(0))); + } + my_errno= save_errno; + DBUG_RETURN (NULL); +} + + +/** + @brief Attach children to a MyISAM MERGE parent table. + + @detail Call a callback function for each child table. + The callback returns the MyISAM table handle of the child table. + Check table definition match. + + @param[in] m_info MERGE parent table structure + @param[in] handle_locking if contains HA_OPEN_FOR_REPAIR, warn about + incompatible child tables, but continue + @param[in] callback function to call for each child table + @param[in] callback_param data pointer to give to the callback + + @return status + @retval 0 OK + @retval != 0 Error +*/ + +int myrg_attach_children(MYRG_INFO *m_info, int handle_locking, + MI_INFO *(*callback)(void*), + void *callback_param) +{ + ulonglong file_offset; + MI_INFO *myisam; + int rc; + int errpos; + int save_errno; + uint idx; + uint child_nr; + uint key_parts; + uint min_keys; + DBUG_ENTER("myrg_attach_children"); + DBUG_PRINT("myrg", ("handle_locking: %d", handle_locking)); + + rc= 1; + errpos= 0; + file_offset= 0; + LINT_INIT(key_parts); + min_keys= 0; + child_nr= 0; + while ((myisam= (*callback)(callback_param))) + { + DBUG_PRINT("myrg", ("child_nr: %u table: '%s'", + child_nr, myisam->filename)); + DBUG_ASSERT(child_nr < m_info->tables); + + /* Special handling when the first child is attached. */ + if (!child_nr) + { + m_info->reclength= myisam->s->base.reclength; + min_keys= myisam->s->base.keys; + key_parts= myisam->s->base.key_parts; + if (!m_info->rec_per_key_part) + { + if(!(m_info->rec_per_key_part= (ulong*) + my_malloc(key_parts * sizeof(long), MYF(MY_WME)))) + goto err; + errpos= 1; + } + } + + /* Add MyISAM table info. */ + m_info->open_tables[child_nr].table= myisam; + m_info->open_tables[child_nr].file_offset= (my_off_t) file_offset; + file_offset+= myisam->state->data_file_length; + + /* Check table definition match. */ + if (m_info->reclength != myisam->s->base.reclength) + { + DBUG_PRINT("error", ("definition mismatch table: '%s' repair: %d", + myisam->filename, + (handle_locking & HA_OPEN_FOR_REPAIR))); + my_errno= HA_ERR_WRONG_MRG_TABLE_DEF; + if (handle_locking & HA_OPEN_FOR_REPAIR) + { + myrg_print_wrong_table(myisam->filename); + continue; + } + goto err; + } + + m_info->options|= myisam->s->options; + m_info->records+= myisam->state->records; + m_info->del+= myisam->state->del; + m_info->data_file_length+= myisam->state->data_file_length; + if (min_keys > myisam->s->base.keys) + min_keys= myisam->s->base.keys; + for (idx= 0; idx < key_parts; idx++) + m_info->rec_per_key_part[idx]+= (myisam->s->state.rec_per_key_part[idx] / + m_info->tables); + child_nr++; + } + + if (my_errno == HA_ERR_WRONG_MRG_TABLE_DEF) + goto err; + if (sizeof(my_off_t) == 4 && file_offset > (ulonglong) (ulong) ~0L) + { + my_errno= HA_ERR_RECORD_FILE_FULL; + goto err; + } + /* Don't mark table readonly, for ALTER TABLE ... UNION=(...) to work */ + m_info->options&= ~(HA_OPTION_COMPRESS_RECORD | HA_OPTION_READ_ONLY_DATA); + m_info->keys= min_keys; + m_info->last_used_table= m_info->open_tables; + m_info->children_attached= TRUE; + DBUG_RETURN(0); + +err: + save_errno= my_errno; + switch (errpos) { + case 1: + my_free((char*) m_info->rec_per_key_part, MYF(0)); + m_info->rec_per_key_part= NULL; + } + my_errno= save_errno; + DBUG_RETURN(1); +} + + +/** + @brief Detach children from a MyISAM MERGE parent table. + + @param[in] m_info MERGE parent table structure + + @note Detach must not touch the children in any way. + They may have been closed at ths point already. + All references to the children should be removed. + + @return status + @retval 0 OK +*/ + +int myrg_detach_children(MYRG_INFO *m_info) +{ + DBUG_ENTER("myrg_detach_children"); + if (m_info->tables) + { + /* Do not attach/detach an empty child list. */ + m_info->children_attached= FALSE; + bzero((char*) m_info->open_tables, m_info->tables * sizeof(MYRG_TABLE)); + } + m_info->records= 0; + m_info->del= 0; + m_info->data_file_length= 0; + m_info->options= 0; + DBUG_RETURN(0); +} +
| Thread | ||
|---|---|---|
| • bk commit into 5.1 tree (istruewing:1.2576) BUG#26379 | Ingo Struewing | 15 Oct |
© 1995-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
Page generated in 1.600 sec. using MySQL 5.1.14-beta-log