List:Commits« Previous MessageNext Message »
From:Roy Lyseng Date:November 22 2011 11:55am
Subject:bzr push into mysql-trunk branch (roy.lyseng:3490 to 3491) Bug#13390138
View as plain text  
 3491 Roy Lyseng	2011-11-22
      Bug#13390138: crash in memcpy from join_cache::write_record_data
      
      This is a similar case as bug#13106350 and bug#13383857. The problem
      here is that a derived table participates in more than one join buffer,
      and the previous enhancements were not designed to handle that.
      
      The query that fails is optimized to this table order:
      
      t1 - t2:bnl - t3 - t4:bnl
      
      All tables here are derived tables, :bnl means that Block Nested Loop
      join buffering is applied for the table. In addition,
      DuplicatesWeedout semi-join strategy is applied for all tables t1 - t4.
      
      Join buffering is first set up for t2, meaning that rows from t1 are
      copied to the join buffer. Next, join buffering is set up for t4,
      meaning that concatenated rows from t1||t2||t3 are copied to the buffer.
      Notice that t1 is part of both join buffers, and t1 being an outer table
      needs a rowid added to the join buffer. A copy object is created for
      each rowid to be copied, and late binding to the table buffer is needed
      because this is a derived table. However, the current code is only
      capable of recording one "late binding" object per join_tab, so we
      fail to bind the table buffer for the first copy object.
      
      The fix is to chain rowid copy objects per join_tab and bind to
      rowid buffers for all chained objects.
      
      Notice also that if t3 above would apply join buffering as well,
      incremental join buffers would be used, and t1 would not have to
      participate in two join buffers.
      
      A MyISAM table with only one row was needed to trigger this
      particular case.
      
      mysql-test/t/derived.test
        Added test case for bug#13390138.
      
      mysql-test/r/derived.result
        Added test case results for bug#13390138.
      
      sql/sql_join_cache.cc
        In create_remaining_fields, chain rowid copy objects.
        Delete references to get_rowid.
      
      sql/sql_select.h
        Added field next_copy_rowid in st_cache_field to chain copy objects.
        Enhanced bind_buffer() to bind multiple chained copy buffers.
        Deleted get_rowid, as it appeared not to be used
        (join_tab->keep_current_rowid is used for same purpose).

    modified:
      mysql-test/r/derived.result
      mysql-test/t/derived.test
      sql/sql_join_cache.cc
      sql/sql_select.h
 3490 Roy Lyseng	2011-11-18
      Bug#13339643: Assertion in JOIN::flatten_subqueries on second execution
      
      Bug#11764757 (former 57623) introduced a DBUG_ASSERT, assuming that a
      join_nest containing a nested join never had the prep_on_expr field set
      when flattening subqueries. Thus, the permanent transformation of this
      field could not be done here, but was postponed to simplify_joins().
      
      However, this bug shows that in the case of a viewed table, the
      join_nest contains a nested join, and prep_on_expr is already set.
      Thus, the assertion is proven wrong, and it should be removed.
      
      mysql-test/include/subquery_sj.inc
        Added test case for bug#13339643
      
      mysql-test/r/subquery_sj_all.result
      mysql-test/r/subquery_sj_all_bka.result
      mysql-test/r/subquery_sj_all_bka_nixbnl.result
      mysql-test/r/subquery_sj_all_bkaunique.result
      mysql-test/r/subquery_sj_dupsweed.result
      mysql-test/r/subquery_sj_dupsweed_bka.result
      mysql-test/r/subquery_sj_dupsweed_bka_nixbnl.result
      mysql-test/r/subquery_sj_dupsweed_bkaunique.result
      mysql-test/r/subquery_sj_firstmatch.result
      mysql-test/r/subquery_sj_firstmatch_bka.result
      mysql-test/r/subquery_sj_firstmatch_bka_nixbnl.result
      mysql-test/r/subquery_sj_firstmatch_bkaunique.result
      mysql-test/r/subquery_sj_loosescan.result
      mysql-test/r/subquery_sj_loosescan_bka.result
      mysql-test/r/subquery_sj_loosescan_bka_nixbnl.result
      mysql-test/r/subquery_sj_loosescan_bkaunique.result
      mysql-test/r/subquery_sj_mat.result
      mysql-test/r/subquery_sj_mat_bka.result
      mysql-test/r/subquery_sj_mat_bka_nixbnl.result
      mysql-test/r/subquery_sj_mat_bkaunique.result
      mysql-test/r/subquery_sj_mat_nosj.result
      mysql-test/r/subquery_sj_none.result
      mysql-test/r/subquery_sj_none_bka.result
      mysql-test/r/subquery_sj_none_bka_nixbnl.result
      mysql-test/r/subquery_sj_none_bkaunique.result
        Added test case results for bug#13339643
      
      sql/sql_select.cc
        Removed DBUG_ASSERT that failed when view was specified.

    modified:
      mysql-test/include/subquery_sj.inc
      mysql-test/r/subquery_sj_all.result
      mysql-test/r/subquery_sj_all_bka.result
      mysql-test/r/subquery_sj_all_bka_nixbnl.result
      mysql-test/r/subquery_sj_all_bkaunique.result
      mysql-test/r/subquery_sj_dupsweed.result
      mysql-test/r/subquery_sj_dupsweed_bka.result
      mysql-test/r/subquery_sj_dupsweed_bka_nixbnl.result
      mysql-test/r/subquery_sj_dupsweed_bkaunique.result
      mysql-test/r/subquery_sj_firstmatch.result
      mysql-test/r/subquery_sj_firstmatch_bka.result
      mysql-test/r/subquery_sj_firstmatch_bka_nixbnl.result
      mysql-test/r/subquery_sj_firstmatch_bkaunique.result
      mysql-test/r/subquery_sj_loosescan.result
      mysql-test/r/subquery_sj_loosescan_bka.result
      mysql-test/r/subquery_sj_loosescan_bka_nixbnl.result
      mysql-test/r/subquery_sj_loosescan_bkaunique.result
      mysql-test/r/subquery_sj_mat.result
      mysql-test/r/subquery_sj_mat_bka.result
      mysql-test/r/subquery_sj_mat_bka_nixbnl.result
      mysql-test/r/subquery_sj_mat_bkaunique.result
      mysql-test/r/subquery_sj_mat_nosj.result
      mysql-test/r/subquery_sj_none.result
      mysql-test/r/subquery_sj_none_bka.result
      mysql-test/r/subquery_sj_none_bka_nixbnl.result
      mysql-test/r/subquery_sj_none_bkaunique.result
      sql/sql_select.cc
=== modified file 'mysql-test/r/derived.result'
--- a/mysql-test/r/derived.result	2011-11-14 10:48:09 +0000
+++ b/mysql-test/r/derived.result	2011-11-22 11:53:21 +0000
@@ -1835,3 +1835,79 @@ AND outr.col_varchar_key <> 'r'
 x
 DROP TABLE t1, t2, t3;
 #
+#
+# Bug#13390138: crash in memcpy from join_cache::write_record_data
+#
+CREATE TABLE t1 (
+col_varchar_key varchar(1),
+col_varchar_nokey varchar(1),
+KEY col_varchar_key (col_varchar_key)
+) ENGINE=MyISAM;
+INSERT INTO t1 VALUES ('r','r');
+CREATE TABLE t2 (
+col_varchar_key varchar(1),
+col_varchar_nokey varchar(1),
+KEY col_varchar_key (col_varchar_key)
+);
+INSERT INTO t2 VALUES
+(NULL,NULL),
+('r','r');
+CREATE TABLE t3 (
+col_int_key int,
+col_varchar_key varchar(1),
+col_varchar_nokey varchar(1),
+KEY col_int_key (col_int_key),
+KEY col_varchar_key (col_varchar_key, col_int_key)
+);
+INSERT INTO t3 VALUES
+(9,'f','f'),
+(4,'y','y'),
+(3,'u','u'),
+(2,'m','m'),
+(NULL,NULL,NULL),
+(2,'o','o'),
+(NULL,'r','r'),
+(6,'m','m'),
+(7,'q','q'),
+(6,'c','c');
+explain SELECT grandparent.col_varchar_nokey AS g1
+FROM (SELECT * FROM t3) AS grandparent
+WHERE grandparent.col_varchar_nokey IN
+(SELECT parent.col_varchar_key AS p1
+FROM (SELECT * FROM t2) AS parent
+WHERE grandparent.col_varchar_key IN (
+SELECT child1.col_varchar_key AS c1
+FROM (SELECT * FROM t1) AS child1
+LEFT JOIN (SELECT * FROM t2) AS child2
+ON child1.col_varchar_nokey <> child2.col_varchar_key
+)
+AND grandparent.col_int_key IS UNKNOWN
+)
+ORDER BY grandparent.col_varchar_nokey;
+id	select_type	table	type	possible_keys	key	key_len	ref	rows	Extra
+1	PRIMARY	<derived2>	ref	auto_key0	auto_key0	5	const	2	Using where; Using index; Using temporary; Using filesort; Start temporary
+1	PRIMARY	<derived6>	ALL	NULL	NULL	NULL	NULL	1	Using where; Using join buffer (Block Nested Loop)
+1	PRIMARY	<derived4>	ref	auto_key0	auto_key0	4	grandparent.col_varchar_nokey	2	
+1	PRIMARY	<derived7>	ALL	NULL	NULL	NULL	NULL	2	Using where; End temporary; Using join buffer (Block Nested Loop)
+7	DERIVED	t2	ALL	NULL	NULL	NULL	NULL	2	
+6	DERIVED	t1	system	NULL	NULL	NULL	NULL	1	
+4	DERIVED	t2	ALL	NULL	NULL	NULL	NULL	2	
+2	DERIVED	t3	ALL	NULL	NULL	NULL	NULL	10	
+SELECT grandparent.col_varchar_nokey AS g1
+FROM (SELECT * FROM t3) AS grandparent
+WHERE grandparent.col_varchar_nokey IN
+(SELECT parent.col_varchar_key AS p1
+FROM (SELECT * FROM t2) AS parent
+WHERE grandparent.col_varchar_key IN (
+SELECT child1.col_varchar_key AS c1
+FROM (SELECT * FROM t1) AS child1
+LEFT JOIN (SELECT * FROM t2) AS child2
+ON child1.col_varchar_nokey <> child2.col_varchar_key
+)
+AND grandparent.col_int_key IS UNKNOWN
+)
+ORDER BY grandparent.col_varchar_nokey;
+g1
+r
+DROP TABLE t1, t2, t3;
+#

=== modified file 'mysql-test/t/derived.test'
--- a/mysql-test/t/derived.test	2011-11-14 10:48:09 +0000
+++ b/mysql-test/t/derived.test	2011-11-22 11:53:21 +0000
@@ -1207,3 +1207,68 @@ eval $query;
 DROP TABLE t1, t2, t3;
 
 --echo #
+
+--echo #
+--echo # Bug#13390138: crash in memcpy from join_cache::write_record_data
+--echo #
+
+CREATE TABLE t1 (
+  col_varchar_key varchar(1),
+  col_varchar_nokey varchar(1),
+  KEY col_varchar_key (col_varchar_key)
+) ENGINE=MyISAM;
+
+INSERT INTO t1 VALUES ('r','r');
+
+CREATE TABLE t2 (
+  col_varchar_key varchar(1),
+  col_varchar_nokey varchar(1),
+  KEY col_varchar_key (col_varchar_key)
+);
+
+INSERT INTO t2 VALUES
+ (NULL,NULL),
+ ('r','r');
+
+CREATE TABLE t3 (
+  col_int_key int,
+  col_varchar_key varchar(1),
+  col_varchar_nokey varchar(1),
+  KEY col_int_key (col_int_key),
+  KEY col_varchar_key (col_varchar_key, col_int_key)
+);
+
+INSERT INTO t3 VALUES
+ (9,'f','f'),
+ (4,'y','y'),
+ (3,'u','u'),
+ (2,'m','m'),
+ (NULL,NULL,NULL),
+ (2,'o','o'),
+ (NULL,'r','r'),
+ (6,'m','m'),
+ (7,'q','q'),
+ (6,'c','c');
+
+let $query=
+SELECT grandparent.col_varchar_nokey AS g1
+FROM (SELECT * FROM t3) AS grandparent
+WHERE grandparent.col_varchar_nokey IN
+ (SELECT parent.col_varchar_key AS p1
+  FROM (SELECT * FROM t2) AS parent
+  WHERE grandparent.col_varchar_key IN (
+    SELECT child1.col_varchar_key AS c1
+    FROM (SELECT * FROM t1) AS child1
+      LEFT JOIN (SELECT * FROM t2) AS child2
+      ON child1.col_varchar_nokey <> child2.col_varchar_key
+    )
+    AND grandparent.col_int_key IS UNKNOWN
+  )
+ORDER BY grandparent.col_varchar_nokey;
+
+eval explain $query;
+eval $query;
+
+DROP TABLE t1, t2, t3;
+
+--echo #

=== modified file 'sql/sql_join_cache.cc'
--- a/sql/sql_join_cache.cc	2011-11-14 10:48:09 +0000
+++ b/sql/sql_join_cache.cc	2011-11-22 11:53:21 +0000
@@ -68,7 +68,7 @@ uint add_flag_field_to_join_cache(uchar 
   copy->type= 0;
   copy->field= 0;
   copy->referenced_field_no= 0;
-  copy->get_rowid= NULL;
+  copy->next_copy_rowid= NULL;
   (*field)++;
   return length;    
 }
@@ -127,7 +127,7 @@ uint add_table_data_fields_to_join_cache
       }
       copy->field= *fld_ptr;
       copy->referenced_field_no= 0;
-      copy->get_rowid= NULL;
+      copy->next_copy_rowid= NULL;
       copy++;
       (*field_cnt)++;
       used_fields--;
@@ -367,7 +367,10 @@ void JOIN_CACHE:: create_remaining_field
       copy->type= 0;
       copy->field= 0;
       copy->referenced_field_no= 0;
-      copy->get_rowid= NULL;
+      copy->next_copy_rowid= NULL;
+      // Chain rowid copy objects belonging to same join_tab
+      if (tab->copy_current_rowid != NULL)
+        copy->next_copy_rowid= tab->copy_current_rowid;
       tab->copy_current_rowid= copy;
       length+= copy->length;
       data_field_count++;
@@ -1088,12 +1091,6 @@ uint JOIN_CACHE::write_record_data(uchar
     }
     else
     {
-      if (copy->get_rowid)
-      {
-        /* SemiJoinDuplicateElimination: get the rowid into table->ref */
-        copy->get_rowid->file->position(copy->get_rowid->record[0]);
-      }
-
       switch (copy->type) {
       case CACHE_VARSTR1:
         /* Copy the significant part of the short varstring field */ 

=== modified file 'sql/sql_select.h'
--- a/sql/sql_select.h	2011-11-11 12:49:36 +0000
+++ b/sql/sql_select.h	2011-11-22 11:53:21 +0000
@@ -692,12 +692,18 @@ typedef struct st_cache_field {
     trailing sequence of offsets.
   */ 
   uint referenced_field_no; 
-  TABLE *get_rowid; /**< only for ROWID fields used for Duplicate Elimination */
+  /// Used to chain rowid copy objects belonging to one join_tab
+  st_cache_field *next_copy_rowid;
   /* The remaining structure fields are used as containers for temp values */
   uint blob_length; /**< length of the blob to be copied */
   uint offset;      /**< field offset to be saved in cache buffer */
 
-  void bind_buffer(uchar *buffer) { str= buffer; }
+  void bind_buffer(uchar *buffer)
+  {
+    if (next_copy_rowid != NULL)
+      next_copy_rowid->bind_buffer(buffer);
+    str= buffer;
+  }
 } CACHE_FIELD;
 
 

No bundle (reason: useless for push emails).
Thread
bzr push into mysql-trunk branch (roy.lyseng:3490 to 3491) Bug#13390138Roy Lyseng22 Nov