bk commit into 5.0 tree (gshchepa:1.2610) BUG#36005

[<gshchepa]

Below is the list of changes that have just been committed into a local
5.0 repository of gshchepa.  When gshchepa does a push these changes
will be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html

ChangeSet@stripped, 2008-04-23 02:27:23+05:00, gshchepa@stripped +3 -0
  Fixed bug#36005: server crashes inside NOT IN clause subquery with
                   impossible WHERE/HAVING clause
                   (subselect_single_select_engine::exec).
  
  Allocation and initialization of joined table list t1, t2... of
  subqueries like:
  
      NOT IN (SELECT ... FROM t1,t2,... WHERE 0)
  
  is optimized out, however server tries to traverse this list.

  mysql-test/r/subselect3.result@stripped, 2008-04-23 02:23:53+05:00, gshchepa@stripped +9 -0
    Added test case for bug#36005.

  mysql-test/t/subselect3.test@stripped, 2008-04-23 02:23:54+05:00, gshchepa@stripped +13 -0
    Added test case for bug#36005.

  sql/sql_select.cc@stripped, 2008-04-23 02:23:55+05:00, gshchepa@stripped +1 -0
    Fixed bug#36005.
    
    1. JOIN::prepare initializes JOIN::table counter (actually a size
       of the JOIN::join_tab array) and sets it to a number of joined tables.
    
    2. The make_join_statistics function (when called from JOIN::optimize)
       allocates and fills the JOIN::join_tab array.
       However, when optimizing subselect has impossible (definite false)
       WHERE or HAVING clause, optimizer skips call to make_join_statistics
       and leaves JOIN::join_tab == NULL.
    
    3. subselect_single_select_engine::exec does traversal of the JOIN::join_tab
       array and the server dies because array is not allocated but array
       counter is greater than 0.
    
    The JOIN::optimize method has been modified to reset the JOIN::table
    counter to 0 in cause of impossible WHERE/HAVING clause.

diff -Nrup a/mysql-test/r/subselect3.result b/mysql-test/r/subselect3.result
--- a/mysql-test/r/subselect3.result	2008-03-14 22:55:55 +04:00
+++ b/mysql-test/r/subselect3.result	2008-04-23 02:23:53 +05:00
@@ -770,4 +770,13 @@ SELECT ROW(1, 2) IN (SELECT t1.a, 2 FROM
 ROW(1, 2) IN (SELECT t1.a, 2 FROM t2)
 1
 DROP TABLE t1, t2;
+CREATE TABLE t1 (a INT);
+INSERT INTO t1 VALUES (1),(2),(3);
+CREATE TABLE t2 SELECT * FROM t1;
+SELECT 1 FROM t1 WHERE t1.a NOT IN (SELECT 1 FROM t1, t2 WHERE 0);
+1
+1
+1
+1
+DROP TABLE t1, t2;
 End of 5.0 tests
diff -Nrup a/mysql-test/t/subselect3.test b/mysql-test/t/subselect3.test
--- a/mysql-test/t/subselect3.test	2008-03-14 22:55:58 +04:00
+++ b/mysql-test/t/subselect3.test	2008-04-23 02:23:54 +05:00
@@ -605,4 +605,17 @@ SELECT ROW(1, 2) IN (SELECT t1.a, 2 FROM
 
 DROP TABLE t1, t2;
 
+#
+# Bug #36005: crash in subselect with single row
+#             (subselect_single_select_engine::exec)
+#
+
+CREATE TABLE t1 (a INT);
+INSERT INTO t1 VALUES (1),(2),(3);
+CREATE TABLE t2 SELECT * FROM t1;
+
+SELECT 1 FROM t1 WHERE t1.a NOT IN (SELECT 1 FROM t1, t2 WHERE 0);
+
+DROP TABLE t1, t2;
+
 --echo End of 5.0 tests
diff -Nrup a/sql/sql_select.cc b/sql/sql_select.cc
--- a/sql/sql_select.cc	2008-03-29 00:01:01 +04:00
+++ b/sql/sql_select.cc	2008-04-23 02:23:55 +05:00
@@ -832,6 +832,7 @@ JOIN::optimize()
                             "Impossible HAVING" : "Impossible WHERE"));
       zero_result_cause=  having_value == Item::COND_FALSE ?
                            "Impossible HAVING" : "Impossible WHERE";
+      tables= 0;
       error= 0;
       DBUG_RETURN(0);
     }