List:Commits« Previous MessageNext Message »
From:Sergey Glukhov Date:April 24 2012 7:36am
Subject:bzr push into mysql-trunk branch (sergey.glukhov:3730 to 3731) Bug#13581962
View as plain text  
 3731 Sergey Glukhov	2012-04-24
      Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
      In Item_func_signed::fix_length_and_dec max_length is inherited
      from the max length of the argument. It can be huge value in case
      of LONGTEXT, for example, and leads to OOM in some situations.
      The fix is to use the minimum of args[0]->length and max possible
      number of digits for the int type.
     @ mysql-test/r/cast.result
        test case
     @ mysql-test/r/ctype_utf8mb4.result
        test case
     @ mysql-test/t/cast.test
        test case
     @ mysql-test/t/ctype_utf8mb4.test
        test case
     @ sql/item_func.h
        limit max length by MY_INT64_NUM_DECIMAL_DIGITS

    modified:
      mysql-test/r/cast.result
      mysql-test/r/ctype_utf8mb4.result
      mysql-test/t/cast.test
      mysql-test/t/ctype_utf8mb4.test
      sql/item_func.h
 3730 Vasil Dimov	2012-04-24
      Implement WL#6175 Introduce persistent statistics ON/OFF switch
      
      A new table_option is added, similarly to e.g. DELAY_KEY_WRITE. Modified files are:
        include/my_base.h
        sql/handler.h
        sql/lex.h
        sql/sql_show.cc
        sql/sql_table.cc
        sql/sql_yacc.yy
      The addition of the new option is purely mechanical - copy every occurence of
      DELAY_KEY_WRITE and rename it to the new STATS_PERSISTENT option. The .frm file
      format is not changed.
      
      In the code before this WL the logic was: every time stats would be updated in
      pre-PS MySQL (e.g. MySQL 5.5) do this: try to fetch stats for that table from
      persistent stats tables (e.g. check if PS is ON for that table), if fetch
      succeeds, then this is it. If fetch fails (due to e.g. no rows for that table in
      the stats storage, PS is OFF) then calculate the stats using the old transient
      statistics method.
      
      Below by "PS enabled table" or "table has PS set to ON" we mean that either it
      been explicitly flagged as STATS_PERSISTENT=1 by CREATE/ALTER TABLE or that its
      STATS_PERSISTENT= flag is not set ('default') and the global
      innodb_stats_persistent is ON.
      
      With this WL we change the logic to: every time stats would be updated in pre-PS
      MySQL do this: check if the table has PS flag set to ON, if yes, then do nothing
      (except if the stats are not initialized (e.g. first open table), in which case
      fetch them from disk), else calculate the stats using the old transient method.
      
      So we introduce a new per-table flag and store it in the .frm file. Upon open
      table we copy that flag into dict_table_t::stat_persistent to ease
      checking the value from InnoDB internal code.
      
      We introduce a new state for a table and its indexes' stats: 'unknown' which
      means that stats have not been calculated after opening the table and should not
      be accessed when in this state. It is also unknown whether the table's
      persistent stats flag is ON or OFF. This idea comes from Marko and is based on
      the fact that stats are only needed by MySQL and are requested after a call to
      ::open(). So whenever we open a table internally in InnoDB (table not being
      opened by MySQL) we leave the stats in 'unknown' state. When ::open() is called,
      then we peek the value of the PS flag from table->s->db_create_options, save it
      in the newly introduced table->stat_persistent and do the
      corresponding action to initialize the stats -
      either fetch them from disk if PS=ON or recalc them if PS=OFF. This is done by
      the newly introduced function dict_stats_init(). The new 'unknown' state is
      designated by table->stat_initialized being FALSE.
      
      When the PS flag is changed by the user with ALTER TABLE, we need to set the new
      value in table->stat_persistent - this is done by the new function
      dict_stats_set_persistent() which is also used to set the value of the flag
      initially from inside dict_stats_init() (called from ::open() and ::create()).
      
      A new function is introduced: dict_stats_is_persistent_enabled(table) which
      checks whether PS is switched ON/OFF for the given table. This function is very
      fast because it just checks if the bits are set in
      table->stat_persistent (compared to the slow SELECTing from the stats tables
      that was done before).
      
      In row_update_statistics_if_needed() if this is PS-enabled table, then do
      nothing. This function may be called very often. Also do nothing if the stats
      are in 'unknown' state - this may happen when the table has been opened
      internally from within InnoDB (e.g. SYS_TABLES or a foreign key table, not
      opened by MySQL) and heavy DML is being done on it.
      
      ha_innobase::info() -> ::info_low() indirection is no loner needed and is
      removed. With this WL we have just one method - ::info() which checks if the
      executed command is ANALYZE and does its logic internally.
      
      Given that stats initialization is now done in ::open() and ::create() after a
      call to dict_table_open_on_name() and not in that latter function, this means
      that this function becomes the same as dict_table_open_on_name_low() and
      dict_table_open_on_name_no_stats() is no longer needed. Thus, remove
      dict_table_open_on_name() and dict_table_open_on_name_no_stats() and rename
      dict_table_open_on_name_low() to dict_table_open_on_name().
      
      The option innodb_analyze_is_persistent is no longer needed and is removed.
      
      Approved by:	Jimmy (rb:912)

    removed:
      mysql-test/suite/sys_vars/r/innodb_analyze_is_persistent_basic.result
      mysql-test/suite/sys_vars/t/innodb_analyze_is_persistent_basic.test
    added:
      mysql-test/suite/innodb/include/innodb_stats_table_flag.inc
      mysql-test/suite/innodb/include/innodb_stats_table_flag_analyze.inc
      mysql-test/suite/innodb/r/innodb_stats_create_on_corrupted.result
      mysql-test/suite/innodb/r/innodb_stats_create_table.result
      mysql-test/suite/innodb/r/innodb_stats_fetch.result
      mysql-test/suite/innodb/r/innodb_stats_fetch_corrupted.result
      mysql-test/suite/innodb/r/innodb_stats_fetch_nonexistent.result
      mysql-test/suite/innodb/r/innodb_stats_flag_global_off.result
      mysql-test/suite/innodb/r/innodb_stats_flag_global_on.result
      mysql-test/suite/innodb/r/innodb_upd_stats_if_needed_not_inited.result
      mysql-test/suite/innodb/r/innodb_ut_format_name.result
      mysql-test/suite/innodb/t/innodb_stats_create_on_corrupted.test
      mysql-test/suite/innodb/t/innodb_stats_create_table.test
      mysql-test/suite/innodb/t/innodb_stats_fetch.test
      mysql-test/suite/innodb/t/innodb_stats_fetch_corrupted.test
      mysql-test/suite/innodb/t/innodb_stats_fetch_nonexistent.test
      mysql-test/suite/innodb/t/innodb_stats_flag_global_off-master.opt
      mysql-test/suite/innodb/t/innodb_stats_flag_global_off.test
      mysql-test/suite/innodb/t/innodb_stats_flag_global_on-master.opt
      mysql-test/suite/innodb/t/innodb_stats_flag_global_on.test
      mysql-test/suite/innodb/t/innodb_upd_stats_if_needed_not_inited.test
      mysql-test/suite/innodb/t/innodb_ut_format_name.test
      mysql-test/suite/sys_vars/r/innodb_stats_persistent_basic.result
      mysql-test/suite/sys_vars/t/innodb_stats_persistent_basic.test
      storage/innobase/include/dict0stats.ic
    modified:
      include/my_base.h
      mysql-test/suite/innodb/include/innodb_stats.inc
      mysql-test/suite/innodb/r/innodb-index-online.result
      mysql-test/suite/innodb/r/innodb_bug11933790.result
      mysql-test/suite/innodb/r/innodb_bug12429573.result
      mysql-test/suite/innodb/r/innodb_stats.result
      mysql-test/suite/innodb/r/innodb_stats_drop_locked.result
      mysql-test/suite/innodb/t/innodb-blob.test
      mysql-test/suite/innodb/t/innodb-index-online.test
      mysql-test/suite/innodb/t/innodb_bug11933790.test
      mysql-test/suite/innodb/t/innodb_bug12429573.test
      mysql-test/suite/innodb/t/innodb_stats.test
      mysql-test/suite/innodb/t/innodb_stats_drop_locked.test
      mysql-test/suite/perfschema/r/digest_table_full.result
      mysql-test/suite/perfschema/r/statement_digest.result
      mysql-test/suite/perfschema/r/statement_digest_consumers.result
      mysql-test/suite/perfschema/r/statement_digest_long_query.result
      scripts/mysql_system_tables.sql
      sql/handler.h
      sql/lex.h
      sql/sql_show.cc
      sql/sql_table.cc
      sql/sql_yacc.yy
      storage/innobase/api/api0api.cc
      storage/innobase/btr/btr0cur.cc
      storage/innobase/dict/dict0dict.cc
      storage/innobase/dict/dict0load.cc
      storage/innobase/dict/dict0stats.cc
      storage/innobase/fts/fts0fts.cc
      storage/innobase/fts/fts0opt.cc
      storage/innobase/fts/fts0que.cc
      storage/innobase/handler/ha_innodb.cc
      storage/innobase/handler/ha_innodb.h
      storage/innobase/handler/handler0alter.cc
      storage/innobase/handler/i_s.cc
      storage/innobase/include/dict0dict.h
      storage/innobase/include/dict0dict.ic
      storage/innobase/include/dict0mem.h
      storage/innobase/include/dict0stats.h
      storage/innobase/include/fts0priv.h
      storage/innobase/include/srv0srv.h
      storage/innobase/include/ut0ut.h
      storage/innobase/pars/pars0pars.cc
      storage/innobase/row/row0ftsort.cc
      storage/innobase/row/row0ins.cc
      storage/innobase/row/row0mysql.cc
      storage/innobase/row/row0sel.cc
      storage/innobase/row/row0upd.cc
      storage/innobase/srv/srv0srv.cc
      storage/innobase/ut/ut0ut.cc
=== modified file 'mysql-test/r/cast.result'
--- a/mysql-test/r/cast.result	2012-04-13 11:50:32 +0000
+++ b/mysql-test/r/cast.result	2012-04-24 07:35:13 +0000
@@ -469,3 +469,19 @@ Warnings:
 Warning	1301	Result of cast_as_char() was larger than max_allowed_packet (2048) - truncated
 SET @@GLOBAL.max_allowed_packet=default;
 End of 5.1 tests
+#
+# Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+#
+CREATE TABLE t1 AS SELECT CONCAT(CAST(REPEAT('9', 1000) AS SIGNED)),
+CONCAT(CAST(REPEAT('9', 1000) AS UNSIGNED));
+Warnings:
+Warning	1292	Truncated incorrect INTEGER value: '99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999'
+Warning	1292	Truncated incorrect INTEGER value: '99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999'
+SHOW CREATE TABLE t1;
+Table	Create Table
+t1	CREATE TABLE `t1` (
+  `CONCAT(CAST(REPEAT('9', 1000) AS SIGNED))` varchar(21) NOT NULL DEFAULT '',
+  `CONCAT(CAST(REPEAT('9', 1000) AS UNSIGNED))` varchar(21) NOT NULL DEFAULT ''
+) ENGINE=MyISAM DEFAULT CHARSET=latin1
+DROP TABLE t1;
+End of 5.5 tests

=== modified file 'mysql-test/r/ctype_utf8mb4.result'
--- a/mysql-test/r/ctype_utf8mb4.result	2012-03-08 14:22:07 +0000
+++ b/mysql-test/r/ctype_utf8mb4.result	2012-04-24 07:35:13 +0000
@@ -2601,6 +2601,21 @@ t2	CREATE TABLE `t2` (
 ) ENGINE=MyISAM DEFAULT CHARSET=latin1
 DROP TABLE t1, t2;
 #
+# Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+#
+CREATE TABLE t1(f1 LONGTEXT CHARACTER SET utf8mb4);
+INSERT INTO t1 VALUES ('a');
+SELECT @a:= CAST(f1 AS SIGNED) FROM t1
+UNION ALL
+SELECT CAST(f1 AS SIGNED) FROM t1;
+@a:= CAST(f1 AS SIGNED)
+0
+0
+Warnings:
+Warning	1292	Truncated incorrect INTEGER value: 'a'
+Warning	1292	Truncated incorrect INTEGER value: 'a'
+DROP TABLE t1;
+#
 # End of 5.5 tests
 #
 #

=== modified file 'mysql-test/t/cast.test'
--- a/mysql-test/t/cast.test	2011-11-01 11:52:24 +0000
+++ b/mysql-test/t/cast.test	2012-04-24 07:35:13 +0000
@@ -296,3 +296,14 @@ disconnect newconn;
 SET @@GLOBAL.max_allowed_packet=default;
 
 --echo End of 5.1 tests
+
+--echo #
+--echo # Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+--echo #
+
+CREATE TABLE t1 AS SELECT CONCAT(CAST(REPEAT('9', 1000) AS SIGNED)),
+                          CONCAT(CAST(REPEAT('9', 1000) AS UNSIGNED));
+SHOW CREATE TABLE t1;
+DROP TABLE t1;
+
+--echo End of 5.5 tests

=== modified file 'mysql-test/t/ctype_utf8mb4.test'
--- a/mysql-test/t/ctype_utf8mb4.test	2012-03-05 13:18:39 +0000
+++ b/mysql-test/t/ctype_utf8mb4.test	2012-04-24 07:35:13 +0000
@@ -1812,6 +1812,17 @@ SHOW CREATE TABLE t2;
 DROP TABLE t1, t2;
 
 --echo #
+--echo # Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+--echo #
+
+CREATE TABLE t1(f1 LONGTEXT CHARACTER SET utf8mb4);
+INSERT INTO t1 VALUES ('a');
+SELECT @a:= CAST(f1 AS SIGNED) FROM t1
+UNION ALL
+SELECT CAST(f1 AS SIGNED) FROM t1;
+DROP TABLE t1;
+
+--echo #
 --echo # End of 5.5 tests
 --echo #
 

=== modified file 'sql/item_func.h'
--- a/sql/item_func.h	2012-04-20 06:48:15 +0000
+++ b/sql/item_func.h	2012-04-24 07:35:13 +0000
@@ -528,12 +528,18 @@ public:
 class Item_func_signed :public Item_int_func
 {
 public:
-  Item_func_signed(Item *a) :Item_int_func(a) {}
+  Item_func_signed(Item *a) :Item_int_func(a) 
+  {
+    unsigned_flag= 0;
+  }
   const char *func_name() const { return "cast_as_signed"; }
   longlong val_int();
   longlong val_int_from_str(int *error);
   void fix_length_and_dec()
-  { fix_char_length(args[0]->max_char_length()); unsigned_flag=0; }
+  {
+    fix_char_length(std::min<uint32>(args[0]->max_char_length(),
+                                     MY_INT64_NUM_DECIMAL_DIGITS));
+  }
   virtual void print(String *str, enum_query_type query_type);
   uint decimal_precision() const { return args[0]->decimal_precision(); }
 };
@@ -542,14 +548,11 @@ public:
 class Item_func_unsigned :public Item_func_signed
 {
 public:
-  Item_func_unsigned(Item *a) :Item_func_signed(a) {}
-  const char *func_name() const { return "cast_as_unsigned"; }
-  void fix_length_and_dec()
+  Item_func_unsigned(Item *a) :Item_func_signed(a)
   {
-    fix_char_length(std::min<uint32>(args[0]->max_char_length(),
-                                     DECIMAL_MAX_PRECISION + 2));
-    unsigned_flag=1;
+    unsigned_flag= 1;
   }
+  const char *func_name() const { return "cast_as_unsigned"; }
   longlong val_int();
   virtual void print(String *str, enum_query_type query_type);
 };

No bundle (reason: useless for push emails).
Thread
bzr push into mysql-trunk branch (sergey.glukhov:3730 to 3731) Bug#13581962Sergey Glukhov24 Apr