3731 Sergey Glukhov 2012-04-24
Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
In Item_func_signed::fix_length_and_dec max_length is inherited
from the max length of the argument. It can be huge value in case
of LONGTEXT, for example, and leads to OOM in some situations.
The fix is to use the minimum of args[0]->length and max possible
number of digits for the int type.
@ mysql-test/r/cast.result
test case
@ mysql-test/r/ctype_utf8mb4.result
test case
@ mysql-test/t/cast.test
test case
@ mysql-test/t/ctype_utf8mb4.test
test case
@ sql/item_func.h
limit max length by MY_INT64_NUM_DECIMAL_DIGITS
modified:
mysql-test/r/cast.result
mysql-test/r/ctype_utf8mb4.result
mysql-test/t/cast.test
mysql-test/t/ctype_utf8mb4.test
sql/item_func.h
3730 Vasil Dimov 2012-04-24
Implement WL#6175 Introduce persistent statistics ON/OFF switch
A new table_option is added, similarly to e.g. DELAY_KEY_WRITE. Modified files are:
include/my_base.h
sql/handler.h
sql/lex.h
sql/sql_show.cc
sql/sql_table.cc
sql/sql_yacc.yy
The addition of the new option is purely mechanical - copy every occurence of
DELAY_KEY_WRITE and rename it to the new STATS_PERSISTENT option. The .frm file
format is not changed.
In the code before this WL the logic was: every time stats would be updated in
pre-PS MySQL (e.g. MySQL 5.5) do this: try to fetch stats for that table from
persistent stats tables (e.g. check if PS is ON for that table), if fetch
succeeds, then this is it. If fetch fails (due to e.g. no rows for that table in
the stats storage, PS is OFF) then calculate the stats using the old transient
statistics method.
Below by "PS enabled table" or "table has PS set to ON" we mean that either it
been explicitly flagged as STATS_PERSISTENT=1 by CREATE/ALTER TABLE or that its
STATS_PERSISTENT= flag is not set ('default') and the global
innodb_stats_persistent is ON.
With this WL we change the logic to: every time stats would be updated in pre-PS
MySQL do this: check if the table has PS flag set to ON, if yes, then do nothing
(except if the stats are not initialized (e.g. first open table), in which case
fetch them from disk), else calculate the stats using the old transient method.
So we introduce a new per-table flag and store it in the .frm file. Upon open
table we copy that flag into dict_table_t::stat_persistent to ease
checking the value from InnoDB internal code.
We introduce a new state for a table and its indexes' stats: 'unknown' which
means that stats have not been calculated after opening the table and should not
be accessed when in this state. It is also unknown whether the table's
persistent stats flag is ON or OFF. This idea comes from Marko and is based on
the fact that stats are only needed by MySQL and are requested after a call to
::open(). So whenever we open a table internally in InnoDB (table not being
opened by MySQL) we leave the stats in 'unknown' state. When ::open() is called,
then we peek the value of the PS flag from table->s->db_create_options, save it
in the newly introduced table->stat_persistent and do the
corresponding action to initialize the stats -
either fetch them from disk if PS=ON or recalc them if PS=OFF. This is done by
the newly introduced function dict_stats_init(). The new 'unknown' state is
designated by table->stat_initialized being FALSE.
When the PS flag is changed by the user with ALTER TABLE, we need to set the new
value in table->stat_persistent - this is done by the new function
dict_stats_set_persistent() which is also used to set the value of the flag
initially from inside dict_stats_init() (called from ::open() and ::create()).
A new function is introduced: dict_stats_is_persistent_enabled(table) which
checks whether PS is switched ON/OFF for the given table. This function is very
fast because it just checks if the bits are set in
table->stat_persistent (compared to the slow SELECTing from the stats tables
that was done before).
In row_update_statistics_if_needed() if this is PS-enabled table, then do
nothing. This function may be called very often. Also do nothing if the stats
are in 'unknown' state - this may happen when the table has been opened
internally from within InnoDB (e.g. SYS_TABLES or a foreign key table, not
opened by MySQL) and heavy DML is being done on it.
ha_innobase::info() -> ::info_low() indirection is no loner needed and is
removed. With this WL we have just one method - ::info() which checks if the
executed command is ANALYZE and does its logic internally.
Given that stats initialization is now done in ::open() and ::create() after a
call to dict_table_open_on_name() and not in that latter function, this means
that this function becomes the same as dict_table_open_on_name_low() and
dict_table_open_on_name_no_stats() is no longer needed. Thus, remove
dict_table_open_on_name() and dict_table_open_on_name_no_stats() and rename
dict_table_open_on_name_low() to dict_table_open_on_name().
The option innodb_analyze_is_persistent is no longer needed and is removed.
Approved by: Jimmy (rb:912)
removed:
mysql-test/suite/sys_vars/r/innodb_analyze_is_persistent_basic.result
mysql-test/suite/sys_vars/t/innodb_analyze_is_persistent_basic.test
added:
mysql-test/suite/innodb/include/innodb_stats_table_flag.inc
mysql-test/suite/innodb/include/innodb_stats_table_flag_analyze.inc
mysql-test/suite/innodb/r/innodb_stats_create_on_corrupted.result
mysql-test/suite/innodb/r/innodb_stats_create_table.result
mysql-test/suite/innodb/r/innodb_stats_fetch.result
mysql-test/suite/innodb/r/innodb_stats_fetch_corrupted.result
mysql-test/suite/innodb/r/innodb_stats_fetch_nonexistent.result
mysql-test/suite/innodb/r/innodb_stats_flag_global_off.result
mysql-test/suite/innodb/r/innodb_stats_flag_global_on.result
mysql-test/suite/innodb/r/innodb_upd_stats_if_needed_not_inited.result
mysql-test/suite/innodb/r/innodb_ut_format_name.result
mysql-test/suite/innodb/t/innodb_stats_create_on_corrupted.test
mysql-test/suite/innodb/t/innodb_stats_create_table.test
mysql-test/suite/innodb/t/innodb_stats_fetch.test
mysql-test/suite/innodb/t/innodb_stats_fetch_corrupted.test
mysql-test/suite/innodb/t/innodb_stats_fetch_nonexistent.test
mysql-test/suite/innodb/t/innodb_stats_flag_global_off-master.opt
mysql-test/suite/innodb/t/innodb_stats_flag_global_off.test
mysql-test/suite/innodb/t/innodb_stats_flag_global_on-master.opt
mysql-test/suite/innodb/t/innodb_stats_flag_global_on.test
mysql-test/suite/innodb/t/innodb_upd_stats_if_needed_not_inited.test
mysql-test/suite/innodb/t/innodb_ut_format_name.test
mysql-test/suite/sys_vars/r/innodb_stats_persistent_basic.result
mysql-test/suite/sys_vars/t/innodb_stats_persistent_basic.test
storage/innobase/include/dict0stats.ic
modified:
include/my_base.h
mysql-test/suite/innodb/include/innodb_stats.inc
mysql-test/suite/innodb/r/innodb-index-online.result
mysql-test/suite/innodb/r/innodb_bug11933790.result
mysql-test/suite/innodb/r/innodb_bug12429573.result
mysql-test/suite/innodb/r/innodb_stats.result
mysql-test/suite/innodb/r/innodb_stats_drop_locked.result
mysql-test/suite/innodb/t/innodb-blob.test
mysql-test/suite/innodb/t/innodb-index-online.test
mysql-test/suite/innodb/t/innodb_bug11933790.test
mysql-test/suite/innodb/t/innodb_bug12429573.test
mysql-test/suite/innodb/t/innodb_stats.test
mysql-test/suite/innodb/t/innodb_stats_drop_locked.test
mysql-test/suite/perfschema/r/digest_table_full.result
mysql-test/suite/perfschema/r/statement_digest.result
mysql-test/suite/perfschema/r/statement_digest_consumers.result
mysql-test/suite/perfschema/r/statement_digest_long_query.result
scripts/mysql_system_tables.sql
sql/handler.h
sql/lex.h
sql/sql_show.cc
sql/sql_table.cc
sql/sql_yacc.yy
storage/innobase/api/api0api.cc
storage/innobase/btr/btr0cur.cc
storage/innobase/dict/dict0dict.cc
storage/innobase/dict/dict0load.cc
storage/innobase/dict/dict0stats.cc
storage/innobase/fts/fts0fts.cc
storage/innobase/fts/fts0opt.cc
storage/innobase/fts/fts0que.cc
storage/innobase/handler/ha_innodb.cc
storage/innobase/handler/ha_innodb.h
storage/innobase/handler/handler0alter.cc
storage/innobase/handler/i_s.cc
storage/innobase/include/dict0dict.h
storage/innobase/include/dict0dict.ic
storage/innobase/include/dict0mem.h
storage/innobase/include/dict0stats.h
storage/innobase/include/fts0priv.h
storage/innobase/include/srv0srv.h
storage/innobase/include/ut0ut.h
storage/innobase/pars/pars0pars.cc
storage/innobase/row/row0ftsort.cc
storage/innobase/row/row0ins.cc
storage/innobase/row/row0mysql.cc
storage/innobase/row/row0sel.cc
storage/innobase/row/row0upd.cc
storage/innobase/srv/srv0srv.cc
storage/innobase/ut/ut0ut.cc
=== modified file 'mysql-test/r/cast.result'
--- a/mysql-test/r/cast.result 2012-04-13 11:50:32 +0000
+++ b/mysql-test/r/cast.result 2012-04-24 07:35:13 +0000
@@ -469,3 +469,19 @@ Warnings:
Warning 1301 Result of cast_as_char() was larger than max_allowed_packet (2048) - truncated
SET @@GLOBAL.max_allowed_packet=default;
End of 5.1 tests
+#
+# Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+#
+CREATE TABLE t1 AS SELECT CONCAT(CAST(REPEAT('9', 1000) AS SIGNED)),
+CONCAT(CAST(REPEAT('9', 1000) AS UNSIGNED));
+Warnings:
+Warning 1292 Truncated incorrect INTEGER value: '99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999'
+Warning 1292 Truncated incorrect INTEGER value: '99999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999'
+SHOW CREATE TABLE t1;
+Table Create Table
+t1 CREATE TABLE `t1` (
+ `CONCAT(CAST(REPEAT('9', 1000) AS SIGNED))` varchar(21) NOT NULL DEFAULT '',
+ `CONCAT(CAST(REPEAT('9', 1000) AS UNSIGNED))` varchar(21) NOT NULL DEFAULT ''
+) ENGINE=MyISAM DEFAULT CHARSET=latin1
+DROP TABLE t1;
+End of 5.5 tests
=== modified file 'mysql-test/r/ctype_utf8mb4.result'
--- a/mysql-test/r/ctype_utf8mb4.result 2012-03-08 14:22:07 +0000
+++ b/mysql-test/r/ctype_utf8mb4.result 2012-04-24 07:35:13 +0000
@@ -2601,6 +2601,21 @@ t2 CREATE TABLE `t2` (
) ENGINE=MyISAM DEFAULT CHARSET=latin1
DROP TABLE t1, t2;
#
+# Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+#
+CREATE TABLE t1(f1 LONGTEXT CHARACTER SET utf8mb4);
+INSERT INTO t1 VALUES ('a');
+SELECT @a:= CAST(f1 AS SIGNED) FROM t1
+UNION ALL
+SELECT CAST(f1 AS SIGNED) FROM t1;
+@a:= CAST(f1 AS SIGNED)
+0
+0
+Warnings:
+Warning 1292 Truncated incorrect INTEGER value: 'a'
+Warning 1292 Truncated incorrect INTEGER value: 'a'
+DROP TABLE t1;
+#
# End of 5.5 tests
#
#
=== modified file 'mysql-test/t/cast.test'
--- a/mysql-test/t/cast.test 2011-11-01 11:52:24 +0000
+++ b/mysql-test/t/cast.test 2012-04-24 07:35:13 +0000
@@ -296,3 +296,14 @@ disconnect newconn;
SET @@GLOBAL.max_allowed_packet=default;
--echo End of 5.1 tests
+
+--echo #
+--echo # Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+--echo #
+
+CREATE TABLE t1 AS SELECT CONCAT(CAST(REPEAT('9', 1000) AS SIGNED)),
+ CONCAT(CAST(REPEAT('9', 1000) AS UNSIGNED));
+SHOW CREATE TABLE t1;
+DROP TABLE t1;
+
+--echo End of 5.5 tests
=== modified file 'mysql-test/t/ctype_utf8mb4.test'
--- a/mysql-test/t/ctype_utf8mb4.test 2012-03-05 13:18:39 +0000
+++ b/mysql-test/t/ctype_utf8mb4.test 2012-04-24 07:35:13 +0000
@@ -1812,6 +1812,17 @@ SHOW CREATE TABLE t2;
DROP TABLE t1, t2;
--echo #
+--echo # Bug#13581962 HIGH MEMORY USAGE ATTEMPT, THEN CRASH WITH LONGTEXT, UNION, USER VARIABLE
+--echo #
+
+CREATE TABLE t1(f1 LONGTEXT CHARACTER SET utf8mb4);
+INSERT INTO t1 VALUES ('a');
+SELECT @a:= CAST(f1 AS SIGNED) FROM t1
+UNION ALL
+SELECT CAST(f1 AS SIGNED) FROM t1;
+DROP TABLE t1;
+
+--echo #
--echo # End of 5.5 tests
--echo #
=== modified file 'sql/item_func.h'
--- a/sql/item_func.h 2012-04-20 06:48:15 +0000
+++ b/sql/item_func.h 2012-04-24 07:35:13 +0000
@@ -528,12 +528,18 @@ public:
class Item_func_signed :public Item_int_func
{
public:
- Item_func_signed(Item *a) :Item_int_func(a) {}
+ Item_func_signed(Item *a) :Item_int_func(a)
+ {
+ unsigned_flag= 0;
+ }
const char *func_name() const { return "cast_as_signed"; }
longlong val_int();
longlong val_int_from_str(int *error);
void fix_length_and_dec()
- { fix_char_length(args[0]->max_char_length()); unsigned_flag=0; }
+ {
+ fix_char_length(std::min<uint32>(args[0]->max_char_length(),
+ MY_INT64_NUM_DECIMAL_DIGITS));
+ }
virtual void print(String *str, enum_query_type query_type);
uint decimal_precision() const { return args[0]->decimal_precision(); }
};
@@ -542,14 +548,11 @@ public:
class Item_func_unsigned :public Item_func_signed
{
public:
- Item_func_unsigned(Item *a) :Item_func_signed(a) {}
- const char *func_name() const { return "cast_as_unsigned"; }
- void fix_length_and_dec()
+ Item_func_unsigned(Item *a) :Item_func_signed(a)
{
- fix_char_length(std::min<uint32>(args[0]->max_char_length(),
- DECIMAL_MAX_PRECISION + 2));
- unsigned_flag=1;
+ unsigned_flag= 1;
}
+ const char *func_name() const { return "cast_as_unsigned"; }
longlong val_int();
virtual void print(String *str, enum_query_type query_type);
};
No bundle (reason: useless for push emails).
| Thread |
|---|
| • bzr push into mysql-trunk branch (sergey.glukhov:3730 to 3731) Bug#13581962 | Sergey Glukhov | 24 Apr |
