3204 Guilhem Bichot 2010-09-20
a Python script to validate that the traces found in a result file are JSON-compliant
(currently they are not because " and newlines in queries are not escaped)
added:
WL4800_validate_json.py
modified:
WL4800_Doxyfile
WL4800_extract_plans.py
WL4800_select_to_explain_test.py
sql/opt_trace.cc
sql/opt_trace.h
unittest/gunit/opt_trace-t.cc
3203 Guilhem Bichot 2010-09-18
1) implementing the main remaining pieces of WL#5257:
1.1) @@optimizer_trace_offset, @@optimizer_trace_limit.
@@optimizer_trace_offset is a signed variable, so I had to add support for
this in MySQL (so far all integer system variables were unsigned, it was impossible
to set a system variable to a negative value, it would be rounded to 0).
1.2) using stored routines implies that one statement has sub-statements; so far
all of the statement's and sub-statements' traces were concatenated in one
single row of INFORMATION_SCHEMA.OPTIMIZER_TRACE. This prevented
@@optimizer_trace_offset/limit to function. Fixed by making each statement and
sub-statement generate a new row in INFORMATION_SCHEMA.OPTIMIZER_TRACE.
1.3) @@optimizer_trace_max_mem_size. If allocating memory for extending a trace
fails because of optimizer_trace_max_mem_size, report missing bytes in
INFORMATION_SCHEMA.OPTIMIZER_TRACE.MISSING_BYTES_BEYOND_MAX_MEM_SIZE.
If OS' malloc()/realloc() fail, report failure in
INFORMATION_SCHEMA.OPTIMIZER_TRACE.OS_MALLOC_ERROR.
1.4) mtr tests of the new variables
2) test of optimizer_trace vs {query with a stored function, or a stored procedure,
or a trigger}.
3) test of optimizer_trace vs {a view, or an INFORMATION_SCHEMA table}.
4) test that first and second EXECUTE of a prepared statement give the same trace.
@ include/my_getopt.h
utility function
@ include/mysql/plugin.h
comments
@ include/mysql/plugin_audit.h.pp
layout of space changed in .h, so changes in .h.pp
@ include/mysql/plugin_ftparser.h.pp
layout of space changed in .h, so changes in .h.pp
@ mysql-test/include/have_optimizer_trace.inc
optimizer_trace tests must not fail if binary doesn't support it
@ mysql-test/include/optimizer_trace.inc
new tests
@ mysql-test/lib/My/SafeProcess/safe_process.cc
temporary fix (should not be pushed to main trees);
without it, some valgrind "at shutdown" errors (memory leaks)
are missed because kill-9 prevents valgrind from finding/printing them
@ mysql-test/r/bug42620.result
update result due to new columns in I_S.OPTIMIZER_TRACE
@ mysql-test/r/mysqld--help-notwin.result
update result due to new variables
@ mysql-test/r/optimizer_trace_ps_prot.result
result is identical to no_prot, with the exception of 3 subquery
transformations as explained at the start of optimizer_trace.inc
@ mysql-test/suite/sys_vars/t/optimizer_trace_limit_basic.test
basic test of reading/writing the optimizer_trace_limit variable
@ mysql-test/suite/sys_vars/t/optimizer_trace_max_mem_size_basic.test
basic test of reading/writing the optimizer_trace_max_mem_size variable
@ mysql-test/suite/sys_vars/t/optimizer_trace_offset_basic.test
basic test of reading/writing the optimizer_trace_offset variable
@ mysql-test/suite/sys_vars/t/optimizer_trace_offset_max-master.opt
this should cap "SET optimizer-trace-offset" at values between -50 and 50
@ mysql-test/suite/sys_vars/t/optimizer_trace_offset_max.test
test that --maximum-optimizer-trace-offset caps values as intended
@ mysys/my_getopt.c
reducing code size by using a utility function, max_of_int_range()
@ sql/CMakeLists.txt
to-be-unit-tested object files should be linked into sqlgunitlib
@ sql/item_func.cc
support for SHOW_SIGNED_LONG. The "unsigned_flag" is what makes
-1 be displaid as -1 and not a huge positive number in "SELECT @@var"
@ sql/item_subselect.cc
unused variable
@ sql/mysqld.cc
unimportant change (get_tail() is now safe to call even
if trace is not started)
@ sql/opt_range.cc
more disabling of range optimizer's tracing. Fixing compiler warnings.
Removing unneeded todo comments.
@ sql/opt_trace.cc
see comments in opt_trace.h
@ sql/set_var.cc
see explanation in commit comment of sql/sys_vars.h
@ sql/sql_class.h
new variables optimizer_trace_offset, optimizer_trace_limit,
optimizer_trace_max_mem_size.
@ sql/sql_plugin.h
support for SHOWing _signed_ long integer variables (in SHOW VARIABLES,
in SELECT @@thevar): request this by using SHOW_SIGNED_LONG.
@ sql/sql_select.cc
removing unneeded todo comments
@ sql/sql_show.cc
the first change is not needed today (there are no SHOW_SIGNED_LONG variables
in SHOW STATUS) but good for the future.
The second change is what makes -1 be displaid as -1 and not a huge positive number
in "SHOW VARIABLES"
@ sql/sys_vars.cc
new variables optimizer_trace_offset, optimizer_trace_limit,
optimizer_trace_max_mem_size.
@ sql/sys_vars.h
Support for _signed_ integer system variables. This makes do_check()
more complicated, as code cannot be the same as for unsigned.
The third parameter of throw_bounds_warning(), named 'fixed', was
here 'var->save_result.ulonglong_value != uv' which tests whether
there was rounding due to the allowed range defined with VALID_RANGE()
(see sys_vars.cc); that rounding is done by getopt_ull_limit_value().
But there is another 'fixing', which is that a negative value is rounded
to 0; this is done earlier with
if (var->value->unsigned_flag)
...
else
uv= (ulonglong) (v < 0 ? 0 : v);
The passed value of 'fixed' didn't reflect this rounding, so
throw_bounds_warning() had to explicitely detect it with the
part after || :
if (fixed || (!is_unsigned && v < 0))
By changing the parameter to be
var->save_result.ulonglong_value != (ulonglong)v
which detects whether any change happened between input value
(v) and final value (var->save_result.ulonglong_value) we can
just test 'fixed' in throw_bounds_warning().
@ unittest/gunit/CMakeLists.txt
build opt_trace unit test
@ unittest/gunit/Makefile.am
build opt_trace unit test
@ unittest/gunit/opt_trace-t.cc
sql/ unit tests live should be put in unittest/gunit nowadays
added:
mysql-test/include/have_optimizer_trace.inc
mysql-test/suite/sys_vars/r/optimizer_trace_limit_basic.result
mysql-test/suite/sys_vars/r/optimizer_trace_max_mem_size_basic.result
mysql-test/suite/sys_vars/r/optimizer_trace_offset_basic.result
mysql-test/suite/sys_vars/r/optimizer_trace_offset_max.result
mysql-test/suite/sys_vars/t/optimizer_trace_limit_basic.test
mysql-test/suite/sys_vars/t/optimizer_trace_max_mem_size_basic.test
mysql-test/suite/sys_vars/t/optimizer_trace_offset_basic.test
mysql-test/suite/sys_vars/t/optimizer_trace_offset_max-master.opt
mysql-test/suite/sys_vars/t/optimizer_trace_offset_max.test
renamed:
sql/opt_trace-t.cc => unittest/gunit/opt_trace-t.cc
modified:
WL4800_TODO.txt
include/my_getopt.h
include/mysql/plugin.h
include/mysql/plugin_audit.h.pp
include/mysql/plugin_ftparser.h.pp
mysql-test/include/optimizer_trace.inc
mysql-test/lib/My/SafeProcess/safe_process.cc
mysql-test/r/bug42620.result
mysql-test/r/mysqld--help-notwin.result
mysql-test/r/optimizer_trace_no_prot.result
mysql-test/r/optimizer_trace_ps_prot.result
mysql-test/suite/sys_vars/t/optimizer_trace_basic.test
mysql-test/t/bug42620.test
mysql-test/t/optimizer_trace_bugs.test
mysys/my_getopt.c
sql/CMakeLists.txt
sql/item_func.cc
sql/item_subselect.cc
sql/mysqld.cc
sql/opt_range.cc
sql/opt_trace.cc
sql/opt_trace.h
sql/set_var.cc
sql/sql_class.h
sql/sql_plugin.h
sql/sql_select.cc
sql/sql_show.cc
sql/sys_vars.cc
sql/sys_vars.h
unittest/gunit/CMakeLists.txt
unittest/gunit/Makefile.am
unittest/gunit/opt_trace-t.cc
=== modified file 'WL4800_Doxyfile'
--- a/WL4800_Doxyfile 2010-09-05 16:08:07 +0000
+++ b/WL4800_Doxyfile 2010-09-20 08:12:18 +0000
@@ -1,5 +1,20 @@
# Doxyfile 1.6.1
+# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+
# This file describes the settings to be used by the documentation system
# doxygen (www.doxygen.org) for a project
#
=== modified file 'WL4800_extract_plans.py'
--- a/WL4800_extract_plans.py 2010-05-24 18:38:10 +0000
+++ b/WL4800_extract_plans.py 2010-09-20 08:12:18 +0000
@@ -1,5 +1,20 @@
#! /usr/bin/python
+# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+
import sys
usage = """
=== modified file 'WL4800_select_to_explain_test.py'
--- a/WL4800_select_to_explain_test.py 2010-09-04 17:40:51 +0000
+++ b/WL4800_select_to_explain_test.py 2010-09-20 08:12:18 +0000
@@ -1,5 +1,20 @@
#! /usr/bin/python
+# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+
import sys, re
usage = """
=== added file 'WL4800_validate_json.py'
--- a/WL4800_validate_json.py 1970-01-01 00:00:00 +0000
+++ b/WL4800_validate_json.py 2010-09-20 08:12:18 +0000
@@ -0,0 +1,65 @@
+#! /usr/bin/python
+
+# Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; version 2 of the License.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+
+import sys
+
+usage = """
+Usage:
+ %s <a_file>
+
+It will verify that all optimizer traces of a_file (usually a_file
+is a .result or .reject file which contains
+SELECT * FROM OPTIMIZER_TRACE; ) are JSON-compliant
+""" % sys.argv[0]
+
+if len(sys.argv) != 2:
+ print usage
+ sys.exit(1)
+
+import json, re
+
+def check(trace, first_trace_line):
+ s = "".join(trace)
+ try:
+ json.loads(s)
+ except:
+ print "error at line", first_trace_line
+ print sys.exc_info()
+ print s
+ else:
+ print "ok at line", first_trace_line
+ print
+
+all = open(sys.argv[1]).readlines()
+
+first_trace_line = trace_line = 0
+trace = None
+for l in all:
+ trace_line += 1
+ if l == "{\n":
+ assert first_trace_line == 0
+ trace = []
+ first_trace_line = trace_line
+ if l.startswith("}\t") or l.startswith("}\n"): # end of current trace
+ assert first_trace_line != 0
+ trace.append("}") # eliminate any following columns of table (OS_MALLOC_ERROR etc)
+ check(trace, first_trace_line)
+ first_trace_line = 0
+ if first_trace_line != 0:
+ # eliminate /* */ (not valid JSON)
+ no_comment = re.sub("/\*.*\*/", "", l)
+ trace.append(no_comment)
=== modified file 'sql/opt_trace.cc'
--- a/sql/opt_trace.cc 2010-09-18 16:25:43 +0000
+++ b/sql/opt_trace.cc 2010-09-20 08:12:18 +0000
@@ -1,4 +1,4 @@
-/* Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
+/* Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
=== modified file 'sql/opt_trace.h'
--- a/sql/opt_trace.h 2010-09-18 16:25:43 +0000
+++ b/sql/opt_trace.h 2010-09-20 08:12:18 +0000
@@ -1,4 +1,4 @@
-/* Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
+/* Copyright (c) 2009, 2010, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
=== modified file 'unittest/gunit/opt_trace-t.cc'
--- a/unittest/gunit/opt_trace-t.cc 2010-09-18 16:25:43 +0000
+++ b/unittest/gunit/opt_trace-t.cc 2010-09-20 08:12:18 +0000
@@ -1,4 +1,4 @@
-/* Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
+/* Copyright (c) 2010, Oracle and/or its affiliates. All rights reserved.
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
Attachment: [text/bzr-bundle] bzr/guilhem@mysql.com-20100920081218-3r0d16cbtwuup7hp.bundle
| Thread |
|---|
| • bzr push into mysql-next-mr-bugfixing branch (guilhem:3203 to 3204) | Guilhem Bichot | 20 Sep |
