Author: paul
Date: 2006-03-09 22:25:11 +0100 (Thu, 09 Mar 2006)
New Revision: 1544
Log:
r3817@kite-hub: paul | 2006-03-09 15:23:32 -0600
Initial import of Wiki material into mysqltest manual.
Some markup added, more to come.
Modified:
trunk/
trunk/mysqltest/mysqltest.xml
Property changes on: trunk
___________________________________________________________________
Name: svk:merge
- b5ec3a16-e900-0410-9ad2-d183a3acac99:/mysqldoc-local/mysqldoc/trunk:8509
bf112a9c-6c03-0410-a055-ad865cd57414:/mysqldoc-local/mysqldoc/trunk:3795
+ b5ec3a16-e900-0410-9ad2-d183a3acac99:/mysqldoc-local/mysqldoc/trunk:8509
bf112a9c-6c03-0410-a055-ad865cd57414:/mysqldoc-local/mysqldoc/trunk:3817
Modified: trunk/mysqltest/mysqltest.xml
===================================================================
--- trunk/mysqltest/mysqltest.xml 2006-03-09 18:20:46 UTC (rev 1543)
+++ trunk/mysqltest/mysqltest.xml 2006-03-09 21:25:11 UTC (rev 1544)
@@ -1530,7 +1530,7 @@
<refsynopsisdiv>
<cmdsynopsis>
- <command>mysql-test-run
[<replaceable>options</replaceable>]
[<replaceable>test_case</replaceable>] ...</command>
+ <command>mysql-test-run
[<replaceable>options</replaceable>]
[<replaceable>test_name</replaceable>] ...</command>
</cmdsynopsis>
</refsynopsisdiv>
@@ -1561,15 +1561,15 @@
</para>
<programlisting>
-shell> <userinput>mysql-test-run
[<replaceable>options</replaceable>]
[<replaceable>test_case</replaceable>] ...</userinput>
+shell> <userinput>mysql-test-run
[<replaceable>options</replaceable>]
[<replaceable>test_name</replaceable>] ...</userinput>
</programlisting>
<para>
- For each <replaceable>test_case</replaceable> argument,
+ For each <replaceable>test_name</replaceable> argument,
<command>mysql-test-run</command> runs [Q: the test found in
the
-
<filename>t/<replaceable>test_case</replaceable>.test</filename>
- file?] With no <replaceable>test_case</replaceable>
+
<filename>t/<replaceable>test_name</replaceable>.test</filename>
+ file?] With no <replaceable>test_name</replaceable>
arguments, <command>mysql-test-run</command> [Q: does what?
Runs all <filename>.test</filename> files in the
<filename>t</filename> subdirectory?]
@@ -5238,4 +5238,1018 @@
</chapter>
+ <chapter id="mysqltest-wiki">
+
+ <title><command>mysqltest</command> Stuff from the
Wiki</title>
+
+ <section id="test-framework">
+
+ <title>The <command>mysqltest</command> Test
Framework</title>
+
+ <para>
+ The MySQL test framework is used primarily to run the MySQL
+ server regression suite. The test cases are for most part SQL
+ commands, mixed by some MySQL Test language constructs to
+ control how the test is to be run and how the result is to be
+ verified.
+ </para>
+
+ <para>
+ The application that run the test suite is named
+ <command>mysql-test-run</command> and is located in the
+ <filename>mysql-test</filename> directory of both source and
+ binary MySQL server distributions.
+ </para>
+
+ <para>
+ The test cases are located in the
+ <filename>mysql-test/t</filename> directory. Each test file is
+ usually refered to as one test, but usually consist of a
+ sequence of related tests. An unexpected failure of a single
+ statement in the test file, will make this test fail.
+ </para>
+
+ <para>
+ The <command>mysql-test-run</command> application will start
+ MySQL servers, restart it if a specific test case needs
+ different start arguments, and present the test result. You may
+ hear referenses to the mysqltest application, this is the
+ application that is called by <command>mysql-test-run</command>
+ to run a single test case, intepret the test language
+ constructs, and send SQL commands to the server.
+ </para>
+
+ <para>
+ There is two ways a test can fail. The test engine check the
+ result codes from executing each SQL command. If the failure is
+ unexpected, the test case will fail.
+ </para>
+
+ <para>
+ The other way it can fail, is by producing the incorrect
+ trace/protocol output. When running the test, it writes out the
+ result from SHOW and other commands to a file. This file is then
+ compared to the expected trace found in the
+ <filename>mysql-test/r</filename> directory. If it is different,
+ the test case will fail and the difference will be presented for
+ evaluation.
+ </para>
+
+ <para>
+ This way of comparing the result trace, puts some restrictions
+ on how a test case can be written. You can't have a result that
+ varies from run to run, like the use the current time. If the
+ data that varies is unimportant for the evaluation there are
+ ways to instruct the test engine to replace those fields in the
+ output with fixed data.
+ </para>
+
+ <para>
+ Because the test cases are written mostly in SQL in a text file,
+ there is no direct support to do test cases that are written in
+ C, Java or other languages. This is not the scope of this test
+ framework. But the test framework support executing your own
+ scripts, initiating with your own data and in a test case
+ execute any program so it may be used for some other testing
+ than just SQL statements.
+ </para>
+
+ </section>
+
+ <section id="mysqltest-tutorial">
+
+ <title><command>mysqltest</command> Tutorial</title>
+
+ <section>
+
+ <title>Writing a test case</title>
+
+ <para>
+ You write test cases in any text editor that use LF (linefeed)
+ as the end-of-line character. File names are in lower case
+ ASCII, with no spaces.
+ </para>
+
+ <para>
+ We are adding support for multiple <quote>suites.</quote>
+ Until then, all test cases are to be located in
+ <filename>mysql-test/t</filename>.
+ </para>
+
+ <para>
+ One test file can be a collection of tests that you think
+ belongs together. But this also means that if one of the tests
+ fail, the whole test case will fail. While this makes it
+ tempting to write each small test into a single file, this
+ will be to inefficient and make the test runs unbearably slow.
+ So make the test files not too big, not too small.
+ </para>
+
+ <para>
+ To make us talk about the same thing, let us define
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ One <quote>test file</quote> is one <quote>test
+ case.</quote>
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ One <quote>test case</quote> might contain a <quote>test
+ sequence</quote> (that is, a number of individual tests
+ that are grouped together in the same test file).
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ Also, a test case (i.e. a test file) is to be self contained.
+ This means you are not allowed to create and/or fill a table
+ in one test case and depend on this in a later test case. If
+ you have some common initiation to be done, you put this into
+ a file in the <filename>mysq-test/include</filename>
+ directory, and in each test case that needs it, you do a
+ <literal>--source include/xxxxx.inc</literal>. We take this by
+ example. Lets say you add a test case called
+ <literal>foo</literal>. You then create a file
+ <filename>mysql-test/t/foo.test</filename>.
+ </para>
+
+ <para>
+ A comment in a test case can be started with the '#'
+ character, or the "--" characters. Note that the first word
+ after the "--" might be a command to the test engine itself,
+ More about that later.
+ </para>
+
+ <para>
+ It is safest to use the '#' character for comments, not to
+ accidently run a mysq-test directive/command. For example
+ writing "-- End or test 43" will give an error message because
+ "end" is something that <command>mysqltest</command> thinks is
+ a directive/command.
+ </para>
+
+ <para>
+ Note that because all our test cases will be published to all
+ Internet, be careful that your comments inside the test cases
+ contain no confidential information.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Naming rules for tables, views etc</title>
+
+ <para>
+ The test cases might be used to run against a production
+ server (we will generally not do that, but our customers
+ might). To reduce the risk that by running the test suite,
+ important tables are altered or destroyed, you are to name
+ your tables, views and other objects like
+ </para>
+
+<programlisting>
+table names : t1, t2, t3.....
+view names : v1, v2, v3.....
+</programlisting>
+
+ <para>
+ Look in the other test cases for examples of this. Of course
+ you can name field names and other objects inside tables as
+ you wish. If you don't have a special reason to, you are to
+ use the default database already created for you 'test'.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Cleaning up from an old run</title>
+
+ <para>
+ For efficiency our test engine will not start with a clean new
+ database for running each test case. So a test case generally
+ starts with a "cleaning up section". Let us assume we are to
+ use two tables in our new test case <literal>foo</literal>.
+ You then begin your test file with
+ </para>
+
+<programlisting>
+#
+# Test of XXXXX
+#
+--disable_warnings
+drop table if exists t1,t2;
+--enable_warnings
+</programlisting>
+
+ <para>
+ The "disable_warnings" command instructs the test engine not
+ to log any warnings until "enable_warnings" or the test case
+ is ended. For some unknown reason MySQL choose to write a
+ warning if the table 't1' or 't2' exists. Surrounding this
+ part with the disable/enable warnings makes the test case
+ output the same if the tables exists or not before the test is
+ started. Now we are free to put in any SQL commands that
+ create, use and drop the tables 't1' and 't2'. The rule is
+ that you are to have a corresponding drop of each table you
+ create in the test.
+ </para>
+
+ <para>
+ So lets put in some SQL code into this test case
+ </para>
+
+<programlisting>
+create table t1 (
+ Period smallint(4) unsigned zerofill default '0000' not null,
+ Varor_period smallint(4) unsigned default '0' not null
+);
+
+insert into t1 values (9410,9412);
+
+select period from t1;
+select * from t1;
+select t1.* from t1;
+
+drop table t1;
+</programlisting>
+
+ </section>
+
+ <section>
+
+ <title>Determining whether a test fails</title>
+
+ <para>
+ Note that the test code we just wrote contains no checks of
+ the result. The test will report a failure for one of two
+ reasons
+ </para>
+
+ <itemizedlist>
+
+ <listitem>
+ <para>
+ A SQL statement failed with an error
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ The result is not what was expected
+ </para>
+ </listitem>
+
+ </itemizedlist>
+
+ <para>
+ So we are to specify the expected result somehow. You do this
+ by running the test with the flag "--record", like
+ </para>
+
+<programlisting>
+% cd mysql-test
+% ./mysql-test-run --record foo
+</programlisting>
+
+ <para>
+ [Q: Is the <quote>protocol</quote> file the result file or the
+ reject file? I have seen the term applied to both.]
+ </para>
+
+ <para>
+ This will create a file
+ <filename>mysql-test/r/foo.result</filename>, also called the
+ <quote>protocol file,</quote> with the content
+ </para>
+
+<programlisting>
+drop table if exists t1,t2;
+create table t1 (
+Period smallint(4) unsigned zerofill default '0000' not null,
+Varor_period smallint(4) unsigned default '0' not null
+);
+insert into t1 values (9410,9412);
+select period from t1;
+period
+9410
+select * from t1;
+Period Varor_period
+9410 9412
+select t1.* from t1;
+Period Varor_period
+9410 9412
+drop table t1;
+</programlisting>
+
+ <para>
+ If we look at this file, it contains the commands in the
+ <filename>foo.test</filename> file together with the output
+ from the select commands. The output is the table headings and
+ fields separated by the TAB character. So now you are to
+ inspect this file, decide if the result is the expected one,
+ and let it be part of your test case.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Running test cases</title>
+
+ <para>
+ You will normally either run the test case from within a
+ source tree, or on an installed server. In source tree you
+ will be located in the <filename>mysql-test</filename>
+ directory. The installed server also have a
+ <filename>mysql-test</filename> directory. The program running
+ the test suite, <command>mysql-test-run</command>, will figure
+ out if you are in a source tree, or in an installed directory
+ tree.
+ </para>
+
+ <para>
+ You will not run against a started server. Instead the
+ <command>mysql-test-run</command> program will start the
+ server or servers needed on ports that doesn't conflicts with
+ a production server, if you happen to have one on the same
+ machine.
+ </para>
+
+ <para>
+ You can run our newly created test case <literal>foo</literal>
+ as part of the whole suite, or run it by itself like
+ </para>
+
+<programlisting>
+% ./mysql-test-run foo
+</programlisting>
+
+ <para>
+ If you have a family of test cases, prefixed with something
+ common, you can use
+ </para>
+
+<programlisting>
+% ./mysql-test-run --do-tests=mytest
+</programlisting>
+
+ </section>
+
+ <section>
+
+ <title>Mutilple users on the same machine, or shared
directories</title>
+
+ <para>
+ If you are multiple users that run test on the same machine,
+ you need to specify the ports to the
+ <command>mysql-test-run</command> program, so that your run
+ doesn't conflict with others. You add unique port arguments to
+ <command>mysql-test-run</command>, such as
+ <option>--no-manager --master_port=3911
+ --slave_port=3927</option>.
+ </para>
+
+ <para>
+ Also note that only one person can run the
+ <command>mysql-test-run</command> program in the same
+ <filename>mysql-test</filename> directory on a shared drive,
+ at the same time. The <filename>mysql-test/var</filename>
+ directory created can't be shared between mutiple users.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>How to get more information to check the result
against</title>
+
+ <para>
+ The test engine by default only output things from "select",
+ "show" and other SQL commands you expect to produce output,
+ i.e. commands that create a data set. But it can be instructed
+ to be more or less verbose, including other data as well.
+ </para>
+
+ <para>
+ Lets say we are a bit more interested how many rows are
+ effected by an operation, we add "--enable_info" to the test
+ case before we create the table. Now, after we rerun the test
+ with "--record", we get some more information in our result
+ file
+ </para>
+
+<programlisting>
+drop table if exists t1,t2;
+create table t1 (
+Period smallint(4) unsigned zerofill default '0000' not null,
+Varor_period smallint(4) unsigned default '0' not null
+);
+affected rows: 0
+insert into t1 values (9410,9412);
+affected rows: 1
+select period from t1;
+period
+9410
+affected rows: 1
+select * from t1;
+Period Varor_period
+9410 9412
+affected rows: 1
+select t1.* from t1;
+Period Varor_period
+9410 9412
+affected rows: 1
+drop table t1;
+</programlisting>
+
+ <para>
+ Options can in general be enabled and disabled for different
+ parts of the test file. Lets say we are interested in the
+ internals of the database as well. We can then enable the
+ output of "meta data" using "--enable_metadata". The output
+ now begins to be a bit verbose, but as said earlier, this can
+ be enabled and disabled just for a part of the test where it
+ interests you to know more.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>How to get less information</title>
+
+ <para>
+ You might be initializing a table, where we don't really
+ expect a failure. You can then temporarily disable the
+ recording of the command lines executed with
+ "--disable_query_log" and then enable the recording again with
+ "--enable_query_log". You can disable the recording of the
+ output from executing commands using "--disable_result_log"
+ and enable again with "--enable_result_log".
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>What if the output varies?</title>
+
+ <para>
+ In general you are to avoid having tests that give different
+ output for each run, or for differences in how the mysql
+ binaries are created, operating system etc. The first example
+ that comes to mind is the use of the current date and time.
+ The test engine have no way of verifying that it is correct.
+ </para>
+
+ <para>
+ Now, you might have a test that have a column that will vary,
+ but you are not really interested in what is in that column
+ anyway. The test engine let you specify that you want to
+ replace whatever is in that column with a string. This makes
+ the output from our run the same each time.
+ </para>
+
+ <para>
+ Just to illustrate the functionality, we add after our first
+ insert, the row
+ </para>
+
+<programlisting>
+insert into t1 values (date_format(now(), '%s'),9999);
+</programlisting>
+
+ <para>
+ We now record and run the test case
+ </para>
+
+<programlisting>
+% ./mysql-test-run --record foo
+% ./mysql-test-run foo
+</programlisting>
+
+ <para>
+ Now we will, most likely, get a failure and the test engine
+ will display the difference between what we got and the
+ expected result, like
+ </para>
+
+<programlisting>
+Below are the diffs between actual and expected results:
+-------------------------------------------------------
+*** r/foo.result Thu Jan 20 18:38:37 2005
+--- r/foo.reject Thu Jan 20 18:39:00 2005
+***************
+*** 11,25 ****
+ select period from t1;
+ period
+ 9410
+! 0037
+ affected rows: 2
+ select * from t1;
+ Period Varor_period
+ 9410 9412
+! 0037 9999
+ affected rows: 2
+ select t1.* from t1;
+ Period Varor_period
+ 9410 9412
+! 0037 9999
+ affected rows: 2
+--- 11,25 ----
+ select period from t1;
+ period
+ 9410
+! 0059
+ affected rows: 2
+ select * from t1;
+ Period Varor_period
+ 9410 9412
+! 0059 9999
+ affected rows: 2
+ select t1.* from t1;
+ Period Varor_period
+ 9410 9412
+! 0059 9999
+ affected rows: 2
+-------------------------------------------------------
+</programlisting>
+
+ <para>
+ If we are not really interested in the first column, one way
+ to solve this by using the directive/command
+ "--replace_column". The duration of the effect of this
+ directive/command, is the next SQL statement, so we need one
+ before each select call, like
+ </para>
+
+<programlisting>
+--replace_column 1 SECONDS
+select period from t1;
+--replace_column 1 SECONDS
+select * from t1;
+--replace_column 1 SECONDS
+select t1.* from t1;
+</programlisting>
+
+ <para>
+ Note that "SECONDS" could be any string, it is just to display
+ something. If we now record again and run the test, we will
+ succeed each time. The result file now looks like
+ </para>
+
+<programlisting>
+drop table if exists t1,t2;
+create table t1 (
+Period smallint(4) unsigned zerofill default '0000' not null,
+Varor_period smallint(4) unsigned default '0' not null
+);
+affected rows: 0
+insert into t1 values (9410,9412);
+affected rows: 1
+insert into t1 values (date_format(now(), '%s'),9999);
+affected rows: 1
+select period from t1;
+period
+SECONDS
+SECONDS
+affected rows: 2
+select * from t1;
+Period Varor_period
+SECONDS 9412
+SECONDS 9999
+affected rows: 2
+select t1.* from t1;
+Period Varor_period
+SECONDS 9412
+SECONDS 9999
+affected rows: 2
+</programlisting>
+
+ <para>
+ If the result may vary, but just between some know values, you
+ can use "--replace_result" to match the different result, and
+ replace them with a common string, making the result file the
+ same even if the result varies some.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Expected failures</title>
+
+ <para>
+ A good test suite not only test when things go right, but also
+ that we get the correct error message, when it goes wrong. The
+ test engine let you specify "expected failures". Lets say we
+ try to create 't1' again without dropping it in between, like
+ </para>
+
+<programlisting>
+--disable_warnings
+drop table if exists t1,t2;
+--enable_warnings
+
+create table t1 (
+ Period smallint(4) unsigned zerofill default '0000' not null,
+ Varor_period smallint(4) unsigned default '0' not null
+);
+
+insert into t1 values (9410,9412);
+
+select period from t1;
+select * from t1;
+select t1.* from t1;
+
+create table t1 (something smallint(4));
+</programlisting>
+
+ <para>
+ This gives a failure and an error
+ </para>
+
+<programlisting>
+At line 19: query 'create table t1 (something smallint(4))'
+ failed: 1050: Table 't1' already exists
+</programlisting>
+
+ <para>
+ We put in a "--error 1050" just before the second "create
+ table", to mark that this is an expected failure, and that we
+ expected it to return the MySQL error code 1050. If we look at
+ the end of the result file we now have
+ </para>
+
+<programlisting>
+create table t1 (something smallint(4));
+ERROR 42S01: Table 't1' already exists
+</programlisting>
+
+ <para>
+ So we still can match on the expected error, but it will not
+ terminate the test case run because of that error. The "42S01"
+ is the SQLSTATE code. Alternatively you might match on that
+ error code, by using the 'S' prefix, like "--error S42S01".
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Setting server options</title>
+
+ <para>
+ Many server options can be set from within the test case, like
+ </para>
+
+<programlisting>
+set sql_warnings=1;
+set sql_mode=no_auto_value_on_zero;
+</programlisting>
+
+ <para>
+ But sometimes you need to restart the server to use some
+ options in the form of extra command line arguments. The test
+ engine will restart the server just before running a test
+ case, if there exists a file
+
<filename>mysql-test/t/<replaceable>test_name</replaceable>-master.opt</filename>
+ with the options you want to use. After the test case is
+ completed, the server will be restarted again, with the
+ default arguments.
+ </para>
+
+ </section>
+
+ </section>
+
+ <section id="mysqltest-test-cases">
+
+ <title>More on Writing Test Cases</title>
+
+ <section>
+
+ <title>Running Test Case Overview</title>
+
+ <para>
+ This is a brief description of how to write test cases and the
+ language used to write them. Normally you run the test suite,
+ when developing.
+ </para>
+
+ <para>
+ For example:
+ </para>
+
+<programlisting>
+% cd mysql-test
+% mysql-test-run
+</programlisting>
+
+ <para>
+ Or:
+ </para>
+
+<programlisting>
+% mysql-test-run --local --force
+</programlisting>
+
+ <para>
+ Use --force if you want mysql-test to continue on failure of a
+ test case
+ </para>
+
+ <para>
+ For a single test case or group or test cases(the .test
+ extension exluded)
+ </para>
+
+<programlisting>
+% mysql-test-run --local
--do-test=<replaceable>test_name_prefix</replaceable>
+</programlisting>
+
+ <para>
+ This can be a single test case or a many tests with the same
+ prefix.
+ </para>
+
+ <para>
+ Examples:
+ </para>
+
+<programlisting>
+% mysql-test-run --local --do-test=rpl_abcd
+</programlisting>
+
+ <para>
+ This will run the rpl_abcd.test
+ </para>
+
+<programlisting>
+% mysql-test-run --local --do-test=rpl
+</programlisting>
+
+ <para>
+ This will run all the tests that start with rpl.
+ </para>
+
+ <para>
+ Please view the README file in the mysql-test directory for
+ more details about how to invoke the test script
+ <command>mysql-test-run</command>. The
+ <command>mysql-test-run</command> is setting up the
+ environment for calling the program mysqltest and start a
+ server etc. mysqltest is the program that will read the test
+ case language text and start managers, create connections,
+ send queries to a server etc.
+ </para>
+
+ <para>
+ The test case language is a mix of commands that the
+ <command>mysqltest</command> program understand and SQL. What
+ it doesn't understand is assumed to be SQL queries and are
+ sent to the database server. This makes the test case language
+ familiar to those that know how to write SQL and powerful
+ enough to add the control needed to write test cases.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Writing Test Cases Basics</title>
+
+ <para>
+ The base principle is that the test script compares the output
+ from running the tests with the expected result. This is just
+ a diff between the output and a file the test writer provide.
+ With this simplistic way of comparing you can't handle
+ variations in the output from running the script but the test
+ language provide ways to suppress unwanted output making this
+ usable. The steps to write a new test case are
+ </para>
+
+ <orderedlist>
+
+ <listitem>
+ <para>
+ Change directories to the test directory
+
<filename>mysql-<replaceable>version</replaceable>/mysql-test</filename>
+ </para>
+
+<programlisting>
+% cd mysql-<replaceable>version</replaceable>/mysql-test
+</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>
+ Create the test in a file
+ "t/<replaceable>test_name</replaceable>.test"
+ </para>
+ </listitem>
+
+ <listitem>
+ <para>
+ Create an empty result file
+ </para>
+
+<programlisting>
+% touch r/<replaceable>test_name</replaceable>.result
+</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>
+ Run the test
+ </para>
+
+<programlisting>
+% ./mysql-test-run <replaceable>test_name</replaceable>
+</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>
+ The failing test will create a file
+ "r/<replaceable>test_name</replaceable>.reject". Examine
+ this file and if it looks ok you copy its content to the
+ result file
+ </para>
+
+<programlisting>
+% cp r/<replaceable>test_name</replaceable>.reject
r/<replaceable>test_name</replaceable>.result
+</programlisting>
+ </listitem>
+
+ <listitem>
+ <para>
+ Run the test again, this time it should succeed
+ </para>
+
+<programlisting>
+% ./mysql-test-run <replaceable>test_name</replaceable>
+</programlisting>
+ </listitem>
+
+ </orderedlist>
+
+ <para>
+ The environment can also be set up so the
+ <command>mysqltest</command> program can be called directly.
+ For more information about options to the
+ <command>mysqltest</command> program run
+ </para>
+
+<programlisting>
+% mysqltest --help
+</programlisting>
+
+ <para>
+ There are also some notes about <command>mysqltest</command>
+ in the README in this directory.
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Sample Test Case</title>
+
+ <para>
+ Small sample test case
+ </para>
+
+<programlisting>
+--disable_warnings
+DROP TABLE IF EXISTS t1;
+--enable_warnings
+SET SQL_WARNINGS=1;
+
+CREATE TABLE t1 (a INT);
+INSERT INTO t1 VALUES (1);
+INSERT INTO t1 VALUES ("hej");
+</programlisting>
+
+ <para>
+ The first rows try to cleanup from possibly earlier runs of
+ the script by dropping the table "t1". We disable storing
+ warnings (--disable_warnings) in the output during this
+ because we are not interested to know if the table "t1" was
+ there or not. Then we enable storing warnings again
+ (--enable_warnings) and enable verbose warnings in MySQL using
+ the SET SQL_WARNINGS=1;" line.
+ </para>
+
+ <para>
+ We create the table "t1" and try some operations. The creation
+ of the table and first insertion will not create any warnings
+ but the last line will. We collect the output from running
+ this test and add this to the result file
+ </para>
+
+<programlisting>
+DROP TABLE IF EXISTS t1;
+SET SQL_WARNINGS=1;
+CREATE TABLE t1 (a INT);
+INSERT INTO t1 VALUES (1);
+INSERT INTO t1 VALUES ("hej");
+Warnings:
+Warning 1265 Data truncated for column 'a' at row 1
+</programlisting>
+
+ <para>
+ Note that we see the commands as well in the result. This can
+ be disabled with the "--disable_query_log" test language
+ command. There are several options how to control the amount
+ of output from running the tests. If there was an failure this
+ will be reported to the screen. You can see the actual output
+ from the last unsuccessful run of the testcase in the test
+ case reject file
+ "r/<replaceable>test_name</replaceable>.reject".
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Test Environment Overview</title>
+
+ <para>
+ There are two ways the same test environment should handle.
+ One is to talk to an installed running server. The second case
+ is to be able to start a server from the binaries in the
+ source tree, i.e. an uninstalled MySQL server. Because of this
+ in general you don't start a server and create a connection
+ from the test case, this is done by the
+ <command>mysql-test-run</command> script before your test case
+ is run.
+ </para>
+
+ <para>
+ FIXME add the most important options to use calling
+ <command>mysql-test-run</command>
+ </para>
+
+ </section>
+
+ <section>
+
+ <title>Language Overview</title>
+
+ <para>
+ FIXME make descriptions shorter by introducing a syntax to
+ inform that an argument could be a test language variable. Why
+ not all?
+ </para>
+
+ <para>
+ FIXME some environment variables are passed from the
+ environment to test language variables, list them. Why not
+ all??
+ </para>
+
+ <para>
+ The following is a description of the language used in
+ .test-files.
+ </para>
+
+ <para>
+ The <command>mysqltest</command> program is a small
+ interpreter that interpret a special test language. Lines that
+ it doesn't recognize are assumed to be SQL queries that can be
+ sent to the MySQL server. In effect this makes the test files
+ look like SQL with some small extensions.
+ </para>
+
+ <para>
+ Lines starting with a "#" in the test files are comments.
+ </para>
+
+ <para>
+ Commands to the test engine itself are often put into
+ comments, like --cmd. This sometimes leads to conflicts with
+ comments that start with the same word as a command.
+ </para>
+
+ <para>
+ FIXME what is @filename ?!
+ </para>
+
+ </section>
+
+ </section>
+
+ </chapter>
+
</book>
| Thread |
|---|
| • svn commit - mysqldoc@docsrva: r1544 - in trunk: . mysqltest | paul | 9 Mar |