Hi Brian,
On Jan 25, 2007, at 4:31 PM, Brian Aker wrote:
> Hi!
>
> On Jan 25, 2007, at 7:09 AM, Paul McCullagh wrote:
>
>> --plugin_dir=/Users/build/Development/mysql-5.1/bk-mysql/storage/
>> example/.libs
>
> That looks like a mistake.
>
>> This is a bit inconvenient for testing other storage engines like
>> PBXT. The 5.1.14 release uses the default, which is "mysql-test/
>> lib/mysql".
>
> How could we better enable the testing of plugins? Scan the storage/
> plugin trees looking for t/ directories? Make mysql-test-run more
> installable?
>
> What are your ideas?
I don't think it is necessary to automatically detect which engines
are to be tested (3rd party 5.1 engines may have there own build
tree, so I don't think they can be found that easily).
I would recommend a simple text file in the mysql-test directory
listing the engines to be tested. 3rd party developers can then just
add their engine to the list (the default would include all engines
configured and built into the server).
I would also add a command line option which overrides the engine
list file, for example:
./mysql-test-run --engines="pbxt"
--engines would also accept a list of engines.
This is convenient because most 3rd party developers are just
interested in testing there own engine. However, if they get an
error, they want to quickly switch to myisam or innodb to check if
the error also occurs with the standard engines. In this way the
developer can decide if it is an engine specific or general MySQL
problem.
Engine specific tests could then be placed in the directories "mysql-
test/t/<engine>" and "mysql-test/r/<engine>" or, even better: "mysql-
test/<engine>/t" and "mysql-test/<engine>/r".
So mysql-test-run would automatically run all tests in the engine
specific directory when instructed to test that engine using the --
engines option (or from the engine list file).
However, I believe that there should be 2 types of tests: generic
tests and engine specific tests. Generic tests should be run for all
engines. So when you run '/mysql-test-run --engines="pbxt"' it must
run the generic tests and the pbxt specific tests.
I currently regard the generic tests as those that do a CREATE TABLE
without an ENGINE= clause. These are the tests that are actually only
run by MyISAM today. But I find the best way to ensure compatibility
is by running these tests on my engine as well.
I do this as follows:
- I configure my engine with --libdir=".../mysql-5.1/mysql-test/lib/
mysql" (so I can just do "make install" after making a change during
testing).
- I add an "INSERT INTO plugin VALUES ('pbxt', 'libpbxt.so');" after
"CREATE TABLE plugin" to "mysql-test/lib/init_db.sql", so that mysqld
loads my plug-in.
- I start mysql-test-run as follows: "./mysql-test-run --mysqld=--
default-storage-engine=pbxt"
- I modify the .test and .result scripts to fit my engine.
So basically I would recommend that mysql-test-run works as follows:
- For each engine in the engine list or --engines parameters list:
* Initialize the environment for that engine:
This would involve adding the "INSERT INTO plugin" for 3rd
party engines.
Check if the engine is present and loaded when mysqld starts.
If the engine cannot be accessed then skip to the next engine
in the list.
* Run the generic tests:
Basically all tests that are not engine specific (see below).
Start mysqld with --default-storage-engine=<engine>
* Run the engine specific tests:
All tests in the mysql-test/<engine>/r/ directory.
In this case I would also start mysqld with --default-storage-
engine=<engine>
Concerning the generic tests:
As mentioned above, all the tests that are currently used for MyISAM
should become generic tests that run with any engine. This involves
quite a few changes.
For example, when SHOW CREATE TABLE is executed the output must be
modified to read "ENGINE=?" instead of "ENGINE=MyISAM", so that any
engine will pass the test.
However, once the initial work has been done, for example, the tests
run with both MyISAM and InnoDB, then 3rd party developers can help
to improve and maintain the tests so that they run with all engines.
Other issues such as: SELECT result order and engine specific
features also need to be considered, but that's basically my thought
on the subject.
Best regards,
Paul
Paul McCullagh, MSc
PrimeBase XT
www.primebase.com/xt
pbxt.blogspot.com
sourceforge.net/projects/pbxt
