Hi Paul,
Your post certainly hits all the present hotspots with mysql-test. I've
been looking into this and here are my observations to date that may
assist you and others.
MySQL Tests does provide a level of management for the new storage
engines. The present approach that some tests are now employing is a
level of refactoring. Here are some existing examples using this
approach. In summary:
* Refactor the actual test to an include file
* Build separate test wrappers and results per storage engine
* Use have_* handling
Two tests to look at in the 5.1 code are that use this are mix2_*.test
and the ps_*.test.
At the recent MySQL Camp the comment was made we could see 50 to 100
storage engines so I can easily see that this approach, while practical
in the early stages does not scale well without manual work. In
addition, the amount of specific tests within say for example
include/mix2.inc does limit certain functionality that may not be
implemented in a particular engine. Shorter tests, and more specific
tests would improve the portability for future engines, and certainly
help particularly when there is an early failure within a long test.
Another thing I've found while trolling the mysql_test_run.pl code is
the support for test suites. There is also some reference in the docs
for syntax.
You could create the following as a demo:
cd $MYSQL_HOME/mysql-test
mkdir -p suite/pbxt/{r,t}
cp r/alias.test suite/pbxt/r
cp r/alias.result suite/pbxt/r
./mysql-test-run.pl --suite=pbxt
Effectively you can have an entire t/ and r/ directory structure under a
suite/[name] subdirectory. This doesn't help in the refactoring process
but does provide an area for extending tests specific for your storage
engine. I've found an issue here with the use of includes, they don't
work if you copy the include directory, you need to repath accordingly.
When it comes to things like FULLTEXT and GEOMETRY I've found the
quickest solution is to add these tests to disabled.def initially
tackling all other test first.
These points really only look at creative ways to help with present
issues, this is certainly a topic that could benefit from a revised
analysis period.
Good Luck.
Regards
Ronald
Paul McCullagh wrote:
> Hi All,
>
> I am just writing a blog on the subject of testing 3rd party engines
> (http://pbxt.blogspot.com/), and was wondering if this subject has
> come up at all before.
>
> When using mysql-test-run to test a different engine, for example:
>
> $ ./mysql-test-run --mysqld=--default-storage-engine=pbxt
>
> you get a lot of failures which are not actually bugs.
>
> For example wherever SHOW CREATE TABLE is used in a test, the result
> verifies that ENGINE=MyISAM.
>
> In my blog I mentioned this and other things that would make testing
> 3rd party engines easier by eliminating unnecessary differences.
>
> If you have a moment, please check it out: http://pbxt.blogspot.com/
>
> Best regards,
>
> Paul
>
>
> --MySQL Internals Mailing List
> For list archives: http://lists.mysql.com/internals
> To unsubscribe:
> http://lists.mysql.com/internals?unsub=1
>
--
Ronald Bradford, Senior Consultant
MySQL Inc, www.mysql.com
1.551.689.2361 (cell)
Are you MySQL certified? www.mysql.com/certification
