Re: [Fwd: bk commit into 5.1 tree (hezx:1.2675)] versional test

[Magnus Svensson]

He Zhenxing skrev:
> Hi Magnus
> 
> Thank you! see comments below.
> 
> 
> On 2008-02-19 Tue 17:15 +0100，Magnus Svensson wrote：
>> He Zhenxing wrote:
>>> Hi Magnus
>>>
>>> Thank you for your nice review! See below.
>>>
>>>
>>> The following is actually what I want, but it doesn't work.
>>>
>>> if (!$have_${_engine_type})
>>> {
>>>   skip requires '$_engine_type' storage engine;
>>> }
>>>
>>> I would be very glad to rewrite this, any suggestion would be
>>> appreciated.
>>>
>> Ok, I checked this today and as you write, there is really no way of 
>> declaring variables with dynamic variable names.
>>
>> My suggestion would be to declare all the variables you need to 
>> initalize in supported_engines.inc, something like:
>> let $have_innodb= `select (support = 'YES' or support = 'DEFAULT' or 
>> support = 'ENABLED') as `TRUE` from information_schema.engines where 
>> engine = 'innodb'`;
>> let $have_federated= `select (support = 'YES' or support = 'DEFAULT' or 
>> support = 'ENABLED') as `TRUE` from information_schema.engines where 
>> engine = 'federated'`;
>> ... and so on
>>
> 
> The information_schema.engines table does not exists for some old
> versions, such as 5.0-maint and 5.1-wl2325-5.0-drop6 used in current
> versional test do not support engines table.
> 

It would be very unfortunate if our tests suite would be limited to the 
feature of the oldest possible MySQL version we can think of. In that 
case we need to find another way around this to make sure that our test 
suite can continue to evolve.

First of all I think we must ask ourself what it is we want to test.

1. For replication we have identified that our latest version should be 
able to be a _slave_ to an old version. That makes it possible to 
upgrade first the slave and then the master. A number of issues with 
that was fixed in the latest version of MySQL Server after Serge setup 
this testing, remember that we can't "go back in time" and fix already 
released versions, we can only make the current version work with 
whatever is out there.

2. What other things do we want check when we say "versional testing"? I 
have not seen any list or thought so much about it. But I assume it can 
be things like:
- new clients against old servers
- old client against new server
- old dumps with new server
  - when the new server starts up it should be able to read the system 
and data files of the old server with data intact.
But I assume that we are now focusing on versional testing for replication?

You mention "5.0-maint and 5.1-wl2325-5.0-drop6 used in current
versional test" - what does that mean? Unless those are released 
versions and used by any customers that want to upgrade, I see no reason 
to test with those. (It does not always have to be a direct upgrade 
path, i.e to go from 5.0.36 to 5.1.21 you might need to go via 5.0.52). 
Of course there are really old version does not have 
information_schema.engines, but what does 5.0-maint and 
5.1-wl2325-5.0-drop6 have to do with that?


Now, how do we make this work in a way that makes it possible to extend 
the test suite in the future while still being backward compatible? I 
don't want to just complain. ;) Let's throw ideas out there how to make 
this work.

1. How to determine if an engine is not supported. You want to set up 
$have_<engine> variable to make it possible to "if($have<engine>)". I 
don't see that actually being used anywhere in the patch and thus I 
would say that we should forget about that one!?

Instead I suggest we rewrite test cases to check if a particular feature 
is supported, or in rare cases do something like "if (`SELECT 
$mysql_version < 50040`)". This has the advantage that we can continue 
to evolve the test suite and when we add a new feature to the server we 
don't run that test on an old version.

2. Problem 2
Let's discuss pros and cons.


Best regards
Magnus