Thanks for the info.
I like the bugs suite idea.
/L
On Mon, Nov 30, 2009 at 02:31:53PM +0300, Serge Kozlov wrote:
> Hi, Lars.
>
> Lars Thalmann пишет:
>> Great work, Serge!
>
> Tanhs :-)
>
>>
>> Some comments:
>>
>> 1. I think we need to document the test cases with what
>> requirement they test. E.g.
>>
>> Tested Requirement: In row-based replication, if the bitfield
>> size on master and slave are different, then a proper error
>> message should be generated on the slave.
>>
>> Tested Requirement: In row-based replication, if the decimal
>> precision is different between master and slave, then error
>> 1535 should be generated.
>>
>> Specifying this also opens up another question: Should the
>> tests be generalized to all types of changes of types or do we
>> already have such tests?
>
> It is just the test case for bug report. We've main test for that in rep
> suite that covers all data types, engines, etc. Tests from 'bugs' just
> check exact steps from bug report.
>
> The advantages of 'bugs' suite are following:
>
> 1. When I file a new bug report I will add new test into 'bugs'.
> This test will fail until it will be fixed. It will be easy 'visible'
> under PB2 or other test tool.
>
> 2. One bug is one test case. We can automate it. E.g. if a test case
> named bugXXXXX failed we can reopen it again.
>
> 3. Who will fix it do not need to create test case if patch is done.
>
> 4. All bugs tests will be a part of regression testing.
>
>
>
>>
>> 2. Is it not better to check error code instead of error message?
>> Do we not have an error code for that error?
>
> The error code is correct but error message has wrong pointing to sizes
> of columns.
>
>
>
>>
>> 3. I think the test should follow the same coding guidelines as
>> the code, i.e. each function should have documentation on what
>> it does.
>
> I sent a draft of 'bugs' suite idea based on some filed bugs in past
> week. If Luis/Alfranio will approve it I'll update files with proper
> documentation. But we post any documentation at end of a perl module as
> perldoc so make sure that you looked there:
> 1. Replication.pm has description for exported methods.
> 2. Bug tests will have description of bug from bug reports.
>
>>
>> Note WL#5151 which will soon be ready. A generalized test for
>> that would be good too.
>>
>> /L
>>
>>
>> On Sun, Nov 29, 2009 at 10:57:03PM +0300, Serge Kozlov wrote:
>>> #At file:///home/ksm/sun/repo/new_bugs/nuts-1/ based on
> revid:luis.soares@stripped
>>>
>>> 364 Serge Kozlov 2009-11-29
>>> 1. Test cases for bugs: bug#48815, bug#48819, bug#48845, bug#48903,
> bug#48908
>>> 2. Added sorting test cases by their file names.
>>> 3. Added test template Replication.pm
>>> added:
>>> lib/My/Nuts/Library/Tests/Replication.pm
>>> suites/bugs/
>>> suites/bugs/bug48815.pm
>>> suites/bugs/bug48819.pm
>>> suites/bugs/bug48845.pm
>>> suites/bugs/bug48903.pm
>>> suites/bugs/bug48908.pm
>>> modified:
>>> bin/Driver.pm
>>>
>>> === modified file 'bin/Driver.pm'
>>> --- a/bin/Driver.pm 2009-11-26 13:15:16 +0000
>>> +++ b/bin/Driver.pm 2009-11-29 19:57:00 +0000
>>> @@ -412,7 +412,7 @@ sub run_driver
>>> }
>>> }
>>> - foreach my $test (@tests)
>>> + foreach my $test (sort @tests)
>>> {
>>> $logger->info ("Clean previous test.\n");
>>> clean_process ();
>>>
>>> === added file 'lib/My/Nuts/Library/Tests/Replication.pm'
>>> --- a/lib/My/Nuts/Library/Tests/Replication.pm 1970-01-01 00:00:00 +0000
>>> +++ b/lib/My/Nuts/Library/Tests/Replication.pm 2009-11-29 19:57:00 +0000
>>> @@ -0,0 +1,213 @@
>>> +package My::Nuts::Library::Tests::Replication;
>>> +use Exporter;
>>> +our @ISA = qw(Exporter My::Nuts::Test);
>>> +our @EXPORT = qw(ok_servers servers get_combination);
>>> +use strict;
>>> +use warnings;
>>> +use My;
>>> +use My::Nuts::Library::Kernel::Manager;
>>> +use My::Nuts::Library::Kernel::Server;
>>> +use My::Nuts::Library::Kernel::Replication;
>>> +use My::Nuts::Library::Kernel::Result;
>>> +use My::Nuts::Library::File;
>>> +use My::Nuts::Properties::BuildProperties;
>>> +use My::Nuts::Test;
>>> +use Class::Std::Utils;
>>> +use Log::Log4perl qw(get_logger :levels); +use Test::More;
>>> +my $logger = get_logger("My::Nuts::Library::Tests::Replication");
>>> +my %server;
>>> +my %counter;
>>> +
>>> +sub get_combination
>>> +{
>>> + my ($self) = @_;
>>> + return $self->get_combination_tag();
>>> +}
>>> +
>>> +sub ok_server
>>> +{
>>> + my ($self, $params) = @_;
>>> + if ( is_defined ($self) == My::FALSE)
>>> + {
>>> + ok ( 0, "Error in the parameters for server" )
>>> + or diag ("Check the parameters (test)");
>>> + return (My::FAILURE);
>>> + }
>>> + my $ret = server($self, $params);
>>> + if (is_server($ret) == My::FALSE || state($ret) != My::RUNNING)
>>> + {
>>> + $logger->fatal("Error allocating server or can not initialize
> plugins. Please, check the logs for more information.");
>>> + die;
>>> + }
>>> + return ( $ret );
>>> +}
>>> +
>>> +sub server
>>> +{
>>> + my ($self, $args) = @_;
>>> + my $ident = ident ($self);
>>> + $counter{$ident} = $counter{$ident} ? ( $counter{$ident} + 1 ) : 1;
>>> + $server{$ident} = get_server("my.server.id" => $counter{$ident});
>>> + $logger->info("deploy server: ($counter{$ident})");
>>> + return (My::UNABLE_TO_DEPLOY) unless ( deploy_server( $server{$ident} )
> == My::SUCCESS );
>>> + $server{$ident}->get_my_properties()->set_my_server_args($args) if
> (defined $args);
>>> + $logger->info("start server: ($counter{$ident})");
>>> + return (My::UNABLE_TO_START) unless (start_server( $server{$ident} ) ==
> My::SUCCESS);
>>> + return ( $server{$ident} );
>>> +}
>>> +
>>> +sub ok_servers
>>> +{
>>> + my @data = @_;
>>> + my @server_list = servers ( @data );
>>> + foreach my $server ( @server_list )
>>> + {
>>> + if ( is_server($server) == My::FALSE || state($server) != My::RUNNING )
>>> + {
>>> + $logger->fatal("Error allocating server. Please, check the logs
> for more information.");
>>> + die;
>>> + }
>>> + }
>>> + #unshift ( @server_list, $data[0] );
>>> + return @server_list;
>>> +}
>>> +
>>> +sub servers
>>> +{
>>> + my $self = shift;
>>> + my @rep_links = @_;
>>> + my %run_servers = ();
>>> + my @out_servers = ();
>>> + my @tags = ();
>>> + my @create_dbs = ();
>>> + my @reps = ();
>>> + foreach my $rep_link ( @rep_links )
>>> + {
>>> + if ( $rep_link =~ m/(.+)\-\>(.+)/i )
>>> + {
>>> + my $master_cfg = $1;
>>> + my $slave_cfg = $2;
>>> + my @server_pair = ();
>>> + foreach my $option ( ($master_cfg, $slave_cfg) )
>>> + {
>>> + if ( $option =~ m/([a-zA-Z0-9]+)/i )
>>> + {
>>> + my $server_name = $1;
>>> + my $arg = undef;
>>> + if ( $option =~ m/\((.+)\)/i )
>>> + {
>>> + $arg = $1;
>>> + }
>>> + if (! defined ( $run_servers{$server_name} ) )
>>> + {
>>> + $run_servers{$server_name} = server ( $self, $arg );
>>> + push ( @out_servers, $run_servers{$server_name} );
>>> + }
>>> + push( @server_pair, $run_servers{$server_name} );
>>> + }
>>> + }
>>> + attach ( $server_pair[0], $server_pair[1] );
>>> + wait_start_replication ( $server_pair[1] );
>>> + push ( @reps, [ $server_pair[0], $server_pair[1] ] );
>>> + }
>>> + else
>>> + {
>>> + @tags = split (", ", $rep_link); + }
>>> + }
>>> + foreach my $tag ( @tags )
>>> + {
>>> + if ($tag =~ m/(.+)\:create\_db\_(.+)/i)
>>> + {
>>> + my $server = $1; + my $dbname = $2; + sql
>>> ( $run_servers{$server}, "DROP DATABASE IF EXISTS $dbname;" );
>>> + sql ( $run_servers{$server}, "CREATE DATABASE $dbname;" );
>>> + foreach my $rep (@reps)
>>> + {
>>> + synchronize ( $rep->[0], $rep->[1] );
>>> + sql ( $rep->[0], "USE $dbname;" );
>>> + sql ( $rep->[1], "USE $dbname;" );
>>> + } + }
>>> + }
>>> + return @out_servers;
>>> +}
>>> +
>>> +sub prepare
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub startup
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub fire
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub shutdown
>>> +{
>>> + return;
>>> +}
>>> +
>>> +1;
>>> +__END__
>>> +
>>> +=over
>>> +
>>> +=back
>>> +
>>> +=head1 My::Nuts::Library::Tests::Replication
>>> +
>>> +=over
>>> +
>>> +=back
>>> +
>>> +=head2 SYNOPSYS
>>> +
>>> +The class creates an environment to develop test cases for replication.
>>> +
>>> +=over
>>> +
>>> +=back
>>> +
>>> +=head2 METHODS
>>> +
>>> +=over +
>>> +=back
>>> +
>>> +=head3 servers ($opt1, $opt2, $opt3 ... $optX)
>>> +
>>> +Description: Returns a list of references to a MySQL Instances.
>>> +
>>> +Each option $optX can have following values:
>>> +
>>> +=over
>>> +
>>> +=item 1. "master_name(mysql command line
>>> arguments)->slave_name(mysql command line arguments)". +Create new
>>> servers named "master_name","slave_name" if they do not exist and
>>> start replication between them.
>>> +
>>> +Sample: "master(--binlog-format=row)->slave_name(--innodb)"
>>> +
>>> +=item 2. "server:created_db_test" where "server", name of a existed
>>> master. +This creates DB named "test" on given server and
>>> synchronizes all slaves pairs.
>>> +
>>> +Sample: "master:created_db_test"
>>> +
>>> +=back
>>> +
>>> +=head3 get_combination ()
>>> +
>>> +Description: Return a combination tag under which the test is running.
>>> +
>>> +=over
>>> +
>>> +=back
>>> +
>>> +=cut
>>>
>>> === added directory 'suites/bugs'
>>> === added file 'suites/bugs/bug48815.pm'
>>> --- a/suites/bugs/bug48815.pm 1970-01-01 00:00:00 +0000
>>> +++ b/suites/bugs/bug48815.pm 2009-11-29 19:57:00 +0000
>>> @@ -0,0 +1,68 @@
>>> +package bugs::bug48815;
>>> +use Exporter;
>>> +our @ISA = qw(Exporter My::Nuts::Library::Tests::Replication);
>>> +use strict;
>>> +use warnings;
>>> +use My;
>>> +use My::Nuts::Library::Kernel::Server;
>>> +use My::Nuts::Library::Kernel::ServerResult;
>>> +use My::Nuts::Library::Kernel::Manager;
>>> +use My::Nuts::Library::Kernel::Result;
>>> +use My::Nuts::Library::Tests::Replication;
>>> +use My::Nuts::Library::Kernel::Replication;
>>> +use Test::More;
>>> +
>>> +sub prepare
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub startup
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub fire
>>> +{
>>> + my ($test) = @_;
>>> + # Setup topology, start server, create test db
>>> + my ($master, $slave, $slave2) = ok_servers($test,
>>> + "master(--binlog-format=row)->slave(--binlog-format=row)",
>>> + "slave(--binlog-format=statement)->slave2(--binlog-format=statement)",
>>> + "master:create_db_test"); + + plan tests => 4;
>>> +
>>> + # Bug code
>>> + ok_sql ( $master, "CREATE TABLE t1 (a INT);");
>>> + ok_sql ( $master, "INSERT INTO t1 VALUES(1);");
>>> + synchronize ( $master, $slave );
>>> + synchronize ( $slave, $slave2 );
>>> + my $query = "SELECT COUNT(*) FROM t1 WHERE a = 1;";
>>> + my $rs = ok_sql( $slave, $query );
>>> + my @rs_data = get_next($rs);
>>> + if (! ok ($rs_data[0] eq "0", "Check result for $query") )
>>> + {
>>> + diag("Expected: 0");
>>> + diag("Got: ".$rs_data[0]);
>>> + }
>>> + + +}
>>> +
>>> +sub shutdown
>>> +{
>>> + return;
>>> +}
>>> +
>>> +1;
>>> +__END__;
>>> +
>>> +=head1 NAME
>>> +
>>> +bugs::bug48815 - Bug#48815
>>> +
>>> +=head1 SYNOPSIS
>>> +
>>> +Test case for Bug#48815
>>> +
>>>
>>> === added file 'suites/bugs/bug48819.pm'
>>> --- a/suites/bugs/bug48819.pm 1970-01-01 00:00:00 +0000
>>> +++ b/suites/bugs/bug48819.pm 2009-11-29 19:57:00 +0000
>>> @@ -0,0 +1,76 @@
>>> +package bugs::bug48819;
>>> +use Exporter;
>>> +our @ISA = qw(Exporter My::Nuts::Library::Tests::Replication);
>>> +use strict;
>>> +use warnings;
>>> +use My;
>>> +use My::Nuts::Library::Kernel::Server;
>>> +use My::Nuts::Library::Kernel::ServerResult;
>>> +use My::Nuts::Library::Kernel::Manager;
>>> +use My::Nuts::Library::Kernel::Result;
>>> +use My::Nuts::Library::Tests::Replication;
>>> +use My::Nuts::Library::Kernel::Replication;
>>> +use Test::More;
>>> +
>>> +sub prepare
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub startup
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub fire
>>> +{
>>> + my ($test) = @_;
>>> + # Setup topology, start server, create test db
>>> + my ($master, $slave) = ok_servers($test,
>>> + "master(--binlog-format=row)->slave(--binlog-format=row)",
>>> + "master:create_db_test"); + + plan tests => 8;
>>> +
>>> + # Bug code
>>> + ok_sql ( $master, "CREATE TABLE t1 (a INT);");
>>> + ok_synchronize ( $master, $slave );
>>> + ok_sql ( $slave, "ALTER TABLE t1 ADD b TIMESTAMP DEFAULT
> CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP;");
>>> + ok_sql ( $master, "INSERT INTO t1 VALUES(0);");
>>> + ok_synchronize ( $master, $slave );
>>> + my $rs = ok_sql( $slave, "SELECT b FROM t1;");
>>> + if (! is_error($rs) )
>>> + {
>>> + my @rs_data = get_next($rs);
>>> + if (! ok( defined ($rs_data[0]), "Check result data") )
>>> + {
>>> + diag("Expected: timestamp");
>>> + diag("Got: undef");
>>> + }
>>> + else
>>> + {
>>> + if (! ok($rs_data[0] ne "0000-00-00 00:00:00", "Check value of column
> a") )
>>> + {
>>> + diag("Expected: current timestamp");
>>> + diag("Got: 0000-00-00 00:00:00");
>>> + }
>>> + }
>>> + }
>>> +}
>>> +
>>> +sub shutdown
>>> +{
>>> + return;
>>> +}
>>> +
>>> +1;
>>> +__END__;
>>> +
>>> +=head1 NAME
>>> +
>>> +bugs::bug48819 - Bug#48819
>>> +
>>> +=head1 SYNOPSIS
>>> +
>>> +Test case for Bug#48819
>>> +
>>>
>>> === added file 'suites/bugs/bug48845.pm'
>>> --- a/suites/bugs/bug48845.pm 1970-01-01 00:00:00 +0000
>>> +++ b/suites/bugs/bug48845.pm 2009-11-29 19:57:00 +0000
>>> @@ -0,0 +1,58 @@
>>> +package bugs::bug48845;
>>> +use Exporter;
>>> +our @ISA = qw(Exporter My::Nuts::Library::Tests::Replication);
>>> +use strict;
>>> +use warnings;
>>> +use My;
>>> +use My::Nuts::Library::Kernel::Server;
>>> +use My::Nuts::Library::Kernel::ServerResult;
>>> +use My::Nuts::Library::Kernel::Manager;
>>> +use My::Nuts::Library::Kernel::Result;
>>> +use My::Nuts::Library::Tests::Replication;
>>> +use My::Nuts::Library::Kernel::Replication;
>>> +use Test::More;
>>> +
>>> +sub prepare
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub startup
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub fire
>>> +{
>>> + my ($test) = @_;
>>> + # Setup topology, start server, create test db
>>> + my ($master, $slave) = ok_servers($test,
>>> + "master(--binlog-format=row)->slave(--binlog-format=row)",
>>> + "master:create_db_test"); + + plan tests => 5;
>>> +
>>> + # Bug code
>>> + ok_sql ( $master, "CREATE TABLE t1 (a CHAR(2));");
>>> + ok_synchronize ( $master, $slave );
>>> + ok_sql ( $slave, "ALTER TABLE t1 CHANGE a a CHAR(1);");
>>> + ok_sql ( $master, "INSERT INTO t1 VALUES('ab');");
>>> + ok_synchronize ( $master, $slave );
>>> +}
>>> +
>>> +sub shutdown
>>> +{
>>> + return;
>>> +}
>>> +
>>> +1;
>>> +__END__;
>>> +
>>> +=head1 NAME
>>> +
>>> +bugs::bug48845 - Bug#48845
>>> +
>>> +=head1 SYNOPSIS
>>> +
>>> +Test case for Bug#48845
>>> +
>>>
>>> === added file 'suites/bugs/bug48903.pm'
>>> --- a/suites/bugs/bug48903.pm 1970-01-01 00:00:00 +0000
>>> +++ b/suites/bugs/bug48903.pm 2009-11-29 19:57:00 +0000
>>> @@ -0,0 +1,65 @@
>>> +package bugs::bug48903;
>>> +use Exporter;
>>> +our @ISA = qw(Exporter My::Nuts::Library::Tests::Replication);
>>> +use strict;
>>> +use warnings;
>>> +use My;
>>> +use My::Nuts::Library::Kernel::Server;
>>> +use My::Nuts::Library::Kernel::ServerResult;
>>> +use My::Nuts::Library::Kernel::Manager;
>>> +use My::Nuts::Library::Kernel::Result;
>>> +use My::Nuts::Library::Tests::Replication;
>>> +use My::Nuts::Library::Kernel::Replication;
>>> +use Test::More;
>>> +
>>> +sub prepare
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub startup
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub fire
>>> +{
>>> + my ($test) = @_;
>>> + # Setup topology, start server, create test db
>>> + my ($master, $slave) = ok_servers($test,
>>> + "master(--binlog-format=row)->slave(--binlog-format=row)",
>>> + "master:create_db_test"); + + plan tests => 5;
>>> +
>>> + # Bug code
>>> + ok_sql ( $master, "CREATE TABLE t1 (a DECIMAL(9,3));");
>>> + ok_synchronize ( $master, $slave );
>>> + ok_sql ( $slave, "ALTER TABLE t1 CHANGE a a DECIMAL(9,4);");
>>> + ok_sql ( $master, "INSERT INTO t1 VALUES (-123456.123),
> (654321.456);");
>>> + synchronize ( $master, $slave );
>>> + my $sql_errno = retrieve_slave_status( $slave, "Last_SQL_Errno" );
>>> + my $sql_state = retrieve_slave_status( $slave, "Slave_SQL_Running" );
>>> + if (! ok ( $sql_state.",".$sql_errno eq "No,1535", "Check state of SQL
> Thread and the error" ) )
>>> + {
>>> + diag ( "Expected: Slave_SQL_Running: No, Last_SQL_Errno: 1535" );
>>> + diag ( "Got: Slave_SQL_Running: $sql_state, Last_SQL_Errno: $sql_errno" );
>>> + }
>>> +}
>>> +
>>> +sub shutdown
>>> +{
>>> + return;
>>> +}
>>> +
>>> +1;
>>> +__END__;
>>> +
>>> +=head1 NAME
>>> +
>>> +bugs::bug48903 - Bug#48903
>>> +
>>> +=head1 SYNOPSIS
>>> +
>>> +Test case for Bug#48903
>>> +
>>>
>>> === added file 'suites/bugs/bug48908.pm'
>>> --- a/suites/bugs/bug48908.pm 1970-01-01 00:00:00 +0000
>>> +++ b/suites/bugs/bug48908.pm 2009-11-29 19:57:00 +0000
>>> @@ -0,0 +1,65 @@
>>> +package bugs::bug48908;
>>> +use Exporter;
>>> +our @ISA = qw(Exporter My::Nuts::Library::Tests::Replication);
>>> +use strict;
>>> +use warnings;
>>> +use My;
>>> +use My::Nuts::Library::Kernel::Server;
>>> +use My::Nuts::Library::Kernel::ServerResult;
>>> +use My::Nuts::Library::Kernel::Manager;
>>> +use My::Nuts::Library::Kernel::Result;
>>> +use My::Nuts::Library::Tests::Replication;
>>> +use My::Nuts::Library::Kernel::Replication;
>>> +use Test::More;
>>> +
>>> +sub prepare
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub startup
>>> +{
>>> + return;
>>> +}
>>> +
>>> +sub fire
>>> +{
>>> + my ($test) = @_;
>>> + # Setup topology, start server, create test db
>>> + my ($master, $slave) = ok_servers($test,
>>> + "master(--binlog-format=row)->slave(--binlog-format=row)",
>>> + "master:create_db_test"); + + plan tests => 6;
>>> +
>>> + # Bug code
>>> + ok_sql ( $master, "CREATE TABLE t1 (a BIT(20));");
>>> + ok_synchronize ( $master, $slave );
>>> + ok_sql ( $slave, "ALTER TABLE t1 CHANGE a a BIT(19);");
>>> + ok_sql ( $master, "INSERT INTO t1 VALUES(0);");
>>> + ok_wait_stop_sql_replication($slave);
>>> + my $error_message = retrieve_slave_status($slave, "Last_Error");
>>> + if (! ok ( $error_message =~ m/.+master has size 20.+slave has
>>> size 19.+/ig, + "Check the error message on the slave") )
>>> + {
>>> + diag("Expected: ... master has size 20 ... slave has size 19 ...");
>>> + diag("Got: $error_message");
>>> + }
>>> +}
>>> +
>>> +sub shutdown
>>> +{
>>> + return;
>>> +}
>>> +
>>> +1;
>>> +__END__;
>>> +
>>> +=head1 NAME
>>> +
>>> +bugs::bug48908 - Bug#48908
>>> +
>>> +=head1 SYNOPSIS
>>> +
>>> +Test case for Bug#48908
>>> +
>>>
>>>
>>> --
>>> MySQL Code Commits Mailing List
>>> For list archives: http://lists.mysql.com/commits
>>> To unsubscribe: http://lists.mysql.com/commits?unsub=1
>>>
>>
>
>
> --
> Serge Kozlov, QA Developer
> MySQL AB, Moscow, Russia, www.mysql.com
> Office:
>
> Are you MySQL certified? www.mysql.com/certification
--
Dr Lars Thalmann, Development Manager, Replication & Backup, www.mysql.com
