Hi Jorgen,
As I understand you are working on the final patch. I include below some
comments on the work you done so far.
Jorgen Loland wrote:
> #At file:///localhome/jl208045/mysql/mysql-6.0-backup-32465/
>
> 2727 Jorgen Loland 2008-11-10
> Bug#32465 - Backup: Commit blocker does not block tables in default driver
>
> The patch contains commit blocker tests written for the temporary commit
> blocker solution of setting Global Read Lock before backup prepare phase (fixed in
> bug#39602).
> removed:
> mysql-test/suite/backup/r/backup_commit_blocker.result
> mysql-test/suite/backup/t/backup_commit_blocker.test
> added:
> mysql-test/suite/backup/r/backup_vp_nontx.result
> mysql-test/suite/backup/r/backup_vp_tx.result
> mysql-test/suite/backup/t/backup_vp_nontx.test
> mysql-test/suite/backup/t/backup_vp_tx.test
>
> per-file messages:
> mysql-test/suite/backup/r/backup_commit_blocker.result
> Test replaced with backup_vp_*
> mysql-test/suite/backup/r/backup_vp_nontx.result
> Backup validity point test for non-transactional engine
> mysql-test/suite/backup/r/backup_vp_tx.result
> Backup validity point test for transactional engine
> mysql-test/suite/backup/t/backup_commit_blocker.test
> Test replaced with backup_vp_*
> mysql-test/suite/backup/t/backup_vp_nontx.test
> Backup validity point test for non-transactional engine
> mysql-test/suite/backup/t/backup_vp_tx.test
> Backup validity point test for transactional engine
> === added file 'mysql-test/suite/backup/t/backup_vp_nontx.test'
> --- a/mysql-test/suite/backup/t/backup_vp_nontx.test 1970-01-01 00:00:00 +0000
> +++ b/mysql-test/suite/backup/t/backup_vp_nontx.test 2008-11-10 13:19:08 +0000
> @@ -0,0 +1,221 @@
> +#
> +# This test is one of three validity point tests. See:
> +# backup_vp_nontx.test
> +# backup_vp_mixed.test
The backup_vp_mixed test is not there, so patch looks incomplete or the comment
is missleading.
> +#
> +# The goal of the test should be to ensure the following assumptions
> +# for the behaviour of validy point (VP) hold. Validity point is
> +# implemented using commit blocker (CB).
> +#
> +# a) transactions that commit before VP are in the backup image
> +# b) transactions with ongoing commits block backup from setting
> +# commit blocker. The commit has to complete before backup can set
> +# CB (and ultimately set the VP), and will therefore be in the backup
> +# image
> +# c) transactions that try to commit when backup is about to set VP
> +# are blocked by CB. The commit has to wait for CB to be released
> +# before it can continue, and will therefore not be in the backup
> +# image
> +# d) transactions that commit after VP are not in the backup image
> +#
As said elsewhere, I don't like testing VP properties by looking at the
implementation (CB in this case). I'd reformulate properties b-c) above as the
following property of VP:
b-c) The validity point should not fall in the middle of a half-executed COMMIT
statement.
Formulating it that way would also suggest slightly different testing strategy,
but that is another story...
> +# Note: the transactions have to modify data.
> +#
> +#
> +#
> +
> +--source include/have_debug_sync.inc
> +--source include/not_embedded.inc
> +
> +--disable_warnings
> +SET DEBUG_SYNC= 'RESET';
> +DROP DATABASE IF EXISTS bup_vp_tx;
> +--error 0,1
> +--remove_file $MYSQLTEST_VARDIR/master-data/bup_commit_blocker.bak;
I suggest using @backupdir value here. Can be done like this:
let $bdir=`SELECT @@backupdir`;
--remove_file $bdir/bup_commit_blocker.bak;
> +--enable_warnings
> +
> +CREATE DATABASE bup_commit_blocker;
> +
> +#
> +# Connections used in this test
> +#
> +# con_bup - used to create data, load data, and run the backup
> +# con_ntx1 - used for non-transactional execution
> +# con_ntx2 - used for non-transactional execution
> +#
> +
> +connect (con_bup,localhost,root,,);
> +connect (con_ntx1,localhost,root,,);
> +connect (con_ntx2,localhost,root,,);
> +
> +connection con_bup;
> +
> +--echo
> +--echo Starting Test
> +--echo
> +
> +#
> +# Sequence diagram (not UML), only relevant parts shown
> +#
> +# bup tx1 tx2
> +# | | |
> +# a) | |
> +# (setup) | |
> +# | b) |
> +# | INSERT |
> +# BACKUP <...> |
> +# SET CB <...> |
> +# <###> <...> |
> +# <...> | c)
> +# | | INSERT
> +# SET VP | <###>
> +# RELEASE CB | <###>
> +# | d) <...>
> +# | INSERT |
> +# BUP DONE | |
> +# (results) | |
> +#
> +# Note: Ongoing operations are indicated with <...>
> +# Blocked operations are indicated with <###>
> +#
> +
Nice simplification and clarification :)
> +# Create transaction tables and load them with data.
> +--echo con_bup: Creating tables
> +CREATE TABLE bup_commit_blocker.t1 (col_a CHAR(40)) ENGINE=MEMORY;
> +
> +--echo con_bup: Loading data
> +--echo con_bup: Scenario a) - commited before backup
> +INSERT INTO bup_commit_blocker.t1 VALUES ("01 Some data to test");
> +INSERT INTO bup_commit_blocker.t1 VALUES ("02 Some data to test");
> +INSERT INTO bup_commit_blocker.t1 VALUES ("03 Some data to test");
> +INSERT INTO bup_commit_blocker.t1 VALUES ("04 Some data to test");
> +INSERT INTO bup_commit_blocker.t1 VALUES ("05 Some data to test");
> +
> +--echo
> +--echo con_bup: Show the data before we start backup
> +SELECT * FROM bup_commit_blocker.t1;
> +
> +### CON 1 ###
> + --echo
> + connection con_ntx1;
> +
> + --echo Scenario (b): Insert in progress blocks CB and is included in backup
> + --echo con_ntx1: Start insert and stop it in the middle of processing
> +
> + --echo con_ntx1: Make commit stop in the middle of execution
> + # Will continue once backup has been blocked from setting CB
> + SET DEBUG_SYNC= 'after_insert_locked_tables SIGNAL insert_started
> + WAIT_FOR complete_insert';
> + send INSERT INTO bup_commit_blocker.t1 VALUES ("ntx1: 06 Some data to test");
> +
> +
> +### CON BUP ###
> +--echo
> +connection con_bup;
> +
> +# Backup will be blocked from setting CB by the ongoing insert in
> +# con_ntx1. Backup will signal con_ntx1 that it has been blocked. The
> +# insert will then finish, making backup able to set CB. When CB has
> +# been set, ntx2 will be singaled to try to insert. ntx2 insert will
> +# be blocked by the CB and send a signal that it has been blocked. CB
> +# is then released, but backup is stopped immediately after releasing
> +# CB. ntx2 will now be able to complete the commit, and ntx1 will
> +# issue and complete another insert. Finally, backup is allowed to
> +# complete.
> +
> +--echo con_bup: Activate synchronization points for BACKUP.
> +# Signal when backup is blocked by ongoing commit in con_ntx1
> +SET DEBUG_SYNC= 'wait_lock_global_read_lock SIGNAL complete_insert';
> +
> +# ntx1 insert is completed, so backup can set CB. Just before reaching
> +# VP, signal ntx2 should try to insert. Wait until ntx2 signals it has
> +# been blocked
> +SET DEBUG_SYNC= 'before_backup_data_lock SIGNAL try_insert
> + WAIT_FOR insert_blocked';
> +
> +# ntx2 insert has been blocked. Create VP and release CB, and then wait
> +# while ntx2 and ntx1 inserts.
> +SET DEBUG_SYNC= 'after_backup_binlog SIGNAL commit_unblocked
> + WAIT_FOR finish_bup';
> +
Running BACKUP automatically signals INSERT. I'd prefer to do this explicitly
from a separate connection (the default one). This way, additional actions can
be performed at the time when BACKUP is blocked and before INSERT is signalled.
> +--echo con_bup: Backing up database -- will block with lock
> +send BACKUP DATABASE bup_commit_blocker TO "bup_commit_blocker.bak";
> +
> +### CON 1 ###
> + --echo
> + connection con_ntx1;
> + --echo con_ntx1: Reap insert
> + reap;
> +
> +
> +###########
> +## Below this line happens after BACKUP has taken CB
> +###########
> +
> +### CON 2 ###
> + --echo
> + connection con_ntx2;
> +
> + --echo con_ntx2: Wait until backup has set CB, then try to insert
> + # Wait for backup to set CB
> + SET DEBUG_SYNC= 'now WAIT_FOR try_insert';
> + # Signal to backup that the commit is blocked
> + SET DEBUG_SYNC= 'wait_if_global_read_lock SIGNAL insert_blocked';
> + INSERT INTO bup_commit_blocker.t1 VALUES ("ntx2: Should NOT be in backup");
> +
> + # Test is blocked on INSERT until CB has been released
Can we test this assumption by doing SELECT on t1 (not sure about table locks)?
(cut)
> === added file 'mysql-test/suite/backup/t/backup_vp_tx.test'
> --- a/mysql-test/suite/backup/t/backup_vp_tx.test 1970-01-01 00:00:00 +0000
> +++ b/mysql-test/suite/backup/t/backup_vp_tx.test 2008-11-10 13:19:08 +0000
(cut)
> +### CON BUP ###
> +--echo
> +connection con_bup;
> +
> +# Backup will be blocked from setting CB by the ongoing commit in
> +# con_tx1. Backup will signal con_tx1 that it has been blocked. The
> +# commit will then finish, making backup able to set CB. When CB has
> +# been set, tx2 will be singaled to try to commit. tx2 commit will be
> +# blocked by the CB and send a signal that it has been blocked. CB is
> +# then released, but backup is stopped immediately after releasing CB.
> +# tx2 will now be able to complete the commit, and tx3 will issue and
> +# complete a commit. Finally, backup is allowed to complete.
> +
> +--echo con_bup: Activate synchronization points for BACKUP.
> +# Signal when backup is blocked by ongoing commit in con_tx1
> +SET DEBUG_SYNC= 'wait_lock_global_read_lock SIGNAL commit_go_done';
> +
> +# tx1 commit is completed, so backup can set CB. Just before reaching
> +# VP, signal tx2 should try to commit. Wait until tx2 signals it has
> +# been blocked
> +SET DEBUG_SYNC= 'before_backup_data_lock SIGNAL try_commit
> + WAIT_FOR commit_blocked';
> +
> +# tx2 commit has been blocked. Create VP and release CB, and then wait
> +# while tx2 and tx2 commits.
> +SET DEBUG_SYNC= 'after_backup_binlog SIGNAL commit_unblocked
> + WAIT_FOR finish_bup';
> +
> +--echo con_bup: Backing up database -- will block with lock
> +send BACKUP DATABASE bup_commit_blocker TO "bup_commit_blocker.bak";
> +
> +
> +### CON 1 ###
> + --echo
> + connection con_tx1;
> + --echo con_tx1: Reap commit
> + reap;
> +
You could check that it has committed by selecting from the table.
> +###########
> +## Below this line happens after BACKUP has taken CB
> +###########
> +
> +### CON 2 ###
> + --echo
> + connection con_tx2;
> + --echo con_tx2: Commit request will be blocked by CB
> +
> + # Wait for backup to set CB
> + SET DEBUG_SYNC= 'now WAIT_FOR try_commit';
> +
> + # Signal to backup that the commit is blocked
> + SET DEBUG_SYNC= 'wait_if_global_read_lock SIGNAL commit_blocked';
> +
> + COMMIT;
> +
> + # Test is blocked on COMMIT until CB has been released
You could stop at the time when COMMIT is blocked by CB and verify this by
selecting from the table.
> +
> +### CON 3 ###
> + --echo
> + connection con_tx3;
> + --echo con_tx3: Backup has now released CB. Commit transaction
> +
And perhaps check that trx2 is committed while BACKUP is still running.
> + # Double-check that backup has reached sync point after CB release
> + SET DEBUG_SYNC= 'now WAIT_FOR commit_unblocked';
> + COMMIT;
> + SET DEBUG_SYNC= 'now SIGNAL finish_bup';
> +
(cut)
