From: Andrei Elkin Date: June 8 2011 8:18pm Subject: bzr commit into mysql-next-mr-wl5569 branch (andrei.elkin:3283) WL#5569 List-Archive: http://lists.mysql.com/commits/138892 Message-Id: <201106082018.p58KIF1W026840@mysql1000.dsl.inet.fi> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="===============1942626995==" --===============1942626995== MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline #At file:///home/andrei/MySQL/BZR/2a-23May/WL/mysql-next-mr-wl5569/ based on revid:andrei.elkin@stripped 3283 Andrei Elkin 2011-06-08 wl#5569 MTS The following has been done with this patch. Some cleanup; Sceleton to fix a recovery flaw that discussed with Alfranio; Fixes for Blackhole (reviewer warned!); Finalization of turning STOP-SLAVE to end MTS with consistent state. covering old Load-data related events with possible erroring out; Fixing circular replication. The failure was due to the substituting an ignored event Rotate was not treated as one that changes group_master_log coordinates (therefore MTS-checkpoint). Three tests were blocked for MTS execution with comments why. They (some) might be converted to be MTS-friendly though. @ mysql-test/suite/rpl/t/rpl_bug26395.test making the test executed only in STS (single-threaded slave). @ mysql-test/suite/rpl/t/rpl_cross_version.test making the test executed only in STS (single-threaded slave). @ mysql-test/suite/rpl/t/rpl_packet.test making the test executed only in STS (single-threaded slave). @ mysql-test/suite/rpl/t/rpl_slave_grp_exec.test Test can run in both MTS and STS modes but required changes due to a possibility MTS be stopped with its own error different that a failing STS expects. Another issue *not-sorted-yet* with this test is it requires recovery to work flawlessly. Leaving further care of it to Alfranio. @ sql/binlog.cc Error constant names changed. @ sql/log_event.cc Initialization of the relocated to Log_event::m_mts_event_ends_group; APPEND_BLOCK_EVENT will be accepted but never assigned because the following *old* server EXEC_LOAD_EVENT can't be made supported yet. See rpl_cross_version.test as example how MTS could fail with ER_MTS_CANT_PARALLEL error. Part of sceleton patch for recovery - rli->reset_notified_checkpoint(0). And, making substituting an ignored event server_id=0 Rotate to enter the checkpoint branch with modifying the if condition. @ sql/log_event.h some cleanup to shift a piece of definitions to more appropriate position; Generalizing a special marking for group that is not started with BEGIN and contains multiple events. Although such group deals with Query_log_event, the bool member is moved to the base class and renamed: Log_event::m_mts_event_ends_group. @ sql/rpl_rli.cc Part of sceleton patch for recovery - + if (w->checkpoint_notified) etc. @ sql/rpl_rli.h reset_notified_checkpoint() accepts an arg of how many bits to shift for all workers in their bitmap. @ sql/rpl_rli_pdb.cc just a todo to convert to a piece of code for recovery. @ sql/rpl_rli_pdb.h Recovery related new members are added. @ sql/rpl_slave.cc comments and warnings relating to handling of stopped MTS. @ sql/share/errmsg-utf8.txt Error const:s names are editted; a new error is added. @ storage/blackhole/ha_blackhole.cc Blackhole's rule for slave thread is extended to allow MTS. modified: mysql-test/suite/rpl/t/rpl_bug26395.test mysql-test/suite/rpl/t/rpl_cross_version.test mysql-test/suite/rpl/t/rpl_packet.test mysql-test/suite/rpl/t/rpl_slave_grp_exec.test sql/binlog.cc sql/log_event.cc sql/log_event.h sql/rpl_rli.cc sql/rpl_rli.h sql/rpl_rli_pdb.cc sql/rpl_rli_pdb.h sql/rpl_slave.cc sql/share/errmsg-utf8.txt storage/blackhole/ha_blackhole.cc === modified file 'mysql-test/suite/rpl/t/rpl_bug26395.test' --- a/mysql-test/suite/rpl/t/rpl_bug26395.test 2010-12-19 17:07:28 +0000 +++ b/mysql-test/suite/rpl/t/rpl_bug26395.test 2011-06-08 20:18:08 +0000 @@ -37,6 +37,10 @@ source include/have_innodb.inc; source include/have_debug.inc; source include/master-slave.inc; +# test adapts simulation of incomplete transaction that MTS does not tolerate +# when is stopped. So it reacts with an error whereas the single-threaded is fine. +-- source include/not_mts_slave_parallel_workers.inc + --echo ==== Initialize ==== @@ -67,7 +71,6 @@ source include/sync_slave_io_with_master # Sync slave's SQL thread. sync_with_master 0; - --echo ==== Verify results on slave ==== source include/stop_slave.inc; === modified file 'mysql-test/suite/rpl/t/rpl_cross_version.test' --- a/mysql-test/suite/rpl/t/rpl_cross_version.test 2010-12-19 17:15:12 +0000 +++ b/mysql-test/suite/rpl/t/rpl_cross_version.test 2011-06-08 20:18:08 +0000 @@ -17,6 +17,9 @@ # Todo: release it from not_windows --source include/not_windows.inc +# EXEC_LOAD_EVENT of 4.1 binlog can't be supported +-- source include/not_mts_slave_parallel_workers.inc + # # Bug#31240 load data infile replication between (4.0 or 4.1) and 5.1 fails # === modified file 'mysql-test/suite/rpl/t/rpl_packet.test' --- a/mysql-test/suite/rpl/t/rpl_packet.test 2011-02-27 17:35:25 +0000 +++ b/mysql-test/suite/rpl/t/rpl_packet.test 2011-06-08 20:18:08 +0000 @@ -11,6 +11,13 @@ # max-out size db name source include/master-slave.inc; source include/have_binlog_format_row.inc; + +# TODO: Fixing is handed over to Sergei. +# The test runs slow in MTS mode because of state of MTS at time of the graceful stop. +# In this case MTS can't stop immediately if there is a Worker that received a BEGIN but never COMMIT. +-- source include/not_mts_slave_parallel_workers.inc + + call mtr.add_suppression("Slave I/O: Got a packet bigger than 'max_allowed_packet' bytes, Error_code: 1153"); call mtr.add_suppression("Slave I/O: Got fatal error 1236 from master when reading data from binary log:"); @@ -283,6 +290,7 @@ eval SET @@global.max_allowed_packet= $o DROP TABLE t1; # Clear Last_IO_Error + --source include/stop_slave_sql.inc RESET SLAVE; === modified file 'mysql-test/suite/rpl/t/rpl_slave_grp_exec.test' --- a/mysql-test/suite/rpl/t/rpl_slave_grp_exec.test 2010-12-19 17:07:28 +0000 +++ b/mysql-test/suite/rpl/t/rpl_slave_grp_exec.test 2011-06-08 20:18:08 +0000 @@ -63,8 +63,20 @@ SELECT * FROM t3 ORDER BY a; --connection slave # 1146 = ER_NO_SUCH_TABLE ---let $slave_sql_errno= 1146 ---source include/wait_for_slave_sql_error.inc +# in MTS case error is either of two: +#--let $slave_sql_errno= 1146,1593 +# whereas in the single-threaded case: +#--let $slave_sql_errno= 1146 + +--source include/wait_for_slave_sql_to_stop.inc +let $slave_sql_errno= query_get_value(SHOW SLAVE STATUS, Last_SQL_Errno, 1); +if (`select $slave_sql_errno != 1146 and $slave_sql_errno != 1593`) +{ + --echo Unexpected error: $slave_sql_errno + --die +} + + SHOW TABLES LIKE 't%'; if (`SELECT @@BINLOG_FORMAT = 'ROW'`) { --replace_regex /AA/AA_for_row_or_XX_for_stmt_mixed/ @@ -110,8 +122,18 @@ UPDATE t1 SET b = 'X' WHERE a = 2; --connection slave # 1146 = ER_NO_SUCH_TABLE ---let $slave_sql_errno= 1146 ---source include/wait_for_slave_sql_error.inc +# in MTS case error is either of two: +#--let $slave_sql_errno= 1146,1593 +# whereas in the single-threaded case: +#--let $slave_sql_errno= 1146 + +--source include/wait_for_slave_sql_to_stop.inc +let $slave_sql_errno= query_get_value(SHOW SLAVE STATUS, Last_SQL_Errno, 1); +if (`select $slave_sql_errno != 1146 and $slave_sql_errno != 1593`) +{ + --echo Unexpected error: $slave_sql_errno + --die +} --connection master SELECT * FROM t1 ORDER BY a; @@ -125,6 +147,8 @@ SELECT * FROM t2 ORDER BY a; --source include/stop_slave_io.inc RENAME TABLE t3_bak TO t3; + +# TODO: recovery. Alfranio it fails to recover here. --source include/start_slave.inc --connection master @@ -156,8 +180,19 @@ COMMIT; --connection slave # 1146 = ER_NO_SUCH_TABLE ---let $slave_sql_errno= 1146 ---source include/wait_for_slave_sql_error.inc +# in MTS case error is either of two: +#--let $slave_sql_errno= 1146,1593 +# whereas in the single-threaded case: +#--let $slave_sql_errno= 1146 + +--source include/wait_for_slave_sql_to_stop.inc +let $slave_sql_errno= query_get_value(SHOW SLAVE STATUS, Last_SQL_Errno, 1); +if (`select $slave_sql_errno != 1146 and $slave_sql_errno != 1593`) +{ + --echo Unexpected error: $slave_sql_errno + --die +} + --connection master SELECT * FROM t1 ORDER BY a; === modified file 'sql/binlog.cc' --- a/sql/binlog.cc 2011-05-24 14:29:35 +0000 +++ b/sql/binlog.cc 2011-06-08 20:18:08 +0000 @@ -4550,8 +4550,8 @@ THD::add_to_binlog_updated_dbs(const cha if (binlog_accessed_db_names->elements > MAX_DBS_IN_EVENT_MTS) { push_warning_printf(this, MYSQL_ERROR::WARN_LEVEL_WARN, - ER_UPDATED_DBS_GREATER_MAX, - ER(ER_UPDATED_DBS_GREATER_MAX), + ER_MTS_UPDATED_DBS_GREATER_MAX, + ER(ER_MTS_UPDATED_DBS_GREATER_MAX), MAX_DBS_IN_EVENT_MTS); return; } === modified file 'sql/log_event.cc' --- a/sql/log_event.cc 2011-06-06 10:51:19 +0000 +++ b/sql/log_event.cc 2011-06-08 20:18:08 +0000 @@ -675,7 +675,7 @@ Log_event::Log_event(THD* thd_arg, uint1 Log_event::Log_event() :temp_buf(0), exec_time(0), flags(0), crc(0), thd(0), - checksum_alg(BINLOG_CHECKSUM_ALG_UNDEF) + checksum_alg(BINLOG_CHECKSUM_ALG_UNDEF), m_mts_event_ends_group(FALSE) { server_id= ::server_id; /* @@ -695,7 +695,7 @@ Log_event::Log_event() Log_event::Log_event(const char* buf, const Format_description_log_event* description_event) :temp_buf(0), exec_time(0), cache_type(Log_event::EVENT_INVALID_CACHE), - crc(0), checksum_alg(BINLOG_CHECKSUM_ALG_UNDEF) + crc(0), checksum_alg(BINLOG_CHECKSUM_ALG_UNDEF), m_mts_event_ends_group(FALSE) { #ifndef MYSQL_CLIENT thd = 0; @@ -2359,7 +2359,7 @@ Log_event::continue_group(Relay_log_info bool Log_event::contains_partition_info() { - return get_type_code() == TABLE_MAP_EVENT || + return (get_type_code() == TABLE_MAP_EVENT) || (get_type_code() == QUERY_EVENT && !ends_group() && !starts_group()) || (get_type_code() == EXECUTE_LOAD_QUERY_EVENT); } @@ -2537,7 +2537,7 @@ Slave_worker *Log_event::get_slave_worke DBUG_ASSERT(rli->curr_group_assigned_parts.elements > 0 || ret_worker->id == 0); } - else // int_, rand_, user_ var:s + else // int_, rand_, user_ var:s, load-data events { Log_event *ptr_curr_ev= this; @@ -2545,7 +2545,8 @@ Slave_worker *Log_event::get_slave_worke get_type_code() == RAND_EVENT || get_type_code() == USER_VAR_EVENT || get_type_code() == ROWS_QUERY_LOG_EVENT || - get_type_code() == BEGIN_LOAD_QUERY_EVENT); + get_type_code() == BEGIN_LOAD_QUERY_EVENT || + get_type_code() == APPEND_BLOCK_EVENT); insert_dynamic(&rli->curr_group_da, (uchar*) &ptr_curr_ev); @@ -2629,6 +2630,8 @@ Slave_worker *Log_event::get_slave_worke strcpy(ptr_g->checkpoint_relay_log_name, rli->get_group_relay_log_name()); ptr_g->checkpoint_relay_log_pos= rli->get_group_relay_log_pos(); + ptr_g->shifted= ret_worker->bitmap_shifted; + ret_worker->bitmap_shifted= 0; ret_worker->checkpoint_notified= TRUE; } ptr_g->checkpoint_seqno= rli->checkpoint_seqno; @@ -2878,13 +2881,15 @@ int Log_event::apply_event(Relay_log_inf wrappped with BEGIN/COMMIT or preceeded by User|Int|Random- var. MTS has to stop to suggest restart in the permanent sequential mode. */ + rli->report(ERROR_LEVEL, ER_MTS_CANT_PARALLEL, + ER(ER_MTS_CANT_PARALLEL), + get_type_code(), c_rli->get_event_relay_log_name(), + c_rli->get_event_relay_log_pos()); + + /* Coordinator cant continue, it marks MTS group status accordingly */ + c_rli->mts_group_status= Relay_log_info::MTS_KILLED_GROUP; - // TODO: improve err msg - rli->report(ERROR_LEVEL, ER_SLAVE_FATAL_ERROR, - ER(ER_SLAVE_FATAL_ERROR), - "Can't execute all binlog event in parallel mode"); - - // destroy possible buffered events of the current group prior to exit + /* destroy deferred events */ for (uint k= 0; k < rli->curr_group_da.elements; k++) { delete *(Log_event**) dynamic_array_ptr(&rli->curr_group_da, k); @@ -3760,7 +3765,7 @@ Query_log_event::Query_log_event(const c auto_increment_increment(1), auto_increment_offset(1), time_zone_len(0), lc_time_names_number(0), charset_database_number(0), table_map_for_update(0), master_data_written(0), - mts_accessed_dbs(OVER_MAX_DBS_IN_EVENT_MTS), m_mts_query_ends_group(FALSE) + mts_accessed_dbs(OVER_MAX_DBS_IN_EVENT_MTS) { ulong data_len; uint32 tmp; @@ -6507,7 +6512,7 @@ int Rotate_log_event::do_update_pos(Rela if ((server_id != ::server_id || rli->replicate_same_server_id) && !is_relay_log_event() && ((!rli->is_parallel_exec() && !rli->is_in_group()) || - rli->mts_group_status == Relay_log_info::MTS_NOT_IN_GROUP)) + rli->mts_group_status != Relay_log_info::MTS_IN_GROUP)) { mysql_mutex_lock(&rli->data_lock); DBUG_PRINT("info", ("old group_master_log_name: '%s' " @@ -6532,7 +6537,7 @@ int Rotate_log_event::do_update_pos(Rela mysql_mutex_unlock(&rli->data_lock); rli->flush_info(TRUE); // todo: error branch if (rli->is_parallel_exec()) - rli->reset_notified_checkpoint(); + rli->reset_notified_checkpoint(0); /* Reset thd->variables.option_bits and sql_mode etc, because this could be the signal of === modified file 'sql/log_event.h' --- a/sql/log_event.h 2011-06-05 17:01:51 +0000 +++ b/sql/log_event.h 2011-06-08 20:18:08 +0000 @@ -1099,37 +1099,6 @@ public: { return thd ? thd->db : 0; } - - /* - The method returns a list of updated by the event databases. - Other than in the case of Query-log-event the list is just one item. - */ - virtual List* mts_get_dbs(MEM_ROOT *mem_root) - { - List *res= new List; - res->push_back(strdup_root(mem_root, get_db())); - return res; - } - - /* - returns the number of updated by the event databases. - In other than Query-log-event case that's one. - */ - virtual uint8 mts_number_dbs() { return 1; } - - /* - Event can be exceptionally marked to force its execution. - in isolation from any other Workers. - Other than Query-log-event class should not have any implementation - of this method. - */ - virtual void mts_do_isolate_event() { DBUG_ASSERT(0); } - - /* - Verifying whether event is marked to execute in isolation. - */ - virtual bool mts_is_event_isolated() { return FALSE; } - #else Log_event() : temp_buf(0) {} /* avoid having to link mysqlbinlog against libpthread */ @@ -1257,6 +1226,10 @@ public: @note There are incompatile combinations such the referred event is wrapped with BEGIN/COMMIT. Such cases should be identified by the caller and treates as an error. + + Notice, even though the func returns TRUE, some events + like old LOAD-DATA rooted EXEC_LOAD_EVENT can't run even + in isolated parallel mode and MTS would have to stop. @return TRUE if despite permanent parallel execution mode an event needs applying in a real isolation that is sequentially. @@ -1270,11 +1243,9 @@ public: get_type_code() == LOAD_EVENT || get_type_code() == SLAVE_EVENT || get_type_code() == CREATE_FILE_EVENT || - get_type_code() == APPEND_BLOCK_EVENT || - get_type_code() == EXEC_LOAD_EVENT || get_type_code() == DELETE_FILE_EVENT || get_type_code() == NEW_LOAD_EVENT || - + get_type_code() == EXEC_LOAD_EVENT || get_type_code() == FORMAT_DESCRIPTION_EVENT|| get_type_code() == INCIDENT_EVENT; @@ -1318,6 +1289,45 @@ public: */ Slave_worker *get_slave_worker_id(Relay_log_info *rli); + /* + The method returns a list of updated by the event databases. + Other than in the case of Query-log-event the list is just one item. + */ + virtual List* mts_get_dbs(MEM_ROOT *mem_root) + { + List *res= new List; + res->push_back(strdup_root(mem_root, get_db())); + return res; + } + + /* + returns the number of updated by the event databases. + In other than Query-log-event case that's one. + */ + virtual uint8 mts_number_dbs() { return 1; } + + /* + Event can be exceptionally marked to force its execution. + in isolation from any other Workers. + Other than Query-log-event class should not have any implementation + of this method. + */ + /* + Event can be indentified as a group terminator and such fact + is memoried by the function. + */ + virtual void mts_do_isolate_event() + { + DBUG_ASSERT(get_type_code() == QUERY_EVENT || + get_type_code() == EXEC_LOAD_EVENT || + get_type_code() == EXECUTE_LOAD_QUERY_EVENT); + m_mts_event_ends_group= TRUE; + } + /* + Verifying whether event is marked to execute in isolation. + */ + virtual bool mts_is_event_isolated() { return m_mts_event_ends_group; } + /** Apply the event to the database. @@ -1451,6 +1461,8 @@ protected: non-zero. The caller shall decrease the counter by one. */ virtual enum_skip_reason do_shall_skip(Relay_log_info *rli); + + bool m_mts_event_ends_group; #endif }; @@ -1908,13 +1920,6 @@ public: uchar mts_accessed_dbs; char mts_accessed_db_names[MAX_DBS_IN_EVENT_MTS][NAME_LEN]; - /* - Event can be indentified as a group terminator and such fact - is memoried by the function. - */ - virtual void mts_do_isolate_event() { m_mts_query_ends_group= TRUE; } - virtual bool mts_is_event_isolated() { return m_mts_query_ends_group; } - #ifdef MYSQL_SERVER Query_log_event(THD* thd_arg, const char* query_arg, ulong query_length, @@ -2024,9 +2029,6 @@ public: /* !!! Public in this pat (!strncasecmp(query, STRING_WITH_LEN("ROLLBACK")) && strncasecmp(query, STRING_WITH_LEN("ROLLBACK TO "))); } -private: - - bool m_mts_query_ends_group; }; @@ -3101,6 +3103,13 @@ public: #endif /* MTS executes this event sequentially */ virtual uint8 mts_number_dbs() { return OVER_MAX_DBS_IN_EVENT_MTS; } + virtual List* mts_get_dbs(MEM_ROOT *mem_root) + { + List *res= new List; + res->push_back(strdup_root(mem_root, "")); + return res; + } + private: #if defined(MYSQL_SERVER) && defined(HAVE_REPLICATION) virtual int do_apply_event(Relay_log_info const *rli); === modified file 'sql/rpl_rli.cc' --- a/sql/rpl_rli.cc 2011-06-06 10:51:19 +0000 +++ b/sql/rpl_rli.cc 2011-06-08 20:18:08 +0000 @@ -192,14 +192,19 @@ void Relay_log_info::reset_notified_rela to reset the current bitmap and starts using the clean bitmap indexed from zero of being reset checkpoint_seqno. */ -void Relay_log_info::reset_notified_checkpoint() +void Relay_log_info::reset_notified_checkpoint(ulong shift) { if (!is_parallel_exec()) return; for (uint i= 0; i < workers.elements; i++) { Slave_worker *w= *(Slave_worker **) dynamic_array_ptr(&workers, i); - w->checkpoint_notified= FALSE; + if (w->checkpoint_notified) + { + w->bitmap_shifted= 0; + w->checkpoint_notified= FALSE; + } + w->bitmap_shifted += shift; // to reset at passing the accumulate value into GAQ } checkpoint_seqno= 0; } @@ -1036,7 +1041,7 @@ void Relay_log_info::stmt_done(my_off_t flush_info(is_transactional() ? TRUE : FALSE); // Alfranio todo: error branch if (is_parallel_exec()) - reset_notified_checkpoint(); + reset_notified_checkpoint(0); } } === modified file 'sql/rpl_rli.h' --- a/sql/rpl_rli.h 2011-06-06 10:51:19 +0000 +++ b/sql/rpl_rli.h 2011-06-08 20:18:08 +0000 @@ -537,7 +537,7 @@ public: Coordinator notifies Workers about this event. Coordinator and Workers maintain a bitmap of executed group that is reset with a new checkpoint. */ - void reset_notified_checkpoint(); + void reset_notified_checkpoint(ulong); /** Helper function to do after statement completion. === modified file 'sql/rpl_rli_pdb.cc' --- a/sql/rpl_rli_pdb.cc 2011-06-06 10:51:19 +0000 +++ b/sql/rpl_rli_pdb.cc 2011-06-08 20:18:08 +0000 @@ -225,6 +225,7 @@ bool Slave_worker::commit_positions(Log_ my_free(ptr_g->checkpoint_relay_log_name); ptr_g->checkpoint_relay_log_name= NULL; + // TODO: shift `group_execed' << ptr_g->shifted bitmap_clear_all(&group_execed); } // extract an updated relay-log name to store in Worker's rli. === modified file 'sql/rpl_rli_pdb.h' --- a/sql/rpl_rli_pdb.h 2011-06-05 17:01:51 +0000 +++ b/sql/rpl_rli_pdb.h 2011-06-08 20:18:08 +0000 @@ -144,6 +144,7 @@ typedef struct st_slave_job_group my_off_t checkpoint_relay_log_pos; // T-event lop_pos filled by W for CheckPoint char* checkpoint_relay_log_name; volatile uchar done; // Flag raised by W, read and reset by C + ulong shifted; // shift the last CP bitmap at receiving a new CP } Slave_job_group; #define retrieve_job(from, to) \ @@ -265,6 +266,7 @@ public: long usage_partition; // number of different partitions handled by this worker volatile bool relay_log_change_notified; // Coord sets and resets, W can read volatile bool checkpoint_notified; // Coord sets and resets, W can read + ulong bitmap_shifted; // shift the last bitmap at receiving new CP bool wq_overrun_set; // W monitors its queue usage to incr/decr rli->mts_wqs_overrun /* We need to make this a dynamic field. /Alfranio === modified file 'sql/rpl_slave.cc' --- a/sql/rpl_slave.cc 2011-06-06 10:51:19 +0000 +++ b/sql/rpl_slave.cc 2011-06-08 20:18:08 +0000 @@ -1067,6 +1067,12 @@ static bool io_slave_killed(THD* thd, Ma In the event of deffering decision @rli->last_event_start_time waiting timer is set to force the killed status be accepted upon its expiration. + Notice Multi-Threaded-Slave behaives similarly in that when it's being + stopped and the current group of assigned events has not yet scheduled + completely, Coordinator deferres to accept to leave its read-distribute + state. The above timeout ensures waiting won't last endlessly, and in + such case an error is repoted. + @param thd pointer to a THD instance @param rli pointer to Relay_log_info instance @@ -1139,9 +1145,14 @@ static bool sql_slave_killed(THD* thd, R if (ret == 0) { rli->report(WARNING_LEVEL, 0, + rli->mts_group_status == Relay_log_info::MTS_NOT_IN_GROUP ? "slave SQL thread is being stopped in the middle " "of applying of a group having updated a non-transaction " - "table; waiting for the group completion ... "); + "table; waiting for the group completion ... " + : + "Coordinator thread of multi-threaded slave is being stopped in the middle " + "of assigning a group of events; " + "deferring to exit until the group completion ... "); } else { @@ -1155,7 +1166,8 @@ static bool sql_slave_killed(THD* thd, R { ret= TRUE; rli->report(ERROR_LEVEL, ER_SLAVE_FATAL_ERROR, ER(ER_SLAVE_FATAL_ERROR), - msg_stopped); + rli->mts_group_status == Relay_log_info::MTS_NOT_IN_GROUP ? + msg_stopped : msg_stopped_mts); } } else @@ -3815,6 +3827,12 @@ pthread_handler_t handle_slave_worker(vo { error= slave_worker_exec_job(w, rli); } + + /* + Cleanup after an error requires clear_error() go first. + Otherwise assert(!all) in binlog_rollback() + */ + thd->clear_error(); w->cleanup_context(thd, error); mysql_mutex_lock(&w->jobs_lock); @@ -3854,7 +3872,6 @@ err: if (thd) { - thd->clear_error(); mysql_mutex_lock(&LOCK_thread_count); THD_CHECK_SENTRY(thd); /* @@ -4136,7 +4153,7 @@ bool mts_checkpoint_routine(Relay_log_in } // end of commit_positions */ - rli->reset_notified_checkpoint(); + rli->reset_notified_checkpoint(cnt); end: set_timespec_nsec(rli->last_clock, 0); @@ -4179,6 +4196,7 @@ int slave_start_single_worker(Relay_log_ w->curr_group_exec_parts.elements= 0; w->relay_log_change_notified= FALSE; // the 1st group to contain relaylog name w->checkpoint_notified= FALSE; + w-> bitmap_shifted= 0; w->workers= rli->workers; // shallow copying is sufficient w->this_worker= w; w->wait_jobs= w->trans_jobs= w->stmt_jobs= w->curr_jobs= 0; @@ -5928,6 +5946,9 @@ static Log_event* next_event(Relay_log_i rli->set_future_event_relay_log_pos(my_b_tell(cur_log)); ev->future_event_relay_log_pos= rli->get_future_event_relay_log_pos(); + if (hot_log) + mysql_mutex_unlock(log_lock); + /* MTS checkpoint in the successful read branch */ @@ -5935,11 +5956,10 @@ static Log_event* next_event(Relay_log_i if (rli->is_parallel_exec() && (mts_checkpoint_period != 0 || force)) { ulonglong period= static_cast(mts_checkpoint_period * 1000000ULL); + mysql_mutex_unlock(&rli->data_lock); mts_checkpoint_routine(rli, period, force, TRUE); // TODO: ALFRANIO ERROR + mysql_mutex_lock(&rli->data_lock); } - - if (hot_log) - mysql_mutex_unlock(log_lock); DBUG_RETURN(ev); } DBUG_ASSERT(thd==rli->info_thd); @@ -6066,7 +6086,10 @@ static Log_event* next_event(Relay_log_i do { + mysql_mutex_unlock(log_lock); mts_checkpoint_routine(rli, period, FALSE, FALSE); // TODO: ALFRANIO ERROR + mysql_mutex_lock(log_lock); + set_timespec_nsec(waittime, period); thd->enter_cond(log_cond, log_lock, "Slave has read all relay log; " @@ -6573,8 +6596,8 @@ int start_slave(THD* thd , Master_info* { mi->rli->opt_slave_parallel_workers= 0; push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, - ER_NO_FEATURE_ON_PARALLEL_SLAVE, - ER(ER_NO_FEATURE_ON_PARALLEL_SLAVE), + ER_MTS_FEATURE_IS_NOT_SUPPORTED, + ER(ER_MTS_FEATURE_IS_NOT_SUPPORTED), "UNTIL condtion", "Slave is started in the sequential execution mode."); } @@ -6586,8 +6609,8 @@ int start_slave(THD* thd , Master_info* if (mi->rli->opt_slave_parallel_workers != 0 && slave_trans_retries != 0) { push_warning_printf(thd, MYSQL_ERROR::WARN_LEVEL_NOTE, - ER_NO_FEATURE_ON_PARALLEL_SLAVE, - ER(ER_NO_FEATURE_ON_PARALLEL_SLAVE), + ER_MTS_FEATURE_IS_NOT_SUPPORTED, + ER(ER_MTS_FEATURE_IS_NOT_SUPPORTED), "Temporary failed transaction retry", "Such failure will force the slave to stop."); } === modified file 'sql/share/errmsg-utf8.txt' --- a/sql/share/errmsg-utf8.txt 2011-02-27 17:35:25 +0000 +++ b/sql/share/errmsg-utf8.txt 2011-06-08 20:18:08 +0000 @@ -6456,7 +6456,9 @@ ER_STMT_CACHE_FULL ER_BINLOG_STMT_CACHE_SIZE_GREATER_THAN_MAX eng "Option binlog_stmt_cache_size (%lu) is greater than max_binlog_stmt_cache_size (%lu); setting binlog_stmt_cache_size equal to max_binlog_stmt_cache_size." -ER_NO_FEATURE_ON_PARALLEL_SLAVE +ER_MTS_FEATURE_IS_NOT_SUPPORTED eng "%s is not supported in Parallel Slave. %s" -ER_UPDATED_DBS_GREATER_MAX +ER_MTS_UPDATED_DBS_GREATER_MAX eng "Modified database names number exceeds the maximum %d; the names are not written into the replication event." +ER_MTS_CANT_PARALLEL + eng "Can't execute the current event group in parallel mode running into event %s, relay-log name %s, position %s." === modified file 'storage/blackhole/ha_blackhole.cc' --- a/storage/blackhole/ha_blackhole.cc 2010-10-06 14:34:28 +0000 +++ b/storage/blackhole/ha_blackhole.cc 2011-06-08 20:18:08 +0000 @@ -23,7 +23,7 @@ #include "unireg.h" #include "probes_mysql.h" #include "ha_blackhole.h" -#include "sql_class.h" // THD, SYSTEM_THREAD_SLAVE_SQL +#include "sql_class.h" // THD, SYSTEM_THREAD_SLAVE_* /* Static declarations for handlerton */ @@ -118,7 +118,8 @@ int ha_blackhole::update_row(const uchar { DBUG_ENTER("ha_blackhole::update_row"); THD *thd= ha_thd(); - if (thd->system_thread == SYSTEM_THREAD_SLAVE_SQL && thd->query() == NULL) + if ((thd->system_thread == SYSTEM_THREAD_SLAVE_SQL || + thd->system_thread == SYSTEM_THREAD_SLAVE_WORKER) && thd->query() == NULL) DBUG_RETURN(0); DBUG_RETURN(HA_ERR_WRONG_COMMAND); } @@ -127,7 +128,8 @@ int ha_blackhole::delete_row(const uchar { DBUG_ENTER("ha_blackhole::delete_row"); THD *thd= ha_thd(); - if (thd->system_thread == SYSTEM_THREAD_SLAVE_SQL && thd->query() == NULL) + if ((thd->system_thread == SYSTEM_THREAD_SLAVE_SQL || + thd->system_thread == SYSTEM_THREAD_SLAVE_WORKER) && thd->query() == NULL) DBUG_RETURN(0); DBUG_RETURN(HA_ERR_WRONG_COMMAND); } @@ -146,7 +148,8 @@ int ha_blackhole::rnd_next(uchar *buf) MYSQL_READ_ROW_START(table_share->db.str, table_share->table_name.str, TRUE); THD *thd= ha_thd(); - if (thd->system_thread == SYSTEM_THREAD_SLAVE_SQL && thd->query() == NULL) + if ((thd->system_thread == SYSTEM_THREAD_SLAVE_SQL || + thd->system_thread == SYSTEM_THREAD_SLAVE_WORKER) && thd->query() == NULL) rc= 0; else rc= HA_ERR_END_OF_FILE; @@ -236,7 +239,8 @@ int ha_blackhole::index_read_map(uchar * DBUG_ENTER("ha_blackhole::index_read"); MYSQL_INDEX_READ_ROW_START(table_share->db.str, table_share->table_name.str); THD *thd= ha_thd(); - if (thd->system_thread == SYSTEM_THREAD_SLAVE_SQL && thd->query() == NULL) + if ((thd->system_thread == SYSTEM_THREAD_SLAVE_SQL || + thd->system_thread == SYSTEM_THREAD_SLAVE_WORKER) && thd->query() == NULL) rc= 0; else rc= HA_ERR_END_OF_FILE; @@ -253,7 +257,8 @@ int ha_blackhole::index_read_idx_map(uch DBUG_ENTER("ha_blackhole::index_read_idx"); MYSQL_INDEX_READ_ROW_START(table_share->db.str, table_share->table_name.str); THD *thd= ha_thd(); - if (thd->system_thread == SYSTEM_THREAD_SLAVE_SQL && thd->query() == NULL) + if ((thd->system_thread == SYSTEM_THREAD_SLAVE_SQL || + thd->system_thread == SYSTEM_THREAD_SLAVE_WORKER) && thd->query() == NULL) rc= 0; else rc= HA_ERR_END_OF_FILE; @@ -269,7 +274,8 @@ int ha_blackhole::index_read_last_map(uc DBUG_ENTER("ha_blackhole::index_read_last"); MYSQL_INDEX_READ_ROW_START(table_share->db.str, table_share->table_name.str); THD *thd= ha_thd(); - if (thd->system_thread == SYSTEM_THREAD_SLAVE_SQL && thd->query() == NULL) + if ((thd->system_thread == SYSTEM_THREAD_SLAVE_SQL || + thd->system_thread == SYSTEM_THREAD_SLAVE_WORKER) && thd->query() == NULL) rc= 0; else rc= HA_ERR_END_OF_FILE; --===============1942626995== MIME-Version: 1.0 Content-Type: text/bzr-bundle; charset="us-ascii"; name="bzr/andrei.elkin@stripped" Content-Transfer-Encoding: 7bit Content-Disposition: inline # Bazaar merge directive format 2 (Bazaar 0.90) # revision_id: andrei.elkin@stripped # target_branch: file:///home/andrei/MySQL/BZR/2a-23May/WL/mysql-next-\ # mr-wl5569/ # testament_sha1: 67f4a61c99bc23b2b3966211b910d84c79640fce # timestamp: 2011-06-08 23:18:15 +0300 # source_branch: file:///home/andrei/MySQL/BZR/2a-23May/mysql-trunk/ # base_revision_id: andrei.elkin@stripped\ # j8yk9b45uvirqvf4 # # Begin bundle IyBCYXphYXIgcmV2aXNpb24gYnVuZGxlIHY0CiMKQlpoOTFBWSZTWfPFXdcAETL/gH/VVRR///// /+//4P////5gIJ0Md59871vtXSSJuex8e3d5g++xikpsfXS7B9Bo9G9nbvtfAygKNgZCvbd8zvfe 69r59ts5EhKtavZr1jVLG2As9zuvudyXXd9ds5SlOGQkNCYRpgiYmU2pmmgmTakyaaaGjQ9QAaAY Q0xqCSIBMgQJqaaZTRP1T9CgDRkAAAANAAAAGgBCRM1PUyCE2mFBoAAAAAAAAAACQoQQhpppMU9R 5NT9Kempmp6aI9R6hoANAGgAABo9QRSIAgNJhMmmjSYjTTRoTIp7SnqHpphT1PKaNPUaBmmhPUwi iCAQBIyYJMkNlTZqh4oeptT1AA0PUaZBpkAG1NCAyDR9DEIgvF16mTkk6F26HN53ciwODw5Oa948 6bmaRm6pqJ9zDbQOxYXeVmCfP+ULf3EvQr2fuRXKg6pwFZCnj+6F0IUQkej3X+dLfR4t8er+oXJ/ qX72iSY9hh13h7ggUO1tJX/iAQ6bGkijnmWBMCrnBQPbm3s1S0aJY5CupPJxC1NBiSBERLWxoNnp msxoBTixWsL5fwnBmD0HEoCiU3HJncb1ws+8F0FbhT1NZMmYIh4jZO/V3ZsSnLF9TlAjWM39mz4Y uG2OUHKs/wQQE2S0YC7GjJJmBRYRhkbWcQdlTYY0ZDs+Q3WLZReL33pMm0P7eoSrJh6vCG5JdJW8 SLrVQwpUpEg2pOAmbrb+FHNlxeTU6nmFE49WuaomKAUYUPpgAPxhzwA8YCUAlJG+JMLEEMAHYOFO LmwjhPzfN3sIUjrzH/ZCY7LfB+7dYieI9C/M/o2uaGA3ejbEt8S1CTG2xDYNpsbTabQm02DYDY2h sC741/CSWz2U5pvpuqWZSaqDYXkA3cC9VulXqGanQoXo6mJTAl0LqhJq2Ch8S9wrtFUZX0ugHlaR SsHgFZygPHEAoxkNiLlWHplDWtcJTpIRDgzeFCDldcHWMMGwdpsDrD2ZMIWcaWXJbBY5Li18byNF mYOjLNu8pCzHssaRo9BBGyh8cjWdc+YA7wcAV6hi06uFU9Tv+HuGTZlk4EZe+uh97qZ5LV5Q1Bpn jy7XT07itnt3NkIOE2guk3F5WAizek+Y7XlIK4gDirXuWhMLODkgOSrHI9ZVCQsnYy6MfZhGNPMt CsulDnJvDzmdSP5poNG1G0cYbWLvr7XLcpNp5Xdj1T9uee+XfZ2xWnu4wy4RXybtrmU5wWKWUnhn e6sYfhtPCFXAvXDa+V1Di+22OlHrbrFjLJ8GnO5QXk8BLyNm0uKybmZCPg1JMga2VE4ZqJIieRt/ tg0xysSHrSukiWmsFWxjNtdDPJ56StrwE+upyelm71yit6VrGFKqQsm9ptGzZlMhooM1QbXi2WUr HFosVmmdFyUy7a4IsbNBntbQzyyjq4s+MV3+c6eICJNhaN0BLhwNvYMchg53rFgjGw4wGVkoQW+X VpRsgRSge51yfn8GqlWjLpgq3R9Z0vpSNh8zKV+nvHyo7pL3ZfqzzzxB+wbtbjqd52btkrBrwZQ4 7LnjRt+r6eeUs87s8MeaJl4OqMdAS2XPSPhWUo3+AWe0pz084+r7S3torbTdPhyZjsEte9kPCot2 F/AZcWRJkb9iTB40lT1gcG0Ko3P/DkIMeXjPKQu/Bk+xu3Iuoz7jLfs1uqcq1l0F1AzGlyIIEJ83 Kpy8m7o97j3IHQOV3e7g8fJ0sLOBzRTWEOVG05Ov0+pLWkZ1aBKPh8fLvZNsjdqbnO+gBYzxUEht DaQEnrJB2HRCOgTFwGXSZuUU9IuE5VPmGCff78ek2K8rvuI/5DyMHB1bffl2aEpJvGWCDiKoePj6 RhnfEE2ew0gsCYUuoUUeFc0y/N5QuFO15wlMprDSWTPiqiMUEx/bad3fAHkFcuUkdpAIzUgrrEI3 HVDnMDRC2JLGGBaEzrxGceE0nrB8AKnvc3RuX4GUKzNKJ+Z9rDQQydDJgdDDIZvuQG1q7QPibT0E Bq+IDWSGQdhIHd3So9r4Jwk0HKxlNwzesva7AA+D69djsxAVNw8QsGmupxhY6dnvrqdzaDlhCxkW QIYAsBmZbBveQNRJ4+852KWcxzLBzjutDZrGxLrowTtH4BFQkbp2dAOAaLeWKOQ7KrKFn6kAYoWL jUKUrWig0FTdabBDLETR6Ob0YDytSktGcTnkT5QPAcB+w/HzznABZrzC8eEiUSIWWKIKEbZHAIEC i3d3j1UpkABsSo3xQRxsszQwjDwOzSVGm0XNLBrMwq7mF79W7pCmWmnZkhqyYtxh+n7KDT6yQZHa DIozC5oQyhLKKS72WdCAkAPiIsaF2zRyLUtAkwykMQY0cmmrgyBDYmWzMHiUKHErkeBnYRLjiBGz o7FbEVLGUlTX3E4gBFxwADYimJs2GrAxihQ6ECUpK2i7ycjAwOd/s0KkRjhYzbgMMxqW4dT3Nd9E OCW+msDMwrJB5GWYNdjy7aVLOx9WCDkwk9+CkEDBkC39GpU7MLoaMOWkUAv+joSIlQ3+2e87ale4 uX4sGqKbjc8yOLg32IQMQNHXboYF4v1KFHZmGrRpRjnC8cK7+W2vd3U4WtxwmkToNI7I3QGxGmnk SLEn3NQB67iq6RdlHiqumGEwzefliFub80Z8RQvkZtqfWUlfIrDRt+EIRCY07aEFLMboZPE17UdL y7ZAA+7cLcDmdhb1wot0huKbJZOGiCn/zfBjnN+1Xec7HU3nQ7RbHYbHM2IP1EnhiVAVhZHoWAAs exbb8XEsMaauztWHPtq1Hfi9Hk21nhC0GMrQVsukhAiy1ACKkslUtFj6UYlaktGxLpaasyMGkQQW aLs0LkfPGNWdAvUj1RXiVPg0k4mbQxGHjj36YnJrdvBzTEkMuMxiqZRNI8iDtMAMom1NDgr15wHg xY9I8czFL9h4Y2w5SgQddfudjgFe81NxB8rwLl2jk5vjQdH2OlwOD84+tw8HMd/Pn1OlHsnoJhj2 VmZrEZQk61vSu4SAKPrMzNCxCRgEtiA3tC3dVI5hpMZdpNetUljCd6L0Q72njy2XcZLhqM++Z3Dr EsbI4LY21Jdw6wTKZbnHY2IxWzddvMbETDXhrCJojjc4PDFDaaiSPBMVAqMegqdsCRUiZFCGhpYH Crt2gDAd8AYtZ0cZ1nUawdZuux1s1VtpntD3OV8SLGMGhK1Vbm1UXKrnzuI87C7e40IVNb7MMm7S T78HtMCw9xOVORqSJlGk2KW8Ni5iWDS/Jo+M3Jp8oRylOQQW0zBc2AonYDkKhyhXUe2Y2xxNic8p 0WpwNs6lDStoX1iRiN7wepkzMEJpK0vFo8HwsdDY/A7ihscF5lCZMDEDuDzdnFyu706Ti0JHiHPa /KxrW6GSnZeUrCCFykWclApzFGogMyYTucFdAEb1KkSfEyMGRFYLl8veDRSU1SphegDzFyGg1aO4 Gg3Z7VOqxsSADbiMRPfrIZCr5i4eS8s8dPDI4A11PTrqMcbEcsoGu5E7iOZ7dp2wzoIMTCq5pkyN bCVKSrocMDlSNDIqZjjqa6mxcuVOwhMsRImRXM5nkrniutpDsclrNvAu2M+l5nMaq2CuVkUYU6Nd dUV3EpTabdOzpJBQU8uwxmIYYAGZNbOuGkkZXnaOCASpXBMTxketBXFTI4J0UqRFEkazG0RpXUyO S1rSk3NBlFqwPFfDiT3O7y3O9Tu2xuakE9oROZc1GJFWlIyK2LEoLLKDJLa5BSIHVUUy5vxDhA0C YxY6qh3h3gdPBMvSsLxXLHBnbPpFtJMfMtaOvEbUtWh85i9yAQKWgQc4XsCwZRzcyWZMC5yD0gTi YKFH7D0Wjw43M85LLI9qtszIeYw/IJG5M5bDj1kbVCE8WqV4dVKZzqWLl1lGzmDWZkADiXPFfZKM Du8fAgeiwBrAqcTqcjUcc73FIzHQXFz6zWcZqNIGm+i3Ct0YqF6VKzp5mqdHAHmUAHCXpRMilMAH NjYhRomdDOZcsADFNN5vl0IjkskzDB3w3rmdhMhEwVNw0G3no5QmMOPc1HJWjaRazau+Rc6tEqdy md/fUkQhyJGODrAHbG17bU1nnrVba2QOzK2k1Ulk1CqRsAiqLDiBDg0FmWjykbQPo6Fy4ZGBQvta xDp4ABMzu60dnnDoWiYGUSk7aygrIfMsPmkdjkuJaqDVVyBzMi+CpjLs4t5z0gVIhdcDByFeQYxc 9xxi4VGUAAZfklMR4z7K71E+W9SBEBIEyNWLcQ607o7hYJ49CjjlTkLjSKxnmYt1aq4mdKrAAUCJ Kqg0xlspyIme6plc7HiIWJd2kgC1Vmh0nU1vvMxsXrjPBt1p0Fbqadrt1wsmX23bafDvFh/XVxdH 99hh+hKQs6SrnJZt/VplqEwsEJi1xK0KSEjBjQ22NJGZIXL6gl3+ATE8YACku728VPmOoDWj859A fQ/kHLf/T8+gLP1BcBifpQKwsC3zxsCIhGBIgib/cfbeXpfTZQMzOUBIeX9syU0/lbsztofT95ew WNaRCv6B9p+kRtANqach9w8oHLqP7WQX7SlHQTB9/5/n+f6Dz6NDc28N9USkW3Tndk3A1jZqS6mY NFLwOY2lxTSl7IoZrk5i4qv6TOyRw6TWXBKE15cmbAabw4H+g7xl3IEpbXxOQ/qkNbLY8yrm2DcQ t06nDrVWvWWUOQQ7HBWu4oUJKrIOoWGhI/Le5xNbByaSf7h50nmNKyDR0Ok67jQRFkcwhzXkSkdO s9qHTAgOwSobI9zkmg4HCoYFU5kswAa5jrZEyYDo7cm77dhTfR+5GB8CqSjkh4BAepIdiuUT6eJC 84cNb7myUZRh4V1RhrB3ywJEEQqWAyMkhOVzTtk2pqeDwyHU2NAhlOAHbo7g7EcJR3S4jYq2GMUd X31934eZ3eRiEkv3oDtjuPGk9PBxAlvPUjZI39UvUoFoBsshc2IRsO9qlUchU0j4+g+s7v1p9aaV iYEDi1QUWSWgQoKfgjYCx9Aw2KH42FLqH7EDYWALYJb6bWQPzljUjC2QGeu5auyYSJrRNH7RLWLf 0twSuRiHDIMRb2GRZXF1bIlg3uRgGAYgmoBsJrETKL3VxcyZVo5WEZ7Vj3DXPePBbAVn9+orPoLC wm/UfUfgHLYbPW5SYbzIGM/xMyaCgvvJTP7/ilpoMJPjBqL+GH3MJuQmINHNyq9BFrwkDxNJI0GU kVbG+5WBC8MP2Jr3uQOd6oNLqJ+SL8n1SSVHWAGf8AmoZezKV6SElJ9MG6beUp2QktTQcEf1nSrA svsoagux5nXeQhHX9x2dWm8bDBLC+gmqzuCv/TnPeZPva7aZXspznedsAQZtQ1l+l2MUtEME8Ru8 tvGMoY1uS6xbvxFyrjB7iMLhVJJ0qZxMIhRzrQzYCXJlfr61KdZu295TOdFJ2mrynmMZvA9HceBU WAY974GIgPRitOByec4XGMgyV7wttdBYBjJyHYzYzGZ6nbVs645ZTD7yIiI0DOrCjMwEsgfsZ25V 9Jrd7iSpv1OZm9HQ3eiT51H54AGIAlRR5B43dsxDsoTfu5j2GNvM0LlS5UofA9p8Rgewsb6WTE8S XY+MzOz8f7a6iopUPZuwPAVrDixkYadi1EumVHKMSfUe6gd1BiFo5znkGcIq4MWhOkFOAnXtLCnC aCmx1eO4IhQhNokW4tjSggy56O9forcDtEH3Ui0g5IuzwvF8lLyNVw87lsGLolzWVmmhcHjILGCw E+6UJtyAmwKNyFXV1pEVrdQKDE2jf2M4BWXrSCFl6hJyzA55DnLTF+3Niox5z4/WZEeO66jYGVhv hFqeREnzKmCKXIAiYCHrKI6MhgAXFmPOTK6hLBWqSGU0HUSNXA6DadRp55CXG8uMTvag0mcI530T dhzG0qSDtdBq6+kJwBIycTz60DjARGZ5FTz7ib3zp3mfqe1WtuqS58PJtAncX3PCsCQcXKWcI5Qm sLCeCTnWztjJv6FSN1ZqrMzKTWEKGYrKHR1QERiUQgZ4sCe9r7HSimypeRR7/olPTNwszO4QvWbX MQd3LtLv7tbpaRizk76ZNzOFjMoXrDDEIAdbTk2CHuO0uxnn3eY4jCnwGPAGYbp3ePpsLQeOoU5A BqakCC+UQFhaFGU7CmRm4PEsqNrpzC7RbQ5IAzwzroBX0piC1QJaBCJczPSAbEIeqhqMtyUpHszP aW96RoNpOx3OGQWYfCHE0BUuBWsq3oeNOBqZ1Feqzs9m+6oNsG3Uw2aWd98xMIdiXPyBm1fczPHG yLeaWLaWXHhMC7ix3+fRDd7kUTx6EyZxaAKgZIoFrQEkWMJ6DCbTcM9N6b+1YDgBvSDa3PeKk/Hn dwQDBZGS7MSCvNXHP5l+SZUBc2y6RNpAxJrGu5QVG2qgHllItgORKYBgjVQlqgIhyOd1MJeGCt8T ZccchOaI6o+uSSE8u4J47BDs2LIhMWhLlxR5ZGWad6EEBqy8uuJ9fdHhZQFxAcIArV37S8+EKlEB iihCz1OdMaTWFMznc45kp8GwFg9ZCR2AMr5FRCB47kqWGAKsRRFnPO4PMYgGIX8IDzORd8DuPWd5 6TzCTJm0DqPSWHpMY4yRgniVlZ3FhVyJeYqCqZWXOQ4lvnMhuK1fIXEsGog0GsAsbgJKe0Xt7RYF 0K2Qd7B0ucDW860KcGMIMdPU5Oz672AO3HGrU7ws6KeURAxHrgA0KTsS+whMX8CcHF3+lluA4GQA c3ENRAnVBkxHifzP5/lHvPkqb3C4DzMkzgYlKl/5lJCr0uD2gTe7y7RiCAh7yD3UQDRRFAlkB1u+ voAC8rgV8Zvr42vqc5YhYSKAIPxkoc77JIREQhEKR11i4MUcSFLJBIiq57GDMYgCoKuLJZDxltke 9oKZH1GZ6Ow8Hd7kAOmQHpBgnjRJFSEihq8X5Wwk16n7NnkUvYnv+R2eA80Chxle19x1SgSLs/KO h7fjJloBUpo5QCmWwc3A8Hp5vF09+PFBZ9HWSwaaEQ6Qh9vGQJip/rmxQ8R/3qWpAjJqzcqb8/Np jb7+Q8uOVFdVM67xG+zvqXSMjiapKbyb9BA5K1KDhk7uy6UuBEyEITEKwewMz2s9HVkdQGEJleqE OmUiII5tW6pApaIe4+seOFU747XYCcwd70sY4xTuQwYr6+Zt1C66E0cWkxCwZHe89j7XpAvMnlZO 2s0t/PyVuh73LJXo7ahs7wPkvgZAQBKe97X9sohd4Ko7kBTkG3vk1QFuE3HXYvNUybxZvON25RGl PV8wp3haFbnvrgAwbKrYONqXQMKkIMQpDN+c4LkyZ3czz1kERKPhlVDGZmoDLUVsb90tuG4W81pH zLCbxBoLN5Bf4Khw1ctUzy028B8rSFjyP2vKd/MAFXpwxZiVYVfqvFSigxZZDByLRRAGN/LqbbRu SbTaiJ00XTIpRHQmWyBu88Fe10OGXJztbtCNt7atYWb7cbfbsoyHc6wRn7GTY9B5xEey4BJRj3oQ HZs72+JHHN4zMK/H+c7TRwqYXNlZsR8Gf1mKbYftPqAo12elgMB1jXVWMMAxA6vy+FD/3KWrV61I WD6wgl650PsCqd4Xe2Ef342PKGfRl53TOyRo2hpploti0I6VUWs/krJQtNBrahViCBlhcX5HBtDV XAVG3s52JYMqi1ygskIBIrMDkqgIikwlIrIqStAr5GFK4gfkdcEKwtoUghSP4gAyaIdrfLP783eG G5hc2bImBsnxe54cRNrgd5YCMuSZdZrZVmFGJtEA7cBRjYkNMohqlMmxCDGS+tvoaib0hnQ7A62F MsNfskh3RU0dlneyA7wAzFe9qSuxC2lr+OAczkLzkiAIKvLqt0h4Qj6IG9UxEKfI+0HY9CVHDMhu 26H1THJAYjHtglewDAiI9ECtNDnHjxCVayXGn6K1gMguokSewoQbjQ02SJRdYgZqEtAVy28xFeb8 YYhpfY7Hki9hrGD2NyYhjZD9hOnDkpij8CYlbnyyvFMY8bbsLJwBXrIgnZpNYUbu3sIc8LOGROEw xZRQ9rhmC3AKS5MiQF4wHRm8qe+RJ8ZCwAQEC4NRQJlVm7gL9oYLovPMBseOQzqDfiHb8YMXuWsJ 8Ma53ktAnotrbP8JV+83PTueDSe5084Eng7qB6HkfWb2kQdLAdLyAYOA8jJ3Ae6zI7KSIiIjmlKA bbBto2omyIYxplpDupxVSgzcIUfhCIE1T0mvNwt17xx1uylyMowmIvExj2ixCFAvRfqgyxOOGHmq lw8iCxXNVp0gMi2vOHDmCaYtKBWDIYAgT33UuJMukzAdJTex4dVdkIWrieQNZ2tOQgCD95ef0+r5 p/5cnLEwT26G69z+d1gaEwTpamsnFCJqAm236oHHGsrQOkeIpXCfIHU3wQkxlH5ikikikbrI3JHR SZA4wNOuQAfYwhBCJqwNeJJyImS9JqlXVKvzA1ZpDY2HCa2cpi5C7sZMkrS3M9rC/+rqNqkTsgUG jsSgq4UhimWPRFtD04OWTuuphDHJsMkx/xUTjHEp30M32PqH1VlGeRJDLB4LAlo5YV+t1tVefAnC XREORsPtaM87SotA8VqKhdkcyu+xfhQmAmc0giA+Z3bhgA49piLs80ekOSV100HWIGqZi9aVlRnt r4xNcQlsu0kkkkkkkkkk88t4bgN9FkF2sIKIrFA3FoFuJLcilCAk3UaVqTEQFSkqdkB2VGzRKbBk xFAyBvfwLtwhI21hA4mGRdOnIMQ8ypjtDct4F2g2dWq+LMgmzg+RuKuolk3Zg5hStFnvqNzBJiFk bhAhzJkhrTRsdgdTbrkKyXuB0O80hu8ZjCGVwwXlprFVddIVusIaNsKBnN1dDzuxz727xgtozSHC dE3GyWDQyVzBDNYM4LYKI4D42qFyCYS1nRymIIikHNgzDy4uIxDPm2cejHVMYTE0zS0LMV5MKel5 AptLakO7GOEJAniHYEhTckAtJTyDV3QhiLcQ4Rc4od9pCd2jRME/vLsJAsZocAJmwB74ETO2TF+L iGqe4bk6NJ3wOHZId2idWvzPBQQaZwJEK/bxZNXBHMIULvDQvUiPJtMNYziCYpTaPjmcWKIHKGQD RwPbZKJpDqlj5/3Wz53elPc4AYN70usD5XweI6i2HasRIh5PlACsf3MMdiZQ4OiluA9b27+YrdGW m3yRH+1ZcZi60sPQyZW0QiEiA0IChiyPKcw5oaGkp3mPmU7z236WRKudKXJ18rF5QTA0j5wL3e7+ h02zeca8NjFz6GM6YQKG+Xkeo5WD2kABlYA3xUTGo1JXHzA3Pa6hlaY4AcEqge6TGrjYMiIIsm1f L7uojeOhudT4tg03yyuxkBqYymmQGEO97+Gh5LLms2EQxcFL8zS48gJHB+EifU9T5vI0B4vtHMKC sR6nri7mv3uBh6+lCtISJ0aSsGEjLwlNy0OJm0BC2HHvYdXLHlNb4AZntYAyPQL0A2h5ioZhLGGT cYg5WjgUK4ERrdmyxyuR6VpO01/yR1wRVtp1n/i7kinChIeeKu64 --===============1942626995==--