| List: | Commits | « Previous MessageNext Message » | |
| From: | rsomla | Date: | May 11 2007 12:20am |
| Subject: | bk commit into 5.1 tree (rafal:1.2526) | ||
| View as plain text | |||
Below is the list of changes that have just been committed into a local 5.1 repository of rafal. When rafal does a push these changes will be propagated to the main repository and, within 24 hours after the push, to the public repository. For information on how to access the public repository see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html ChangeSet@stripped, 2007-05-11 00:20:29+02:00, rafal@quant.(none) +22 -0 WL#3327: This is a main rewrite of backup kernel meta-data handling code. Also, more documentation is added. The main idea of the meta-data handling framework is to have a specialized classes for handling each meta-data item type. For items of type X there is class meta::X, derived from meta::Item, defining how to create such item and how to save it in the archive. Additionaly there is Archive_info::X_item class derived from Archive_info::Item, which determines how a data item is identified inside backup archive's catalogue. Class Archive_info is used for describing contents of a backup archive. Its specialization Backup_info has methods for adding items to the catalogue. There are provisions for handling item dependencies but general dependencies are not handled yet. However, care is taken to save databases before tables so that they are restored first. Limitations of this code (compared to previous version): - only tables and databased are saved, - SHOW BACKUP ARCHIVE command is not supported, - The summary of BACKUP/RESTORE command doesn't contain info about databases and number of tables in each. BitKeeper/deleted/.del-tables.cc@stripped, 2007-05-10 15:45:10+02:00, rafal@quant.(none) +0 -0 Delete: sql/backup/tables.cc BitKeeper/deleted/.del-tables.h@stripped, 2007-05-10 15:43:20+02:00, rafal@quant.(none) +0 -0 Delete: sql/backup/tables.h sql/backup/Makefile.am@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +1 -2 Tables.{cc,h} are no longer used. Code is moved to archive.cc sql/backup/api_types.h@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +123 -12 - Add file comment. - Add comparison operators to Table_ref and Db_ref. - Introduce concept of invalid table/db reference. sql/backup/archive.cc@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +599 -22 - Code for writing/reading archive header and catalogue now moved here to Archive_info::{save,read} methods. - Format in which meta-data items are saved defined by Archive_info::Item::save() method. Archive_info::Item::create_from_stream() is an item reading counterpart. - Implementation of Image_info::Tables class moved here (previously in tables.cc). sql/backup/archive.h@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +283 -182 - Change of terminology: Image -> Archive, Subimage -> Image. - Replace Backup_subimage and Restore_subimage classes by a single Image_info class. - Class Image_info and parts of what was before in Backup_image_info and Restore_image_info is now turned into a single Archive_info class. - List of archive's images is now stored in an array (with a fixed limit on number of images - currently 256). - Class Archive_info::Item and its subclasses defined for storing a list of archive's meta-data items. - The data type used to store list of tables inside Image_info object is now defined inside the Image_info class (Image_info::Tables). Before, Tables class was defined in a separate tables.{h,cc} files. - Previous Native_{backup,restore}_info classes implemented as a single Native_image class. sql/backup/backup_aux.h@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +19 -1 Add helper functions declarations. sql/backup/backup_engine.h@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +5 -83 Move definition of Buffer class to api_types.h. sql/backup/backup_kernel.h@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +187 -10 Add declarations for functions and data-structures forming the backup kernel API. sql/backup/be_default.cc@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +39 -36 - Move description of the default backup driver here. - Rename class describing a default engine image to Default_image and move its definition to be_default.cc sql/backup/be_default.h@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +34 -77 - Move description of the default backup to be_default.cc. - Add implementation of Default_image class. sql/backup/be_nodata.cc@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +6 -0 - Add file description. sql/backup/be_nodata.h@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +6 -0 - Add file description. sql/backup/data_backup.cc@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +431 -269 - Added documentation. - Move method implementations outside classes. - Update code to accomodate for changes in class definitions. sql/backup/meta_backup.cc@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +241 -339 - Functions for writing/reading archive header and catalogue are moved to the methods of Archive_info class. - The meta-data is saved in a separate section of backup archive using methods of meta::Item and derived classes. The complete format of an entry is determined by Archive_info::Item::save() method and methods defined in derived classes. - The order in which meta-data items are saved is determined inside the Backup_info instance (and can be easily changed). - When restoring tables (in the future other per-database items) it is not assumed that theyr names are qualified with database name. Instead, the current database is set accordingly when creating these items. This makes it easier to restore such items to a database with different name. - Apart from tables and databases, other meta-data items which were supprted in Chucks code are not yet implemented into this framework. sql/backup/meta_backup.h@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +118 -0 Define meta::Item class and its specializations for tables and databases. sql/backup/meta_backup.h@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +0 -0 sql/backup/sql_backup.cc@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +904 -481 - Add new execute_backup_command() function called from the parser and calling other functions to perform correct operation. - Functions do_backup(), do_restore() are replaced by mysql_backup() and mysql_restore(). Part of the functionality (opening stream, reading image info) is moved into execute_backup_commad(). - Collecting databases and tables to be bcked-up is now implemented by methods of the Backup_info class. - Constructing CREATE statements is now delegated to class meta::Item and its subclasses. Similar for code creating and deleting meta-data items. - Change interface to send_summary() function -- now it gets all information from Archive_info instance. sql/backup/stream.h@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +9 -4 Cosmetic changes. sql/backup/string_pool.cc@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +36 -37 Replace operator<< and operator>> with StringPool::save(), StringPool::read() functions. sql/backup/string_pool.h@stripped, 2007-05-11 00:20:25+02:00, rafal@quant.(none) +3 -2 Replace operator<< and operator>> with StringPool::save(), StringPool::read() functions. sql/sql_parse.cc@stripped, 2007-05-11 00:20:23+02:00, rafal@quant.(none) +22 -61 Move all code related to backup/restore statement execution to execute_backup_command() function defined in backup/sql_backup.cc. sql/sql_yacc.yy@stripped, 2007-05-11 00:20:24+02:00, rafal@quant.(none) +1 -9 Represent '*' wildcard by an empty Lex->db_list. # This is a BitKeeper patch. What follows are the unified diffs for the # set of deltas contained in the patch. The rest of the patch, the part # that BitKeeper cares about, is below these diffs. # User: rafal # Host: quant.(none) # Root: /ext/mysql/bk/backup/prototype --- 1.635/sql/sql_parse.cc 2007-05-11 00:20:36 +02:00 +++ 1.636/sql/sql_parse.cc 2007-05-11 00:20:36 +02:00 @@ -28,11 +28,10 @@ #include "events.h" #include "event_data_objects.h" -#ifndef EMBEDDED_LIBRARY -#include "backup/backup_kernel.h" -#endif #include "sql_trigger.h" +int execute_backup_command(THD*, LEX*); // defined in backup/sql_backup.cc + /* Used in error handling only */ #define SP_TYPE_STRING(LP) \ ((LP)->sphead->m_type == TYPE_ENUM_FUNCTION ? "FUNCTION" : "PROCEDURE") @@ -1856,62 +1855,24 @@ break; } #endif - case SQLCOM_SHOW_ARCHIVE: - { - backup::Location *loc; - LEX_STRING where = lex->backup_dir; - loc= backup::Location::find(where, TRUE); - if (!loc) - goto error; // TODO: report something meaningfull - backup::show_archive_catalog(thd,*loc); - loc->free(); - break; - } + case SQLCOM_SHOW_ARCHIVE: case SQLCOM_BACKUP: case SQLCOM_RESTORE: { #ifndef EMBEDDED_LIBRARY - backup::Location *loc; - - // TODO: make sanity checks on parameters. - - LEX_STRING where = lex->backup_dir; - if (lex->sql_command==SQLCOM_BACKUP) - loc= backup::Location::find(where, FALSE); - else - loc= backup::Location::find(where, TRUE); - if (!loc) - goto error; // TODO: report something meaningfull - if (lex->sql_command==SQLCOM_BACKUP) - backup::do_backup(thd,*loc, &lex->db_list); - else - backup::do_restore(thd,*loc); - loc->free(); - + /* + Sure, it adds one extra level of indirection (one more switch + inside execute_backup_command) but fortunatelly backup/restore + statements are not time critical and we can isolate all related + code into the backup/ source dir. + */ + execute_backup_command(thd,lex); + // TODO: error detection and reporting #endif -/* - DBUG_ASSERT(first_table == all_tables && first_table != 0); - if (check_table_access(thd, SELECT_ACL, all_tables, 0) || - check_global_access(thd, FILE_ACL)) - goto error; */ /* purecov: inspected */ -/* thd->enable_slow_log= opt_log_slow_admin_statements; - res = mysql_backup_table(thd, first_table); - select_lex->table_list.first= (byte*) first_table; - lex->query_tables=all_tables; -*/ -/* DBUG_ASSERT(first_table == all_tables && first_table != 0); - if (check_table_access(thd, INSERT_ACL, all_tables, 0) || - check_global_access(thd, FILE_ACL)) - goto error; -*/ /* purecov: inspected */ -/* thd->enable_slow_log= opt_log_slow_admin_statements; - res = mysql_restore_table(thd, first_table); - select_lex->table_list.first= (byte*) first_table; - lex->query_tables=all_tables; -*/ break; } + case SQLCOM_ASSIGN_TO_KEYCACHE: { DBUG_ASSERT(first_table == all_tables && first_table != 0); @@ -5215,20 +5176,20 @@ { #ifndef NO_EMBEDDED_ACCESS_CHECKS if (mqh_used && thd->user_connect && - check_mqh(thd, lex->sql_command)) + check_mqh(thd, lex->sql_command)) { - thd->net.error = 0; + thd->net.error = 0; } else #endif { - if (thd->net.report_error) - { + if (thd->net.report_error) + { delete lex->sphead; lex->sphead= NULL; - } - else - { + } + else + { /* Binlog logs a string starting from thd->query and having length thd->query_length; so we set thd->query_length correctly (to not @@ -5243,9 +5204,9 @@ (thd->query_length= (ulong)(lex->found_semicolon - thd->query))) thd->query_length--; /* Actually execute the query */ - mysql_execute_command(thd); - query_cache_end_of_result(thd); - } + mysql_execute_command(thd); + query_cache_end_of_result(thd); + } } lex->unit.cleanup(); } --- 1.547/sql/sql_yacc.yy 2007-05-11 00:20:37 +02:00 +++ 1.548/sql/sql_yacc.yy 2007-05-11 00:20:37 +02:00 @@ -5605,15 +5605,7 @@ database_list: '*' - { - LEX_STRING wild_buff; - if (!(wild_buff.str= alloc_root(YYTHD->mem_root, 3))) - YYABORT; - strmake(wild_buff.str, "*", 1); - if (Lex->db_list.push_back((LEX_STRING*) - sql_memdup(&wild_buff, sizeof(LEX_STRING)))) - YYABORT; - } + {} | ident { if (Lex->db_list.push_back((LEX_STRING*) --- 1.15/sql/backup/be_default.cc 2007-05-11 00:20:37 +02:00 +++ 1.16/sql/backup/be_default.cc 2007-05-11 00:20:37 +02:00 @@ -14,11 +14,46 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */ -#include "mysql_priv.h" -#include "backup_engine.h" -#include "tables.h" // for build_table_list() -#include "be_default.h" +/** + * @file + * + * @brief Implements the default backup engine. + * + * This file contains the default backup algorithm (also called a "driver" + * in the online backup terminology. The default backup algorithm may be + * used in place of an engine-specific driver if one does not exist or if + * chosen by the user. + * + * The default backup algorithm is a blocking algorithm that locks all of + * the tables given at the start of the backup/restore process. Once all of + * the data is backed up or restored, the locks are removed. The default + * backup is a row-level backup and therefore does not backup the indexes + * or any of the engine-specific files. + * + * The classes in this file use the namespace "default_backup" to distinguish + * these classes from other backup drivers. The backup functionality is + * contained in the backup class shown below. Similarly, the restore + * functionality is contained in the restore class below. + * + * The format of the backup is written as a series of data blocks where the + * first block contains the record buffer (table->record[0]). This is + * followed by one or more blocks containing the blob (text) fields one per + * block. This sequence is repeated for each row in the table. + * <pre> + * blknum contents + * ------------------------------------------------------ + * 1 (byte) copy record_buffer read from handler + * ------------------------------------------------------ + * 2...n (uint32) size of blob data to read + * (byte) blob data for blob + * ------------------------------------------------------ + * </pre> + */ + +#include "../mysql_priv.h" + #include "backup_aux.h" +#include "be_default.h" /* TODO list: @@ -704,35 +739,3 @@ } /* default_backup namespace */ - -/********************************************************************* - - Classes describing backup image created using default backup engine. - - *********************************************************************/ - -namespace backup { - -bool Default_backup_info::add(const Table_ref &tbl, const ::handlerton*) -{ - // accept any table ... - DBUG_PRINT("backup",("adding table %s to default image",tbl.name().ptr())); - tables.add(tbl); - return TRUE; -} - - -bool Default_backup_info::do_write_description(OStream &) -{ - // no extra information needed -// return(false); - return TRUE; -} - -Default_restore_info* -Default_restore_info::create_from_stream(uint no, version_t ver, IStream &) -{ - return new Default_restore_info(no, ver); -} - -} // backup namespace --- 1.26/sql/backup/sql_backup.cc 2007-05-11 00:20:37 +02:00 +++ 1.27/sql/backup/sql_backup.cc 2007-05-11 00:20:37 +02:00 @@ -1,354 +1,485 @@ +/** + @file + + Implementation of the backup kernel API. + */ + +/* + TODO: + + - Handle SHOW BACKUP ARCHIVE command. + - Add database list to the backup/restore summary. + - Implement meta-data freeze during backup/restore. + - Error feedback to client. + - Improve logic selecting backup driver for a table. + - Handle other types of meta-data in Backup_info methods. + - Handle item dependencied when adding new items. + - Handle other kinds of backup locations (far future). + */ + #include "../mysql_priv.h" -#include "archive.h" -#include "backup_kernel.h" +#include "backup_aux.h" #include "stream.h" +#include "backup_kernel.h" +#include "meta_backup.h" +#include "archive.h" #include "be_default.h" -#include "backup_aux.h" -//#define TEST yes +#define TEST yes //int sortcmp(const String *s,const String *t, CHARSET_INFO *cs); namespace backup { +// TODO: handle charsets consistently CHARSET_INFO *default_charset= table_alias_charset; -/************************************************* - * - * BACKUP - * - *************************************************/ +IStream* open_for_read(const Location&); +OStream* open_for_write(const Location&); -// FIXME -#define DBUG_ERROR(E) { my_error(ER_NO,MYF(0),"ala","ala"); \ - DBUG_RETURN(1); } +bool send_summary(THD*,const Backup_info&); +bool send_summary(THD*,const Restore_info&); +} -// Returns tmp table containing records from a given I_S table -TABLE* get_schema_table(THD *thd, ST_SCHEMA_TABLE *st); +/** + Call backup kernel API to execute backup related SQL statement. -IStream* open_for_read(const Location&); -OStream* open_for_write(const Location&); -bool collect_tables_to_backup(THD *,Backup_info &); -bool collect_databases_to_backup(THD *thd, Backup_info &info); -bool write_header(const Backup_info&,OStream&); -bool write_catalog(const Backup_info&,OStream&); -bool write_table_data(const Backup_info&,OStream&); -bool get_table_metadata(THD *thd, TABLE_LIST *tbl, - List<String> *buffer); -bool get_db_metadata(THD *thd, const char *db, - List<String> *buffer); -bool send_summary(List<schema_ref> *catalog, bool backup, - size_t header_size, size_t data_size); - -/* - Implements BACKUP command -- called from sql_parse.cc -*/ + @param lex results of parsing the statement. + */ -int do_backup(THD *thd, const Location &loc, List <LEX_STRING> *db_list) +int +execute_backup_command(THD *thd, LEX *lex) { - DBUG_ENTER("backup::do_backup"); - size_t header_size= 0; - size_t data_size= 0; + DBUG_ENTER("execute_backup_command"); + DBUG_ASSERT(thd && lex); - bool stream_opened= FALSE; + backup::Location *loc= backup::Location::find(lex->backup_dir); - // Open backup stream... + if (!loc) + DBUG_RETURN(-1); - OStream *backup_stream= open_for_write(loc); + int res= -1; - if(!backup_stream) - DBUG_RETURN(backup::ERROR); + switch (lex->sql_command) { - stream_opened= TRUE; + case SQLCOM_SHOW_ARCHIVE: + case SQLCOM_RESTORE: + { + backup::IStream *stream= backup::open_for_read(*loc); + + if (!stream) + DBUG_RETURN(-2); + + backup::Restore_info info(*stream); + + if (lex->sql_command == SQLCOM_SHOW_ARCHIVE) + { +# ifndef TEST + res= mysql_show_archive(thd,info); +# endif + ::send_ok(thd); + } + else + { + // TODO: freeze all DDL operations - Backup_info info; - info.db_list= db_list; - List_iterator<LEX_STRING> db_names(*info.db_list); - LEX_STRING *dbstr= db_names++; - if (!my_strcasecmp(system_charset_info, - dbstr->str, "*")) + info.restore_all_dbs(); + res= mysql_restore(thd,info,*stream); + + info.total_size += info.header_size; + // TODO: unfreeze DDL + + send_summary(thd,info); + } + + stream->close(); + break; + } + + case SQLCOM_BACKUP: { - info.db_list->empty(); - collect_databases_to_backup(thd, info); + backup::OStream *stream= backup::open_for_write(*loc); + + if (!stream) + DBUG_RETURN(-2); + + backup::Backup_info info(thd); + + // TODO: freeze all DDL operations + + if (lex->db_list.is_empty()) + info.add_all_dbs(); // backup all databases + else + info.add_dbs(lex->db_list); // backup databases specified by user + + info.close(); + + res= mysql_backup(thd,info,*stream); + + // TODO: unfreeze DDL + + send_summary(thd,info); + + // TODO: report errors to the client + + stream->close(); + + backup::IStream *is= backup::open_for_read(*loc); + if (is) + { + dump_stream(*is); + is->close(); + } + + break; } - if(!collect_tables_to_backup(thd,info)) - goto error; + default: DBUG_ASSERT(FALSE); + } - DBUG_PRINT("backup",("Writing image header")); + loc->free(); - backup_stream->bytes= 0; // must initialize to 0 + //::send_ok(thd); + DBUG_RETURN(res); +} - write_header(info,*backup_stream); +/************************************************* + * + * BACKUP + * + *************************************************/ - DBUG_PRINT("backup",("Writing image catalog")); - write_catalog(info,*backup_stream); +// Declarations for functions used in backup operation - header_size= backup_stream->bytes; +namespace backup { - DBUG_PRINT("backup",("Writing backup images")); +// defined in meta_backup.cc +bool write_meta_data(THD*, Backup_info&, OStream&); - if(!write_table_data(info,*backup_stream)) - goto error; +// defined in data_backup.cc +bool write_table_data(THD*, Backup_info&, OStream&); + +} // backup namespace + + +/** + Create backup archive. +*/ + +int mysql_backup(THD *thd, + backup::Backup_info &info, + backup::OStream &s) +{ + DBUG_ENTER("mysql_backup"); + + DBUG_ASSERT(info.is_valid()); - data_size= backup_stream->bytes -header_size; + size_t start_bytes= s.bytes; + + DBUG_PRINT("backup",("Writing image header")); - DBUG_PRINT("backup",("Backup image written")); + info.save(s); - // send feedback to client - send_summary(&info.catalog, TRUE, header_size, data_size); + DBUG_PRINT("backup",("Writing meta-data")); + + backup::write_meta_data(thd,info,s); + + DBUG_PRINT("backup",("Writing table")); + + if(!backup::write_table_data(thd,info,s)) + goto error; + + DBUG_PRINT("backup",("Backup done.")); + + info.total_size= s.bytes - start_bytes; goto end; error: DBUG_PRINT("backup",("Error creating backup image")); - // FIXME: report errors to the client end: - if( stream_opened ) - backup_stream->close(); - - //send_ok(thd); DBUG_RETURN(0); } /* - Send a summary of the backup/restore operation to the client. + Logic for selecting image format for a given table location: - SYNOPSIS - send_summary() - List<schema_ref> *catalog - The archive catalog - bool backup - TRUE = backup, FALSE = restore - size_t header_size - size in bytes of header read/written - size_t data_size - size in bytes of data read/written + 1. If tables from that location have been already stored in one of the + sub-images then choose that subimage. - DESCRIPTION - This procedure sends a condensed summary to the client detailing - the number of tables in each database backed up or restored and - the number of bytes written/read. + 2. If location has "native" backup format, put it in a new sub-image with + that format. - RETURNS - 0 - no errors. - -1 - error. -*/ -bool send_summary(List<schema_ref> *catalog, bool backup, - size_t header_size, size_t data_size) + 3. Otherwise check if one of the existing sub-images would accept table from + this location. + + 4. When everything else fails, use default backup format. + */ + +namespace backup { + + +class Backup_info::Table_ref: + public backup::Table_ref { - Protocol *protocol= ::current_thd->protocol; // client comms - List<Item> field_list; // list of fields to send - Item *item; // field item - char buf[255]; // buffer for data - String op_str; // operations string - String print_str; // string to print + String m_db_name; + String m_name; + const ::handlerton *m_hton; + + public: + + Table_ref(TABLE*); + const ::handlerton *hton() const + { return m_hton; } +}; - DBUG_ENTER("backup::sending summary"); - op_str.length(0); - if (backup) - op_str.append("Backup Summary"); - else - op_str.append("Restore Summary"); +int Backup_info::find_image(const Backup_info::Table_ref &tbl) +{ + DBUG_ENTER("Backup_info::find_image"); - DBUG_PRINT("backup", ("sending %s", op_str.c_ptr())); + const ::handlerton *hton= tbl.hton(); + Image_info *img; - field_list.push_back(item= new Item_empty_string(op_str.c_ptr(),255)); // TODO: use varchar field - item->maybe_null= TRUE; - protocol->send_fields(&field_list, Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF); + DBUG_ASSERT(hton); - size_t total_size= header_size + data_size; + DBUG_PRINT("backup",("Adding table %s.%s (%s,%p) to archive.", + tbl.db().name().ptr(), + tbl.name().ptr(), + ::ha_resolve_storage_engine_name(hton), + hton->get_backup_engine) ); - /* - Loop through the catalog and list the contents by - database. - */ - List<schema_ref> cat1= *catalog; - List_iterator<schema_ref> cat(cat1); - schema_ref *tmp; + #ifndef USE_DEFAULT_BACKUP - while ((tmp= cat++)) - { - uint howmuch= tmp->tables.elements; + uint last_image_no= images[0]? img_count-1 : img_count; - my_snprintf(buf,sizeof(buf),"%s %3u %s in database %s.", - backup? "Backed up" : "Restored", - howmuch, howmuch > 1 ? "tables" : " table", - tmp->db_name.c_ptr()); + // Point 3: try existing images but not the default one. - protocol->prepare_for_resend(); - protocol->store(buf,system_charset_info); - protocol->write(); - } + for (uint no=1; no <= last_image_no && no < 256 ; ++no) + { + img= images[no]; - my_snprintf(buf,sizeof(buf)," "); - protocol->prepare_for_resend(); - protocol->store(buf,system_charset_info); - protocol->write(); + // An image object decides if it can handle given table or not + if( img && img->accept(tbl,hton) ) + DBUG_RETURN(no); + } - my_snprintf(buf,sizeof(buf)," header = %-8lu bytes",(unsigned long)header_size); - protocol->prepare_for_resend(); - protocol->store(buf,system_charset_info); - protocol->write(); + // Point 2: try native backup of table's storage engine. - my_snprintf(buf,sizeof(buf)," data = %-8lu bytes",(unsigned long)data_size); - protocol->prepare_for_resend(); - protocol->store(buf,system_charset_info); - protocol->write(); + if (hton->get_backup_engine) + { + uint no= ++last_image_no; - my_snprintf(buf,sizeof(buf)," --------------"); - protocol->prepare_for_resend(); - protocol->store(buf,system_charset_info); - protocol->write(); + DBUG_ASSERT(no<256); - my_snprintf(buf,sizeof(buf)," total %-8lu bytes", (unsigned long)total_size); - protocol->prepare_for_resend(); - protocol->store(buf,system_charset_info); - protocol->write(); + img= images[no]= new Native_image(*this,hton); - send_eof(::current_thd); - DBUG_RETURN(0); + DBUG_ASSERT(images[no]); + DBUG_PRINT("backup",("%s image added to archive",img->name())); + img_count++; + + // native image should accept all tables from its own engine + bool res= img->accept(tbl,hton); + DBUG_ASSERT(res); + + DBUG_RETURN(no); + } + + #endif + + // Point 4: try default backup engine. + + if (!images[0]) + { + img= images[0]= new Default_image(*this); + DBUG_ASSERT(img); + DBUG_PRINT("backup",("Default image added to archive")); + img_count++; + } + + if (img->accept(tbl,hton)) + DBUG_RETURN(0); + + DBUG_PRINT("backup",("Table ignored.")); + DBUG_RETURN(-1); } -// Collect tables to be backed-up and fill Backup_info structure describing backup image. -// Currently we try to backup all tables in 'test' database (but only tables for which -// appropriate backup engine is found will be included in the image). -bool collect_tables_to_backup(THD *thd,Backup_info &info) +} // backup namespace + + +/**************************** + + Backup_info implementation + + ****************************/ + +namespace backup { + +// Returns tmp table containing records from a given I_S table +TABLE* get_schema_table(THD *thd, ST_SCHEMA_TABLE *st); + +/** + Create @c Backup_info structure and prepare it for populationg with meta-data + items. + + When adding a complete database to the archive, all its tables are added. + These are found by reading INFORMATION_SCHEMA.TABLES table. The table is + opened here so that it is ready for use in @c add_db_items() method. It is + closed when the structure is closed with the @c close() method. + */ +Backup_info::Backup_info(THD *thd): + m_thd(thd), i_s_tables(NULL), /* m_tables(NULL), */ m_state(INIT), + m_items(NULL), m_last_item(NULL), m_last_db(NULL) { - LEX_STRING *tmp; - schema_ref *cat; - String db_name; + i_s_tables= get_schema_table(m_thd, ::get_schema_table(SCH_TABLES)); +} - DBUG_ENTER("backup::collect_tables_to_backup"); +Backup_info::~Backup_info() +{ + close(); + //close_tables(); + m_state= DONE; - // open and scan I_S.TABLES table + Item_iterator it(*this); + Item *i; - TABLE *i_s_tables = get_schema_table(thd, ::get_schema_table(SCH_TABLES) ); + while ((i=it++)) + delete i; - if( !i_s_tables ) - { - DBUG_PRINT("backup",("get_schema_table returned NULL!")); - DBUG_ERROR(ERROR); - } + m_items= NULL; +} - ::handler *ha = i_s_tables->file; +/** + Close the structure after populating it with items. - // TODO: locking + When meta-data is written, we need to open archive's tables to be able + to read their definitions. Tables are opened here so that the + structure is ready for creation of the archive. - /* - Build catalog for writing... - For each database in the db_list, - create a new catalog struct - fill with tables and their metadata - add catalog to list - */ - List_iterator<LEX_STRING> databases(*info.db_list); + FIXME: opening all tables at once consumes too much resources when there + is a lot of tables (opened file descriptors!). Find a better solution. + */ +bool Backup_info::close() +{ + if(i_s_tables) + ::free_tmp_table(m_thd,i_s_tables); + i_s_tables= NULL; - table_info *ti; - while ((tmp= databases++)) + if (m_state == INIT) + m_state= READY; + + return TRUE; //open_tables(); +} + +/** + Specialization of @c Db_ref which takes db name from @c LEX_STRING structure + or reads it from INFORMATION_SCHEMA.SCHEMATA table. + */ + +class Backup_info::Db_ref: public backup::Db_ref +{ + String m_name; + + public: + + Db_ref(const LEX_STRING &s): + backup::Db_ref(m_name), + m_name(s.str,s.length,::system_charset_info) + {} + + Db_ref(TABLE *t): backup::Db_ref(m_name) { - /* - Save this database in the catalog along with metadata - */ - cat= new schema_ref(); // create a new catalog struct - //cat->db_name= new (current_thd->mem_root) String(); - cat->db_name.length(0); - cat->db_name.append(tmp->str); // save the database name + t->field[1]->val_str(&m_name); + } +}; - DBUG_PRINT("backup", ("Creating database metadata.")); - get_db_metadata(thd, cat->db_name.c_ptr(), &cat->db_metadata); - DBUG_PRINT("backup", ("Database metadata created.")); - ha->ha_rnd_init(TRUE); +/** + Add to archive all databases in the list. + */ +bool Backup_info::add_dbs(List<LEX_STRING> &dbs) +{ + List_iterator<LEX_STRING> it(dbs); + LEX_STRING *s; - // TODO: use conditions, do it properly - while( ! ha->rnd_next(i_s_tables->record[0]) ) + while ((s= it++)) + { + Db_ref db(*s); + + if (db_exists(db)) { - Table tbl(i_s_tables); - db_name= tbl.db().name(); - if (tbl && !my_strcasecmp(system_charset_info, - db_name.c_ptr(), tmp->str) && - my_strcasecmp(system_charset_info, - db_name.c_ptr(), "information_schema") && - my_strcasecmp(system_charset_info, - db_name.c_ptr(), "mysql")) - { - DBUG_PRINT("backup", ("Found table %s for database %s", - tbl.name().ptr(), tmp->str)); - // Backup_info::add method selects/creates sub-image appropriate for storing given table - info.add(tbl,tbl.hton()); - /* - Add the table to this database's table list - */ - ti= new table_info(); - ti->table_name.length(0); - ti->table_name.append(tbl.name()); - /* - Get the metadata for the table - */ - TABLE_LIST *ptr= - build_table_list_str((char *)tbl.name().ptr(), db_name.c_ptr(), TL_READ); - get_table_metadata(thd, ptr, &ti->table_metadata); - /* - Save the table_info to the tables list for this database - */ - cat->tables.push_back(ti); - } - } - info.catalog.push_back(cat); //add this catalog to the list of catalogs - } + Db_item *it= add_db(db); - DBUG_PRINT("backup", ("No more tables in I_S")); + if (!it) + return FALSE; - ha->ha_rnd_end(); + bool res= add_db_items(*it); - ::free_tmp_table(thd,i_s_tables); + if (!res) + return FALSE; + } + else + DBUG_PRINT("backup",("warning: skipping unknown database %s",s->str)); + } - DBUG_RETURN(TRUE); + return TRUE; } -bool collect_databases_to_backup(THD *thd, Backup_info &info) -{ - LEX_STRING *tmp; - String db_name; - - DBUG_ENTER("backup::collect_databases_to_backup"); +static const LEX_STRING is_string = { C_STRING_WITH_LEN("information_schema") }; +static const LEX_STRING mysql_string = { C_STRING_WITH_LEN("mysql") }; - // open and scan I_S.TABLES table +/** + Add to archive all instance's databases (except the internal ones). + */ - TABLE *db_table = get_schema_table(thd, ::get_schema_table(SCH_SCHEMATA)); +bool Backup_info::add_all_dbs() +{ + ::TABLE *db_table = get_schema_table(m_thd, ::get_schema_table(SCH_SCHEMATA)); if (!db_table) { - DBUG_PRINT("backup",("get_schema_table returned NULL!")); - DBUG_ERROR(ERROR); + DBUG_PRINT("backup",("Can't open I_S.SCHEMATA table!")); + return FALSE; } ::handler *ha = db_table->file; // TODO: locking + DBUG_PRINT("backup", ("Adding all databases to the archive.")); + + const Db_ref is_db(is_string); + const Db_ref mysql_db(mysql_string); + ha->ha_rnd_init(TRUE); - // TODO: use conditions, do it properly while (!ha->rnd_next(db_table->record[0])) { - db_name.length(0); - db_table->field[1]->val_str(&db_name); - if (my_strcasecmp(system_charset_info, - db_name.c_ptr(), "information_schema") && - my_strcasecmp(system_charset_info, - db_name.c_ptr(), "mysql")) + Db_ref db(db_table); + + if (db==is_db || db==mysql_db) { - DBUG_PRINT("backup", ("Found database %s", db_name.ptr())); - tmp= (LEX_STRING *)thd->alloc(sizeof(LEX_STRING)); - tmp->str= my_malloc(MAX_ALIAS_NAME, MYF(MY_WME)); - strcpy(tmp->str, db_name.c_ptr()); - info.db_list->push_back(tmp); + DBUG_PRINT("backup",(" Skipping internal database %s",db.name().ptr())); + continue; } + + DBUG_PRINT("backup", (" Found database %s", db.name().ptr())); + + Db_item *it= add_db(db); + + if (!it) + return FALSE; + + bool res= add_db_items(*it); + + if (!res) + return FALSE; } DBUG_PRINT("backup", ("No more databases in I_S")); @@ -356,278 +487,198 @@ ha->ha_rnd_end(); db_table->file->close(); - DBUG_RETURN(TRUE); -} - -/* - Logic for selecting sub-image format for a given location: - 1. If tables from that location have been already stored in one of the - sub-images then choose that subimage. - - 2. If location has "native" backup format, put it in a new sub-image with - that format. - - 3. Otherwise check if one of the existing sub-images would accept table from - this location. + return TRUE; +} - 4. When everything else fails, use default backup format. +/** + Insert a @c Db_item into the meta-data items list. */ - -bool Backup_info::add(const Table_ref &tbl, const ::handlerton *hton) +Backup_info::Db_item* +Backup_info::add_db(const backup::Db_ref &db) { - DBUG_ASSERT(hton); + StringPool::Key key= db_names.add(db.name()); + Db_item *di= new Db_item(*this,key); -/* - DBUG_PRINT("backup",("Adding table %s.%s (%s,%p) to archive.", - tbl.db().name().ptr(), - tbl.name().ptr(), - ::ha_resolve_storage_engine_name(hton), - hton->get_backup_engine) ); -*/ - Backup_subimage *img; - - #ifdef USE_DEFAULT_BACKUP - - // Point 4 + // insert db item before all table items - img = images.head(); - - if (!img) - { - img= new Default_backup_info(img_count++); - images.push_back(img); - } - - DBUG_ASSERT(img); - - return img->add(tbl,hton); + if (!m_last_db) + { + di->next= m_items; + m_items= m_last_db= di; + if (!m_last_item) + m_last_item= m_items; + } + else + { + DBUG_ASSERT(m_items); - #else + di->next= m_last_db->next; + if (m_last_item == m_last_db) + m_last_item= di; + m_last_db->next= di; + } - // Point 3 + return di; +} - List_iterator<Backup_subimage> it(images); +/** + Add to archive all items belonging to a given database. + */ +bool Backup_info::add_db_items(Db_item &db) +{ + DBUG_ASSERT(is_valid()); - while( (img= it++) ) - { - // A sub-image decides if it can handle given table or not - if( img != default_image && img->add(tbl,hton) ) - { - table_count++; - return TRUE; - } - } + // TODO: add other items (views, stored routines,...) - // Point 2 + // add all tables from db. - if (hton->get_backup_engine) - { - bool res; + ::handler *ha= i_s_tables->file; - img= new Native_backup_info(img_count++,hton); - DBUG_ASSERT(img); - images.push_back(img); - DBUG_PRINT("backup",("%s image added to archive",img->name())); + // TODO: do it properly: use SELECT conditions - res= img->add(tbl,hton); - DBUG_ASSERT(res); // native image should accept all tables from its own engine + ha->ha_rnd_init(TRUE); - table_count++; + while (!ha->rnd_next(i_s_tables->record[0])) + { + const Backup_info::Table_ref t(i_s_tables); - return TRUE; - } + if( t.db() == db ) + { + DBUG_PRINT("backup", ("Found table %s for database %s", + t.name().ptr(), t.db().name().ptr())); - if (!default_image) - { - default_image= new Default_backup_info(img_count++); - DBUG_ASSERT(default_image); - images.push_back(default_image); - DBUG_PRINT("backup",("Default image added to archive")); - } + // add_table method selects/creates sub-image appropriate for storing given table + Table_item *ti= add_table(db,t); + DBUG_ASSERT(ti); - if( default_image->add(tbl,hton) ) - { - table_count++; - return TRUE; - } + add_table_items(*ti); + } + } - DBUG_PRINT("backup",("Table ignored.")); - return FALSE; + ha->ha_rnd_end(); - #endif + return TRUE; } -} // backup namespcae +/// Gets table identity from a row of I_S.TABLES table -/************************************************* - * - * RESTORE - * - *************************************************/ +Backup_info::Table_ref::Table_ref(TABLE *t): + backup::Table_ref(m_db_name,m_name) +{ + String engine_name; -namespace backup { + t->field[1]->val_str(&m_db_name); + t->field[2]->val_str(&m_name); + t->field[4]->val_str(&engine_name); -bool read_header(IStream&,Restore_info&); -bool read_catalog(IStream&,Restore_info&); -bool restore_metadata(THD *thd, const Restore_info &info); -bool restore_table_data(const Restore_info&,IStream&); + LEX_STRING name_lex; -/* - Get the metadata for the backup file specified and display - it to the client interface. + name_lex.str= const_cast<char*>(engine_name.ptr()); + name_lex.length= engine_name.length(); - SYNOPSIS - show_archive_catalog() - THD *thd - The current thread instance. - Location loc - The name of the archive with path. + m_hton= ::ha_resolve_by_name(::current_thd,&name_lex); + DBUG_ASSERT(m_hton); +} - DESCRIPTION - This procedure opens the archive and extracts the metadata - for the databases and tables stored in the archive. A simple - list of the tables by database is sent to the client. +/** + Add @c Table_item to archive's list of meta-data items. + */ - RETURNS - 0 - no errors. - -1 - database cannot be read or doesn't exist. -*/ -int show_archive_catalog(THD *thd, const Location &loc) +Backup_info::Table_item* +Backup_info::add_table(Db_item &db, + const Table_ref &t) { - Protocol *protocol= thd->protocol; - List<Item> field_list; - Item *item; + int no= find_image(t); - DBUG_ENTER("backup::show_archive_catalog"); - field_list.push_back(item= new Item_empty_string("Database",255)); - field_list.push_back(item= new Item_empty_string("Table",255)); - field_list.push_back(item= new Item_empty_string("Metadata",255)); - item->maybe_null= TRUE; - protocol->send_fields(&field_list, Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF); - Restore_info info; + DBUG_ASSERT(no >= 0 && images[no]); - IStream *backup_str= open_for_read(loc); + int tno= images[no]->tables.add(t); + DBUG_ASSERT(tno >= 0); + table_count++; - if (!backup_str) - DBUG_RETURN(-1); + Table_item *ti= new Table_item(*this,no,tno); - // for debug purposes -- grep for 'stream: ' in server trace - dump_stream(*backup_str); + // add to the end of items list - backup_str->rewind(); + ti->next= NULL; + if (m_last_item) + m_last_item->next= ti; + m_last_item= ti; + if (!m_items) + m_items= ti; - bool res= read_header(*backup_str,info); - DBUG_ASSERT(res); + return ti; +} - res= read_catalog(*backup_str,info); - DBUG_ASSERT(res); +/** + Add to archive all items belonging to a given table. + */ +bool Backup_info::add_table_items(Table_item&) +{ + // FIXME: complete this + return TRUE; +} - backup_str->close(); - // send catalog to client +/********** meta-item iterator *************/ - /* - Loop through the catalog and list the contents by - database. - */ - List<schema_ref> cat1= info.catalog; - List_iterator<schema_ref> catalog(cat1); - schema_ref *tmp; - while ((tmp= catalog++)) - { - List_iterator<table_info> tbls(tmp->tables); - table_info *t; - while ((t= tbls++)) - { - /* - Send the following data: - database name (from tmp->db_name) - table name (from t->table_name) - CREATE TABLE SQL for the table (first string in table_metadata) - */ - protocol->prepare_for_resend(); - protocol->store(tmp->db_name.c_ptr(),system_charset_info); - protocol->store(t->table_name.c_ptr(),system_charset_info); - List_iterator<String> tbl_meta(t->table_metadata); - String *tmp= tbl_meta++; //just get the first string which is create - protocol->store(tmp->c_ptr(),system_charset_info); - protocol->write(); - } - } - send_eof(thd); - DBUG_RETURN(OK); -} - -int do_restore(THD *thd, const Location &loc) +Backup_info::Item* Backup_info::Item_iterator::operator++(int) { - DBUG_ENTER("restore::do_restore"); + m_prev= m_curr; - size_t header_size= 0; - size_t data_size= 0; + if (m_curr) + m_curr= m_curr->next; - Restore_info info; - bool res; + return m_prev; +} - IStream *backup_str= open_for_read(loc); - if (!backup_str) - goto error; +} // backup namespcae - dump_stream(*backup_str); // for debug purposes -- grep for 'stream: ' in server trace +/************************************************* + * + * RESTORE + * + *************************************************/ - backup_str->rewind(); +// Declarations of functions used in restore operation - res= read_header(*backup_str,info); - DBUG_ASSERT(res); +namespace backup { - res= read_catalog(*backup_str,info); - DBUG_ASSERT(res); +bool restore_meta_data(THD*, Restore_info&, IStream&); +bool restore_table_data(THD*, Restore_info&, IStream&); - header_size= backup_str->bytes; +} // backup namespace - DBUG_PRINT("restore",("Got archive header")); - DBUG_PRINT("restore",("== version: %d",info.ver)); - { - List_iterator<Restore_subimage> it(info.images); - Restore_subimage *img; - for( uint no=1 ; (img= it++) ; no++ ) - { - DBUG_PRINT("restore",(" %s image (%d)",img->name(), no)); - for( uint tno=0 ; tno < img->tables.count() ; tno++ ) - DBUG_PRINT("restore",(" table %s.%s", - img->tables[tno].db().name().ptr(), - img->tables[tno].name().ptr()) - ); - } - } +// pre: archive info has been already read and the stream is positioned +// at archive meta-data - DBUG_PRINT("restore",("== end of catalog")); +int mysql_restore(THD *thd, backup::Restore_info &info, backup::IStream &s) +{ + DBUG_ENTER("mysql_restore"); - res= restore_metadata(thd, info); - DBUG_ASSERT(res); + size_t start_bytes= s.bytes; - DBUG_PRINT("restore",("All tables created, restoring data")); + DBUG_PRINT("restore",("Restoring meta-data")); - res= restore_table_data(info,*backup_str); + bool res= backup::restore_meta_data(thd, info, s); DBUG_ASSERT(res); - data_size= backup_str->bytes -header_size; + DBUG_PRINT("restore",("Restoring table data")); - DBUG_PRINT("restore",("Done.")); - - backup_str->close(); + res= backup::restore_table_data(thd,info,s); + DBUG_ASSERT(res); - // send feedback to client - send_summary(&info.catalog, FALSE, header_size, data_size); + DBUG_PRINT("restore",("Done.")); - error: + info.total_size= s.bytes - start_bytes; - DBUG_RETURN(OK); + DBUG_RETURN(0); } -} // backup namespace - /************************************************* * * BACKUP LOCATIONS @@ -643,7 +694,7 @@ enum_type type() const { return SERVER_FILE; } - File_loc(const my_string p) + File_loc(const char *p) { path.append(p); } // files_charset_info }; @@ -684,36 +735,23 @@ return open_stream<OStream>(loc); } +/** + Find location described by a string. + The string is taken from the "TO ..." clause of BACKUP/RESTORE commands. + This function parses the string and creates instance of @c Location class + describing the location or NULL if string doesn't describe any valid location. + + Currently the only supported type of location is a file on the server host. + The string is supposed to contain a path to such file. + + @note No checks on the location are made at this stage. In particular the + location might not physically exist. In the future methods performing such + checks can be added to @Location class. + */ Location* -Location::find(const LEX_STRING &where, bool read) +Location::find(const LEX_STRING &where) { - //char buff[FN_REFLEN]; - //int length; - - //length = dirname_part(buff, where.str); - ///* - // If reading, file must exist. - // If writing, path must exist. - //*/ - //if (read) - //{ - // if (my_access(where.str, F_OK) || (length == where.length)) - // { - // my_error(ER_CANT_OPEN_FILE, MYF(0), where.str, - // ER_CANT_OPEN_FILE); - // return NULL; - // } - //} - //else - //{ - // if (my_access(buff, (F_OK|W_OK))) - // { - // my_error(ER_CANT_CREATE_FILE, MYF(0), where.str, - // ER_CANT_CREATE_FILE); - // return NULL; - // } - //} return new File_loc(where.str); } @@ -721,17 +759,122 @@ } // backup namespace - /************************************************* - * - * HELPER FUNCTIONS - * + + Helper functions + *************************************************/ TABLE *create_schema_table(THD *thd, TABLE_LIST *table_list); // defined in sql_show.cc namespace backup { +/** + Send a summary of the backup/restore operation to the client. + + The data about the operation is taken from filled @c Archive_info + structure. Parameter @c backup determines if this was backup or + restore operation. + + TODO: Add list of databases in the output. +*/ + +bool send_summary(THD *thd, const Archive_info &info, bool backup) +{ + Protocol *protocol= ::current_thd->protocol; // client comms + List<Item> field_list; // list of fields to send + Item *item; // field item + char buf[255]; // buffer for data + String op_str; // operations string + String print_str; // string to print + + DBUG_ENTER("backup::send_summary"); + + DBUG_PRINT(backup?"backup":"restore", ("sending summary")); + + op_str.length(0); + if (backup) + op_str.append("Backup Summary"); + else + op_str.append("Restore Summary"); + + + field_list.push_back(item= new Item_empty_string(op_str.c_ptr(),255)); // TODO: use varchar field + item->maybe_null= TRUE; + protocol->send_fields(&field_list, Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF); + + /* + Loop through the catalog and list the contents by + database. + + List<schema_ref> cat1= *catalog; + List_iterator<schema_ref> cat(cat1); + schema_ref *tmp; + + for (Archive_info::Db_iterator it(info); it; ++it) + while ((tmp= cat++)) + { + uint howmuch= 0; + + for (Archive_info::DItem_iterator dit(it); dit; ++dit) + howmuch++; + + my_snprintf(buf,sizeof(buf),"%s %3u %s in database %s.", + backup? "Backed up" : "Restored", + howmuch, howmuch != 1 ? "tables" : " table", + it->name().c_ptr()); + + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + } + + my_snprintf(buf,sizeof(buf)," "); + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + */ + + my_snprintf(buf,sizeof(buf)," header = %-8lu bytes",(unsigned long)info.header_size); + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + + my_snprintf(buf,sizeof(buf)," meta-data = %-8lu bytes",(unsigned long)info.meta_size); + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + + my_snprintf(buf,sizeof(buf)," data = %-8lu bytes",(unsigned long)info.data_size); + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + + my_snprintf(buf,sizeof(buf)," --------------"); + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + + my_snprintf(buf,sizeof(buf)," total %-8lu bytes", (unsigned long)info.total_size); + protocol->prepare_for_resend(); + protocol->store(buf,system_charset_info); + protocol->write(); + + send_eof(thd); + DBUG_RETURN(0); +} + +inline +bool send_summary(THD *thd, const Backup_info &info) +{ return send_summary(thd,info,TRUE); } + +inline +bool send_summary(THD *thd, const Restore_info &info) +{ return send_summary(thd,info,FALSE); } + + +/// Open given table in @c INFORMATION_SCHEMA database. + // TODO: Use select condition to get only selected rows from a table // CHECK: prepare_select_* functions (sql_help.cc) @@ -777,8 +920,288 @@ return t; } +/// Build linked @c TABLE_LIST list from a list stored in @c Table_list object. + +// FIXME: build list with the same order as in input +// Actually, should work fine with reversed list as long as we use the reversed +// list both in table writing and reading. + +TABLE_LIST *build_table_list(const Table_list &tables, thr_lock_type lock) +{ + TABLE_LIST *tl= NULL; + + for( uint tno=0; tno < tables.count() ; tno++ ) + { + TABLE_LIST *ptr= (TABLE_LIST*)sql_alloc(sizeof(TABLE_LIST)); + DBUG_ASSERT(ptr); + bzero(ptr,sizeof(TABLE_LIST)); + + Table_ref tbl= tables[tno]; + + ptr->alias= ptr->table_name= const_cast<char*>(tbl.name().ptr()); + ptr->db= const_cast<char*>(tbl.db().name().ptr()); + ptr->lock_type= lock; + + // and add it to the list + + ptr->next_global= ptr->next_local= + ptr->next_name_resolution_table= tl; + tl= ptr; + tl->table= ptr->table; + } + + return tl; +} + } // backup namespace +/**************************** + + Old code to be integrated into the current framework. + + ****************************/ + +#if FALSE + +/* + Get the metadata for the backup file specified and display + it to the client interface. + + SYNOPSIS + show_archive_catalog() + THD *thd - The current thread instance. + Location loc - The name of the archive with path. + + DESCRIPTION + This procedure opens the archive and extracts the metadata + for the databases and tables stored in the archive. A simple + list of the tables by database is sent to the client. + + RETURNS + 0 - no errors. + -1 - database cannot be read or doesn't exist. +*/ +int mysql_show_archive(THD *thd, const backup::Location &loc) +{ + Protocol *protocol= thd->protocol; + List<Item> field_list; + Item *item; + + DBUG_ENTER("backup::show_archive_catalog"); + field_list.push_back(item= new Item_empty_string("Database",255)); + field_list.push_back(item= new Item_empty_string("Table",255)); + field_list.push_back(item= new Item_empty_string("Metadata",255)); + item->maybe_null= TRUE; + protocol->send_fields(&field_list, Protocol::SEND_NUM_ROWS | Protocol::SEND_EOF); + backup::Restore_info info; + + backup::IStream *backup_str= backup::open_for_read(loc); + + if (!backup_str) + DBUG_RETURN(-1); + + // for debug purposes -- grep for 'stream: ' in server trace + backup::dump_stream(*backup_str); + + backup_str->rewind(); + + bool res= backup::read_header(*backup_str,info); + DBUG_ASSERT(res); + + res= backup::read_catalog(*backup_str,info); + DBUG_ASSERT(res); + + backup_str->close(); + + // send catalog to client + + /* + Loop through the catalog and list the contents by + database. + */ + List<backup::schema_ref> cat1= info.catalog; + List_iterator<backup::schema_ref> catalog(cat1); + backup::schema_ref *tmp; + while ((tmp= catalog++)) + { + List_iterator<backup::table_info> tbls(tmp->tables); + backup::table_info *t; + while ((t= tbls++)) + { + /* + Send the following data: + database name (from tmp->db_name) + table name (from t->table_name) + CREATE TABLE SQL for the table (first string in table_metadata) + */ + protocol->prepare_for_resend(); + protocol->store(tmp->db_name.c_ptr(),system_charset_info); + protocol->store(t->table_name.c_ptr(),system_charset_info); + List_iterator<String> tbl_meta(t->table_metadata); + String *tmp= tbl_meta++; //just get the first string which is create + protocol->store(tmp->c_ptr(),system_charset_info); + protocol->write(); + } + } + send_eof(thd); + DBUG_RETURN(backup::OK); +} + +#endif + +#if FALSE + +bool collect_tables_to_backup(THD *,Backup_info &); +bool collect_databases_to_backup(THD *thd, Backup_info &info); + + +// Collect tables to be backed-up and fill Backup_info structure describing backup image. +// Currently we try to backup all tables in 'test' database (but only tables for which +// appropriate backup engine is found will be included in the image). + +bool collect_tables_to_backup(THD *thd,Backup_info &info) +{ + LEX_STRING *tmp; + schema_ref *cat; + String db_name; + + DBUG_ENTER("backup::collect_tables_to_backup"); + + // open and scan I_S.TABLES table + + TABLE *i_s_tables = get_schema_table(thd, ::get_schema_table(SCH_TABLES) ); + + if( !i_s_tables ) + { + DBUG_PRINT("backup",("get_schema_table returned NULL!")); + DBUG_ERROR(ERROR); + } + + ::handler *ha = i_s_tables->file; + + // TODO: locking + + /* + Build catalog for writing... + For each database in the db_list, + create a new catalog struct + fill with tables and their metadata + add catalog to list + */ + List_iterator<LEX_STRING> databases(*info.db_list); + + table_info *ti; + while ((tmp= databases++)) + { + /* + Save this database in the catalog along with metadata + */ + cat= new schema_ref(); // create a new catalog struct + //cat->db_name= new (current_thd->mem_root) String(); + cat->db_name.length(0); + cat->db_name.append(tmp->str); // save the database name + + DBUG_PRINT("backup", ("Creating database metadata.")); + get_db_metadata(thd, cat->db_name.c_ptr(), &cat->db_metadata); + DBUG_PRINT("backup", ("Database metadata created.")); + + ha->ha_rnd_init(TRUE); + + // TODO: use conditions, do it properly + while( ! ha->rnd_next(i_s_tables->record[0]) ) + { + Table tbl(i_s_tables); + db_name= tbl.db().name(); + if (tbl && !my_strcasecmp(system_charset_info, + db_name.c_ptr(), tmp->str) && + my_strcasecmp(system_charset_info, + db_name.c_ptr(), "information_schema") && + my_strcasecmp(system_charset_info, + db_name.c_ptr(), "mysql")) + { + DBUG_PRINT("backup", ("Found table %s for database %s", + tbl.name().ptr(), tmp->str)); + // Backup_info::add method selects/creates sub-image appropriate for storing given table + info.add(tbl,tbl.hton()); + /* + Add the table to this database's table list + */ + ti= new table_info(); + ti->table_name.length(0); + ti->table_name.append(tbl.name()); + /* + Get the metadata for the table + */ + TABLE_LIST *ptr= + build_table_list_str((char *)tbl.name().ptr(), db_name.c_ptr(), TL_READ); + get_table_metadata(thd, ptr, &ti->table_metadata); + /* + Save the table_info to the tables list for this database + */ + cat->tables.push_back(ti); + } + } + info.catalog.push_back(cat); //add this catalog to the list of catalogs + } + + DBUG_PRINT("backup", ("No more tables in I_S")); + + ha->ha_rnd_end(); + + ::free_tmp_table(thd,i_s_tables); + + DBUG_RETURN(TRUE); +} + +bool collect_databases_to_backup(THD *thd, Backup_info &info) +{ + LEX_STRING *tmp; + String db_name; + + DBUG_ENTER("backup::collect_databases_to_backup"); + + // open and scan I_S.TABLES table + + TABLE *db_table = get_schema_table(thd, ::get_schema_table(SCH_SCHEMATA)); + + if (!db_table) + { + DBUG_PRINT("backup",("get_schema_table returned NULL!")); + DBUG_ERROR(ERROR); + } + + ::handler *ha = db_table->file; + + // TODO: locking + + ha->ha_rnd_init(TRUE); + + // TODO: use conditions, do it properly + while (!ha->rnd_next(db_table->record[0])) + { + db_name.length(0); + db_table->field[1]->val_str(&db_name); + if (my_strcasecmp(system_charset_info, + db_name.c_ptr(), "information_schema") && + my_strcasecmp(system_charset_info, + db_name.c_ptr(), "mysql")) + { + DBUG_PRINT("backup", ("Found database %s", db_name.ptr())); + tmp= (LEX_STRING *)thd->alloc(sizeof(LEX_STRING)); + tmp->str= my_malloc(MAX_ALIAS_NAME, MYF(MY_WME)); + strcpy(tmp->str, db_name.c_ptr()); + info.db_list->push_back(tmp); + } + } + + DBUG_PRINT("backup", ("No more databases in I_S")); + + ha->ha_rnd_end(); + + db_table->file->close(); + DBUG_RETURN(TRUE); +} +#endif --- 1.5/sql/backup/Makefile.am 2007-05-11 00:20:37 +02:00 +++ 1.6/sql/backup/Makefile.am 2007-05-11 00:20:37 +02:00 @@ -31,19 +31,18 @@ be_nodata.cc \ archive.cc \ stream.cc \ - tables.cc \ string_pool.cc noinst_HEADERS = \ api_types.h \ backup_kernel.h \ backup_engine.h \ + meta_backup.h \ backup_aux.h \ be_default.h \ be_nodata.h \ archive.h \ stream.h \ - tables.h \ string_pool.h \ map.h --- 1.4/sql/backup/api_types.h 2007-05-11 00:20:37 +02:00 +++ 1.5/sql/backup/api_types.h 2007-05-11 00:20:37 +02:00 @@ -1,6 +1,12 @@ #ifndef _BACKUP_API_TYPES_H #define _BACKUP_API_TYPES_H +/** + @file + + Declarations of common data types used in the backup API's + */ + /* Note: Structures defined in this file use String class which introduces dependency on its implementation defined in sql/ directory. Thus to correctly @@ -59,25 +65,35 @@ class Db_ref { - const String &m_name; + const String *m_name; public: + // Construct invalid reference + Db_ref(): m_name(NULL) + {} + + const bool is_valid() const + { return m_name != NULL; } + const String& name() const - { return m_name; } + { return *m_name; } const String& catalog() const { return my_null_string; } + bool operator==(const Db_ref &db) const + { return stringcmp(m_name,&db.name())==0; } + + bool operator!=(const Db_ref &db) const + { return ! this->operator==(db); } + protected: // Constructors are made protected as clients of this class are // not supposed to create instances (see comment inside Table_ref) - Db_ref(const String &name): m_name(name) - {} - - Db_ref(const Db_ref &db): m_name(db.m_name) + Db_ref(const String &name): m_name(&name) {} friend class Table_ref; @@ -87,15 +103,31 @@ class Table_ref { const Db_ref m_db; - const String &m_name; + const String *m_name; public: + // Construct invalid reference + Table_ref(): m_name(NULL) + {} + + const bool is_valid() const + { return m_name != NULL; } + const Db_ref& db() const { return m_db; } const String& name() const - { return m_name; } + { return *m_name; } + + bool operator==(const Table_ref &t) const + { + return m_db == t.db() && + stringcmp(m_name,&t.name()) == 0; + } + + bool operator!=(const Table_ref &db) const + { return ! this->operator==(db); } // Produce string identifying the table (e.g. for error reporting) //operator String() const; @@ -110,10 +142,8 @@ */ Table_ref(const String &db, const String &name): - m_db(db), m_name(name) + m_db(db), m_name(&name) {} - - friend class Tables; // internal implementation of Table_list interface }; @@ -156,7 +186,88 @@ virtual uint count() const =0; }; -class Engine; // forward declaration + +/** + @class Buffer + + @brief Used for data transfers between backup kernel and backup/restore + drivers. + + Apart from allocated memory a Buffer structure contains fields informing about + its size and holding other information about contained data. Buffers are + created and memory is allocated by backup kernel. It is also kernel's + responsibility to write contents of buffers to a backup stream. + + Data created by a backup driver is opaque to the kernel. However, to support + selective restores, each block of data can be assigned to one of the tables + being backed-up. This is done by setting <code>table_no</code> member of the + buffer structure to the number of the table to which this data belongs. Tables + are numbered from 1 acoording to their position in the list passed when driver + is created (<code>m_tables</code> member of <code>Driver</code> class). If + some of the data doesn't correspond to any particular table, then + <code>table_no</code> should be set to 0. + + This way, driver can create several "streams" of data blocks. For each table + there is a stream corresponding to that table and there is one "shared stream" + consisting of blocks with <code>table_no</code> set to 0. Upon restore, kernel + sends to a restore driver only blocks corresponding to the tables being + restored plus all the blocks from the shared stream. + + For example, consider backing-up three tables t1, t2 and t3. Data blocks + produced by a backup driver are divided into four streams: + <pre> + #0: shared data + #1: data for table t1 + #2: data for table t2 + #3: data for table t3 + </pre> + When a user restores tables t1 and t3, only blocks from streams #0, #1 and #3 + will be sent to a restore driver, but not the ones from stream #2. + + Using this approach, backup engine can arrange its backup image data in the + way which best suits its internal data representation. If needed, all data can + be put in the shared stream #0, so that all of it will be sent back to + a restore driver. On the other hand, if possible, backup data can be + distributed into per table streams to reduce the amount of data transferred + upon a selective restore. + + Backup driver signals end of data in a given stream by setting + <code>buf.last</code> flag to TRUE when get_data(buf) fills the last block of + data from that stream (otherwise <code>buf.last</code> should be FALSE). This + should be done for each stream used by the driver. Upon restore, kernel sets + <code>buf.last</code> to TRUE when sending to a restore driver the last block + of data from a stream. + + A driver learns about the size of a buffer provided by the kernel from its + <code>size</code> member. It does not have to fill the buffer completely. + It should update the <code>size</code> member to reflect the actual size + of the data in the buffer. It is possible to return no data in which case + <code>size</code> should be zero. Such empty buffers are ignored by the + kernel (no data is writen to the archive). + */ + +struct Buffer +{ + size_t size; ///< size of the buffer (of memory block pointed by data). + uint table_no; ///< Number of the table to which data in the buffer belongs. + bool last; ///< <code>TRUE</code> if this is last block of data in the stream. + byte *data; ///< Pointer to data area. + + Buffer(): data(NULL),size(0),table_no(0),last(FALSE) + {} + + void reset(size_t len) + { + size= len; + table_no= 0; + last= FALSE; + } +}; + +// forward declaration +class Engine; +class Backup_driver; +class Restore_driver; } // backup namespace --- 1.5/sql/backup/archive.cc 2007-05-11 00:20:37 +02:00 +++ 1.6/sql/backup/archive.cc 2007-05-11 00:20:37 +02:00 @@ -1,53 +1,625 @@ #include "../mysql_priv.h" -#include "archive.h" + +/** + @file + + Implementation of @c Archive_info and related classes. + */ + +/* + TODO: + + - Add to Archive_info storage for other meta-data items. + - Make existing storage solutions more rational. + - Make reading code resistant to unknown image formats or meta-data types + (or, assume it is handled by format version number). + - Improve Image_info::Tables implementation (use some existing data structure). + - Add more information to backup archive header. + + */ + +#include "backup_engine.h" +#include "be_default.h" #include "backup_aux.h" +#include "archive.h" + +/*************************************** + + Implementation of Archive_info class + + ***************************************/ namespace backup { -bool find_db_in_list(List<LEX_STRING> *db_list, String db_name) +Archive_info::~Archive_info() +{ + for (uint i=0; i<256; ++i) + if (images[i]) + { + delete images[i]; + images[i]= NULL; + } +} + + +/** + Write header and catalogue of a backup archive. + + Header forms the first chunk of archive. Currently it contains archive's + format version number followed by a list of table data images used in the + archive. + + Next chunk contains the pool of database names. After that there is one chunk + per image containing list of tables whose data is saved in that image. + <pre> + ===================== + version number } + --------------------- } header + image descriptions } + ===================== + db names } + ===================== } + tables of image 1 } + ===================== } + } catalogue + ... } + } + ===================== } + tables of image N } + ===================== + </pre> + In the picture "====" denotes chunk boundaries. Number of images is known + from the header. + + The format in which image descriptions are saved is determined by + @c Image_info::write_description() method. + + For list of databases and tables the format in which they are saved is + defined by @c StringPool::save() and @c Archive_info::Tbles::save() methods + respectively. + + TODO: Add more info to the header. E.g. version number of the server on + which archive was created. + */ + +stream_result::value Archive_info::save(OStream &s) +{ + DBUG_ENTER("Archive_info::save"); + + size_t start_bytes= s.bytes; + stream_result::value res; + + res= s.write2int(ver); + if (res != stream_result::OK) + DBUG_RETURN(res); + + // list of images + DBUG_PRINT("backup",(" writing image list")); + + uint ino; + Image_info *img; + + for (ino=0; ino <= img_count ; ++ino) + if ((img= images[ino])) + { + DBUG_PRINT("backup",(" %2d: %s image",ino,img->name())); + res= img->write_description(s); + if (res != stream_result::OK) + DBUG_RETURN(res); + } + + res= s.end_chunk(); + if (res != stream_result::OK) + DBUG_RETURN(res); + + // catalogue + DBUG_PRINT("backup",(" writing archive catalog")); + + // db names + res= db_names.save(s); + if (res != stream_result::OK) + DBUG_RETURN(res); + + // table lists + for (ino=0; ino <= img_count ; ++ino) + if ((img= images[ino])) + { + DBUG_PRINT("backup",(" saving %s image's tables",img->name())); + res= img->tables.save(s); + if (res != stream_result::OK) + DBUG_RETURN(res); + } + + header_size= s.bytes - start_bytes; + + DBUG_RETURN(res); +} + +/** + Fill @c Archive_info structure reading data from backup archive header and + catalogue. + */ +stream_result::value Archive_info::read(IStream &s) { - bool found= false; - List_iterator<LEX_STRING> databases(*db_list); - LEX_STRING *tmp; - while ((tmp= databases++)) - if (!my_strcasecmp(system_charset_info, tmp->str, db_name.c_ptr())) - return(TRUE); - return(found); + DBUG_ENTER("Archive_info::read"); + + size_t start_bytes= s.bytes; + stream_result::value res; + version_t ver; + + // read version number + res= s.read2int(ver); + if (res != stream_result::OK) + DBUG_RETURN(stream_result::ERROR); // neither stream nor chunk should end here + + if (ver != Archive_info::ver) + DBUG_RETURN(stream_result::ERROR); // wrong version number + // (in future handle backward compatibility) + + DBUG_PRINT("restore",(" reading image list")); + + uint ino; + Image_info *img; + + for (ino=0 ; + ino<256 && (img= Image_info::create_from_stream(*this,s)) ; + ino++) + { + images[ino]= img; + if (!img) + DBUG_RETURN(stream_result::ERROR); + DBUG_PRINT("restore",(" %2d: %s image",ino,img->name())); + } + + img_count= ino; + table_count= 0; + + if (res == stream_result::EOS) // empty archive + DBUG_RETURN(res); + + // move to the catalogue + res= s.next_chunk(); + if (res != stream_result::OK) + DBUG_RETURN(stream_result::ERROR); // neither stream nor chunk should end here + + DBUG_PRINT("restore",(" reading catalog (%d images)",img_count)); + + // read db_names pool + res= db_names.read(s); + if (res != stream_result::OK) + DBUG_RETURN(stream_result::ERROR); // neither stream nor chunk should end here + + // read tables + for (uint ino=0; ino < img_count; ++ino) + { + img= images[ino]; + DBUG_PRINT("restore",(" reading %s image's tables (ino=%d)",img->name(),ino)); + res= img->tables.read(s); + if (ino < img_count-1 && res != stream_result::OK || + res == stream_result::ERROR) + DBUG_RETURN(stream_result::ERROR); // neither stream nor chunk should end here + table_count+= img->tables.count(); + DBUG_PRINT("restore",(" finished reading tables")); + } + + header_size= s.bytes - start_bytes; + + DBUG_RETURN(res); } } // backup namespace +/********************************** + + Write/read image descriptions + + **********************************/ -// Implementation of default backup image type namespace backup { +/** + Write entry describing (format of) a table data image. -bool Native_backup_info::add(const Table_ref &tbl, const ::handlerton *hton) + Entry has the form: + <pre> + | type | version | image description | + </pre> + where type is a byte holding Image_info::image_type value, version is 2 byte + integer holding image format version. The format of optional image description + is determined by @c X::do_write_description() method where X is a subclass of + Image_info corresponding to given image type. + */ +stream_result::value +Image_info::write_description(OStream &s) { - // Accept all tables stored in the storage engine creating this format. - if( hton == m_hton ) // this assumes handlertons are single instance objects! + // TODO: write description length here + s.writebyte(type()); + s.write2int(ver); + return do_write_description(s); +} + +/** + Create @c Image_info instance from a saved entry describing it. + + TODO: Handle unknown image types (always skip correct amount of bytes). + */ +Image_info* Image_info::create_from_stream(Archive_info &info, IStream &s) +{ + uint ver; + byte t; + + if (s.readbyte(t) != stream_result::OK) + return NULL; + + if (s.read2int(ver) != stream_result::OK) + return NULL; + + switch (image_type(t)) { + + case DEFAULT_IMAGE: + return Default_image::create_from_stream(ver,info,s); + + case NATIVE_IMAGE: + return Native_image::create_from_stream(ver,info,s); + + default: + return NULL; + } +} + +} // backup namespace + + +/******************* + + Serialization of meta-data items + + *******************/ + +namespace backup { + +/** + Write an entry describing single meta-data item. + + Entry has format: + <pre> + | type | id data | create data | + </pre> + Type is a single byte holding meta::Item::enum_type value. Id data is + used to determine which item (from the archive catalogue) the entry + corresponds to. Create data is used to create the item. + + The format of id data and create data for item of type X is determined + by methods @c Archive_info::X_item::save_id() and @c meta::X::save(), + respectively. + + @see @c write_meta_data() for information about the format of the meta-data + section of bakcup archive. + */ +stream_result::value Archive_info::Item::save(THD *thd, OStream &s) +{ + byte b= meta().type(); + + stream_result::value res=s.writebyte(b); + TEST_WR_RES(res); + + res= save_id(s); + TEST_WR_RES(res); + + return meta().save(thd,s); +} + +/** + Create meta-data item from a saved entry. + + This function reads the type byte and calls @c create_from_stream method of + corresponding class to create the item. + + TODO: handle unknow entry types (so that they can be skipped). + */ +Archive_info::Item* +Archive_info::Item::create_from_stream(const Archive_info &info, + IStream &s) +{ + byte b; + Item *i; + stream_result::value res; + + res= s.readbyte(b); + + if (res!=stream_result::OK) + return NULL; + + switch (meta::Item::enum_type(b)) { + + case meta::Item::DB: + i= Db_item::create_from_stream(info,s); + break; + + case meta::Item::TABLE: + i= Table_item::create_from_stream(info,s); + break; + + default: return NULL; + + } + + if (i == NULL) + return NULL; + + res= i->meta().read(s); + TEST_RD_RES(res); + + return i; +} + +// Db items + +stream_result::value Archive_info::Db_item::save_id(OStream &s) +{ + uint k= key; + DBUG_PRINT("backup",(" saving db-item (%d)",k)); + return s.writeint(k); +} + +Archive_info::Db_item* +Archive_info::Db_item::create_from_stream(const Archive_info &i,IStream &s) +{ + uint k; + stream_result::value res; + + res= s.readint(k); + TEST_RD_RES(res); + + return new Db_item(i,k); +} + +// Table items + +stream_result::value Archive_info::Table_item::save_id(OStream &s) +{ + DBUG_PRINT("backup",(" saving table-item (%d,%d)",img,pos)); + stream_result::value res= s.writeint(img); + TEST_WR_RES(res); + return s.writeint(pos); +} + +Archive_info::Table_item* +Archive_info::Table_item::create_from_stream(const Archive_info &i,IStream &s) +{ + uint img,no; + stream_result::value res; + + res= s.readint(img); + TEST_RD_IRES(res); + + res= s.readint(no); + TEST_RD_RES(res); + + return new Table_item(i,img,no); +} + +} // backup namespace + + +/********************************** + + Implementation of Image_info::Tables + + **********************************/ + +namespace backup { + +// TODO: use better implementation (red-black tree from mysys?) + +struct Image_info::Tables::node { + StringPool::Key db; + String name; + node *next; + + node(const Image_info::Tables &t, + const StringPool::Key &k, + const String &nm): db(k), next(NULL) { - DBUG_PRINT("backup",("adding table %s to %s image",tbl.name().ptr(),m_name)); - tables.add(tbl); - return TRUE; + name.copy(nm); + } +}; + +Image_info::Tables::~Tables() +{ + for( node *ptr= m_head ; ptr ; ) + { + node *n=ptr; + ptr= n->next; + delete n; + }; +} + +int Image_info::Tables::add(const backup::Table_ref &t) +{ + StringPool::Key k= m_db_names.add(t.db().name()); + + if( !k.is_valid() ) return -1; + + return add(k,t.name()); +} + +int Image_info::Tables::add(const StringPool::Key &k, const String &name) +{ + node *n= new node(*this,k,name); + + if( !n ) return -1; + + if( m_head == NULL ) + { + m_count=1; + m_head= m_last= n; } else - return FALSE; + { + m_count++; + m_last->next= n; + m_last= n; + }; + + return m_count-1; +} + +Image_info::Tables::node* +Image_info::Tables::find_table(uint pos) const +{ + DBUG_ASSERT(pos < m_count); + + node *ptr; + + for( ptr= m_head ; ptr && pos ; ptr= ptr->next ) + pos--; + + //if( !ptr ) ptr= m_last; + + return ptr; +} + +inline +Table_ref Image_info::Tables::operator[](uint pos) const +{ + // Get access to backup::Table_ref protected constructor + + struct Table_ref: public backup::Table_ref + { + Table_ref(const Image_info::Tables &list, Image_info::Tables::node &n): + backup::Table_ref(list.m_db_names[n.db],n.name) + {} + }; + + node *ptr= find_table(pos); + DBUG_ASSERT(ptr); + + return Table_ref(*this,*ptr); +} + +/****************** + + Serialization for Image_info::Tables class + + ******************/ + +/** + Save list of tables in a backup stream. + + The format used assumes that a pool of database names is stored elsewhere. + Thus for each table only the key of the database is stored as var-length + integer followed by table name. + + The list is stored in a single stream chunk which determines its end. + */ +stream_result::value +Image_info::Tables::save(OStream &s) +{ + stream_result::value res; + + for( Tables::node *n= m_head ; n ; n= n->next ) + { + res= s.writeint(n->db); + res= s.writestr(n->name); + }; + + return s.end_chunk(); } +/** + Read a list from a backup stream. -bool Native_backup_info::do_write_description(OStream &s) + @pre Stream is positioned at the first entry of the saved list. + @post Stream is positioned at the beginning of next chunk or at its end. + */ +stream_result::value +Image_info::Tables::read(IStream &s) +{ + DBUG_ENTER("Image_info::Tables::read"); + + stream_result::value res; + + do + { + String name; + uint k,tno; + + res= s.readint(k); + res= s.readstr(name); + + if( res == stream_result::OK ) + { + tno= add(k,name); + DBUG_PRINT("restore",("got next table %s.%s (pos %d, dbkey %d)", + (*this)[tno].db().name().ptr(), + (*this)[tno].name().ptr(),tno,k)); + } + } + while (res == stream_result::OK); + + DBUG_ASSERT(res == stream_result::EOC || res == stream_result::EOS); + + res= s.next_chunk(); + + DBUG_RETURN(res); +} + +} // backup namespace + +/************************************** + + Native image type definition + + **************************************/ + +namespace backup { + +Backup_driver* Native_image::get_backup_driver() +{ + Backup_driver *drv; + + result_t res= m_be->get_backup(Driver::PARTIAL,tables,drv); + + DBUG_ASSERT(res == OK); + DBUG_ASSERT(drv); + + return drv; +} + +Restore_driver* Native_image::get_restore_driver() +{ + Restore_driver *drv; + + result_t res= m_be->get_restore(ver,Driver::PARTIAL,tables,drv); + + DBUG_ASSERT(res == OK); + DBUG_ASSERT(drv); + + return drv; +} + +/* + For native image its format (apart from the version number) is determined + by the storage engine whose backup driver created it. Thus we save the name + of storage engine. + + TODO: add more information here. Eg the version number of the storage engine. + */ +stream_result::value +Native_image::do_write_description(OStream &s) { String name(::ha_resolve_storage_engine_name(m_hton),default_charset); - return s.writestr(name) == stream_result::OK; + return s.writestr(name); } -Native_restore_info* -Native_restore_info::create_from_stream(uint no, version_t ver, IStream &s) +Native_image* +Native_image::create_from_stream(version_t ver, Archive_info &info, IStream &s) { String name; + Native_image *img; if( s.readstr(name) == stream_result::OK ) { @@ -58,7 +630,12 @@ ::handlerton *hton= ::ha_resolve_by_name(::current_thd,&name_lex); - return new Native_restore_info(no, hton,ver); + img= new Native_image(info,hton); + DBUG_ASSERT(ver <= img->ver); // TODO: deal with different versions + + img->ver= ver; + + return img; } else return NULL; --- 1.9/sql/backup/archive.h 2007-05-11 00:20:37 +02:00 +++ 1.10/sql/backup/archive.h 2007-05-11 00:20:37 +02:00 @@ -1,285 +1,386 @@ #ifndef _BACKUP_ARCHIVE_H #define _BACKUP_ARCHIVE_H -#include "backup_engine.h" -#include "tables.h" -#include "stream.h" +/** + @file -namespace backup { + Data types used to read, write and manipulate backup archives. + */ -// Data common to Backup_subimage and Restore_subimage +#include <backup/api_types.h> +#include <backup/string_pool.h> +#include <backup/stream.h> +#include <backup/backup_engine.h> +#include <backup/meta_backup.h> -struct Subimage_info -{ - enum image_type {NATIVE_IMAGE, DEFAULT_IMAGE}; +namespace backup { - uint no; - Tables tables; +// Forward declaration for a class describing an image inside backup archive. +class Image_info; - Subimage_info(uint no): no(no) - {} +typedef Image_info* Img_list[256]; ///< List (vector) of image descriptions. - virtual version_t ver() const =0; - virtual image_type type() const =0; +/** + Describes contents of a backup archive. - virtual ~Subimage_info() - {} + This class stores a catalogue of a backup archive, that is, description of + all items stored in the archive (currently only databases and tables). It also + determines how to save and read the catalogue to/from a backup stream. - virtual const char* name() const - { return "<Unknown Driver>"; } -}; + Only item names are stored in the catalouge. Other item data is stored + in the meta-data part of an archive and in case of tables, their data is + stored in images created by backup drivers. -class Backup_subimage: public Subimage_info -{ - protected: - - Backup_driver *m_drv; // TODO: ensure that driver is freed + The @c images member stores a list of @c Image_info objects describing the + images included in the archive. Each image description contains a list of + tables stored in that image (note that no table can be stored in more than + one image). - public: + To save space, we have a separate pool of database names (@c db_names member). + In table references, only the key of the database name is stored, not the + whole name. - size_t init_size; + When reading or writing backup archive, statistics about the size of its parts + is stored in the members of this class for later reporting. - Backup_subimage(uint no): - Subimage_info(no), - m_drv(NULL), init_size(Driver::UNKNOWN_SIZE) - {} + TODO: Define methods for browsing contents of the archive. + */ +struct Archive_info +{ + static const version_t ver=1; + uint img_count; ///< number of images in the archive + uint table_count; ///< total number of tables in the archive + + size_t total_size; ///< size of processed backup archive + size_t header_size; ///< size of archive's header (after reading or writing an archive) + size_t meta_size; ///< size of archive's meta-data (after reading or writing an archive) + size_t data_size; ///< size of archive's table data images (after reading or writing an archive) + + // Classes representing various types of meta-data items. + + class Item; + class Db_item; + class Table_item; + + /* + class Item_iterator; // for iterating over all meta-data items + class Db_iterator; // iterates over databases in archive + class Ditem_iterator; // iterates over per-db items + */ + + Img_list images; ///< list of archive's images + + /// Write archive's header and save the catalogue. + stream_result::value save(OStream&); + /// Read the header and catalogue from a stream. + stream_result::value read(IStream&); - virtual bool add(const Table_ref &tbl, const ::handlerton *hton) - { return FALSE; } + virtual ~Archive_info(); - result_t create_driver() - { - result_t res= do_create_driver(); - DBUG_ASSERT(res == OK); - DBUG_ASSERT(m_drv); - init_size= m_drv->init_size(); + protected: - return res; - } + // Item *m_items; - // TODO: change driver method to return pointer (to signal problems). - Backup_driver& driver() + Archive_info(): + img_count(0), table_count(0), + total_size(0), header_size(0), meta_size(0), data_size(0) { - if(!m_drv) - create_driver(); - - return *m_drv; + for (uint i=0; i<256; ++i) + images[i]= NULL; } - bool write_description(OStream &s); + // storage for meta-data items - protected: + StringPool db_names; ///< Pool of database names. - virtual bool do_write_description(OStream&) =0; - virtual result_t do_create_driver() =0; + private: + friend class Image_info; }; -class Restore_subimage: public Subimage_info +/** + Describes an image of table data stored in a backup archive. + + An instance of this class: + - informs about the type of image, + - stores list of tables whose data is keept in the image, + - provides methods for creating backup and restore drivers to write/read the + image, + - determines which tables can be stored in the image, + - defines how image's format is described inside backup archive + (via @c do_write_description() method) + */ +struct Image_info { - protected: + enum image_type {NATIVE_IMAGE, DEFAULT_IMAGE}; - Restore_driver *m_drv; // TODO: ensure that driver is freed + virtual image_type type() const =0; ///< Return type of the image. + version_t ver; ///< Image format version. - public: + /// Create backup driver for the image. + virtual Backup_driver* get_backup_driver() =0; + /// Create restore driver for the image. + virtual Restore_driver* get_restore_driver() =0; - version_t m_ver; + size_t init_size; ///< Size of the initial data transfer (estimate). This is + ///< meaningfull only after a call to get_backup_driver(). - Restore_subimage(uint no, version_t ver): - Subimage_info(no), m_drv(NULL), m_ver(ver) - {} + /// Write header entry describing the image. + stream_result::value write_description(OStream&); + /// Create instance of @c Image_info described by an entry in backup stream. + static Image_info* create_from_stream(Archive_info&, IStream&); - version_t ver() const { return m_ver; } + /// Determine if a table stored in given engine can be saved in this image. + virtual bool accept(const Table_ref&, const ::handlerton*) =0; + + /// Return name identifying the image in debug messages. + virtual const char* name() const + { return "<Unknown Driver>"; } + + virtual ~Image_info() + {} - result_t create_driver() + /* + Implementation of Table_list interface used to store the + list of tables of an image. Database names are stored in + external StringPool + */ + class Tables: public Table_list { - result_t res= do_create_driver(); - DBUG_ASSERT(res == OK); - DBUG_ASSERT(m_drv); - return res; - } + public: - Restore_driver* driver() - { - if(!m_drv) - create_driver(); + Tables(StringPool &db_names): + m_db_names(db_names), + m_head(NULL),m_last(NULL),m_count(0) + {} + ~Tables(); - return m_drv; - } + int add(const backup::Table_ref&); - static Restore_subimage* create_from_stream(uint,IStream&); + backup::Table_ref operator[](uint pos) const; + //::TABLE_LIST* get_table_ptr(uint pos) const; -protected: + uint count() const + { return m_count; } - virtual result_t do_create_driver() =0; + stream_result::value save(OStream&); + stream_result::value read(IStream&); -}; + private: + struct node; -// Info about (contents of) a global backup image. + int add(const StringPool::Key&, const String&); + node* find_table(uint pos) const; -struct Image_info -{ - version_t ver; - uint img_count; // number of subimages - uint table_count; // total number of tables + StringPool &m_db_names; + node *m_head, *m_last; + uint m_count; + friend struct Table_ref; + friend class Archive_info::Table_item; + }; - Image_info(version_t v): ver(v), img_count(0), table_count(0) - {} -}; + Tables tables; ///< List of tables stored in the image. -// Information about databases and their tables. -struct table_info -{ - String table_name; // table name - List<String> table_metadata; // metadata for table - ~table_info() - { table_metadata.delete_elements(); } -}; + protected: -struct schema_ref -{ - String db_name; // the name of the database - List<String> db_metadata; // metadata for the database - List<table_info> tables; // the list of tables - ~schema_ref() - { - db_metadata.delete_elements(); - tables.delete_elements(); - } + Image_info(Archive_info &info): + init_size(Driver::UNKNOWN_SIZE), tables(info.db_names) + {} + + /** + Write image specific data describing it. + + Method redefined in subclasses corresponding to different image types. + */ + virtual stream_result::value do_write_description(OStream&) =0; }; -bool find_db_in_list(List<LEX_STRING> *db_list, String db_name); +/** + Represents a meta-data item in a backup archive. + + Instances of this class: -struct Backup_info: public Image_info + - identify a meta-data item inside backup archive, + - provide storage for a corresponding meta::Item instance, + - write item identification data to a backup stream. + + For each type of meta-data there is a specialized subclass of + @c Archive_info::Item implementing the above tasks. Each subclass has static + @c create_from_stream() method which can create class instance using an + identity stored in a stream. For examples, see @c Archive_info::Table_item + class. + + Class @c Archive_info::Item defines the format of an entry describing a + meta-data item inside the meta-data part of an archive. Such entry is created + by @c Archive_info::save() method. These entries are read by + @c Restore_info::read_item() method. + */ + +class Archive_info::Item { - List<Backup_subimage> images; - Backup_subimage *default_image; - List<LEX_STRING> *db_list; // list of databases - List<schema_ref> catalog; // list of tables by database + protected: + + const Archive_info *const m_info; ///< Pointer to an archive info instance to + ///< which this item belongs. + Item *next; ///< Used to create a linked list of all meta-data items. + + public: + + virtual ~Item() {} - Backup_info(): Image_info(7), default_image(NULL) + virtual meta::Item& meta() =0; ///< Return reference to the @c meta::Item + ///< instance. + + stream_result::value save(THD*,OStream&); ///< Write entry describing the item. + + /// Create item from a stream entry. + static Item* create_from_stream(const Archive_info&, IStream&); + + protected: + + Item(const Archive_info &i): m_info(&i), next(NULL) {} - ~Backup_info() - { - images.delete_elements(); - catalog.delete_elements(); - } + /// Save data identifying the item inside the archive. + virtual stream_result::value save_id(OStream&) =0; - // Add table stored in a given storage engine - bool add(const Table_ref &tbl, const ::handlerton *hton); + friend class Archive_info; + friend class Backup_info; + friend class Restore_info; }; -struct Restore_info: public Image_info +/** + Specialization of @c Archive_info::Item representing a database. + + A database is identified by a key into Archive_info::db_names string pool. + Using the key one can read database name from the pool. The key is saved + as a var-lenght coded integer. + */ +class Archive_info::Db_item: + public Item, public meta::Db, public Db_ref { - List<Restore_subimage> images; - List<schema_ref> catalog; // list of tables by database + StringPool::Key key; - Restore_info(): Image_info(7) + Db_item(const Archive_info &i, const StringPool::Key &key): + Archive_info::Item(i), Db_ref(m_info->db_names[key]), key(key) {} - ~Restore_info() - { - images.delete_elements(); - catalog.delete_elements(); - } -}; + meta::Item& meta() + { return *this; } -} // backup namespace + /// Get the name from @c db_names pool. + const char* sql_name() const + { return m_info->db_names[key].ptr(); } + public: -/************************************************************ + stream_result::value save_id(OStream&); + /// Create instance reading its identity from a stream. + static Db_item* create_from_stream(const Archive_info&,IStream&); - Classes describing native backup image + friend class Archive_info; + friend class Backup_info; +}; - ************************************************************/ -namespace backup { +/** + Specialization of @c Archive_info::Item representing a table. -class Native_backup_info: public Backup_subimage + A table is identified by its position inside the table list of + one of archive's images. Its identity is saved as two var-length coded + integers: first being the image number and second the table position inside + image's table list. + */ +class Archive_info::Table_item: + public Item, public meta::Table, public Table_ref { - const ::handlerton *m_hton; - Engine *m_be; - - char m_name[256]; + uint img; ///< Image in which this table is saved. + uint pos; ///< Position of the table in image's table list. - public: + Table_item(const Archive_info &i, uint no, uint tno): + Archive_info::Item(i), Table_ref(i.images[no]->tables[tno]), + img(no), pos(tno) + {} - Native_backup_info(uint no, const ::handlerton *hton): - Backup_subimage(no), m_hton(hton) - { - DBUG_ASSERT(hton); - DBUG_ASSERT(hton->get_backup_engine); - hton->get_backup_engine(const_cast< ::handlerton* >(hton),m_be); - DBUG_ASSERT(m_be); + meta::Item& meta() + { return *this; } - my_snprintf(m_name,sizeof(m_name),"%s's driver", - ::ha_resolve_storage_engine_name(hton)); - } + const char* sql_name() const + { return m_info->images[img]->tables[pos].name().ptr(); } - version_t ver() const - { return m_be->version(); } + /// Table is a per-db item -- indicate to which database it belongs. + const Db_ref in_db() + { return m_info->images[img]->tables[pos].db(); } - image_type type() const - { return NATIVE_IMAGE; } + public: - result_t do_create_driver() - { - result_t res= OK; + stream_result::value save_id(OStream&); + /// Create instance reading its identity from a stream. + static Table_item* create_from_stream(const Archive_info&,IStream&); + + // Implementation of methods used in meta::Table. +/* + TABLE_LIST *open(THD*) + { return m_info->images[img]->tables.get_table_ptr(pos); } + virtual bool close() + { return TRUE; } +*/ + friend class Backup_info; +}; - DBUG_ASSERT(m_be); +} // backup namespace - res= m_be->get_backup(Driver::PARTIAL, tables,m_drv); - DBUG_ASSERT(res == OK); - DBUG_ASSERT(m_drv); +/************************************************************ - return res; - } + Class describing native backup image - const char* name() const - { return m_name; } + ************************************************************/ - bool add(const Table_ref&, const ::handlerton*); - bool do_write_description(OStream&); -}; +namespace backup { -class Native_restore_info: public Restore_subimage +/** + Specialization of @c Image_info for images created by native backup drivers. + */ +class Native_image: public Image_info { - const ::handlerton *m_hton; - Engine *m_be; + const ::handlerton *m_hton; ///< Pointer to storage engine. + Engine *m_be; ///< Pointer to the native backup engine. + + char m_name[256]; ///< Used to identify image in debug messages. public: - Native_restore_info(uint no, const ::handlerton *hton, version_t ver): - Restore_subimage(no,ver), m_hton(hton) + Native_image(Archive_info &info, const ::handlerton *hton): + Image_info(info), m_hton(hton) { DBUG_ASSERT(hton); DBUG_ASSERT(hton->get_backup_engine); hton->get_backup_engine(const_cast< ::handlerton* >(hton),m_be); DBUG_ASSERT(m_be); + + ver= m_be->version(); + + my_snprintf(m_name,sizeof(m_name),"%s's driver", + ::ha_resolve_storage_engine_name(hton)); } image_type type() const { return NATIVE_IMAGE; } - result_t do_create_driver() - { - result_t res= OK; - - DBUG_ASSERT(m_be); - - res= m_be->get_restore(m_ver, Driver::PARTIAL, tables,m_drv); + const char* name() const + { return m_name; } - DBUG_ASSERT(res == OK); - DBUG_ASSERT(m_drv); + Backup_driver* get_backup_driver(); + Restore_driver* get_restore_driver(); - return res; - } + stream_result::value do_write_description(OStream&); + static Native_image* create_from_stream(version_t,Archive_info&,IStream&); - static Native_restore_info* create_from_stream(uint, version_t, IStream&); + bool accept(const Table_ref&, const ::handlerton *hton) + { return hton == m_hton; }; // this assumes handlertons are single instance objects! }; } // backup namespace --- 1.6/sql/backup/backup_engine.h 2007-05-11 00:20:37 +02:00 +++ 1.7/sql/backup/backup_engine.h 2007-05-11 00:20:37 +02:00 @@ -16,7 +16,7 @@ #include <backup/api_types.h> #ifndef VIEW_ALGORITHM_UNDEFINED - #error backup.h must be included after table.h + #error backup_engine.h must be included after table.h #endif namespace backup { @@ -65,7 +65,7 @@ @param flags (in) additional info about backup operation. @param tables (in) list of tables to be backed-up. - @param eng (out) pointer to backup driver instance. + @param drv (out) pointer to backup driver instance. @return Error code or <code>backup::OK</code> on success. */ @@ -87,7 +87,7 @@ @param version (in) version of the backup image. @param flags (in) additional info about restore operation. @param tables (in) list of tables to be restored. - @param eng (out) pointer to restore driver instance. + @param drv (out) pointer to restore driver instance. @return Error code or <code>backup::OK</code> on success. */ @@ -113,84 +113,6 @@ /** - @class Buffer - - @brief Used for data transfers between backup kernel and backup/restore - drivers. - - Apart from allocated memory a Buffer structure contains fields informing about - its size and holding other information about contained data. Buffers are - created and memory is allocated by backup kernel. It is also kernel's - responsibility to write contents of buffers to a backup stream. - - Data created by a backup driver is opaque to the kernel. However, to support - selective restores, each block of data can be assigned to one of the tables - being backed-up. This is done by setting <code>table_no</code> member of the - buffer structure to the number of the table to which this data belongs. Tables - are numbered from 1 acoording to their position in the list passed when driver - is created (<code>m_tables</code> member of <code>Driver</code> class). If - some of the data doesn't correspond to any particular table, then - <code>table_no</code> should be set to 0. - - This way, driver can create several "streams" of data blocks. For each table - there is a stream corresponding to that table and there is one "shared stream" - consisting of blocks with <code>table_no</code> set to 0. Upon restore, kernel - sends to a restore driver only blocks corresponding to the tables being - restored plus all the blocks from the shared stream. - - For example, consider backing-up three tables t1, t2 and t3. Data blocks - produced by a backup driver are divided into four streams: - <pre> - #0: shared data - #1: data for table t1 - #2: data for table t2 - #3: data for table t3 - </pre> - When a user restores tables t1 and t3, only blocks from streams #0, #1 and #3 - will be sent to a restore driver, but not the ones from stream #2. - - Using this approach, backup engine can arrange its backup image data in the - way which best suits its internal data representation. If needed, all data can - be put in the shared stream #0, so that all of it will be sent back to - a restore driver. On the other hand, if possible, backup data can be - distributed into per table streams to reduce the amount of data transferred - upon a selective restore. - - Backup driver signals end of data in a given stream by setting - <code>buf.last</code> flag to TRUE when get_data(buf) fills the last block of - data from that stream (otherwise <code>buf.last</code> should be FALSE). This - should be done for each stream used by the driver. Upon restore, kernel sets - <code>buf.last</code> to TRUE when sending to a restore driver the last block - of data from a stream. - - A driver learns about the size of a buffer provided by the kernel from its - <code>size</code> member. It does not have to fill the buffer completely. - It should update the <code>size</code> member to reflect the actual size - of the data in the buffer. It is possible to return no data in which case - <code>size</code> should be zero. Such empty buffers are ignored by the - kernel (no data is writen to the archive). - */ - -struct Buffer -{ - size_t size; ///< size of the buffer (of memory block pointed by data). - uint table_no; ///< Number of the table to which data in the buffer belongs. - bool last; ///< <code>TRUE</code> if this is last block of data in the stream. - byte *data; ///< Pointer to data area. - - Buffer(): data(NULL),size(0),table_no(0),last(FALSE) - {} - - void reset(size_t len) - { - size= len; - table_no= 0; - last= FALSE; - } -}; - - -/** @class Driver @brief This class contains methods which are common to both types of drivers. @@ -212,8 +134,8 @@ public: /// Types of backup/restore operations. - enum enum_flags { FULL = 0x1, ///< concerns all tables from given storage engine - PARTIAL = 0 ///< backup/restore only selected tables + enum enum_flags { FULL =0x1, ///< concerns all tables from given storage engine + PARTIAL =0 ///< backup/restore only selected tables }; /// Construct from list of tables. The list is stored for future use. --- 1.3/sql/backup/backup_kernel.h 2007-05-11 00:20:37 +02:00 +++ 1.4/sql/backup/backup_kernel.h 2007-05-11 00:20:37 +02:00 @@ -2,13 +2,42 @@ #define _BACKUP_KERNEL_API_H #include <backup/api_types.h> +#include <backup/archive.h> +#include <backup/stream.h> + + +// Called from the big switch in mysql_execute_command() to execute +// backup related statement +int execute_backup_command(THD*, LEX*); + +/** + @file + + Functions and types forming the backup kernel API + + */ + +namespace backup { + +class Archive_info; +class Backup_info; +class Restore_info; + +} // backup namespace + +// Backup kernel API + +int mysql_show_archive(THD*,const backup::Archive_info&); +int mysql_backup(THD*, backup::Backup_info&, backup::OStream&); +int mysql_restore(THD*, backup::Restore_info&, backup::IStream&); + namespace backup { /** Represents a location where backup archive can be stored. - The class is suppose to represent the location on the abstract level + The class is supposed to represent the location on the abstract level so that it is easier to add new types of locations. Currently we support only files on the server's file system. Thus the @@ -38,19 +67,167 @@ @returns NULL if the string doesn't denote a valid location */ - static Location* find(const LEX_STRING&, bool); + static Location* find(const LEX_STRING&); }; -//@{ + /** - Functions used to implement BACKUP/RESTORE SQL statements. - They are called from sql_parse.cc + Specialization of @c Archive_info which adds methods for selecting items + to backup. + + When Backup_info object is created it is empty and ready for adding items + to it. Methods @c add_table() @c add_db(), @c add_dbs() and @c add_all_dbs() + can be used for that purpose (currently only databases and tables are + supported). After populationg info object with items it should be "closed" + with a call to @c close() method. After that it is ready for use as a + description of backup archive to be created. + + A linked list of all meta-data items is pointed by @c m_items member. It + consists of three parts: first all the global items, then all per-database + items and finally all per-table items. Inside each part, items are stored in + dependency order so that if item A depends on B then B is before A in the + list (currently dependencies are not checked). One should iterate through the + meta-data item list using @c Backup_info::Item_iterator class. */ -int show_archive_catalog(THD *thd, const Location &loc); -int do_backup(THD*, const Location&, List <LEX_STRING> *db_list); -int do_restore(THD*, const Location&); -//@} -} // backup namespace +class Backup_info: public Archive_info +{ + class Table_ref; + class Db_ref; + + public: + + Backup_info(THD*); + ~Backup_info(); + + bool add_dbs(List<LEX_STRING>&); + bool add_all_dbs(); + + bool close(); + //bool close_tables(); + bool is_valid() const + { + switch (m_state) { + + case INIT: + return i_s_tables != NULL; + break; + + default: return TRUE; + } + } + + class Item_iterator; // for iterating over all meta-data items + + private: + + /// State of the info structure. + enum {INIT, // structure ready for filling + READY, // structure ready for backup (tables opened) + DONE // tables are closed + } m_state; + + /* Find image to which given table can be added and add it. + Creates new images as needed. + Returns -1 on error. + */ + int find_image(const Table_ref&); + + Db_item* add_db(const backup::Db_ref&); + Table_item* add_table(Db_item&, const Table_ref&); + + bool add_db_items(Db_item&); + bool add_table_items(Table_item&); + + THD *m_thd; + TABLE *i_s_tables; + + //bool open_tables(); + + Item *m_items; + Item *m_last_item; + Item *m_last_db; +}; + +/** + Used to iterate over meta-data items in a @c Backup_info structure. + + Usage: + <pre> + Backup_info info; + for (Backup_info::Item_iterator it(info); it ; it++) + { + it->archive_item_method() + } + </pre> + or + <pre> + Backup_info info; + Backup_info::Item_iterator it(info); + Archive_info::Item *p; + + while ((p=it++)) + { + @<use p here> + } + </pre> + */ +class Backup_info::Item_iterator +{ + Item *m_curr; + Item *m_prev; + + public: + + Item_iterator(const Backup_info &info): m_prev(NULL) + { m_curr= info.m_items; } + + operator bool() const + { return m_curr != NULL; } + + Item* operator++(int); + + Item* operator->() + { DBUG_ASSERT(m_curr); + return m_curr; } +}; + + +/** + Specialization of @c Archive_info which is used to select and restore items + from a backup archive. + + An instance of this class is created by reading backup archive header and it + describes contents of the archive. @c Restore_info methods select which items + should be restored. Instances of @c Restore_info::Item class are created when + reading meta-data info stored in the archive. They are used to restore the + meta-data items (but not the table data, which is done by restore drivers). + + @note This class is not fully implemented. Right now it is not possible to + select items to restore - always all items are restored. + */ + +class Restore_info: public Archive_info +{ + public: + + Restore_info(IStream &s) + { + stream_result::value res= read(s); + DBUG_ASSERT(res != stream_result::ERROR); + } + + bool is_valid() const + { return TRUE; } + + bool restore_all_dbs() + { return TRUE; } + + /// Determine if given item is selected for restore. + bool selected(const Archive_info::Item&) + { return TRUE; } +}; + +} // backup namespace #endif --- 1.4/sql/backup/be_default.h 2007-05-11 00:20:37 +02:00 +++ 1.5/sql/backup/be_default.h 2007-05-11 00:20:37 +02:00 @@ -1,42 +1,15 @@ #ifndef _DEFAULT_BACKUP_H #define _DEFAULT_BACKUP_H -#include "archive.h" - /** - * @file default_backup.h - * - * @brief Contains the default backup algorithm driver. - * - * This file contains the default backup algorithm (also called a "driver" - * in the online backup terminology. The default backup algorithm may be - * used in place of an engine-specific driver if one does not exist or if - * chosen by the user. - * - * The default backup algorithm is a blocking algorithm that locks all of - * the tables given at the start of the backup/restore process. Once all of - * the data is backed up or restored, the locks are removed. The default - * backup is a row-level backup and therefore does not backup the indexes - * or any of the engine-specific files. - * - * The classes in this file use the namespace "default_backup" to distinguish - * these classes from other backup drivers. The backup functionality is - * contained in the backup class shown below. Similarly, the restore - * functionality is contained in the restore class below. - * - * The format of the backup is written as a series of data blocks where the - * first block contains the record buffer (table->record[0]). This is - * followed by one or more blocks containing the blob (text) fields one per - * block. This sequence is repeated for each row in the table. - * - * blknum contents - * ------------------------------------------------------ - * 1 (byte) copy record_buffer read from handler - * ------------------------------------------------------ - * 2...n (uint32) size of blob data to read - * (byte) blob data for blob - * ------------------------------------------------------ + @file + + Declarations for the default backup engine. */ + +#include <backup/backup_engine.h> +#include <backup/archive.h> // to define default backup image class + namespace default_backup { using backup::byte; @@ -75,12 +48,14 @@ * Given a list of tables to be backed-up, create instance of backup * driver which will create backup image of these tables. * + * @param flags (in) additional info about type of backup operation to be + * performed. * @param tables (in) list of tables to be backed-up. - * @param eng (out) pointer to backup driver instance. + * @param drv (out) pointer to backup driver instance. * * @return Error code or <code>backup::OK</code> on success. */ - result_t get_backup(const uint32, const Table_list &tables, Backup_driver* &drv); + result_t get_backup(const uint32 flags, const Table_list &tables, Backup_driver* &drv); /** * Create a default backup restore driver. @@ -89,12 +64,14 @@ * driver which will restore these tables from a backup image. * * @param version (in) version of the backup image. + * @param flags (in) additional info about type of restore operation to be + * performed. * @param tables (in) list of tables to be restored. - * @param eng (out) pointer to restore driver instance. + * @param drv (out) pointer to restore driver instance. * * @return Error code or <code>backup::OK</code> on success. */ - result_t get_restore(version_t ver, const uint32, const Table_list &tables, + result_t get_restore(version_t version, const uint32 flags, const Table_list &tables, Restore_driver* &drv); /** @@ -114,7 +91,7 @@ * a table scan on each table reading the rows and saving the data to the * buffer from the backup algorithm. * - * @see <backup driver> + * @see @<backup driver> */ class Backup: public Backup_driver { @@ -235,7 +212,7 @@ * data for each table by writing the data for the rows from the * buffer given by the backup algorithm. * - * @see <restore driver> + * @see @<restore driver> */ class Restore: public Restore_driver { @@ -334,22 +311,19 @@ /********************************************************************* - Classes describing backup image created using default backup engine. + Default image class *********************************************************************/ namespace backup { -class Default_backup_info: public Backup_subimage +class Default_image: public Image_info { public: - Default_backup_info(uint no): Backup_subimage(no) - {} - - version_t ver() const - { return 1; } + Default_image(Archive_info &info): Image_info(info) + { ver= 1; } image_type type() const { return DEFAULT_IMAGE; } @@ -357,40 +331,23 @@ const char* name() const { return "Default driver"; } - result_t do_create_driver() - { - m_drv= new default_backup::Backup(tables, ::current_thd); + bool accept(const Table_ref&, const ::handlerton*) + { return TRUE; }; // accept all tables - return m_drv? OK : ERROR; - } + Backup_driver* get_backup_driver() + { return new default_backup::Backup(tables,::current_thd); } - bool add(const Table_ref&, const ::handlerton*); - bool do_write_description(OStream&); -}; + Restore_driver* get_restore_driver() + { return new default_backup::Restore(tables,::current_thd); } -class Default_restore_info: public Restore_subimage -{ - public: - - Default_restore_info(uint no, version_t ver): Restore_subimage(no, ver) - {} - - image_type type() const - { return DEFAULT_IMAGE; } - - const char* name() const - { return "Default driver"; } - - result_t do_create_driver() - { - m_drv= new default_backup::Restore(tables, ::current_thd); - - return m_drv? OK : ERROR; - } - - static Default_restore_info* create_from_stream(uint, version_t, IStream&); + stream_result::value do_write_description(OStream&) + { return stream_result::OK; }; // nothing to write + + static Default_image* + create_from_stream(version_t ver, Archive_info &info, + IStream&) + { return new Default_image(info); }; // TODO: check format version }; - } // backup namespace --- 1.3/sql/backup/be_nodata.cc 2007-05-11 00:20:37 +02:00 +++ 1.4/sql/backup/be_nodata.cc 2007-05-11 00:20:37 +02:00 @@ -1,3 +1,9 @@ +/** + @file + + Implementation of a trivial backup engine which stores no data. + */ + #include "../mysql_priv.h" #include "be_nodata.h" --- 1.2/sql/backup/be_nodata.h 2007-05-11 00:20:37 +02:00 +++ 1.3/sql/backup/be_nodata.h 2007-05-11 00:20:37 +02:00 @@ -1,6 +1,12 @@ #ifndef _BE_NODATA_H #define _BE_NODATA_H +/** + @file + + Declarations for the no-data backup engine. + */ + #include "backup/backup_engine.h" namespace backup { --- 1.15/sql/backup/data_backup.cc 2007-05-11 00:20:37 +02:00 +++ 1.16/sql/backup/data_backup.cc 2007-05-11 00:20:37 +02:00 @@ -1,24 +1,44 @@ #include "../mysql_priv.h" +/** + @file + + Code used to backup table data. + + Function @c write_table_data() and @c restore_table_data() use backup/restore + drivers and protocols to create image of the data stored in the tables being + backed up. + */ + #include "backup_engine.h" #include "stream.h" -#include "archive.h" +#include "backup_kernel.h" + + +/*********************************************** + + DATA BACKUP + + ***********************************************/ namespace backup { struct backup_state { - enum value { INACTIVE, - INIT, - WAITING, - PREPARING, - READY, - FINISHING, - DONE, - SHUT_DOWN, + /// State of a single backup driver. + enum value { INACTIVE, ///< Before backup process is started (phase 0). + INIT, ///< During initial data transfer (phase 1). + WAITING, ///< Waiting for other drivers to finish init phase (phase 2). + PREPARING, ///< Preparing for @c lock() call (phase 3). + READY, ///< Ready for @c lock() call (phase 4) + FINISHING, ///< Final data transfer (phase 7). + DONE, ///< Backup complete. + SHUT_DOWN, ///< After @c end() call. ERROR, MAX }; +#ifndef DBUG_OFF + static const char* name[]; struct Initializer @@ -40,11 +60,24 @@ private: static Initializer init; + +#endif + }; +#ifndef DBUG_OFF + const char* backup_state::name[backup_state::MAX]; backup_state::Initializer init; +#endif + +/** + Used to write data blocks to a stream. + + This class defines how buffers are allocated for data transfers + (@c get_buf() method). Each block is written as a separate chunk of data. + */ struct Block_writer { enum result_t { OK, NO_RES, ERROR }; @@ -58,12 +91,10 @@ private: -// uint m_no; OStream &m_str; size_t buf_size; }; - /** @class Backup_pump @@ -78,64 +109,41 @@ struct Backup_pump { - backup_state::value state; - enum { READING, WRITING } mode; - size_t init_size; + backup_state::value state; ///< State of the backup driver. + + enum { READING, ///< Pump is polling driver for data. + WRITING ///< Pump sends data to the stream. + } mode; + + size_t init_size; ///< The estimate returned by backup driver's @c init_data() method. size_t bytes_in, bytes_out; - const char *m_name; + const char *m_name; ///< Name of the driver (for debug purposes). - Backup_pump(Backup_subimage&, Block_writer&); + Backup_pump(uint, Image_info&, Block_writer&); ~Backup_pump(); size_t pump(); - bool prepare(); - bool begin() - { - state= backup_state::INIT; - DBUG_PRINT("backup/data",(" %s enters INIT state",m_name)); - return m_drv.begin(m_buf_size) == backup::OK; - } - - bool end() - { - if (state != backup_state::SHUT_DOWN) - { - DBUG_PRINT("backup/data",(" shutting down %s",m_name)); - m_drv.end(); - m_drv.free(); - state= backup_state::SHUT_DOWN; - } - return TRUE; - } - - - bool lock() - { - DBUG_PRINT("backup/data",(" locking %s",m_name)); - return m_drv.lock() == backup::OK; - } - - bool unlock() - { - DBUG_PRINT("backup/data",(" unlocking %s, goes to FINISHING state",m_name)); - state= backup_state::FINISHING; - return m_drv.unlock() == backup::OK; // FIXME: better handling of errors - } + bool begin(); + bool end(); + bool prepare(); + bool lock(); + bool unlock(); + /// Return the backup driver used by the pump. Backup_driver &drv() const - { return m_drv; } + { return *m_drv; } private: - static const uint m_buf_size= 2048; + static const uint m_buf_size= 2048; ///< Size of buffers used for data transfers. static const uint get_buf_retries= 3; /**< if block writer has no buffers, retry this many times before giving up */ - uint m_drv_no; - Backup_driver &m_drv; - Block_writer &m_bw; - Buffer m_buf; + uint m_drv_no; ///< The number of this image in the backup archive. + Backup_driver *m_drv; ///< Pointer to the backup driver. + Block_writer &m_bw; ///< Block writer used for writing data blocks. + Buffer m_buf; ///< Buffer used for data transfers. /** Pointer to the memory area used as write buffer. @@ -152,6 +160,7 @@ bool drop_buf(); bool write_buf(); + /// Bitmap showing which streams have been closed by the driver. MY_BITMAP m_closed_streams; void mark_stream_closed(uint stream_no) @@ -166,32 +175,6 @@ }; -bool Backup_pump::prepare() -{ - result_t res= m_drv.prelock(); - - state= backup_state::PREPARING; - - switch (res) { - - case READY: - state= backup_state::READY; - - case OK: - res= OK; - break; - - default: - break; - - } - - DBUG_PRINT("backup/data",(" preparing %s, goes to %s state", - m_name,backup_state::name[state])); - - return res == backup::OK; -} - /** @class Scheduler @@ -201,10 +184,14 @@ */ class Scheduler { + class Pump; + class Pump_iterator; + public: + void add(Pump*); void step(); - void add(Backup_subimage&); + void prepare(); void lock(); void unlock(); @@ -224,94 +211,12 @@ private: - // Extend Backup_pump with information abouts its position relative - // to other pumps. - - class Pump: public Backup_pump - { - size_t start_pos; - Block_writer bw; - - Pump(OStream &s, Backup_subimage &img, size_t pos): - Backup_pump(img,bw), start_pos(pos), bw(2048,s) - {} - - friend class Scheduler; - - public: - - size_t pos() const - { return start_pos + bytes_in; } - - }; - - struct Pump_ptr - { - LIST *el; - - Pump* operator->() - { - return el? static_cast<Pump*>(el->data) : NULL; - } - - void operator++() - { - if(el) el= el->next; - } - - operator bool() const - { return el && el->data; } - - void operator=(const Pump_ptr &p) - { el= p.el; } - - Pump_ptr(): el(NULL) - {} - - Pump_ptr(LIST *el): el(el) - {} - }; - LIST *m_pumps, *m_last; - uint m_count; // current number of pumps - size_t m_total; // accumulated position of all drivers - size_t m_init_left; // how much of init data is left (estimate) - uint m_known_count; // no. drivers which can estimate init data size - OStream &m_str; - - void move_pump_to_end(const Pump_ptr &p) - { - DBUG_ASSERT(m_pumps); - if (m_last != p.el) - { - m_pumps= list_delete(m_pumps,p.el); - m_last->next= p.el; - p.el->prev= m_last; - p.el->next= NULL; - m_last= p.el; - } - } - - void remove_pump(Pump_ptr &p) - { - DBUG_ASSERT(p.el); - - if (m_last == p.el) - m_last= m_last->prev; - - if (m_pumps) - { - m_pumps= list_delete(m_pumps,p.el); - m_count--; - } - - if (p) - { - p->end(); - delete static_cast<Pump*>(p.el->data); - my_free((::gptr)p.el,MYF(0)); - } - } + uint m_count; ///< current number of pumps + size_t m_total; ///< accumulated position of all drivers + size_t m_init_left; ///< how much of init data is left (estimate) + uint m_known_count; ///< no. drivers which can estimate init data size + OStream &m_str; ///< stream to which we write Scheduler(OStream &s): init_count(0), prepare_count(0), finish_count(0), @@ -320,10 +225,34 @@ m_str(s) {} - friend bool write_table_data(const Backup_info&, OStream&); + void move_pump_to_end(const Pump_iterator&); + void remove_pump(Pump_iterator&); + friend bool write_table_data(THD*, Backup_info&, OStream&); }; +/** + Extend Backup_pump with information about its position relative + to other pumps. + */ +class Scheduler::Pump: public Backup_pump +{ + size_t start_pos; + Block_writer bw; + + friend class Scheduler; + + public: + + Pump(uint no, Image_info &img, OStream &s): + Backup_pump(no,img,bw), start_pos(0), bw(2048,s) + {} + + size_t pos() const + { return start_pos + bytes_in; } +}; + + #define CHECK_STATE(P,S) \ do { \ @@ -332,47 +261,56 @@ DBUG_ASSERT( (P).state == (S) ); \ } while(0) -// Poll all backup drivers involved in the backup image for their backup -// data and write to a backup stream. -// Currently only one driver is handled. -bool write_table_data(const Backup_info &info, OStream &s) +/** + Save data from tables being backed up. + + Function initializes and controls backup drivers which create the image + of table data. Currently single thread is used and drivers are polled in + a round robin fashion. + */ +bool write_table_data(THD*, Backup_info &info, OStream &s) { DBUG_ENTER("backup::write_table_data"); + info.data_size= 0; + if (info.img_count==0 || info.table_count==0) // nothing to backup DBUG_RETURN(TRUE); Scheduler sch(s); // scheduler instance - List<Backup_subimage> inactive; // list of images not yet being created + List<Scheduler::Pump> inactive; // list of images not yet being created size_t max_init_size=0; // keeps maximal init size for images in inactive list + size_t start_bytes= s.bytes; + DBUG_PRINT("backup/data",("initializing scheduler")); // add unknown "at end" drivers to scheduler, rest to inactive list - List_iterator<Backup_subimage> it(const_cast< List<Backup_subimage>& >(info.images)); - Backup_subimage *img; - - while ((img= it++)) + for (uint no=0; no < info.img_count; ++no) { - result_t res; + Image_info *i= info.images[no]; - res= img->create_driver(); - DBUG_ASSERT(res == OK); + if (!i) + continue; + + Scheduler::Pump *p= new Scheduler::Pump(no,*i,s); - size_t init_size= img->init_size; + DBUG_ASSERT(p); + + size_t init_size= p->init_size; if (init_size == Driver::UNKNOWN_SIZE) { - sch.add(*img); + sch.add(p); } else { if (init_size > max_init_size) max_init_size= init_size; - inactive.push_back(img); + inactive.push_back(p); } } @@ -388,10 +326,12 @@ DBUG_PRINT("backup/data",("-- INIT PHASE --")); - // poll "at end" drivers activating inactive ones on the way - // note: if scheduler is empty and there are images with non-zero - // init size (max_init_size > 0) then enter the loop as one such image - // will be added to the scheduler inside. + /* + poll "at end" drivers activating inactive ones on the way + note: if scheduler is empty and there are images with non-zero + init size (max_init_size > 0) then enter the loop as one such image + will be added to the scheduler inside. + */ while (sch.init_count > 0 || sch.is_empty() && max_init_size > 0) { @@ -400,12 +340,12 @@ if (max_init_size > 0 && sch.init_left() <= max_init_size) { - List_iterator<Backup_subimage> it(inactive); - Backup_subimage *p; + List_iterator<Scheduler::Pump> it(inactive); + Scheduler::Pump *p; size_t second_max= 0; max_init_size= 0; - img= NULL; + Scheduler::Pump *p1= NULL; while ((p= it++)) { @@ -413,25 +353,28 @@ { second_max= max_init_size; max_init_size= p->init_size; - img= p; + p1= p; } } max_init_size= second_max; - sch.add(*img); + sch.add(p1); } + // poll drivers + sch.step(); } // start "at begin" drivers DBUG_PRINT("backup/data",("- activating \"at begin\" drivers")); - List_iterator<Backup_subimage> it1(inactive); + List_iterator<Scheduler::Pump> it1(inactive); + Scheduler::Pump *p; - while ((img= it1++)) - sch.add(*img); + while ((p= it1++)) + sch.add(p); while (sch.init_count > 0) sch.step(); @@ -459,18 +402,60 @@ DBUG_PRINT("backup/data",("-- DONE --")); sch.close(); - DBUG_RETURN(TRUE); + info.data_size= s.bytes - start_bytes; + DBUG_RETURN(TRUE); } +} // backup namespace + /************************************************** - * Implementation of Scheduler + + Implementation of Scheduler + **************************************************/ -void Scheduler::step() +namespace backup { + +/** + Used to iterate over backup pumps of a scheduler. + */ +struct Scheduler::Pump_iterator { + LIST *el; + Pump* operator->() + { + return el? static_cast<Pump*>(el->data) : NULL; + } + void operator++() + { + if(el) el= el->next; + } + + operator bool() const + { return el && el->data; } + + void operator=(const Pump_iterator &p) + { el= p.el; } + + Pump_iterator(): el(NULL) + {} + + Pump_iterator(const Scheduler &sch): el(sch.m_pumps) + {} + +}; + +/** + Pick next backup pump and call its @c pump() method. + + Method updates statistics of number of drivers in each phase which is used + to detect end of a backup process. + */ +void Scheduler::step() +{ /* find pump with least data read. Pick next driver to pump data from. The driver with least data read @@ -479,7 +464,8 @@ TODO: implement this */ - Pump_ptr p(m_pumps); + + Pump_iterator p(*this); /* for (Pump_ptr it(m_pumps); it; ++it) @@ -564,18 +550,17 @@ } - -void Scheduler::add(Backup_subimage &img) +/// Add backup pump to the scheduler. +void Scheduler::add(Pump *p) { size_t avg= m_count? m_total/m_count + 1 : 0; bool res; - Pump *p= new Pump(m_str,img,avg); - DBUG_ASSERT(p); + p->start_pos= avg; DBUG_PRINT("backup/data",("Adding %s to scheduler (at pos %lu)", - img.name(), (unsigned long)avg)); + p->m_name, (unsigned long)avg)); m_pumps= list_cons(p,m_pumps); if (!m_last) @@ -587,9 +572,9 @@ res= p->begin(); DBUG_ASSERT(res); - if (img.init_size != Driver::UNKNOWN_SIZE) + if (p->init_size != Driver::UNKNOWN_SIZE) { - m_init_left += img.init_size; + m_init_left += p->init_size; m_known_count++; } @@ -615,12 +600,53 @@ DBUG_PRINT("backup/data",("total init data size estimate: %lu",(unsigned long)m_init_left)); } +/// Move backup pump to the end of scheduler's list. +void Scheduler::move_pump_to_end(const Pump_iterator &p) +{ + DBUG_ASSERT(m_pumps); + if (m_last != p.el) + { + m_pumps= list_delete(m_pumps,p.el); + m_last->next= p.el; + p.el->prev= m_last; + p.el->next= NULL; + m_last= p.el; + } +} + +/** + Remove backup pump from the scheduler. + + The corresponding backup driver is shut down using @c end() call. + */ +void Scheduler::remove_pump(Pump_iterator &p) +{ + DBUG_ASSERT(p.el); + + if (m_last == p.el) + m_last= m_last->prev; + + if (m_pumps) + { + m_pumps= list_delete(m_pumps,p.el); + m_count--; + } + + if (p) + { + p->end(); + delete static_cast<Pump*>(p.el->data); + my_free((::gptr)p.el,MYF(0)); + } +} + +/// Start prepare phase for all drivers. void Scheduler::prepare() { DBUG_ASSERT(init_count==0); DBUG_PRINT("backup/data",("calling prepare() for all drivers")); - for(Pump_ptr it(m_pumps); it; ++it) + for(Pump_iterator it(*this); it; ++it) { it->prepare(); if (it->state == backup_state::PREPARING) @@ -631,22 +657,24 @@ m_count, init_count, prepare_count, finish_count)); } +/// Lock all drivers. void Scheduler::lock() { DBUG_ASSERT(prepare_count==0); DBUG_PRINT("backup/data",("calling lock() for all drivers")); - for(Pump_ptr it(m_pumps); it; ++it) + for(Pump_iterator it(*this); it; ++it) it->lock(); DBUG_PRINT("backup/data",("driver counts: total=%u, init=%u, prepare=%u, finish=%u.", m_count, init_count, prepare_count, finish_count)); } +/// Unlock all drivers. void Scheduler::unlock() { DBUG_PRINT("backup/data",("calling unlock() for all drivers")); - for(Pump_ptr it(m_pumps); it; ++it) + for(Pump_iterator it(*this); it; ++it) { it->unlock(); if (it->state == backup_state::FINISHING) @@ -654,25 +682,28 @@ } } +/// Shut down backup process. void Scheduler::close() { // shutdown any remaining drivers while (m_count && m_pumps) { - Pump_ptr p(m_pumps); + Pump_iterator p(*this); remove_pump(p); } } /************************************************** - * Implementation of Backup_pump + + Implementation of Backup_pump + **************************************************/ -Backup_pump::Backup_pump(Backup_subimage &img, Block_writer &bw): +Backup_pump::Backup_pump(uint no, Image_info &img, Block_writer &bw): state(backup_state::INACTIVE), mode(READING), - init_size(img.init_size), bytes_in(0), bytes_out(0), - m_drv_no(img.no), m_drv(img.driver()), m_bw(bw), m_buf_head(NULL), + init_size(0), bytes_in(0), bytes_out(0), + m_drv_no(no), m_drv(NULL), m_bw(bw), m_buf_head(NULL), m_buf_retries(0) { m_buf.data= NULL; @@ -681,13 +712,95 @@ 1+img.tables.count(), FALSE); // not thread safe m_name= img.name(); + m_drv= img.get_backup_driver(); + DBUG_ASSERT(m_drv); + init_size= m_drv->init_size(); } Backup_pump::~Backup_pump() { + if (m_drv) + m_drv->free(); bitmap_free(&m_closed_streams); } +/// Initialize backup driver. +bool Backup_pump::begin() +{ + state= backup_state::INIT; + DBUG_PRINT("backup/data",(" %s enters INIT state",m_name)); + return m_drv->begin(m_buf_size) == backup::OK; +} + +/// Shut down the driver. +bool Backup_pump::end() +{ + if (state != backup_state::SHUT_DOWN) + { + DBUG_PRINT("backup/data",(" shutting down %s",m_name)); + m_drv->end(); + state= backup_state::SHUT_DOWN; + } + return TRUE; +} + +/// Start prepare phase for the driver. +bool Backup_pump::prepare() +{ + result_t res= m_drv->prelock(); + + state= backup_state::PREPARING; + + switch (res) { + + case READY: + state= backup_state::READY; + + case OK: + res= OK; + break; + + default: + break; + + } + + DBUG_PRINT("backup/data",(" preparing %s, goes to %s state", + m_name,backup_state::name[state])); + return res == backup::OK; +} + +/// Request VP from the driver. +bool Backup_pump::lock() +{ + DBUG_PRINT("backup/data",(" locking %s",m_name)); + return m_drv->lock() == backup::OK; +} + +/// Unlock the driver after VP creation. +bool Backup_pump::unlock() +{ + DBUG_PRINT("backup/data",(" unlocking %s, goes to FINISHING state",m_name)); + state= backup_state::FINISHING; + return m_drv->unlock() == backup::OK; // FIXME: better handling of errors +} + +/** + Poll the driver for next block of data and/or write data to stream. + + Depending on the current mode in which the pump is operating (@c mode member) + the backup driver is polled for image data or data obtained before is written + to the stream. Answers from drivers @c get_data() method are interpreted and + the state of the driver is updated accordingly. + + Each block of data obtained from a driver is prefixed with the image + number and the table number stored in @c buf.table_no before it is written to + the stream: + <pre> + | img no | table no | data from the driver | + </pre> + */ + // FIXME: return number of bytes *written* to stream. size_t Backup_pump::pump() @@ -751,7 +864,7 @@ DBUG_ASSERT(m_buf_head); - result_t res= m_drv.get_data(m_buf); + result_t res= m_drv->get_data(m_buf); switch (res) { @@ -842,84 +955,61 @@ return howmuch; } +} // backup namespace -/************************************************** - * Implementation of Block_writer - **************************************************/ - -Block_writer::result_t -Block_writer::get_buf(Buffer &buf) -{ - buf.table_no= 0; - buf.last= FALSE; - buf.size= buf_size; - buf.data= m_str.get_window(buf.size); - - if (!buf.data) - return NO_RES; - - return OK; -} - -Block_writer::result_t -Block_writer::write_buf(const Buffer &buf) -{ - stream_result::value res= m_str.write_window(buf.size); - m_str.end_chunk(); - return res == stream_result::OK ? OK : ERROR; -} - -Block_writer::result_t -Block_writer::drop_buf(Buffer &buf) -{ - m_str.drop_window(); - buf.data= NULL; - buf.size= 0; - return OK; -} +/*********************************************** -} // backup namespace + DATA RESTORE + ***********************************************/ namespace backup { -// Read backup image data from a backup stream and forward it to restore drivers. -// This simple implementation handles only one restore driver. - -bool restore_table_data(const Restore_info &info, IStream &s) +/** + Read backup image data from a backup stream and forward it to restore drivers. + */ +bool restore_table_data(THD*, Restore_info &info, IStream &s) { DBUG_ENTER("restore::restore_table_data"); enum { READING, SENDING, DONE, ERROR } state= READING; + // FIXME: when selective restores are implemented, check that + // nothing was *selected* to be restored. + if (info.img_count==0 || info.table_count==0) // nothing to restore DBUG_RETURN(TRUE); - Restore_driver* drv[32]; - uint drv_count; + result_t res; + Restore_driver* drv[256]; - List_iterator<Restore_subimage> it(const_cast<Restore_info&>(info).images); - Restore_subimage *img; + DBUG_ASSERT(info.img_count < 256); - for(drv_count=0; (img= it++) && drv_count < 32 ; ++drv_count) + for (uint no=0; no < info.img_count; ++no) { - result_t res; + drv[no]= NULL; + + Image_info *img= info.images[no]; - DBUG_PRINT("restore",("Creating %s (image %u)",img->name(), img->no+1)); - drv[drv_count]= img->driver(); - DBUG_ASSERT(drv[drv_count]); + // note: img can be NULL if it is not used in restore. + if (!img) + continue; - res= drv[drv_count]->begin(0); // TODO: provide correct size + drv[no]= img->get_restore_driver(); + DBUG_ASSERT(drv[no]); + + res= drv[no]->begin(0); // TODO: provide correct size DBUG_ASSERT(res == OK); } Buffer buf; - uint img_no; + uint img_no=0; uint repeats=0, errors= 0; static const uint MAX_ERRORS= 3; static const uint MAX_REPEATS= 7; + size_t start_bytes= s.bytes; // main data reading loop @@ -960,7 +1050,7 @@ case SENDING: - if( img_no < 1 || !drv[img_no-1] ) + if( img_no < 1 || img_no > info.img_count || !drv[img_no-1] ) { DBUG_PRINT("restore",("Skipping data for subimage %u",img_no)); state= READING; @@ -1019,8 +1109,13 @@ if (state != DONE) DBUG_PRINT("restore",("state is %d",state)); - for(uint no=0; no < drv_count; ++no) + info.data_size= s.bytes - start_bytes; + + for(uint no=0; no < info.img_count; ++no) { + if (!drv[no]) + continue; + DBUG_PRINT("restore",("Shutting down driver %u",no)); drv[no]->end(); drv[no]->free(); @@ -1031,4 +1126,71 @@ } // backup namespace + + +/************************************************** + + Implementation of Block_writer + + **************************************************/ + +namespace backup { + +/** + Allocate new buffer for data transfer. + + The buffer size is given by @c buf.size member. + + Current implementation tries to allocate the data transfer bufer in the + stream. It can handle only one buffer at a time. + + @returns @c NO_RES if buffer can not be allocated, @c OK otherwise. + */ +Block_writer::result_t +Block_writer::get_buf(Buffer &buf) +{ + buf.table_no= 0; + buf.last= FALSE; + buf.size= buf_size; + buf.data= m_str.get_window(buf.size); + + if (!buf.data) + return NO_RES; + + return OK; +} + +/** + Write block of data to stream. + + The buffer containing data must be obtained from a previous @c get_buf() call. + After this call, bufer is returned to the buffer pool and can be reused for + other transfers. + */ +Block_writer::result_t +Block_writer::write_buf(const Buffer &buf) +{ + stream_result::value res= m_str.write_window(buf.size); + m_str.end_chunk(); + return res == stream_result::OK ? OK : ERROR; +} + +/** + Return buffer to the buffer pool. + + If a buffer obtained from @c get_buf() is not written to the stream, this + method can return it to the buffer pool so that it can be reused for other + transfers. + */ +Block_writer::result_t +Block_writer::drop_buf(Buffer &buf) +{ + m_str.drop_window(); + buf.data= NULL; + buf.size= 0; + return OK; +} + +} // backup namespace + --- 1.15/sql/backup/meta_backup.cc 2007-05-11 00:20:37 +02:00 +++ 1.16/sql/backup/meta_backup.cc 2007-05-11 00:20:37 +02:00 @@ -1,392 +1,330 @@ -#include "../mysql_priv.h" -#include "sql_show.h" -#include "event_scheduler.h" - -#include "tables.h" -#include "archive.h" -#include "be_default.h" +/** + @file + + Code to save/read meta-data image. + */ + +/* + TODO: + + - Handle events, routines, triggers and other item types (use Chuck's code). + */ + +#include <mysql_priv.h> +#include <sql_show.h> +//#include <event_scheduler.h> +//#include <sp.h> +//#include <sql_trigger.h> +//#include <log.h> + #include "backup_aux.h" -#include "sp.h" -#include "sql_trigger.h" -#include "log.h" +#include "backup_kernel.h" +#include "meta_backup.h" -namespace backup { -/*********************************************************** +/******************************************************* - Writing/reading archive header and catalog. + Functions used by kernel to save/restore meta-data - ***********************************************************/ + *******************************************************/ -bool write_header(const Backup_info &info, OStream &str) -{ - DBUG_ENTER("backup::write_header"); - stream_result::value res= str.write2int(info.ver); - TEST_WR_RES(res); +namespace backup { - DBUG_PRINT("restore",("header written")); +/** + Write meta-data description to backup stream. - DBUG_PRINT("restore",("write images")); + Meta-data information is stored as a sequence of entries, each describing a + single meta-data item. The format of a single entry depends on the type of + item (see @c Archive_info::Item::save() method). The entries are saved in a + single chunk. - List_iterator<Backup_subimage> it(const_cast<List<Backup_subimage>&>(info.images)); - Backup_subimage *img; + The order of entries is important. Items on which other items depend should + be saved first in the sequence. This order is determined by the implementation + of @c Backup_info::Item_iterator class. - // List of sub-images + @note Meta-data description is added to the current chunk of the stream which + is then closed. + */ +bool +write_meta_data(THD *thd, Backup_info &info, OStream &s) +{ + DBUG_ENTER("backup::write_meta_data"); - uint ino; + size_t start_bytes= s.bytes; - for( ino=0 ; (img= it++) ; ino++ ) + for (Backup_info::Item_iterator it(info); it; it++) { - DBUG_PRINT("backup",(" adding %s image to catalog",img->name())); - img->write_description(str); // write description of sub-image format + stream_result::value res= it->save(thd,s); // this calls Archive_info::Item::save() method + DBUG_ASSERT(res != stream_result::ERROR); } - str.end_chunk(); - - DBUG_PRINT("restore",("images written")); + s.end_chunk(); - if( ino == 0 ) - DBUG_RETURN(TRUE); + info.meta_size= s.bytes - start_bytes; - DBUG_PRINT("backup",(" ===")); DBUG_RETURN(TRUE); } -/* - Write the catalog to the stream. - - SYNOPSIS - write_catalog() - Backup_info *info - The backup information for this archive. - OStream str - The stream to write to. - DESCRIPTION - This procedure writes the contents of the catalog to the - stream. Lists are prequalified with the number of elements to - enable faster construction of the catalog on read. - - NOTES - The format of this information is stored in the stream as - follows: +/** + Read meta-data items from a stream and create them if they are selected + for restore. - <int> #schema_refs to read - schema_ref - db_name - <int> #db metadata strings - db_metadata - ... - <int> #table_info structs to read - table_info - table_name - <int> # table metadata strings - table_metadata - ... - ... - ... - - RETURNS - 0 - no errors. - -1 - table or table name missing or not found in string. -*/ -bool write_catalog(const Backup_info &info, OStream &str) + @pre Stream is at the beginning of the meta-data chunk. + @post Stream is at the beginning of the first chunk after the meta-data one. + */ +bool +restore_meta_data(THD *thd, Restore_info &info, IStream &s) { - uint ino; + Archive_info::Item *it; // save pointer to item read form the stream + Db_ref curr_db; // remember the current database - DBUG_ENTER("backup::write_catalog"); - List_iterator<Backup_subimage> it(const_cast<List<Backup_subimage>&>(info.images)); - Backup_subimage *img; + size_t start_bytes= s.bytes; - List<schema_ref> cat= info.catalog; - List_iterator<schema_ref> catalog(cat); - schema_ref *tmp; - str.write2int(cat.elements); // write # of schema_refs to read on restore - while ((tmp= catalog++)) + while ((it= Archive_info::Item::create_from_stream(info,s))) // read next item from the stream { - str.writestr(tmp->db_name.c_ptr()); // write db name - str.write2int(tmp->db_metadata.elements); // write # db metadata strings - String *dbstr; - List_iterator<String> db_meta(tmp->db_metadata); - while ((dbstr= db_meta++)) - str.writestr(dbstr->c_ptr()); // write db metadata string - str.write2int(tmp->tables.elements); // write # table_info structs - table_info *t; - List_iterator<table_info> tbls(tmp->tables); - str.end_chunk(); - while ((t= tbls++)) + DBUG_PRINT("restore",(" got next meta-item.")); + if (info.selected(*it)) // if the item was selected for restore ... { - str.writestr(t->table_name.c_ptr()); - str.write2int(t->table_metadata.elements); // # table metadata strings - String *s; - List_iterator<String> tbl_meta(t->table_metadata); - while ((s= tbl_meta++)) - { - str.writestr(s->c_ptr()); // write table metadata string - str.end_chunk(); - } - } - } + DBUG_PRINT("restore",(" creating it!")); - // Write list of tables + // change the current database if we are going to create a per-db item + // and we are not already in the correct one. - it.rewind(); - for( ino=0 ; (img= it++) ; ino++ ) - { - str << img->tables; // write list of tables stored in the image + const Db_ref db= it->meta().in_db(); + + if (db.is_valid() && (!curr_db.is_valid() || db != curr_db)) + { + DBUG_PRINT("restore",(" changing current db to %s",db.name().ptr())); + curr_db= db; + change_db(thd,db); + } + + bool res= it->meta().create(thd); // This calls method create() in the + // appropriate subclass of meta::Item class + DBUG_ASSERT(res); // TODO: handle errors! + + delete it; + } } - DBUG_PRINT("backup",("catalog written")); + stream_result::value res= s.next_chunk(); - DBUG_RETURN(TRUE); + info.meta_size= s.bytes - start_bytes; + + return res != stream_result::ERROR; } -bool read_header(IStream &str, Restore_info &info) -{ - Restore_subimage *img; - uint ino; +} // backup namespace - DBUG_ENTER("restore::read_header"); - stream_result::value res= str.read2int(info.ver); - TEST_RD_RES(res); - DBUG_PRINT("restore",("header read")); +/********************************************* - DBUG_PRINT("restore",("read images")); + Save/restore for different meta-data items. - for (ino=0 ; (img= Restore_subimage::create_from_stream(ino,str)); ino++) - { - DBUG_PRINT("restore",(" got %s image",img->name())); - info.images.push_back(img); - } + *********************************************/ - info.img_count= ino; +namespace backup { - res= str.next_chunk(); // move to next_chunk +bool silent_exec_query(THD*, String&); - DBUG_PRINT("restore",("images read")); +/** + Write data needed to restore an item. - if( ino == 0 || res != stream_result::OK ) - DBUG_RETURN(res == stream_result::OK); + By default, a complete DDL CREATE statement for the item is saved. + This statement is constructed using @c meta::X::build_create_stmt() method + where meta::X is the class representing the item. The method stores statement + in the @c create_stmt member. + */ - DBUG_RETURN(TRUE); +stream_result::value meta::Item::save(THD *thd, OStream &s) +{ + create_stmt.free(); + + bool res= build_create_stmt(thd); + + DBUG_ASSERT(res); + + return s.writestr(create_stmt); } -/* - Read the catalog from the stream. +/** + Read data written by @c save(). - SYNOPSIS - read_catalog() - Restore_info *info - The restore information for this archive. - OStream str - The stream to write to. + By default, the CREATE statement is read and stored in the @c create_stmt + member. + */ +stream_result::value meta::Item::read(IStream &s) +{ + return s.readstr(create_stmt); +} - DESCRIPTION - This procedure writes the contents of the catalog to the - stream. Lists are prequalified with the number of elements to - enable faster construction of the catalog on read. +/** + Create item. - RETURNS - 0 - no errors. - -1 - table or table name missing or not found in string. -*/ -bool read_catalog(IStream &str, Restore_info &info) + Default implementation executes the statement stored in the @c create_stmt + member. + */ +bool meta::Item::create(THD *thd) { - stream_result::value res; // result from read stream - Restore_subimage *img; // the image read - uint ino; // number of image read - schema_ref *cat; // reference to database data - String db_name; // database name - uint num_schemas= 0; // num of schemas to read - uint num_db_strings= 0; // num of db metadata strings - uint num_table_strings= 0; // num of table metadata strings - uint num_tables= 0; // num of tables to read - table_info *ti; // table data - String *s; // string for reading - - DBUG_ENTER("restore::read_catalog"); - res= str.read2int(num_schemas); // read # of schema_refs to read - TEST_RD_RES(res); - for (uint i= 0; i < num_schemas; i++) - { - cat= new schema_ref(); // create a new catalog struct - str.readstr(cat->db_name); // read db name - num_db_strings= 0; - res= str.read2int(num_db_strings); // read # db metadata strings - TEST_RD_RES(res); - for (uint j= 0; j < num_db_strings; j++) - { - s= new (current_thd->mem_root) String(); - s->length(0); - res= str.readstr(*s); // read db metadata string - TEST_RD_RES(res); - cat->db_metadata.push_back(s); - } - num_tables= 0; - res= str.read2int(num_tables); // read # tables to read - TEST_RD_RES(res); - res= str.next_chunk(); - for (uint j= 0; j < num_tables; j++) - { - ti= new table_info(); - res= str.readstr(ti->table_name); // read table name - TEST_RD_RES(res); - num_table_strings= 0; - res= str.read2int(num_table_strings); // read # table metadata strings - TEST_RD_RES(res); - for (uint k= 0; k < num_table_strings; k++) - { - s= new (current_thd->mem_root) String(); - s->length(0); - res= str.readstr(*s); // read table metadata string - TEST_RD_RES(res); - res= str.next_chunk(); - ti->table_metadata.push_back(s); - } - cat->tables.push_back(ti); - } - info.catalog.push_back(cat); // add db data to catalog - } + if (create_stmt.is_empty()) + { return FALSE; } - // read list of tables + return drop(thd) && silent_exec_query(thd,create_stmt); +} - List_iterator<Restore_subimage> it(info.images); +/** + Destroy item if it exists. - for( ino=0 ; (img= it++) ; ino++ ) - { - res= (str >> img->tables); // read list of tables stored in the image - TEST_RD_RES(res); - info.table_count+= img->tables.count(); - }; + Default implementation executes SQL statement of the form: + <pre> + DROP @<object type> IF EXISTS @<name> + </pre> + strings @<object type> and @<name> are returned by @c X::sql_object_name() + and @c X::sql_name() methods of the class @c meta::X representing the item. + If neccessary, method @c drop() can be overwriten in a specialized class + corresponding to a given type of meta-data item. + */ +bool meta::Item::drop(THD *thd) +{ + const char *ob= sql_object_name(); + DBUG_ASSERT(ob); - DBUG_PRINT("restore",("catalog read")); + String drop_stmt; - if( ino == 0 || res != stream_result::OK ) - DBUG_RETURN( res == stream_result::OK || res == stream_result::EOS ); + drop_stmt.append("DROP "); + drop_stmt.append(ob); + drop_stmt.append(" IF EXISTS "); + drop_stmt.append(sql_name()); - DBUG_RETURN(TRUE); -} + return silent_exec_query(thd,drop_stmt); +}; + +/**** SAVE/RESTORE DATABASES ***********************************/ -// Write/read description of a sub-image. The exact format is determined by specific -// classes derived from Backup_subimage. +/** + Save data needed to create a database. -bool Backup_subimage::write_description(OStream &s) + Currently we don't save anything. A database is always created using + "CREATE DATABASE @<name>" statement. + + TODO: Save info about db charset and collation. Alternatively, save a complete + "CREATE DATABASE ..." statement with all clauses which should work even when + new clauses are added in the future. + */ +stream_result::value meta::Db::save(THD*,OStream&) { - // TODO: write description length here - s.write2int(ver()); - s.writebyte(type()); - return do_write_description(s); + return stream_result::OK; } -// TODO: Handle different and unknown types (always skip correct amount of bytes). +/** + Read data needed to create a database. -Restore_subimage* -Restore_subimage::create_from_stream(uint no, IStream &s) + Nothing to read. We just build the "CREATE DATABASE ..." statement in + @c create_stmt. + */ +stream_result::value meta::Db::read(IStream&) { - version_t ver; - //stream_result::value res= s.read2int(ver); + create_stmt.append("CREATE DATABASE "); + create_stmt.append(sql_name()); + return stream_result::OK; +} - if (s.read2int(ver) != stream_result::OK) - return NULL; - byte t; - //res= s.read_byte(t); +/**** SAVE/RESTORE TABLES ***************************************/ - if (s.readbyte(t) != stream_result::OK) - return NULL; +/** + Build a CREATE steatement for a table. - switch (image_type(t)) { + We use @c store_create_info() function defined in the server. For that + we need to open the table. After building the statement the table is closed to + save resources. Actually, all tables of the thread are closed as we use + @c close_thread_tables() function. + */ +bool meta::Table::build_create_stmt(THD *thd) +{ + TABLE_LIST t, *tl= &t; - case DEFAULT_IMAGE: - return Default_restore_info::create_from_stream(no, ver,s); + bzero(&t,sizeof(TABLE_LIST)); - case NATIVE_IMAGE: - return Native_restore_info::create_from_stream(no, ver,s); + t.db= const_cast<char*>(in_db().name().ptr()); + t.alias= t.table_name= const_cast<char*>(sql_name()); - default: - return NULL; - } + uint cnt; + int res= ::open_tables(thd,&tl,&cnt,0); + DBUG_ASSERT(res==0); + + res= ::store_create_info(thd,&t,&create_stmt,NULL); + + ::close_thread_tables(thd); + + return res == 0; } -/* - Insert the database name into the CREATE statement to form a - complete CREATE TABLE db.table statement. +} // backup namespace - SYNOPSIS - insert_db_in_create() - TABLE_LIST *tbl - The table to operation on. - String create_str - The SQL string for the CREATE TABLE. - DESCRIPTION - This procedure modifies the CREATE string to include the - database name in the table specification. This ensures the SQL - statement is written as CREATE TABLE db.table ... +/************************************************ - RETURNS - 0 - no errors. - -1 - table or table name missing or not found in string. -*/ -bool insert_db_in_create(TABLE_LIST *tbl, String *create_str) + Helper functions + + ************************************************/ + +namespace backup { + +/// Execute SQL query without sending anything to client. + +// Change net.vio idea taken from execute_init_command in sql_parse.cc +// TODO: investigate what happens if query triggers error - there were +// signals that the code might hang in that case (some clean-up needed?) + +bool silent_exec_query(THD *thd, String &query) { - String table_name; // Table name - String db_table_name; // Table name with db 'test.table1' - String tmp; // temporary string - int index= 0; // index of table name in create string - int period= 0; // index of "." in create string + Vio *save_vio= thd->net.vio; - DBUG_ENTER("meta_backup::insert_db_in_create"); + DBUG_PRINT("restore",("executing query %s",query.c_ptr())); - /* - If table is NULL or the table's database name is missing, - abort. - */ - if (!tbl || !tbl->db) - { - DBUG_PRINT("backup", ("table or database not specified: ")); - DBUG_RETURN(-1); - } + thd->net.vio= 0; + thd->net.no_send_error= 0; - /* - Capture the table name then create a database.table string - for inserting into the CREATE statement in replace of the - table name. - */ - table_name.length(0); - table_name.append(tbl->table_name); - db_table_name.length(0); - db_table_name.append(tbl->db); - db_table_name.append("`.`"); - db_table_name.append(table_name); - index= create_str->strstr(table_name); - tmp.length(0); - tmp.append("."); - period= create_str->strstr(tmp); + thd->query= query.c_ptr(); + thd->query_length= query.length(); - /* - If the table name isn't found in the string or if a period appears - in the string before the table name (indicating the db.table is - already in there), abort. - */ - if (!index || ((period > 0) && (period < index))) + thd->set_time(time(NULL)); + pthread_mutex_lock(&::LOCK_thread_count); + thd->query_id= ::next_query_id(); + pthread_mutex_unlock(&::LOCK_thread_count); + + ::mysql_parse(thd,thd->query,thd->query_length); + + if (thd->query_error) { - DBUG_PRINT("backup", ("table name not found in string!")); - DBUG_RETURN(-1); + DBUG_PRINT("restore", + ("error executing query %s!", thd->query)); + DBUG_PRINT("restore", + ("last error (%d): %s",thd->net.last_errno, + thd->net.last_error)); + return FALSE; } - /* - Insert the database.table string at the location - of the table name in the CREATE command. - */ - DBUG_PRINT("backup", ("inserting database %s into %s", - db_table_name.c_ptr(), create_str->c_ptr())); - DBUG_RETURN(create_str->replace(index, table_name.length(), db_table_name)); + thd->net.vio= save_vio; + + return TRUE; } -/*********************************************************** +} // backup namespace + - Writing/reading table defs +/********************************************************** - @todo Handle db names (include them in CREATE TABLE stmts) - @todo Handle other metadata (triggers,...) + Old code to intorporate into above framework: handle + events, routines and triggers + + **********************************************************/ + +namespace backup { - ***********************************************************/ +#if FALSE /* Get the metadata for the table specified. @@ -660,9 +598,6 @@ DBUG_RETURN(0); } -/// Execute SQL query without sending anything to client. - -bool silent_exec_query(THD*, String&); /* Use the catalog read to get the metadata for the databases @@ -741,39 +676,6 @@ DBUG_RETURN(TRUE); } -// Change net.vio idea taken from execute_init_command in sql_parse.cc - -bool silent_exec_query(THD *thd, String &query) -{ - Vio *save_vio= thd->net.vio; - - thd->net.vio= 0; - thd->net.no_send_error= 0; - - thd->query= query.c_ptr(); - thd->query_length= query.length(); - - thd->set_time(time(NULL)); - pthread_mutex_lock(&::LOCK_thread_count); - thd->query_id= ::next_query_id(); - pthread_mutex_unlock(&::LOCK_thread_count); - - ::mysql_parse(thd,thd->query,thd->query_length); - - if (thd->query_error) - { - DBUG_PRINT("restore", - ("error executing query %s!", thd->query)); - DBUG_PRINT("restore", - ("last error (%d): %s",thd->net.last_errno, - thd->net.last_error)); - return FALSE; - } - - thd->net.vio= save_vio; - - return TRUE; -} - +#endif } // backup namespace --- New file --- +++ sql/backup/meta_backup.h 07/05/11 00:20:25 #ifndef _META_BACKUP_H #define _META_BACKUP_H /** @file Declarations of classes used to handle meta-data items. */ #include <backup/api_types.h> #include <backup/stream.h> namespace backup { namespace meta { /** Defines how to backup and restore a meta-data item. This class provides @c create() and @c drop() methods used to create and destroy items. The default implementation uses SQL "CREATE ..." and "DROP ..." statements for that purpose. Its instances determine how to save and read data needed for item creation and provide any other item specific data. For each type of meta-data there is a specialized subclass of @c meta::Item which implements these tasks. */ class Item { public: /// Possible types of meta-data items. enum enum_type {DB, TABLE}; virtual ~Item() {} /// Return type of the item. virtual const enum_type type() const =0; /** For per-db items return the database to which this item belongs. For other items returns an invalid db reference. */ virtual const Db_ref in_db() { return Db_ref(); } /// Save data needed to create the item. virtual stream_result::value save(THD*,OStream&); /// Read data saved by @c save() method. virtual stream_result::value read(IStream&); /// Destroy the item if it exists. virtual bool drop(THD*); /// Create the item. virtual bool create(THD*); protected: String create_stmt; /// Storage for a create statement of the item. /** Return SQL name of the object represented by this item like TABLE or DATABASE. This is used to construct a "DROP ..." statement for the item. */ virtual const char* sql_object_name() const { return NULL; } /** Return name under which this item is known to the server. In case of per-db items the name should *not* be qualified by db name. */ virtual const char* sql_name() const =0; /// Store in @c create_stmt a DDL statement which will create the item. virtual bool build_create_stmt(THD*) { return FALSE; } }; /** Specialization of @c meta::Item representing a database. */ class Db: public Item { const enum_type type() const { return DB; } const char* sql_object_name() const { return "DATABASE"; } // Overwrite default implementations. stream_result::value save(THD*,OStream&); stream_result::value read(IStream&); }; /** Specialization of @c meta::Item representing a table. */ class Table: public Item { const enum_type type() const { return TABLE; } const char* sql_object_name() const { return "TABLE"; } bool build_create_stmt(THD*); }; } // meta namespace } // backup namespace #endif --- 1.8/sql/backup/stream.h 2007-05-11 00:20:37 +02:00 +++ 1.9/sql/backup/stream.h 2007-05-11 00:20:37 +02:00 @@ -1,8 +1,7 @@ #ifndef _BACKUP_STREAM_H_ #define _BACKUP_STREAM_H_ -#include "backup_kernel.h" // for Location class -#include "backup_engine.h" // for Buffer definition +#include <backup/api_types.h> // for Buffer definition /*********************************************************** @@ -682,8 +681,14 @@ DBUG_PRINT(H,("stream op result= %d",(R))); \ DBUG_ASSERT( (R) != stream_result::ERROR ); \ } while(0) -#define TEST_WR_RES(R) TEST_STR_RES("backup",(R)) +#define TEST_STR_OK(H,R) \ + do { \ + if ( (R) != stream_result::OK ) \ + DBUG_PRINT(H,("stream op result= %d",(R))); \ + DBUG_ASSERT( (R) == stream_result::OK || (R) == stream_result::NIL ); \ + } while(0) +#define TEST_WR_RES(R) TEST_STR_OK("backup",(R)) +#define TEST_RD_IRES(R) TEST_STR_OK("restore",(R)) #define TEST_RD_RES(R) TEST_STR_RES("restore",(R)) - #endif /*BACKUP_STREAM_H_*/ --- 1.2/sql/backup/string_pool.cc 2007-05-11 00:20:37 +02:00 +++ 1.3/sql/backup/string_pool.cc 2007-05-11 00:20:37 +02:00 @@ -22,23 +22,54 @@ // Serialization of StringPool -stream_result::value operator>>(IStream &str, StringPool &p) +stream_result::value StringPool::save(OStream &str) +{ + String buf; + uint skip=0 , i=0; + + // TODO: error checking + + for(i=0 ; i < count() ; i++ ) + { + if( entries[i].el != NULL ) + { + if( skip > 0 ) + str.writeint(skip); + skip= 0; + str.writestr(*entries[i].el); + continue; + }; + + if( skip == 0 ) + str.writenil(); + + skip++; + + }; + + if( skip > 0 ) + str.writeint(skip); + + return str.end_chunk(); +} + +stream_result::value StringPool::read(IStream &str) { stream_result::value res; String buf; uint skip, i=0; - p.clear(); + clear(); do { - if( i >= p.size ) break; + if( i >= size ) break; res= str.readstr(buf); switch( res ) { case stream_result::OK: - p.set(i++,buf); + set(i++,buf); break; case stream_result::NIL: @@ -51,41 +82,9 @@ } } while( res == stream_result::OK ); - if( res == stream_result::EOC ) - res= str.next_chunk(); + res= str.next_chunk(); return res; -} - -stream_result::value operator<<(OStream &str, const StringPool &p) -{ - String buf; - uint skip=0 , i=0; - - // TODO: error checking - - for(i=0 ; i < p.count() ; i++ ) - { - if( p.entries[i].el != NULL ) - { - if( skip > 0 ) - str.writeint(skip); - skip= 0; - str.writestr(*p.entries[i].el); - continue; - }; - - if( skip == 0 ) - str.writenil(); - - skip++; - - }; - - if( skip > 0 ) - str.writeint(skip); - - return str.end_chunk(); } } // backup namespace --- 1.1/sql/backup/string_pool.h 2007-05-11 00:20:37 +02:00 +++ 1.2/sql/backup/string_pool.h 2007-05-11 00:20:37 +02:00 @@ -39,8 +39,9 @@ class StringPool: public util::StringPool { - friend stream_result::value operator>>(IStream &str, StringPool &p); - friend stream_result::value operator<<(OStream &str, const StringPool &p); + public: + stream_result::value save(OStream&); + stream_result::value read(IStream&); }; } // backup namespace --- 1.9/sql/backup/backup_aux.h 2007-05-11 00:20:37 +02:00 +++ 1.10/sql/backup/backup_aux.h 2007-05-11 00:20:37 +02:00 @@ -1,7 +1,7 @@ #ifndef _BACKUP_AUX_H #define _BACKUP_AUX_H -//#define USE_DEFAULT_BACKUP +#include <backup/api_types.h> inline bool operator==(const String &s1, const String &s2) @@ -12,6 +12,24 @@ namespace backup { extern CHARSET_INFO *default_charset; + +TABLE_LIST *build_table_list(const Table_list&,thr_lock_type); + + +/// Check if a database exists. + +inline +bool db_exists(const Db_ref &db) +{ + // This is how database existence is checked inside mysql_change_db(). + return ! ::check_db_dir_existence(db.name().ptr()); +} + +inline +bool change_db(THD *thd, const Db_ref &db) +{ + return 0 == ::mysql_change_db(thd,db.name().ptr(),TRUE); +} } // backup namespace
| Thread | ||
|---|---|---|
| • bk commit into 5.1 tree (rafal:1.2526) | rsomla | 11 May |
© 1995-2008 MySQL AB, 2008-2009 Sun Microsystems, Inc.
Page generated in 1.791 sec. using MySQL 5.1.14-beta-log