Below is the list of changes that have just been committed into a local
5.2 repository of guilhem. When guilhem does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html
ChangeSet@stripped, 2007-11-06 20:56:05+01:00, guilhem@stripped +59 -0
WL#866 - Online backup driver for MyISAM.
When this driver (file myisam_backup_log.cc) is to do a backup,
it starts physical logging of changes for target tables (file
mi_log.c), dirtily copies tables, then the physical log.
When this driver does a restore, it copies back tables and the log,
applies the log (file mi_examine_log.c).
The data file is always backed up; for the index file two methods are
available: either only its 64KB header is copied and then restore will
repair indexes, or the whole index file is copied.
Backup starts and runs without disturbing any running or new update
except at the very end: when creating the validity point it locks all
backed-up MyISAM tables with a read lock (so, stalls new update
statements and waits for running update statements to finish).
I recommend to reviewers to read in order myisam_backup_driver.cc,
mi_log.c then other files.
GUILHEM_TODO tags identify the most urgent todos.
include/atomic/nolock.h@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +4 -1
comment
include/keycache.h@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +10 -2
A block now has an optional callback/callback_argument pair which is
called after the block goes to the file. This callback and its argument
are set through key_cache_write().
This is used by MyISAM's physical logging: when a key page
is flushed to the file, the write is also recorded in the physical log,
thanks to such callback.
include/my_global.h@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +5 -3
When testing a C++ file against "gcc -Wold-style-cast", these casts
surfaced. static_cast_C_or_CPP means "a static cast which works in
C and in C++, without warnings".
include/my_sys.h@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +22 -11
New members of IO_CACHE:
* post_write (called by my_b_flush_io_cache()),
* hard_write_error_in_the_past: like the existing 'error' except that
once set it is not reset until cache is reinitialized; 'error' is
reset at next _my_b_write() call for example. Is useful when we want
to lazily (once in a while) monitor if an IO_CACHE got a write error
(and so, file is not usable) instead of monitoring each IO_CACHE write.
These are used by MyISAM's online backup (myisam_backup_driver.cc)
to test if the physical log got a write error.
include/myisam.h@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +18 -3
declarations' update.
mysql-test/mysql-test-run.pl@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +21 -0
ability to call myisamlog from tests (for new test t/myisamlog.test)
mysql-test/r/backup.result@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +152 -43
result update.
Note, for the first test, the increase in backup size which is due to:
- more tables to back up
- the MyISAM native driver copies the index file which is always at
least 1 KB, while the generic locking driver (which was used for MyISAM
so far) copies only rows, so can give a smaller backup.
The message in CHECK TABLE is expected (we copied a table which was
open at this moment).
Note how the checksums after restore are equal to those at the _end_ of
the backup.
For the second test, MyISAM table has no index, and MYD is more
compact that row format used by the default backup driver, so size
decreases. By the way, the second test wanted to test the default
driver: should I change MyISAM to MEMORY there?
mysql-test/r/backup_no_engine.result@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +6 -6
MyISAM driver gives larger backup than default driver because it
backs up the index (which is 1KB in this case).
mysql-test/r/myisamlog.result@stripped, 2007-11-06 20:56:02+01:00, guilhem@stripped +48 -0
result is ok
mysql-test/r/myisamlog.result@stripped, 2007-11-06 20:56:02+01:00, guilhem@stripped +0 -0
mysql-test/t/backup.test@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +123 -17
testing of online backup for MyISAM:
- table 'staff' is augmented (index, more records)
- a second MyISAM table is added, 'tasking2' with indices and
delay_key_write.
- a third MyISAM table is added, 'staff2' to test index rebuilds
done by bulk insert when the table is empty.
A backup is launched but blocked before it creates the validity
point; then we modify MyISAM tables. After restore, we verify
that restored tables are as they originally were at end of backup,
i.e. they contain the modifications.
Too long lines are wrapped.
mysql-test/t/key_cache.test@stripped, 2007-11-06 20:55:59+01:00, guilhem@stripped +3 -3
As the size of the BLOCK_LINK structure has grown (to accomodate
the new flush_callback and flush_callback_arg members), the number
of available blocks for a fixed key cache size has decreased
(a similar fix will be needed for 64-bit systems too)
mysql-test/t/myisamlog-master.opt@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +1 -0
To test the MyISAM logical log, which cannot be enabled on the fly.
mysql-test/t/myisamlog-master.opt@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +0 -0
mysql-test/t/myisamlog.test@stripped, 2007-11-06 20:56:02+01:00, guilhem@stripped +37 -0
Test of MyISAM logical logging and the myisamlog utility:
see if they can repopulate a table.
mysql-test/t/myisamlog.test@stripped, 2007-11-06 20:56:02+01:00, guilhem@stripped +0 -0
mysys/mf_iocache.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +53 -19
Calling the new post_write hook. Note that it does not work with
SEQ_READ_APPEND IO_CACHEs (we don't need a post_write for them now).
Setting IO_CACHE::hard_write_error_in_the_past at the same time we
set IO_CACHE::error to -1 in functions which do writes.
IO_CACHE::hard_write_error_in_the_past is however not reset for
each write, that's its difference from IO_CACHE::error.
mysys/mf_keycache.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +92 -39
A block now has an optional callback/callback_argument pair which is
called after the block goes to the file. This callback and its argument
are set through key_cache_write().
We make sure that all pwrite() call the callback, by replacing
them by a new key_cache_pwrite().
This is used by MyISAM's physical logging.
mysys/my_thr_init.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +12 -5
a dedicated mutex for the MyISAM log (better not use THR_LOCK_myisam
which is already used at every mi_open()/mi_close(), and also it made
coding easier, see comments in _myisam_log_command())
sql/backup/backup_aux.h@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +1 -1
unneeded initialization
sql/backup/data_backup.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +5 -4
bugfix. What happened is that there was the default driver and the MyISAM
native driver.
The first iteration of this loop:
for (uint no=0; no < info.img_count; ++no)
{
(in write_table_data()),
met the default driver, set def_or_snap_used to true,
called get_default_snapshot_tables().
Then the second iteration met the MyISAM driver, def_or_snap_used was
still true (it is cumulative) and as it was used to decide if
get_default_snapshot_tables() should be called, that function was
called, with p->drv() the MyISAM driver. Inside the function, MyISAM
driver was cast to default_backup::Backup*, corrupting memory.
Chuck mentioned that this code will soon go away anyway.
sql/backup/stream.h@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +10 -2
temporary bugfix. Too long comment lines, but the whole
block will go away soon.
sql/mysqld.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +7 -7
emphasizing that the MyISAM log set from the command line is the
logical one.
sql/sql_bitmap.h@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +2 -2
When mysqld starts I get:
==25833== Conditional jump or move depends on uninitialised value(s)
==25833== at 0x8185A2C: Bitmap<64>::is_set(unsigned) const (sql_bitmap.h:189)
==25833== by 0x84FB484: ha_myisam::index_flags(unsigned, unsigned, bool) const (ha_myisam.h:63)
==25833== by 0x82B0444: open_binary_frm(THD*, st_table_share*, unsigned char*, int) (table.cc:1476)
==25833== by 0x82B11D6: open_table_def(THD*, st_table_share*, unsigned) (table.cc:614)
==25833== by 0x82A1508: get_table_share(THD*, TABLE_LIST*, char*, unsigned, unsigned, int*) (sql_base.cc:401)
==25833== by 0x82A1958: get_table_share_with_create(THD*, TABLE_LIST*, char*, unsigned, unsigned, int*) (sql_base.cc:494)
==25833== by 0x82A1C27: open_unireg_entry(THD*, st_table*, TABLE_LIST*, char const*, char*, unsigned, st_mem_root*, unsigned) (sql_base.cc:3294)
==25833== by 0x82A42FA: open_table(THD*, TABLE_LIST*, st_mem_root*, bool*, unsigned) (sql_base.cc:2639)
==25833== by 0x82A505C: open_tables(THD*, TABLE_LIST**, unsigned*, unsigned) (sql_base.cc:3606)
==25833== by 0x82A570A: simple_open_n_lock_tables(THD*, TABLE_LIST*) (sql_base.cc:3866)
==25833== by 0x841EF39: plugin_load(st_mem_root*, int*, char**) (sql_plugin.cc:1352)
==25833== by 0x841F752: plugin_init(int*, char**, int) (sql_plugin.cc:1187)
I cannot reproduce it in a clean tree, but this problem looks to
be unrelated to the MyISAM online backup driver still.
It looks like ha_myisam::keys_with_parts is initialized only in
ha_myisam::open() which is not called above. Bitmap<64>'s default
constructor does not initialize the bitmap, whereas the default
constructor of the any-size Bitmap class does (see at start of file);
this difference does not look normal; I add here an initialization in
Bitmap<64>'s default constructor, which removes the problem. This
problem is specific of 5.2 as ha_myisam::keys_with_parts appeared in 5.2.
sql/sql_cache.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +3 -2
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
sql/sql_class.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +3 -2
For the "locking threads" created by some online backup drivers
(MyISAM one and default lock-based one), THD::awake() must abort locks.
That if() dates from a time where the only possible system threads
were the delayed insert system thread and the slave threads.
sql/sql_class.h@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +2 -1
a new type of system thread: backup utility thread (used by the
MyISAM online backup driver).
Note that Chuck is doing similar work with threads & backup, so that
change will probably go away when merged with Chuck's work which is:
http://lists.mysql.com/commits/36809
sql/sql_load.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +1 -2
cast not needed
sql/sql_repl.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +4 -1
new prototype
sql/sql_repl.h@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +1 -1
new prototype
sql/sql_table.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +3 -0
As start_bulk_insert() has been called, end_bulk_insert() must be
called (bug found thanks to new assertion in mi_locking.c)
storage/myisam/CMakeLists.txt@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +2 -1
include new files in build
storage/myisam/Makefile.am@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +1 -1
include new files in build
storage/myisam/ha_myisam.cc@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +8 -5
MyISAM now has an online backup driver, see myisam_backup_driver.cc.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name.
Fix for doxygen.err warnings.
storage/myisam/ha_myisam.h@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +4 -0
MyISAM now has an online backup driver
storage/myisam/mi_check.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +73 -11
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging.
Note: REPAIR/OPTIMIZE TABLE are not handled by
physical logging; they should be blocked by the MySQL online backup
kernel: when they delete/create/rename files, they are as much
a problem as ALTER TABLE. So we assert for these cases.
At end of INSERT SELECT, if the table was originally empty,
an index rebuild is done. This uses my_pwrite() and my_chsize(),
which we thus have to write to the physical log.
storage/myisam/mi_close.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +12 -2
If we failed to flush the key cache, we need to mark the table as
corrupted (fix for an unlikely bug).
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging.
If the table has logged a MI_LOG_OPEN to the physical log (because it
has logged some writes there), it needs to log a MI_LOG_CLOSE too.
Regarding the added assertion, see revision comment in mi_locking.c.
storage/myisam/mi_create.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +5 -1
Ensure that no table is truncated if it is doing physical
logging. We *could* support TRUNCATE during an online backup of
MyISAM but not without the server's help. But anyway we know that
TRUNCATE is going to be an issue (it deletes InnoDB's consistent
reads, recreates files...) so will be blocked by the backup kernel.
storage/myisam/mi_delete.c@stripped, 2007-11-06 20:56:00+01:00, guilhem@stripped +7 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_delete_all.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +4 -1
Log to the physical log the MI_DELETE_ALL operation
storage/myisam/mi_dynrec.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +23 -13
Log to the physical log the operation of writing bytes to the data file.
Always log _after_ the write (see comment at start of
mi_log_start_physical()).
storage/myisam/mi_examine_log.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +868 -0
This is the examine_log() function taken out of myisamlog.c, as we
now need it in the server too, to restore from an online backup.
It is renamed to mi_examine_log().
examine_log() used global variables of myisamlog.c, that is not possible
now so mi_examine_log() takes in parameter a new structure
MI_EXAMINE_LOG_PARAM whose content corresponds to the old global
variables.
The function is extended to understand new log commands found in
physical logs: MI_LOG_WRITE_BYTES_MYI, MI_LOG_WRITE_BYTES_MYD,
MI_LOG_CHSIZE_MYI, as well as MI_LOG_DELETE_ALL which wasn't replayed
properly (a bug).
Other bugfix: NO_FILEPOS was too small for 64-bit machines.
Other bugfix: MI_LOG_LOCK applying could not work because it didn't
remove the 'flag' added at the end of mi_lock_database().
To see the real code changes I made (because, most of lines are just
moved from myisamlog.c), you should diff mi_examine_log() with
examine_log() from the old myisamlog.c (I recommend it).
storage/myisam/mi_examine_log.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +0 -0
storage/myisam/mi_extra.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +50 -3
When we set up a write cache (HA_EXTRA_WRITE_CACHE) we give the cache
a post_write call, which will make sure that writes to the data file
will go to the physical log if needed.
It is more efficient that logging all my_b_write()s as it generates
less log records, and anyway we needed the post_write callback for
when logging is enabled while the table is in already the middle of
using a write cache, when non-logged my_b_write()s have passed already).
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging
storage/myisam/mi_locking.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +84 -46
Comments explaining why certain writes to the index file needn't go into
the physical log.
New function mi_remap_file_and_write_state_for_unlock(), used when
unlocking a table and in mi_backup_stop_logging_for_tables().
I added DBUG_ASSERT(!(info->opt_flag & WRITE_CACHE_USED)); originally
for online backup purposes, then I saw it fired in some non-backup test
(alter_table.test); turned out to be a forgotten end_bulk_insert() in
sql_table.cc.
If logical log, we now always log MI_LOG_LOCK. Indeed, without it,
the MI_LOG_EXTRA failed to apply when command was HA_EXTRA_CACHE (see
the EACCES error in mi_extra():
missing MI_LOG_LOCK => info->lock_type==F_UNLCK => applying of
MI_LOG_EXTRA failed (so myisamlog.test got such warnings in
mysqltest.log: "Warning: error 13, expected 0 on command extra at 211").
We also make the storage of lock_type portable (independent of
endianess).
storage/myisam/mi_log.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +768 -80
mi_log() can now open a logical (==debug)or physical (==backup) log:
it thus receives new parameters (list of tables...); it can
close logs too. Opening and closing a physical log is more work
than a logical one, so additional functions are called.
Logging functions now use IO_CACHE instead of my_write(), to be
faster.
_myisam_log() merged into _myisam_log_command() (they were very
similar).
If _myisam_log_command() uses small numbers (file descriptor,
file offset buffer length) it stores them in few bytes; if they are big
it stores them in more bytes; how many bytes are used is denoted
by the highest bit of the 'command' (MI_LOG_BIG_NUMBERS).
That and the merge of _myisam_log() and _myisam_log_command()
imply that old myisamlog won't read new logical log, but it's ok: the
MyISAM log was so far only for debugging, advertised as such,
apparently used only by MySQL devs, and had a bug (MI_LOG_DELETE_ALL
was not replayed by myisamlog) which nobody reported.
_myisam_log_record() renamed to _myisam_log_record_logical()
(used only by logical logging).
Logging functions now need to check if the log is opened _inside the
log's mutex_, because the physical log can close at any time (while
the old logical log, was either open or closed for the
lifetime of the MySQL server process).
_myisam_log_command() may log MI_LOG_OPEN on top of its command
(needed for physical logging), if this is a physical log and
the first time this table's share writes to it.
_myisam_log_record() is used only by logical logging, renamed it.
storage/myisam/mi_open.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +37 -13
When opening a table, detect if it must do physical logging.
mi_state_info_write() needs the MYISAM_SHARE now, as it needs to
record the my_pwrite() done to the index file, into the physical log.
Note, for now all calls to mi_state_info_write() have the state
equal to the share's state but future projects may not have that,
that's why the two arguments stay.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name,
indeed physical logging needs this unresolved name, and
sometimes cannot have MI_INFO available, but always has MYISAM_SHARE
(see log_key_cache_flush_physical()). It is also more
efficient (why duplicate the same string in all MI_INFOs for a same
table?).
storage/myisam/mi_page.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +68 -15
When writing a page to the key cache, if the table is doing physical
logging, we pass a callback to the key cache, which will be invoked
when key cache later flushes this page to the file. This callback may
be NULL if we don't want to store index page changes into the physical
log.
It's more efficient to log at flush time than when we write to the
key cache; writes to the same key page accumulate (possibly many
writes to key cache for one flush).
storage/myisam/mi_panic.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +11 -7
update for new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_rrnd.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +1 -3
cosmetic change
storage/myisam/mi_static.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +18 -5
New physical log in MyISAM
storage/myisam/mi_test2.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +1 -1
update for new prototype
storage/myisam/mi_test3.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +2 -2
update for new prototype
storage/myisam/mi_update.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +6 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_write.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +5 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/myisam_backup_driver.cc@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +1875 -0
MyISAM online backup driver. See this file's first lines (namespace
myisam_backup) for a description of how it works.
This is the file which really drives the backup/restore inside MyISAM
(see it as the master which calls functions from other files to
perform some subtasks).
Note that Chuck has taken some code from this file and it is now
duplicated into sql/backup/ files, I should merge the two
to avoid duplication, when Chuck's work is pushed
(http://lists.mysql.com/commits/36809)
storage/myisam/myisam_backup_driver.cc@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +0 -0
storage/myisam/myisam_ftdump.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +2 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/myisamdef.h@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +111 -18
Adding member "physical_logging" to MYISAM_SHARE, tells if the table
is currently doing physical logging or not.
It is read and set with atomic operations: this provides the needed
synchronization between the table-writer thread (if the table is being
written now) (which is the reader of "physical_logging") and the thread
turning physical logging on or off (which is the writer of
"physical_logging").
Adding member "MI_LOG_OPEN_stored_in_physical_log" to MYISAM_SHARE, to
remember if we already stored MI_LOG_OPEN in this physical log for this
table.
A new mutex THR_LOCK_myisam_log to protect MyISAM logs (instead of
THR_LOCK_myisam, for concurrency and coding reasons (explained
in _myisam_log_command()).
New commands which can be stored only in the physical log:
MI_LOG_WRITE_BYTES_MYD, MI_LOG_WRITE_BYTES_MYI, MI_LOG_CHSIZE_MYI.
Updates to myisam_log_* macros to adapt to the fact that logs are now
an IO_CACHE and not a plain OS file.
As most of the logic of myisamlog moved out of myisamlog.c into
mi_examine_log.c, mi_examine_log() cannot use globals of myisamlog.c
anymore; a new structure MI_EXAMINE_LOG_PARAM is introduced instead
(like MI_CHECK). mi_state_info_write() needs the MYISAM_SHARE now, as
it is used by physical logging.
Note: all extern vars touched in this file have their documentation
in the file where they are defined.
storage/myisam/myisamlog.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +74 -624
The main function of this file, examine_log(), is now needed in
the server to be able to do a restore from an online backup,
so moves to mi_examine_log.c.
I changed a meaningless min() to max().
New log commands (MI_LOG_etc) have long names so I widened the
space for displaying their name to 20 characters.
Due to backward-incompatibilities (see mi_log.c) we bump version
number to 2.0.
Reviewer: what do you think about my_use_symdir=1 added to this file?
storage/myisam/myisampack.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +10 -5
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name.
mi_state_info_write() needs MYISAM_SHARE as physical logging does.
storage/myisammrg/ha_myisammrg.cc@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +4 -4
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_info.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +2 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_rrnd.c@stripped, 2007-11-06 20:56:01+01:00, guilhem@stripped +1 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
diff -Nrup a/include/atomic/nolock.h b/include/atomic/nolock.h
--- a/include/atomic/nolock.h 2007-10-09 19:55:11 +02:00
+++ b/include/atomic/nolock.h 2007-11-06 20:55:59 +01:00
@@ -29,7 +29,10 @@
#endif
#ifdef make_atomic_cas_body
-
+/*
+ Type not used so minimal size (emptry struct has different size between C
+ and C++, zero-length array is gcc-specific).
+*/
typedef char my_atomic_rwlock_t __attribute__ ((unused));
#define my_atomic_rwlock_destroy(name)
#define my_atomic_rwlock_init(name)
diff -Nrup a/include/keycache.h b/include/keycache.h
--- a/include/keycache.h 2007-07-25 00:58:05 +02:00
+++ b/include/keycache.h 2007-11-06 20:55:59 +01:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* Key cache variable structures */
+/**
+ @file
+ Key cache API
+*/
#ifndef _keycache_h
#define _keycache_h
@@ -121,10 +124,15 @@ extern uchar *key_cache_read(KEY_CACHE *
extern int key_cache_insert(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length);
+/** Per-block callback called when block is flushed */
+typedef int (*KEYCACHE_POST_WRITE_CALLBACK)(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos);
extern int key_cache_write(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length,
- uint block_length,int force_write);
+ uint block_length, int force_write,
+ KEYCACHE_POST_WRITE_CALLBACK callback,
+ void *callback_arg);
extern int flush_key_blocks(KEY_CACHE *keycache,
int file, enum flush_type type);
extern void end_key_cache(KEY_CACHE *keycache, my_bool cleanup);
diff -Nrup a/include/my_global.h b/include/my_global.h
--- a/include/my_global.h 2007-10-17 15:57:30 +02:00
+++ b/include/my_global.h 2007-11-06 20:55:59 +01:00
@@ -68,9 +68,11 @@
#ifdef __cplusplus
#define C_MODE_START extern "C" {
#define C_MODE_END }
+#define static_cast_C_or_CPP(TYPE) static_cast<TYPE>
#else
#define C_MODE_START
#define C_MODE_END
+#define static_cast_C_or_CPP(TYPE) (TYPE)
#endif
#if defined(_WIN32) || defined(_WIN64) || defined(__WIN32__) || defined(WIN32)
@@ -923,7 +925,7 @@ typedef long long my_ptrdiff_t;
#define my_offsetof(TYPE, MEMBER) \
((size_t)((char *)&(((TYPE *)0x10)->MEMBER) - (char*)0x10))
-#define NullS (char *) 0
+#define NullS static_cast_C_or_CPP(char *)(0)
/* Nowdays we do not support MessyDos */
#ifndef NEAR
#define NEAR /* Who needs segments ? */
@@ -1039,7 +1041,7 @@ typedef ulonglong my_off_t;
#else
typedef unsigned long my_off_t;
#endif
-#define MY_FILEPOS_ERROR (~(my_off_t) 0)
+#define MY_FILEPOS_ERROR (~static_cast_C_or_CPP(my_off_t)(0))
#if !defined(__WIN__)
typedef off_t os_off_t;
#endif
@@ -1076,7 +1078,7 @@ typedef char bool; /* Ordinary boolean
#define INT8(v) (int8) (v)
#define INT16(v) (int16) (v)
#define INT32(v) (int32) (v)
-#define MYF(v) (myf) (v)
+#define MYF(v) static_cast_C_or_CPP(myf)(v)
#ifndef LL
#ifdef HAVE_LONG_LONG
diff -Nrup a/include/my_sys.h b/include/my_sys.h
--- a/include/my_sys.h 2007-10-09 21:42:20 +02:00
+++ b/include/my_sys.h 2007-11-06 20:55:59 +01:00
@@ -13,6 +13,11 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+/**
+ @file
+ mysys library API
+*/
+
#ifndef _my_sys_h
#define _my_sys_h
C_MODE_START
@@ -335,7 +340,8 @@ typedef struct st_dynamic_string
} DYNAMIC_STRING;
struct st_io_cache;
-typedef int (*IO_CACHE_CALLBACK)(struct st_io_cache*);
+typedef int (*IO_CACHE_CALLBACK)(struct st_io_cache *, const uchar *, uint,
+ my_off_t);
#ifdef THREAD
typedef struct st_io_cache_share
@@ -434,21 +440,24 @@ typedef struct st_io_cache /* Used when
*/
enum cache_type type;
/*
- Callbacks when the actual read I/O happens. These were added and
- are currently used for binary logging of LOAD DATA INFILE - when a
- block is read from the file, we create a block create/append event, and
- when IO_CACHE is closed, we create an end event. These functions could,
- of course be used for other things
- */
- IO_CACHE_CALLBACK pre_read;
- IO_CACHE_CALLBACK post_read;
- IO_CACHE_CALLBACK pre_close;
+ Callbacks were added and are currently used for binary logging of LOAD
+ DATA INFILE - when a block is read from the file, we create a block
+ create/append event, and when IO_CACHE is closed, we create an end event;
+ also used to write the MyISAM WRITE_CACHE blocks to the MyISAM backup
+ log. These functions could, of course be used for other things. Note: some
+ callbacks share the same argument ("arg").
+ */
+ IO_CACHE_CALLBACK pre_read; /**< called before reading from disk */
+ IO_CACHE_CALLBACK post_read; /**< called after reading from disk */
+ IO_CACHE_CALLBACK pre_close; /**< called before ending the cache */
+ /** Called after writing to disk; not honoured by SEQ_READ_APPEND */
+ IO_CACHE_CALLBACK post_write;
/*
Counts the number of times, when we were forced to use disk. We use it to
increase the binlog_cache_disk_use status variable.
*/
ulong disk_writes;
- void* arg; /* for use by pre/post_read */
+ void *arg; /**< used by pre/post_read,post_write */
char *file_name; /* if used with 'open_cached_file' */
char *dir,*prefix;
File file; /* file descriptor */
@@ -460,6 +469,8 @@ typedef struct st_io_cache /* Used when
partial.
*/
int seek_not_done,error;
+ /** Cumulative 'error' since last [re]init_io_cache() */
+ int hard_write_error_in_the_past;
/* buffer_length is memory size allocated for buffer or write_buffer */
size_t buffer_length;
/* read_length is the same as buffer_length except when we use async io */
diff -Nrup a/include/myisam.h b/include/myisam.h
--- a/include/myisam.h 2007-08-26 13:31:06 +02:00
+++ b/include/myisam.h 2007-11-06 20:55:59 +01:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* This file should be included when using myisam_funktions */
+/**
+ @file
+ This file should be included when using MyISAM functions.
+*/
#ifndef _myisam_h
#define _myisam_h
@@ -32,6 +35,7 @@ extern "C" {
#endif
#include "my_handler.h"
#include <mysql/plugin.h>
+#include <hash.h>
/*
There is a hard limit for the maximum number of keys as there are only
@@ -256,10 +260,13 @@ typedef struct st_columndef /* column i
#endif
} MI_COLUMNDEF;
+/** Physical logging is always compiled in. Undefine if want to benchmark */
+#define HAVE_MYISAM_PHYSICAL_LOGGING 1
+
/* invalidator function reference for Query Cache */
typedef void (* invalidator_by_filename)(const char * filename);
-extern char * myisam_log_filename; /* Name of logfile */
+extern char * myisam_logical_log_filename;
extern ulong myisam_block_size;
extern ulong myisam_concurrent_insert;
extern my_bool myisam_flush,myisam_delay_key_write,myisam_single_user;
@@ -304,7 +311,15 @@ extern int mi_extra(struct st_myisam_inf
extern int mi_reset(struct st_myisam_info *file);
extern ha_rows mi_records_in_range(MI_INFO *info, int inx,
key_range *min_key, key_range *max_key);
-extern int mi_log(int activate_log);
+/** Open/close actions allowed on a MyISAM log */
+enum enum_mi_log_action
+{
+ MI_LOG_ACTION_OPEN,
+ MI_LOG_ACTION_CLOSE_CONSISTENT, MI_LOG_ACTION_CLOSE_INCONSISTENT
+};
+enum enum_mi_log_type { MI_LOG_PHYSICAL, MI_LOG_LOGICAL };
+extern int mi_log(enum enum_mi_log_action action, enum enum_mi_log_type type,
+ const char *log_filename, const HASH *tables);
extern int mi_is_changed(struct st_myisam_info *info);
extern int mi_delete_all_rows(struct st_myisam_info *info);
extern ulong _mi_calc_blob_length(uint length , const uchar *pos);
diff -Nrup a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl
--- a/mysql-test/mysql-test-run.pl 2007-10-04 13:01:50 +02:00
+++ b/mysql-test/mysql-test-run.pl 2007-11-06 20:55:59 +01:00
@@ -139,6 +139,7 @@ our $exe_mysql;
our $exe_mysqladmin;
our $exe_mysql_upgrade;
our $exe_mysqlbinlog;
+our $exe_myisamlog;
our $exe_mysql_client_test;
our $exe_bug25714;
our $exe_mysqld;
@@ -2009,6 +2010,26 @@ sub environment_setup () {
" --debug=d:t:A,$path_vardir_trace/log/mysqlbinlog.trace";
}
$ENV{'MYSQL_BINLOG'}= $cmdline_mysqlbinlog;
+
+ # ----------------------------------------------------
+ # Setup env so childs can execute myisamlog
+ # ----------------------------------------------------
+
+ my $myisam_path= mtr_file_exists("$glob_basedir/storage/myisam",
+ "$glob_basedir/bin");
+
+ $exe_myisamlog=
+ mtr_exe_exists("$myisam_path/myisamlog");
+
+ my $cmdline_myisamlog=
+ mtr_native_path($exe_myisamlog);
+
+ if ( $opt_debug )
+ {
+ $cmdline_myisamlog .=
+ " -#d:t:A,$path_vardir_trace/log/myisamlog.trace";
+ }
+ $ENV{'MYISAMLOG'}= $cmdline_myisamlog;
# ----------------------------------------------------
# Setup env so childs can execute mysql
diff -Nrup a/mysql-test/r/backup.result b/mysql-test/r/backup.result
--- a/mysql-test/r/backup.result 2007-09-11 11:29:28 +02:00
+++ b/mysql-test/r/backup.result 2007-11-06 20:55:59 +01:00
@@ -14,9 +14,9 @@ CREATE TABLE `building` (
`dir_code` char(4),
`building` char(6)
) ENGINE=archive DEFAULT CHARSET=latin1;
-LOCK TABLES `building` WRITE;
-INSERT INTO `building` VALUES ('N41','1300'),('N01','1453'),('M00','1000'),('N41','1301'),('N41','1305');
-UNLOCK TABLES;
+INSERT INTO `building` VALUES
+('N41','1300'),('N01','1453'),('M00','1000'),
+('N41','1301'),('N41','1305');
DROP TABLE IF EXISTS `directorate`;
Warnings:
Note 1051 Unknown table 'directorate'
@@ -25,9 +25,8 @@ CREATE TABLE `directorate` (
`dir_name` char(30),
`dir_head_id` char(9)
) ENGINE=archive DEFAULT CHARSET=latin1;
-LOCK TABLES `directorate` WRITE;
-INSERT INTO `directorate` VALUES ('N41','Development','333445555'),('N01','Human Resources','123654321'),('M00','Management','333444444');
-UNLOCK TABLES;
+INSERT INTO `directorate` VALUES ('N41','Development','333445555'),
+('N01','Human Resources','123654321'),('M00','Management','333444444');
USE db2;
DROP TABLE IF EXISTS `staff`;
Warnings:
@@ -39,11 +38,26 @@ CREATE TABLE `staff` (
`last_name` char(30),
`sex` char(1),
`salary` int(11),
-`mgr_id` char(9)
+`mgr_id` char(9),
+index(mid_name)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
-LOCK TABLES `staff` WRITE;
-INSERT INTO `staff` VALUES ('333445555','John','Q','Smith','M',30000,'333444444'),('123763153','William','E','Walters','M',25000,'123654321'),('333444444','Alicia','F','St.Cruz','F',25000,NULL),('921312388','Goy','X','Hong','F',40000,'123654321'),('800122337','Rajesh','G','Kardakarna','M',38000,'333445555'),('820123637','Monty','C','Smythe','M',38000,'333445555'),('830132335','Richard','E','Jones','M',38000,'333445555'),('333445665','Edward','E','Engles','M',25000,'333445555'),('123654321','Beware','D','Borg','F',55000,'333444444'),('123456789','Wilma','N','Maxima','F',43000,'333445555');
-UNLOCK TABLES;
+INSERT INTO `staff` VALUES
+('333445555','John','Q','Smith','M',30000,'333444444'),
+('123763153','William','E','Walters','M',25000,'123654321'),
+('333444444','Alicia','F','St.Cruz','F',25000,NULL),
+('921312388','Goy','X','Hong','F',40000,'123654321'),
+('800122337','Rajesh','G','Kardakarna','M',38000,'333445555'),
+('820123637','Monty','C','Smythe','M',38000,'333445555'),
+('830132335','Richard','E','Jones','M',38000,'333445555'),
+('333445665','Edward','E','Engles','M',25000,'333445555'),
+('123654321','Beware','D','Borg','F',55000,'333444444'),
+('123456789','Wilma','N','Maxima','F',43000,'333445555');
+insert into staff select * from staff;
+insert into staff select * from staff;
+insert into staff select * from staff;
+insert into staff select * from staff;
+insert into staff select * from staff;
+create table staff2 like staff;
DROP TABLE IF EXISTS `tasking`;
Warnings:
Note 1051 Unknown table 'tasking'
@@ -52,26 +66,84 @@ CREATE TABLE `tasking` (
`project_number` char(9),
`hours_worked` double(10,2)
) ENGINE=blackhole DEFAULT CHARSET=latin1;
-LOCK TABLES `tasking` WRITE;
-INSERT INTO `tasking` VALUES ('333445555','405',23),('123763153','405',33.5),('921312388','601',44),('800122337','300',13),('820123637','300',9.5),('830132335','401',8.5),('333445555','300',11),('921312388','500',13),('800122337','300',44),('820123637','401',500.5),('830132335','400',12),('333445665','600',300.25),('123654321','607',444.75),('123456789','300',1000);
-UNLOCK TABLES;
+INSERT INTO `tasking` VALUES ('333445555','405',23),
+('123763153','405',33.5),('921312388','601',44),('800122337','300',13),
+('820123637','300',9.5),('830132335','401',8.5),('333445555','300',11),
+('921312388','500',13),('800122337','300',44),('820123637','401',500.5),
+('830132335','400',12),('333445665','600',300.25),('123654321','607',444.75),
+('123456789','300',1000);
+CREATE TABLE `tasking2` (
+`id` char(9),
+`project_number` char(9),
+`hours_worked` double(10,2),
+index(id), index(project_number), index(hours_worked)
+) delay_key_write=1 ENGINE=MyISAM DEFAULT CHARSET=latin1;
+INSERT INTO `tasking2` VALUES
+('333445555','405',23),('123763153','405',33.5),
+('921312388','601',44),('800122337','300',13),('820123637','300',9.5),
+('830132335','401',8.5),('333445555','300',11),('921312388','500',13),
+('800122337','300',44),('820123637','401',500.5),('830132335','400',12),
+('333445665','600',300.25),('123654321','607',444.75),
+('123456789','300',1000);
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+SELECT get_lock("data_prepare", 100);
+get_lock("data_prepare", 100)
+1
BACKUP DATABASE db1,db2 TO 'test.ba';
+use db2;
+insert into staff2 select * from staff;
+checksum table staff2;
+Table Checksum
+db2.staff2 569030976
+checksum table staff;
+Table Checksum
+db2.staff 569030976
+INSERT INTO staff SELECT * FROM staff;
+UPDATE staff SET sex='N' where sex='M';
+DELETE FROM staff WHERE first_name='Edward';
+checksum table staff;
+Table Checksum
+db2.staff 2375145344
+checksum table tasking2;
+Table Checksum
+db2.tasking2 2742373372
+update tasking2 set hours_worked=12 where hours_worked<11;
+checksum table tasking2;
+Table Checksum
+db2.tasking2 778095994
+SELECT release_lock("data_prepare");
+release_lock("data_prepare")
+1
Backup Summary
- header = 52 bytes
- meta-data = 807 bytes
- data = 1217 bytes
+ header = 80 bytes
+ meta-data = 1474 bytes
+ data = 160400 bytes
--------------
- total 2076 bytes
+ total 161954 bytes
+use db2;
+CHECK TABLE staff EXTENDED;
+Table Op Msg_type Msg_text
+db2.staff check status OK
+CHECK TABLE tasking2 EXTENDED;
+Table Op Msg_type Msg_text
+db2.tasking2 check status OK
+CHECK TABLE staff2 EXTENDED;
+Table Op Msg_type Msg_text
+db2.staff2 check status OK
DROP DATABASE db1;
DROP DATABASE db2;
USE mysql;
RESTORE FROM 'test.ba';
Restore Summary
- header = 52 bytes
- meta-data = 807 bytes
- data = 1217 bytes
+ header = 80 bytes
+ meta-data = 1474 bytes
+ data = 160400 bytes
--------------
- total 2076 bytes
+ total 161954 bytes
USE db1;
SHOW TABLES;
Tables_in_db1
@@ -93,26 +165,63 @@ USE db2;
SHOW TABLES;
Tables_in_db2
staff
+staff2
tasking
-SELECT * FROM staff;
-id first_name mid_name last_name sex salary mgr_id
-333445555 John Q Smith M 30000 333444444
-123763153 William E Walters M 25000 123654321
-333444444 Alicia F St.Cruz F 25000 NULL
-921312388 Goy X Hong F 40000 123654321
-800122337 Rajesh G Kardakarna M 38000 333445555
-820123637 Monty C Smythe M 38000 333445555
-830132335 Richard E Jones M 38000 333445555
-333445665 Edward E Engles M 25000 333445555
-123654321 Beware D Borg F 55000 333444444
-123456789 Wilma N Maxima F 43000 333445555
-SHOW CREATE TABLE tasking;
+tasking2
+SHOW CREATE TABLE staff;
Table Create Table
-tasking CREATE TABLE `tasking` (
+staff CREATE TABLE `staff` (
+ `id` char(9) DEFAULT NULL,
+ `first_name` char(20) DEFAULT NULL,
+ `mid_name` char(20) DEFAULT NULL,
+ `last_name` char(30) DEFAULT NULL,
+ `sex` char(1) DEFAULT NULL,
+ `salary` int(11) DEFAULT NULL,
+ `mgr_id` char(9) DEFAULT NULL,
+ KEY `mid_name` (`mid_name`)
+) ENGINE=MyISAM DEFAULT CHARSET=latin1
+checksum table staff;
+Table Checksum
+db2.staff 2375145344
+CHECK TABLE staff EXTENDED;
+Table Op Msg_type Msg_text
+db2.staff check warning 1 client is using or hasn't closed the table properly
+db2.staff check status OK
+SHOW CREATE TABLE tasking2;
+Table Create Table
+tasking2 CREATE TABLE `tasking2` (
`id` char(9) DEFAULT NULL,
`project_number` char(9) DEFAULT NULL,
- `hours_worked` double(10,2) DEFAULT NULL
-) ENGINE=BLACKHOLE DEFAULT CHARSET=latin1
+ `hours_worked` double(10,2) DEFAULT NULL,
+ KEY `id` (`id`),
+ KEY `project_number` (`project_number`),
+ KEY `hours_worked` (`hours_worked`)
+) ENGINE=MyISAM DEFAULT CHARSET=latin1 DELAY_KEY_WRITE=1
+checksum table tasking2;
+Table Checksum
+db2.tasking2 778095994
+CHECK TABLE tasking2 EXTENDED;
+Table Op Msg_type Msg_text
+db2.tasking2 check warning 1 client is using or hasn't closed the table properly
+db2.tasking2 check status OK
+SHOW CREATE TABLE staff2;
+Table Create Table
+staff2 CREATE TABLE `staff2` (
+ `id` char(9) DEFAULT NULL,
+ `first_name` char(20) DEFAULT NULL,
+ `mid_name` char(20) DEFAULT NULL,
+ `last_name` char(30) DEFAULT NULL,
+ `sex` char(1) DEFAULT NULL,
+ `salary` int(11) DEFAULT NULL,
+ `mgr_id` char(9) DEFAULT NULL,
+ KEY `mid_name` (`mid_name`)
+) ENGINE=MyISAM DEFAULT CHARSET=latin1
+checksum table staff2;
+Table Checksum
+db2.staff2 569030976
+CHECK TABLE staff2 EXTENDED;
+Table Op Msg_type Msg_text
+db2.staff2 check status OK
DROP DATABASE IF EXISTS db1;
DROP DATABASE IF EXISTS db2;
DROP DATABASE IF EXISTS bup_default;
@@ -319,19 +428,19 @@ Chuck@linux:~> mysql -uroot -p
Enter password:
BACKUP DATABASE bup_default TO "bup_default.bak";
Backup Summary
- header = 43 bytes
+ header = 50 bytes
meta-data = 890 bytes
- data = 7411 bytes
+ data = 7030 bytes
--------------
- total 8344 bytes
+ total 7970 bytes
DROP DATABASE bup_default;
RESTORE FROM "bup_default.bak";
Restore Summary
- header = 43 bytes
+ header = 50 bytes
meta-data = 890 bytes
- data = 7411 bytes
+ data = 7030 bytes
--------------
- total 8344 bytes
+ total 7970 bytes
SELECT * FROM bup_default.t1;
a
1
diff -Nrup a/mysql-test/r/backup_no_engine.result b/mysql-test/r/backup_no_engine.result
--- a/mysql-test/r/backup_no_engine.result 2007-10-01 16:13:20 +02:00
+++ b/mysql-test/r/backup_no_engine.result 2007-11-06 20:55:59 +01:00
@@ -4,20 +4,20 @@ CREATE TABLE db.t1 (a int, b char(32))
ENGINE=myisam;
BACKUP DATABASE db TO "db.backup";
Backup Summary
- header = 12 bytes
+ header = 19 bytes
meta-data = 120 bytes
- data = 0 bytes
+ data = 1029 bytes
--------------
- total 132 bytes
+ total 1168 bytes
DROP DATABASE db;
CREATE DATABASE db;
RESTORE FROM "db.backup";
Restore Summary
- header = 12 bytes
+ header = 19 bytes
meta-data = 120 bytes
- data = 0 bytes
+ data = 1029 bytes
--------------
- total 132 bytes
+ total 1168 bytes
SHOW TABLES IN db;
Tables_in_db
t1
diff -Nrup a/mysql-test/r/myisamlog.result b/mysql-test/r/myisamlog.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/myisamlog.result 2007-11-06 20:56:02 +01:00
@@ -0,0 +1,48 @@
+create database if not exists mysqltest;
+use mysqltest;
+drop table if exists t1;
+create table t1 (a int, b varchar(100)) engine=myisam;
+insert into t1 values(1,'life'), (2,'file');
+insert into t1 select a*5, concat("A ",b) from t1;
+update t1 set b="A knife" where a=5;
+delete from t1 where a=10;
+select * from t1;
+a b
+1 life
+2 file
+5 A knife
+truncate table t1;
+flush table t1;
+Commands Used count Errors Recover errors
+open 16 0 0
+write 4 0 0
+update 1 0 0
+delete 1 0 0
+close 8 0 0
+extra 72 0 0
+lock 48 0 0
+Total 150 0 0
+select * from t1;
+a b
+flush table t1;
+Trying to update MyISAM files according to log './var/master-data/myisam.log'
+Tables updated successfully
+
+Commands Used count Errors Recover errors
+open 17 0 0
+write 4 0 0
+update 1 0 0
+delete 1 0 0
+close 9 0 0
+extra 75 0 0
+lock 50 0 0
+Total 157 0 0
+select * from t1;
+a b
+1 life
+2 file
+5 A knife
+check table t1 extended;
+Table Op Msg_type Msg_text
+mysqltest.t1 check status OK
+drop table t1;
diff -Nrup a/mysql-test/t/backup.test b/mysql-test/t/backup.test
--- a/mysql-test/t/backup.test 2007-07-06 11:57:13 +02:00
+++ b/mysql-test/t/backup.test 2007-11-06 20:55:59 +01:00
@@ -1,8 +1,15 @@
+# Tests is for the default, consistent snapshot and MyISAM online
+# backup drivers.
+
--source include/have_innodb.inc
connect (backup,localhost,root,,);
connect (restore,localhost,root,,);
+#
+# This test is for the default (Archive+Blackhole) and MyISAM driver.
+#
+
connection backup;
DROP DATABASE IF EXISTS db1;
@@ -27,9 +34,9 @@ CREATE TABLE `building` (
# Dumping data for table `building`
#
-LOCK TABLES `building` WRITE;
-INSERT INTO `building` VALUES ('N41','1300'),('N01','1453'),('M00','1000'),('N41','1301'),('N41','1305');
-UNLOCK TABLES;
+INSERT INTO `building` VALUES
+('N41','1300'),('N01','1453'),('M00','1000'),
+('N41','1301'),('N41','1305');
#
@@ -47,9 +54,8 @@ CREATE TABLE `directorate` (
# Dumping data for table `directorate`
#
-LOCK TABLES `directorate` WRITE;
-INSERT INTO `directorate` VALUES ('N41','Development','333445555'),('N01','Human Resources','123654321'),('M00','Management','333444444');
-UNLOCK TABLES;
+INSERT INTO `directorate` VALUES ('N41','Development','333445555'),
+('N01','Human Resources','123654321'),('M00','Management','333444444');
USE db2;
@@ -65,16 +71,37 @@ CREATE TABLE `staff` (
`last_name` char(30),
`sex` char(1),
`salary` int(11),
- `mgr_id` char(9)
+ `mgr_id` char(9),
+ index(mid_name)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
#
# Dumping data for table `staff`
#
-LOCK TABLES `staff` WRITE;
-INSERT INTO `staff` VALUES ('333445555','John','Q','Smith','M',30000,'333444444'),('123763153','William','E','Walters','M',25000,'123654321'),('333444444','Alicia','F','St.Cruz','F',25000,NULL),('921312388','Goy','X','Hong','F',40000,'123654321'),('800122337','Rajesh','G','Kardakarna','M',38000,'333445555'),('820123637','Monty','C','Smythe','M',38000,'333445555'),('830132335','Richard','E','Jones','M',38000,'333445555'),('333445665','Edward','E','Engles','M',25000,'333445555'),('123654321','Beware','D','Borg','F',55000,'333444444'),('123456789','Wilma','N','Maxima','F',43000,'333445555');
-UNLOCK TABLES;
+INSERT INTO `staff` VALUES
+('333445555','John','Q','Smith','M',30000,'333444444'),
+('123763153','William','E','Walters','M',25000,'123654321'),
+('333444444','Alicia','F','St.Cruz','F',25000,NULL),
+('921312388','Goy','X','Hong','F',40000,'123654321'),
+('800122337','Rajesh','G','Kardakarna','M',38000,'333445555'),
+('820123637','Monty','C','Smythe','M',38000,'333445555'),
+('830132335','Richard','E','Jones','M',38000,'333445555'),
+('333445665','Edward','E','Engles','M',25000,'333445555'),
+('123654321','Beware','D','Borg','F',55000,'333444444'),
+('123456789','Wilma','N','Maxima','F',43000,'333445555');
+# grow the table a bit
+insert into staff select * from staff;
+insert into staff select * from staff;
+insert into staff select * from staff;
+insert into staff select * from staff;
+insert into staff select * from staff;
+
+# and make an empty table (INSERT SELECT works differently if the
+# destination table is empty: it disables indices (see
+# ha_myisam::start_bulk_insert()) and thus repairs them, which backup
+# has to tolerate.
+create table staff2 like staff;
#
# Table structure for table `tasking`
@@ -91,12 +118,82 @@ CREATE TABLE `tasking` (
# Dumping data for table `tasking`
#
-LOCK TABLES `tasking` WRITE;
-INSERT INTO `tasking` VALUES ('333445555','405',23),('123763153','405',33.5),('921312388','601',44),('800122337','300',13),('820123637','300',9.5),('830132335','401',8.5),('333445555','300',11),('921312388','500',13),('800122337','300',44),('820123637','401',500.5),('830132335','400',12),('333445665','600',300.25),('123654321','607',444.75),('123456789','300',1000);
-UNLOCK TABLES;
+INSERT INTO `tasking` VALUES ('333445555','405',23),
+('123763153','405',33.5),('921312388','601',44),('800122337','300',13),
+('820123637','300',9.5),('830132335','401',8.5),('333445555','300',11),
+('921312388','500',13),('800122337','300',44),('820123637','401',500.5),
+('830132335','400',12),('333445665','600',300.25),('123654321','607',444.75),
+('123456789','300',1000);
+
+CREATE TABLE `tasking2` (
+ `id` char(9),
+ `project_number` char(9),
+ `hours_worked` double(10,2),
+ index(id), index(project_number), index(hours_worked)
+) delay_key_write=1 ENGINE=MyISAM DEFAULT CHARSET=latin1;
+
+--
+-- Dumping data for table `tasking2`
+--
+
+INSERT INTO `tasking2` VALUES
+('333445555','405',23),('123763153','405',33.5),
+('921312388','601',44),('800122337','300',13),('820123637','300',9.5),
+('830132335','401',8.5),('333445555','300',11),('921312388','500',13),
+('800122337','300',44),('820123637','401',500.5),('830132335','400',12),
+('333445665','600',300.25),('123654321','607',444.75),
+('123456789','300',1000);
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+insert into tasking select * from tasking;
+
+connection restore;
+# make sure we can update MyISAM tables before the backup finishes,
+# to test backup in online conditions
+SELECT get_lock("data_prepare", 100);
+
+connection backup;
+send BACKUP DATABASE db1,db2 TO 'test.ba';
+connection restore;
+# Must wait to know when backup has entered lock.
+let $wait_condition = SELECT state = "debug_sync_point: data_prepare"
+ FROM INFORMATION_SCHEMA.PROCESSLIST
+ WHERE info LIKE "backup database %";
+# Until Chuck has pushed the cset where backup thread sets its
+# processlist info http://lists.mysql.com/commits/37013 ,
+# such smart wait is impossible
+#--source include/wait_condition.inc
+sleep 2;
+
+# At this point, MyISAM tables are copied, but we will do a
+# modification to them before BACKUP ends. Then we will verify that
+# the backup does contain this modification.
+
+use db2;
+insert into staff2 select * from staff;
+checksum table staff2;
+checksum table staff;
+INSERT INTO staff SELECT * FROM staff;
+UPDATE staff SET sex='N' where sex='M';
+DELETE FROM staff WHERE first_name='Edward';
+checksum table staff;
+checksum table tasking2;
+update tasking2 set hours_worked=12 where hours_worked<11;
+checksum table tasking2;
+# let BACKUP finish now
+SELECT release_lock("data_prepare");
+
+connection backup;
+reap;
+use db2;
+# verify that backup didn't corrupt MyISAM tables
+CHECK TABLE staff EXTENDED;
+CHECK TABLE tasking2 EXTENDED;
+CHECK TABLE staff2 EXTENDED;
-BACKUP DATABASE db1,db2 TO 'test.ba';
DROP DATABASE db1;
DROP DATABASE db2;
@@ -120,14 +217,23 @@ SELECT * FROM directorate;
USE db2;
SHOW TABLES;
-SELECT * FROM staff;
-SHOW CREATE TABLE tasking;
+# verify that restored MyISAM tables are ok and correspond to the
+# validity point
+SHOW CREATE TABLE staff;
+checksum table staff;
+CHECK TABLE staff EXTENDED;
+SHOW CREATE TABLE tasking2;
+checksum table tasking2;
+CHECK TABLE tasking2 EXTENDED;
+SHOW CREATE TABLE staff2;
+checksum table staff2;
+CHECK TABLE staff2 EXTENDED;
DROP DATABASE IF EXISTS db1;
DROP DATABASE IF EXISTS db2;
#
-# This test is for the default and snapshot online backup drivers
+# This test is for the MyISAM and snapshot online backup drivers
#
connection backup;
diff -Nrup a/mysql-test/t/key_cache.test b/mysql-test/t/key_cache.test
--- a/mysql-test/t/key_cache.test 2007-05-31 20:04:51 +02:00
+++ b/mysql-test/t/key_cache.test 2007-11-06 20:55:59 +01:00
@@ -71,7 +71,7 @@ show status like 'key_blocks_used';
# Following results differs on 64 and 32 bit systems because of different
# pointer sizes, which takes up different amount of space in key cache
---replace_result 1812 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
+--replace_result 1799 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
insert into t1 values (1, 'qqqq'), (11, 'yyyy');
@@ -84,7 +84,7 @@ update t1 set p=2 where p=1;
update t2 set i=2 where i=1;
show status like 'key_blocks_used';
---replace_result 1808 KEY_BLOCKS_UNUSED 1789 KEY_BLOCKS_UNUSED 1670 KEY_BLOCKS_UNUSED 1814 KEY_BLOCKS_UNUSED 1820 KEY_BLOCKS_UNUSED
+--replace_result 1795 KEY_BLOCKS_UNUSED 1789 KEY_BLOCKS_UNUSED 1670 KEY_BLOCKS_UNUSED 1814 KEY_BLOCKS_UNUSED 1820 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
cache index t1 key (`primary`) in keycache1;
@@ -146,7 +146,7 @@ cache index t1,t2 in default;
drop table t1,t2,t3;
show status like 'key_blocks_used';
---replace_result 1812 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
+--replace_result 1799 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
diff -Nrup a/mysql-test/t/myisamlog-master.opt b/mysql-test/t/myisamlog-master.opt
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/myisamlog-master.opt 2007-11-06 20:56:01 +01:00
@@ -0,0 +1 @@
+--log-isam
diff -Nrup a/mysql-test/t/myisamlog.test b/mysql-test/t/myisamlog.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/myisamlog.test 2007-11-06 20:56:02 +01:00
@@ -0,0 +1,37 @@
+# Test of MyISAM logical logging and the myisamlog utility:
+# see if they can repopulate a table.
+
+disable_warnings;
+create database if not exists mysqltest;
+use mysqltest;
+drop table if exists t1;
+enable_warnings;
+# CREATE generates no entry in the logical log
+create table t1 (a int, b varchar(100)) engine=myisam;
+
+# we put some data into the table
+insert into t1 values(1,'life'), (2,'file');
+insert into t1 select a*5, concat("A ",b) from t1;
+update t1 set b="A knife" where a=5;
+delete from t1 where a=10;
+select * from t1;
+
+# wipe it out (TRUNCATE generates no entry in the logical log)
+truncate table t1;
+# close the table (as we are going to change it out of the server process)
+flush table t1;
+
+# look at log
+exec $MYISAMLOG $MYSQLTEST_VARDIR/master-data/myisam.log ;
+# should not have changed the table
+select * from t1;
+flush table t1;
+
+# apply log to empty table
+--replace_result $MYSQL_TEST_DIR .
+exec $MYISAMLOG -F $MYSQLTEST_VARDIR/master-data -u $MYSQLTEST_VARDIR/master-data/myisam.log ;
+
+# reopen the table and verify that content is back
+select * from t1;
+check table t1 extended;
+drop table t1;
diff -Nrup a/mysys/mf_iocache.c b/mysys/mf_iocache.c
--- a/mysys/mf_iocache.c 2007-05-10 11:59:25 +02:00
+++ b/mysys/mf_iocache.c 2007-11-06 20:56:00 +01:00
@@ -129,6 +129,9 @@ init_functions(IO_CACHE* info)
}
+/* FUNCTIONS TO SET UP OR RESET A CACHE */
+
+
/*
Initialize an IO_CACHE object
@@ -166,7 +169,7 @@ int init_io_cache(IO_CACHE *info, File f
info->file= file;
info->type= TYPE_NOT_SET; /* Don't set it until mutex are created */
info->pos_in_file= seek_offset;
- info->pre_close = info->pre_read = info->post_read = 0;
+ info->pre_close= info->pre_read= info->post_read= info->post_write= NULL;
info->arg = 0;
info->alloced_buffer = 0;
info->buffer=0;
@@ -278,7 +281,7 @@ int init_io_cache(IO_CACHE *info, File f
/* End_of_file may be changed by user later */
info->end_of_file= end_of_file;
- info->error=0;
+ info->error= info->hard_write_error_in_the_past= 0;
info->type= type;
init_functions(info);
#ifdef HAVE_AIOWAIT
@@ -405,7 +408,7 @@ my_bool reinit_io_cache(IO_CACHE *info,
}
}
info->type=type;
- info->error=0;
+ info->error= info->hard_write_error_in_the_past= 0;
init_functions(info);
#ifdef HAVE_AIOWAIT
@@ -422,6 +425,8 @@ my_bool reinit_io_cache(IO_CACHE *info,
} /* reinit_io_cache */
+/* FUNCTIONS TO DO READS FROM THE CACHE */
+
/*
Read buffered.
@@ -1471,14 +1476,19 @@ int _my_b_get(IO_CACHE *info)
uchar buff;
IO_CACHE_CALLBACK pre_read,post_read;
if ((pre_read = info->pre_read))
- (*pre_read)(info);
+ (*pre_read)(info, NULL, 0, 0);
if ((*(info)->read_function)(info,&buff,1))
return my_b_EOF;
if ((post_read = info->post_read))
- (*post_read)(info);
+ (*post_read)(info, NULL, 0, 0);
return (int) (uchar) buff;
}
+/* FUNCTIONS TO DO WRITES TO THE CACHE */
+
+#define set_hard_write_error(CACHE) \
+ ((CACHE)->error= (CACHE)->hard_write_error_in_the_past= -1)
+
/*
Write a byte buffer to IO_CACHE and flush to disk
if IO_CACHE is full.
@@ -1496,7 +1506,7 @@ int _my_b_write(register IO_CACHE *info,
if (info->pos_in_file+info->buffer_length > info->end_of_file)
{
my_errno=errno=EFBIG;
- return info->error = -1;
+ return set_hard_write_error(info);
}
rest_length= (size_t) (info->write_end - info->write_pos);
@@ -1520,13 +1530,15 @@ int _my_b_write(register IO_CACHE *info,
*/
if (my_seek(info->file,info->pos_in_file,MY_SEEK_SET,MYF(0)))
{
- info->error= -1;
+ set_hard_write_error(info);
return (1);
}
info->seek_not_done=0;
}
if (my_write(info->file, Buffer, length, info->myflags | MY_NABP))
- return info->error= -1;
+ return set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, length, info->pos_in_file);
#ifdef THREAD
/*
@@ -1571,7 +1583,7 @@ int my_b_append(register IO_CACHE *info,
*/
DBUG_ASSERT(!info->share);
#endif
-
+ DBUG_ASSERT(info->post_write == NULL); /* unsupported */
lock_append_buffer(info);
rest_length= (size_t) (info->write_end - info->write_pos);
if (Count <= rest_length)
@@ -1591,7 +1603,7 @@ int my_b_append(register IO_CACHE *info,
if (my_write(info->file,Buffer, length, info->myflags | MY_NABP))
{
unlock_append_buffer(info);
- return info->error= -1;
+ return set_hard_write_error(info);
}
Count-=length;
Buffer+=length;
@@ -1644,12 +1656,21 @@ int my_block_write(register IO_CACHE *in
{
/* Of no overlap, write everything without buffering */
if (pos + Count <= info->pos_in_file)
- return my_pwrite(info->file, Buffer, Count, pos,
- info->myflags | MY_NABP);
+ {
+ int ret= my_pwrite(info->file, Buffer, Count, pos,
+ info->myflags | MY_NABP);
+ if (unlikely(ret))
+ set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, Count, pos);
+ return ret;
+ }
/* Write the part of the block that is before buffer */
length= (uint) (info->pos_in_file - pos);
if (my_pwrite(info->file, Buffer, length, pos, info->myflags | MY_NABP))
- info->error= error= -1;
+ error= set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, length, pos);
Buffer+=length;
pos+= length;
Count-= length;
@@ -1710,7 +1731,7 @@ int my_b_flush_io_cache(IO_CACHE *info,
if (info->file == -1)
{
if (real_open_cached_file(info))
- DBUG_RETURN((info->error= -1));
+ DBUG_RETURN(set_hard_write_error(info));
}
LOCK_APPEND_BUFFER;
@@ -1738,29 +1759,42 @@ int my_b_flush_io_cache(IO_CACHE *info,
MY_FILEPOS_ERROR)
{
UNLOCK_APPEND_BUFFER;
- DBUG_RETURN((info->error= -1));
+ DBUG_RETURN(set_hard_write_error(info));
}
if (!append_cache)
info->seek_not_done=0;
}
- if (!append_cache)
- info->pos_in_file+=length;
info->write_end= (info->write_buffer+info->buffer_length-
((pos_in_file+length) & (IO_SIZE-1)));
if (my_write(info->file,info->write_buffer,length,
info->myflags | MY_NABP))
- info->error= -1;
+ set_hard_write_error(info);
else
info->error= 0;
if (!append_cache)
{
+ /*
+ This post_write is really POST-write; callers depend on this! So
+ always call it after writing to the file, not before.
+ */
+ if (info->post_write)
+ (*(info->post_write))(info, info->write_buffer,
+ length, info->pos_in_file);
+ /*
+ The addition below will make the info->pos_in_file be the end of
+ written block; whereas the value we needed in post_write is the
+ value before the addition. That's why we called post_write before
+ this.
+ */
+ info->pos_in_file+=length;
set_if_bigger(info->end_of_file,(pos_in_file+length));
}
else
{
info->end_of_file+=(info->write_pos-info->append_read_pos);
DBUG_ASSERT(info->end_of_file == my_tell(info->file,MYF(0)));
+ DBUG_ASSERT(info->post_write == NULL); /* unsupported */
}
info->append_read_pos=info->write_pos=info->write_buffer;
@@ -1814,7 +1848,7 @@ int end_io_cache(IO_CACHE *info)
if ((pre_close=info->pre_close))
{
- (*pre_close)(info);
+ (*pre_close)(info, NULL, 0, 0);
info->pre_close= 0;
}
if (info->alloced_buffer)
diff -Nrup a/mysys/mf_keycache.c b/mysys/mf_keycache.c
--- a/mysys/mf_keycache.c 2007-10-09 19:55:13 +02:00
+++ b/mysys/mf_keycache.c 2007-11-06 20:56:00 +01:00
@@ -13,30 +13,33 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- These functions handle keyblock cacheing for ISAM and MyISAM tables.
-
- One cache can handle many files.
- It must contain buffers of the same blocksize.
- init_key_cache() should be used to init cache handler.
-
- The free list (free_block_list) is a stack like structure.
- When a block is freed by free_block(), it is pushed onto the stack.
- When a new block is required it is first tried to pop one from the stack.
- If the stack is empty, it is tried to get a never-used block from the pool.
- If this is empty too, then a block is taken from the LRU ring, flushing it
- to disk, if neccessary. This is handled in find_key_block().
- With the new free list, the blocks can have three temperatures:
- hot, warm and cold (which is free). This is remembered in the block header
- by the enum BLOCK_TEMPERATURE temperature variable. Remembering the
- temperature is neccessary to correctly count the number of warm blocks,
- which is required to decide when blocks are allowed to become hot. Whenever
- a block is inserted to another (sub-)chain, we take the old and new
- temperature into account to decide if we got one more or less warm block.
- blocks_unused is the sum of never used blocks in the pool and of currently
- free blocks. blocks_used is the number of blocks fetched from the pool and
- as such gives the maximum number of in-use blocks at any time.
+/**
+ @file
+ These functions handle keyblock cacheing for ISAM and MyISAM tables.
+
+ One cache can handle many files.
+ It must contain buffers of the same blocksize.
+ init_key_cache() should be used to init cache handler.
+
+ The free list (free_block_list) is a stack like structure.
+ When a block is freed by free_block(), it is pushed onto the stack.
+ When a new block is required it is first tried to pop one from the stack.
+ If the stack is empty, it is tried to get a never-used block from the pool.
+ If this is empty too, then a block is taken from the LRU ring, flushing it
+ to disk, if neccessary. This is handled in find_key_block().
+ With the new free list, the blocks can have three temperatures:
+ hot, warm and cold (which is free). This is remembered in the block header
+ by the enum BLOCK_TEMPERATURE temperature variable. Remembering the
+ temperature is neccessary to correctly count the number of warm blocks,
+ which is required to decide when blocks are allowed to become hot. Whenever
+ a block is inserted to another (sub-)chain, we take the old and new
+ temperature into account to decide if we got one more or less warm block.
+ blocks_unused is the sum of never used blocks in the pool and of currently
+ free blocks. blocks_used is the number of blocks fetched from the pool and
+ as such gives the maximum number of in-use blocks at any time.
+*/
+/*
Key Cache Locking
=================
@@ -212,6 +215,8 @@ struct st_block_link
uint hits_left; /* number of hits left until promotion */
ulonglong last_hit_time; /* timestamp of the last hit */
KEYCACHE_CONDVAR *condvar; /* condition variable for 'no readers' event */
+ KEYCACHE_POST_WRITE_CALLBACK post_write;/**< Called when block is flushed */
+ void *post_write_arg; /**< post_write's argument*/
};
KEY_CACHE dflt_key_cache_var;
@@ -768,6 +773,29 @@ void end_key_cache(KEY_CACHE *keycache,
} /* end_key_cache */
+/**
+ Does a my_pwrite() to the file and then calls callback. Arguments are those
+ of my_pwrite() plus the callback and its argument.
+
+ @note The callback is really POST-write; callers depend on this! So always
+ call it after writing to the file, not before.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 write or callback failed
+*/
+static inline int key_cache_pwrite(int Filedes, const uchar *Buffer,
+ uint Count, my_off_t offset, myf MyFlags,
+ KEYCACHE_POST_WRITE_CALLBACK callback,
+ void *callback_arg)
+{
+ int ret= my_pwrite(Filedes, Buffer, Count, offset, MyFlags);
+ if (callback)
+ ret|= (*callback)(callback_arg, Buffer, Count, offset);
+ return ret;
+}
+
+
#ifdef THREAD
/*
@@ -2205,11 +2233,22 @@ restart:
The call is thread safe because only the current
thread might change the block->hash_link value
*/
- error= my_pwrite(block->hash_link->file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
+ error= key_cache_pwrite(block->hash_link->file,
+ block->buffer + block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos +
+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ block->post_write,
+ block->post_write_arg);
+ /*
+ post_write was called above, block is now non-dirty; block
+ keeps its post_write, but it's ok: post_write will be used
+ only at next flush; next flush implies that block became dirty
+ again, which implies that key_cache_write() was called again,
+ which implies that post_write was overwritten with a new
+ value. No need to reset post_write to NULL now.
+ */
keycache_pthread_mutex_lock(&keycache->cache_lock);
/* Block status must not have changed. */
@@ -2952,6 +2991,10 @@ int key_cache_insert(KEY_CACHE *keycache
length length of the buffer
dont_write if is 0 then all dirty pages involved in writing
should have been flushed from key cache
+ callback a function which should be called after the block
+ has been written to its file (i.e. only at flush
+ time); NULL if no function
+ callback_arg argument which will be passed to 'callback'
RETURN VALUE
0 if a success, 1 - otherwise.
@@ -2971,7 +3014,9 @@ int key_cache_write(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length,
uint block_length __attribute__((unused)),
- int dont_write)
+ int dont_write,
+ KEYCACHE_POST_WRITE_CALLBACK callback,
+ void *callback_arg)
{
my_bool locked_and_incremented= FALSE;
int error=0;
@@ -2988,7 +3033,9 @@ int key_cache_write(KEY_CACHE *keycache,
/* Force writing from buff into disk. */
keycache->global_cache_w_requests++;
keycache->global_cache_write++;
- if (my_pwrite(file, buff, length, filepos, MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, buff, length, filepos,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ callback, callback_arg))
DBUG_RETURN(1);
/* purecov: end */
}
@@ -3063,8 +3110,10 @@ int key_cache_write(KEY_CACHE *keycache,
/* Used in the server. */
keycache->global_cache_write++;
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- if (my_pwrite(file, (uchar*) buff, read_length, filepos + offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, (uchar*) buff, read_length,
+ filepos + offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ callback, callback_arg))
error=1;
keycache_pthread_mutex_lock(&keycache->cache_lock);
}
@@ -3156,6 +3205,8 @@ int key_cache_write(KEY_CACHE *keycache,
*/
if (!(block->status & BLOCK_ERROR))
{
+ block->post_write= callback;
+ block->post_write_arg= callback_arg;
#if !defined(SERIALIZED_READ_FROM_CACHE)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
#endif
@@ -3232,8 +3283,9 @@ no_key_cache:
keycache->global_cache_write++;
if (locked_and_incremented)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- if (my_pwrite(file, (uchar*) buff, length, filepos,
- MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, (uchar*) buff, length, filepos,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ callback, callback_arg))
error=1;
if (locked_and_incremented)
keycache_pthread_mutex_lock(&keycache->cache_lock);
@@ -3459,11 +3511,12 @@ static int flush_cached_blocks(KEY_CACHE
(BLOCK_READ | BLOCK_IN_FLUSH | BLOCK_CHANGED | BLOCK_IN_USE));
block->status|= BLOCK_IN_FLUSHWRITE;
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- error= my_pwrite(file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
+ error= key_cache_pwrite(file,
+ block->buffer+block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ block->post_write, block->post_write_arg);
keycache_pthread_mutex_lock(&keycache->cache_lock);
keycache->global_cache_write++;
if (error)
diff -Nrup a/mysys/my_thr_init.c b/mysys/my_thr_init.c
--- a/mysys/my_thr_init.c 2007-08-21 18:09:43 +02:00
+++ b/mysys/my_thr_init.c 2007-11-06 20:56:00 +01:00
@@ -13,9 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Functions to handle initializating and allocationg of all mysys & debug
- thread variables.
+/**
+ @file
+ Functions for initialization and allocation of all mysys & debug
+ thread variables.
*/
#include "mysys_priv.h"
@@ -29,8 +30,12 @@ pthread_key(struct st_my_thread_var*, TH
pthread_key(struct st_my_thread_var, THR_KEY_mysys);
#endif /* USE_TLS */
pthread_mutex_t THR_LOCK_malloc,THR_LOCK_open,
- THR_LOCK_lock,THR_LOCK_isam,THR_LOCK_myisam,THR_LOCK_heap,
- THR_LOCK_net, THR_LOCK_charset, THR_LOCK_threads, THR_LOCK_time;
+ THR_LOCK_lock,THR_LOCK_isam,THR_LOCK_heap, THR_LOCK_net,
+ THR_LOCK_charset, THR_LOCK_threads, THR_LOCK_time;
+/** For insert/delete in the list of MyISAM open tables */
+pthread_mutex_t THR_LOCK_myisam;
+/** For writing to the MyISAM logs */
+pthread_mutex_t THR_LOCK_myisam_log;
pthread_cond_t THR_COND_threads;
uint THR_thread_count= 0;
uint my_thread_end_wait_time= 5;
@@ -143,6 +148,7 @@ my_bool my_thread_global_init(void)
pthread_mutex_init(&THR_LOCK_lock,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_isam,MY_MUTEX_INIT_SLOW);
pthread_mutex_init(&THR_LOCK_myisam,MY_MUTEX_INIT_SLOW);
+ pthread_mutex_init(&THR_LOCK_myisam_log,MY_MUTEX_INIT_SLOW);
pthread_mutex_init(&THR_LOCK_heap,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_net,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_charset,MY_MUTEX_INIT_FAST);
@@ -209,6 +215,7 @@ void my_thread_global_end(void)
pthread_mutex_destroy(&THR_LOCK_lock);
pthread_mutex_destroy(&THR_LOCK_isam);
pthread_mutex_destroy(&THR_LOCK_myisam);
+ pthread_mutex_destroy(&THR_LOCK_myisam_log);
pthread_mutex_destroy(&THR_LOCK_heap);
pthread_mutex_destroy(&THR_LOCK_net);
pthread_mutex_destroy(&THR_LOCK_time);
diff -Nrup a/sql/backup/backup_aux.h b/sql/backup/backup_aux.h
--- a/sql/backup/backup_aux.h 2007-10-01 16:13:20 +02:00
+++ b/sql/backup/backup_aux.h 2007-11-06 20:56:00 +01:00
@@ -66,7 +66,7 @@ inline int free_table_list(TABLE_LIST *a
if (all_tables)
{
TABLE_LIST *tbl= all_tables;
- TABLE_LIST *prev= tbl;
+ TABLE_LIST *prev;
while (tbl != NULL)
{
prev= tbl;
diff -Nrup a/sql/backup/data_backup.cc b/sql/backup/data_backup.cc
--- a/sql/backup/data_backup.cc 2007-09-11 11:29:29 +02:00
+++ b/sql/backup/data_backup.cc 2007-11-06 20:56:00 +01:00
@@ -418,12 +418,13 @@ int write_table_data(THD*, Backup_info &
inactive.push_back(p);
}
- if (!def_or_snap_used)
- def_or_snap_used= ((i->type() == Image_info::DEFAULT_IMAGE) ||
- (i->type() == Image_info::SNAPSHOT_IMAGE));
- if (def_or_snap_used)
+ if ((i->type() == Image_info::DEFAULT_IMAGE) ||
+ (i->type() == Image_info::SNAPSHOT_IMAGE))
+ {
+ def_or_snap_used= true;
get_default_snapshot_tables(&p->drv(), NULL,
&table_list, &table_list_last);
+ }
}
/*
diff -Nrup a/sql/backup/stream.h b/sql/backup/stream.h
--- a/sql/backup/stream.h 2007-09-11 11:29:29 +02:00
+++ b/sql/backup/stream.h 2007-11-06 20:56:00 +01:00
@@ -14,8 +14,16 @@ namespace backup {
// TODO: remove the upper limit on chunk size
-const size_t io_buffer_size= 1024*1024; // buffer size for kernel/driver data transfers
-const size_t max_chunk_size= 2*1024*1024; // upper limit on a single chunk size
+/*
+ kept under 64k until chunk's length is not stored in 2 bytes anymore
+ (see write2int(len) in stream_result::value OStream::end_chunk() :
+ it stores len in 2 bytes). If chunk's len is > 64k, which happens for
+ MyISAM's log in my tests, I get only len%64k back at restore time, buffers
+ get parsed wrongly, infinite loop etc.
+ Rafal knows, bug will go away with the new stream code.
+*/
+const size_t io_buffer_size= 30*1024; // buffer size for kernel/driver data transfers
+const size_t max_chunk_size= 60*1024; // upper limit on a single chunk size
}
diff -Nrup a/sql/mysqld.cc b/sql/mysqld.cc
--- a/sql/mysqld.cc 2007-10-17 15:57:33 +02:00
+++ b/sql/mysqld.cc 2007-11-06 20:56:00 +01:00
@@ -611,7 +611,7 @@ char *opt_logname, *opt_slow_logname;
/* Static variables */
static bool kill_in_progress, segfaulted;
-static my_bool opt_do_pstack, opt_bootstrap, opt_myisam_log;
+static my_bool opt_do_pstack, opt_bootstrap, opt_myisam_logical_log;
static int cleanup_done;
static ulong opt_specialflag, opt_myisam_block_size;
static char *opt_update_logname, *opt_binlog_index_name;
@@ -3581,8 +3581,8 @@ server.");
pthread_attr_setstacksize(&connection_attrib, NW_THD_STACKSIZE);
#endif
- if (opt_myisam_log)
- (void) mi_log(1);
+ if (opt_myisam_logical_log)
+ (void) mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
/* call ha_init_key_cache() on all key caches to init them */
process_key_caches(&ha_init_key_cache);
@@ -5371,8 +5371,8 @@ Disable with --skip-large-pages.",
(uchar**) &log_error_file_ptr, (uchar**) &log_error_file_ptr, 0, GET_STR,
OPT_ARG, 0, 0, 0, 0, 0, 0},
{"log-isam", OPT_ISAM_LOG, "Log all MyISAM changes to file.",
- (uchar**) &myisam_log_filename, (uchar**) &myisam_log_filename, 0, GET_STR,
- OPT_ARG, 0, 0, 0, 0, 0, 0},
+ (uchar**) &myisam_logical_log_filename, (uchar**)
+ &myisam_logical_log_filename, 0, GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
{"log-long-format", '0',
"Log some extra information to update log. Please note that this option is deprecated; see --log-short-format option.",
0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0},
@@ -7027,7 +7027,7 @@ static void mysql_init_variables(void)
opt_tc_log_file= (char *)"tc.log"; // no hostname in tc_log file name !
opt_secure_auth= 0;
opt_secure_file_priv= 0;
- opt_bootstrap= opt_myisam_log= 0;
+ opt_bootstrap= opt_myisam_logical_log= 0;
mqh_used= 0;
segfaulted= kill_in_progress= 0;
cleanup_done= 0;
@@ -7299,7 +7299,7 @@ mysqld_get_one_option(int optid,
thd_startup_options|=OPTION_BIG_TABLES;
break;
case (int) OPT_ISAM_LOG:
- opt_myisam_log=1;
+ opt_myisam_logical_log=1;
break;
case (int) OPT_UPDATE_LOG:
opt_update_log=1;
diff -Nrup a/sql/sql_bitmap.h b/sql/sql_bitmap.h
--- a/sql/sql_bitmap.h 2007-08-27 10:14:19 +02:00
+++ b/sql/sql_bitmap.h 2007-11-06 20:56:00 +01:00
@@ -156,7 +156,7 @@ template <> class Bitmap<64>
{
ulonglong map;
public:
- Bitmap<64>() { }
+ Bitmap<64>() { init(); }
#if defined(__NETWARE__) || defined(__MWERKS__)
/*
Metwork compiler gives error on Bitmap<64>
@@ -167,7 +167,7 @@ public:
#else
explicit Bitmap<64>(uint prefix_to_set) { set_prefix(prefix_to_set); }
#endif
- void init() { }
+ void init() { clear_all(); }
void init(uint prefix_to_set) { set_prefix(prefix_to_set); }
uint length() const { return 64; }
void set_bit(uint n) { map|= ((ulonglong)1) << n; }
diff -Nrup a/sql/sql_cache.cc b/sql/sql_cache.cc
--- a/sql/sql_cache.cc 2007-09-14 22:09:40 +02:00
+++ b/sql/sql_cache.cc 2007-11-06 20:56:00 +01:00
@@ -2629,8 +2629,9 @@ Query_cache::register_tables_from_list(T
{
char key[MAX_DBKEY_LENGTH];
uint32 db_length;
- uint key_length= filename_2_table_key(key, table->table->filename,
- &db_length);
+ uint key_length=
+ filename_2_table_key(key, table->table->s->unresolv_file_name,
+ &db_length);
(++block_table)->n= ++n;
/*
There are not callback function for for MyISAM, and engine data
diff -Nrup a/sql/sql_class.cc b/sql/sql_class.cc
--- a/sql/sql_class.cc 2007-09-13 10:56:30 +02:00
+++ b/sql/sql_class.cc 2007-11-06 20:56:00 +01:00
@@ -839,8 +839,9 @@ void THD::awake(THD::killed_state state_
if (mysys_var)
{
pthread_mutex_lock(&mysys_var->mutex);
- if (!system_thread) // Don't abort locks
- mysys_var->abort=1;
+ if (system_thread == NON_SYSTEM_THREAD ||
+ system_thread == SYSTEM_THREAD_BACKUP)
+ mysys_var->abort= 1; // abort locks
/*
This broadcast could be up in the air if the victim thread
exits the cond in the time between read and broadcast, but that is
diff -Nrup a/sql/sql_class.h b/sql/sql_class.h
--- a/sql/sql_class.h 2007-09-13 10:56:30 +02:00
+++ b/sql/sql_class.h 2007-11-06 20:56:00 +01:00
@@ -936,7 +936,8 @@ enum enum_thread_type
SYSTEM_THREAD_SLAVE_SQL= 4,
SYSTEM_THREAD_NDBCLUSTER_BINLOG= 8,
SYSTEM_THREAD_EVENT_SCHEDULER= 16,
- SYSTEM_THREAD_EVENT_WORKER= 32
+ SYSTEM_THREAD_EVENT_WORKER= 32,
+ SYSTEM_THREAD_BACKUP= 64
};
diff -Nrup a/sql/sql_load.cc b/sql/sql_load.cc
--- a/sql/sql_load.cc 2007-08-26 07:51:36 +02:00
+++ b/sql/sql_load.cc 2007-11-06 20:56:00 +01:00
@@ -1139,8 +1139,7 @@ READ_INFO::READ_INFO(File file_par, uint
cache.read_function = _my_b_net_read;
if (mysql_bin_log.is_open())
- cache.pre_read = cache.pre_close =
- (IO_CACHE_CALLBACK) log_loaded_block;
+ cache.pre_read= cache.pre_close = log_loaded_block;
#endif
}
}
diff -Nrup a/sql/sql_repl.cc b/sql/sql_repl.cc
--- a/sql/sql_repl.cc 2007-08-26 10:01:10 +02:00
+++ b/sql/sql_repl.cc 2007-11-06 20:56:00 +01:00
@@ -1567,7 +1567,10 @@ err:
}
-int log_loaded_block(IO_CACHE* file)
+int log_loaded_block(IO_CACHE* file,
+ const uchar *buffert __attribute__((unused)),
+ uint count __attribute__((unused)),
+ my_off_t filepos __attribute__((unused)))
{
LOAD_FILE_INFO *lf_info;
uint block_len ;
diff -Nrup a/sql/sql_repl.h b/sql/sql_repl.h
--- a/sql/sql_repl.h 2007-08-16 08:52:44 +02:00
+++ b/sql/sql_repl.h 2007-11-06 20:56:00 +01:00
@@ -66,7 +66,7 @@ typedef struct st_load_file_info
bool wrote_create_file, log_delayed;
} LOAD_FILE_INFO;
-int log_loaded_block(IO_CACHE* file);
+int log_loaded_block(IO_CACHE *, const uchar *, uint, my_off_t);
int init_replication_sys_vars();
#endif /* HAVE_REPLICATION */
diff -Nrup a/sql/sql_table.cc b/sql/sql_table.cc
--- a/sql/sql_table.cc 2007-09-26 17:49:05 +02:00
+++ b/sql/sql_table.cc 2007-11-06 20:56:00 +01:00
@@ -6984,7 +6984,10 @@ copy_data_between_tables(TABLE *from,TAB
(SQL_SELECT *) 0, HA_POS_ERROR, 1,
&examined_rows)) ==
HA_POS_ERROR)
+ {
+ to->file->ha_end_bulk_insert();
goto err;
+ }
};
/* Tell handler that we have values for all columns in the to table */
diff -Nrup a/storage/myisam/CMakeLists.txt b/storage/myisam/CMakeLists.txt
--- a/storage/myisam/CMakeLists.txt 2007-08-06 23:14:47 +02:00
+++ b/storage/myisam/CMakeLists.txt 2007-11-06 20:56:00 +01:00
@@ -32,7 +32,8 @@ SET(MYISAM_SOURCES ft_boolean_search.c
mi_rfirst.c mi_rlast.c mi_rnext.c mi_rnext_same.c mi_rprev.c mi_rrnd.c
mi_rsame.c mi_rsamepos.c mi_scan.c mi_search.c mi_static.c mi_statrec.c
mi_unique.c mi_update.c mi_write.c rt_index.c rt_key.c rt_mbr.c
- rt_split.c sort.c sp_key.c ft_eval.h myisamdef.h rt_index.h mi_rkey.c)
+ rt_split.c sort.c sp_key.c ft_eval.h myisamdef.h rt_index.h mi_rkey.c
+ mi_examine_log.c myisam_backup_driver.cc)
IF(NOT SOURCE_SUBLIBS)
diff -Nrup a/storage/myisam/Makefile.am b/storage/myisam/Makefile.am
--- a/storage/myisam/Makefile.am 2006-12-31 01:06:39 +01:00
+++ b/storage/myisam/Makefile.am 2007-11-06 20:56:00 +01:00
@@ -98,7 +98,7 @@ libmyisam_a_SOURCES = mi_open.c mi_extra
mi_keycache.c mi_preload.c \
ft_parser.c ft_stopwords.c ft_static.c \
ft_update.c ft_boolean_search.c ft_nlq_search.c sort.c \
- ha_myisam.cc \
+ mi_examine_log.c ha_myisam.cc myisam_backup_driver.cc \
rt_index.c rt_key.c rt_mbr.c rt_split.c sp_key.c
CLEANFILES = test?.MY? FT?.MY? isam.log mi_test_all rt_test.MY? sp_test.MY?
diff -Nrup a/storage/myisam/ha_myisam.cc b/storage/myisam/ha_myisam.cc
--- a/storage/myisam/ha_myisam.cc 2007-10-17 15:57:35 +02:00
+++ b/storage/myisam/ha_myisam.cc 2007-11-06 20:56:00 +01:00
@@ -881,7 +881,7 @@ int ha_myisam::repair(THD *thd, MI_CHECK
param.thd= thd;
param.tmpdir= &mysql_tmpdir_list;
param.out_flag= 0;
- strmov(fixed_name,file->filename);
+ strmov(fixed_name,file->s->unresolv_file_name);
// Don't lock tables if we have used LOCK TABLE
if (!thd->locked_tables &&
@@ -1622,11 +1622,11 @@ int ha_myisam::info(uint flag)
if table is symlinked (Ie; Real name is not same as generated name)
*/
data_file_name= index_file_name= 0;
- fn_format(name_buff, file->filename, "", MI_NAME_DEXT,
+ fn_format(name_buff, file->s->unresolv_file_name, "", MI_NAME_DEXT,
MY_APPEND_EXT | MY_UNPACK_FILENAME);
if (strcmp(name_buff, misam_info.data_file_name))
data_file_name=misam_info.data_file_name;
- fn_format(name_buff, file->filename, "", MI_NAME_IEXT,
+ fn_format(name_buff, file->s->unresolv_file_name, "", MI_NAME_IEXT,
MY_APPEND_EXT | MY_UNPACK_FILENAME);
if (strcmp(name_buff, misam_info.index_file_name))
index_file_name=misam_info.index_file_name;
@@ -1909,6 +1909,9 @@ static int myisam_init(void *p)
myisam_hton->create= myisam_create_handler;
myisam_hton->panic= myisam_panic;
myisam_hton->flags= HTON_CAN_RECREATE | HTON_SUPPORT_LOG_TABLES;
+#if !defined(EMBEDDED_LIBRARY) && defined(HAVE_MYISAM_PHYSICAL_LOGGING)
+ myisam_hton->get_backup_engine= myisam_backup_engine;
+#endif
return 0;
}
@@ -1995,8 +1998,8 @@ mysql_declare_plugin_end;
@brief Register a named table with a call back function to the query cache.
@param thd The thread handle
- @param table_key A pointer to the table name in the table cache
- @param key_length The length of the table name
+ @param table_name A pointer to the table name in the table cache
+ @param table_name_len The length of the table name
@param[out] engine_callback The pointer to the storage engine call back
function, currently 0
@param[out] engine_data Engine data will be set to 0.
diff -Nrup a/storage/myisam/ha_myisam.h b/storage/myisam/ha_myisam.h
--- a/storage/myisam/ha_myisam.h 2007-08-26 13:31:22 +02:00
+++ b/storage/myisam/ha_myisam.h 2007-11-06 20:56:00 +01:00
@@ -175,3 +175,7 @@ private:
friend my_bool index_cond_func_myisam(void *arg);
};
+#if !defined(EMBEDDED_LIBRARY) && defined(HAVE_MYISAM_PHYSICAL_LOGGING)
+// If embedded, there is no online backup
+Backup_result_t myisam_backup_engine(handlerton *self, Backup_engine* &be);
+#endif
diff -Nrup a/storage/myisam/mi_check.c b/storage/myisam/mi_check.c
--- a/storage/myisam/mi_check.c 2007-08-13 15:11:15 +02:00
+++ b/storage/myisam/mi_check.c 2007-11-06 20:56:00 +01:00
@@ -85,6 +85,7 @@ static SORT_KEY_BLOCKS *alloc_key_blocks
uint buffer_length);
static ha_checksum mi_byte_checksum(const uchar *buf, uint length);
static void set_data_file_type(SORT_INFO *sort_info, MYISAM_SHARE *share);
+static int chsize_kfile(MI_INFO *info);
void myisamchk_init(MI_CHECK *param)
{
@@ -1548,7 +1549,7 @@ int mi_repair(MI_CHECK *param, register
{
VOID(fputs(" \r",stdout)); VOID(fflush(stdout));
}
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
{
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d",
@@ -1613,6 +1614,11 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ /*
+ File change like this is not handled in physical log. filecopy() above
+ is also not handled.
+ */
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -1871,6 +1877,7 @@ int mi_sort_index(MI_CHECK *param, regis
VOID(my_close(share->kfile,MYF(MY_WME)));
share->kfile = -1;
VOID(my_close(new_file,MYF(MY_WME)));
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->index_file_name,MI_NAME_IEXT,INDEX_TMP_EXT,0,
MYF(0)) ||
mi_open_keyfile(share))
@@ -1976,6 +1983,7 @@ static int sort_one_index(MI_CHECK *para
/* Fill block with zero and write it to the new index file */
length=mi_getint(buff);
bzero((uchar*) buff+length,keyinfo->block_length-length);
+ DBUG_ASSERT(!info->s->physical_logging);
if (my_pwrite(new_file,(uchar*) buff,(uint) keyinfo->block_length,
new_page_pos,MYF(MY_NABP | MY_WAIT_IF_FULL)))
{
@@ -2038,7 +2046,7 @@ int lock_file(MI_CHECK *param, File file
} /* lock_file */
- /* Copy a block between two files */
+ /* Copy a block between two files. Not handled by physical logging. */
int filecopy(MI_CHECK *param, File to,File from,my_off_t start,
my_off_t length, const char *type)
@@ -2390,15 +2398,18 @@ int mi_repair_by_sort(MI_CHECK *param, r
skr=share->base.reloc*share->base.min_pack_length;
#endif
if (skr != sort_info.filelength && !info->s->base.raid_type)
+ {
+ DBUG_ASSERT(!share->physical_logging);
if (my_chsize(info->dfile,skr,0,MYF(0)))
mi_check_print_warning(param,
"Can't change size of datafile, error: %d",
my_errno);
+ }
}
if (param->testflag & T_CALC_CHECKSUM)
info->state->checksum=param->glob_crc;
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d",
my_errno);
@@ -2427,6 +2438,7 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -2911,15 +2923,18 @@ int mi_repair_parallel(MI_CHECK *param,
skr=share->base.reloc*share->base.min_pack_length;
#endif
if (skr != sort_info.filelength && !info->s->base.raid_type)
+ {
+ DBUG_ASSERT(!share->physical_logging);
if (my_chsize(info->dfile,skr,0,MYF(0)))
mi_check_print_warning(param,
"Can't change size of datafile, error: %d",
my_errno);
+ }
}
if (param->testflag & T_CALC_CHECKSUM)
info->state->checksum=param->glob_crc;
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d", my_errno);
@@ -2960,6 +2975,7 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -3936,9 +3952,16 @@ static int sort_insert_key(MI_SORT_PARAM
if (_mi_write_keypage(info, keyinfo, filepos, DFLT_INIT_HITS, anc_buff))
DBUG_RETURN(1);
}
- else if (my_pwrite(info->s->kfile,(uchar*) anc_buff,
- (uint) keyinfo->block_length,filepos, param->myf_rw))
- DBUG_RETURN(1);
+ else
+ {
+ if (my_pwrite(info->s->kfile, anc_buff,
+ keyinfo->block_length, filepos, param->myf_rw))
+ DBUG_RETURN(1);
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(info->s)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, info->s, anc_buff,
+ keyinfo->block_length, filepos);
+ }
DBUG_DUMP("buff",(uchar*) anc_buff,mi_getint(anc_buff));
/* Write separator-key to block in next level */
@@ -4041,9 +4064,17 @@ int flush_pending_blocks(MI_SORT_PARAM *
DFLT_INIT_HITS, key_block->buff))
DBUG_RETURN(1);
}
- else if (my_pwrite(info->s->kfile,(uchar*) key_block->buff,
- (uint) keyinfo->block_length,filepos, myf_rw))
- DBUG_RETURN(1);
+ else
+ {
+ if (my_pwrite(info->s->kfile, key_block->buff,
+ keyinfo->block_length,filepos, myf_rw))
+ DBUG_RETURN(1);
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(info->s)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, info->s,
+ key_block->buff, keyinfo->block_length,
+ filepos);
+ }
DBUG_DUMP("buff",(uchar*) key_block->buff,length);
nod_flag=1;
}
@@ -4323,7 +4354,7 @@ int update_state_info(MI_CHECK *param, M
*/
if (info->lock_type == F_WRLCK)
share->state.state= *info->state;
- if (mi_state_info_write(share->kfile,&share->state,1+2))
+ if (mi_state_info_write(share, share->kfile, &share->state, 1+2))
goto err;
share->changed=0;
}
@@ -4607,4 +4638,35 @@ set_data_file_type(SORT_INFO *sort_info,
mi_setup_functions(&tmp);
share->delete_record=tmp.delete_record;
}
+}
+
+/**
+ Changes the size of an index file, and logs the operation to the physical
+ log if needed.
+
+ The only known case when my_chsize(kfile) can happen on a table doing
+ physical logging, is when the table was empty, bulk insert on it has been
+ done, is going on, it's the end of bulk insert: we are re-enable indices
+ (mi_repair*()): thus my_chsize() is in fact a void operation (file already
+ has grown, starting from empty, info->state->key_file_length is up-to-date
+ and so file already has the requested size). We however log the operation,
+ in case there are unknown cases.
+
+ @param info table
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+static int chsize_kfile(MI_INFO *info)
+{
+ MYISAM_SHARE *share= info->s;
+ my_off_t new_length= info->state->key_file_length;
+ int ret, physical_logging= mi_get_physical_logging_state(share);
+ DBUG_ASSERT(!physical_logging ||
+ my_seek(share->kfile, 0L, MY_SEEK_END, MYF(0)) == new_length);
+ ret= my_chsize(share->kfile, new_length, 0, MYF(0));
+ if (physical_logging && mi_log_index_pages_physical)
+ myisam_log_chsize_kfile_physical(share, new_length);
+ return ret;
}
diff -Nrup a/storage/myisam/mi_close.c b/storage/myisam/mi_close.c
--- a/storage/myisam/mi_close.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_close.c 2007-11-06 20:56:00 +01:00
@@ -52,6 +52,8 @@ int mi_close(register MI_INFO *info)
}
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /* Logically there should not be a WRITE_CACHE at this stage */
+ DBUG_ASSERT(!(info->opt_flag & WRITE_CACHE_USED));
if (end_io_cache(&info->rec_cache))
error=my_errno;
info->opt_flag&= ~(READ_CACHE_USED | WRITE_CACHE_USED);
@@ -67,7 +69,11 @@ int mi_close(register MI_INFO *info)
flush_key_blocks(share->key_cache, share->kfile,
share->temporary ? FLUSH_IGNORE_CHANGED :
FLUSH_RELEASE))
+ {
error=my_errno;
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info); /* Mark that table must be checked */
+ }
if (share->kfile >= 0)
{
/*
@@ -77,7 +83,10 @@ int mi_close(register MI_INFO *info)
may be using the file at this point
*/
if (share->mode != O_RDONLY && mi_is_crashed(info))
- mi_state_info_write(share->kfile, &share->state, 1);
+ mi_state_info_write(share, share->kfile, &share->state, 1);
+ if (share->MI_LOG_OPEN_stored_in_physical_log)
+ _myisam_log_command(&myisam_physical_log, MI_LOG_CLOSE, share,
+ NULL, 0, error);
if (my_close(share->kfile,MYF(0)))
error = my_errno;
}
@@ -93,6 +102,7 @@ int mi_close(register MI_INFO *info)
#ifdef THREAD
thr_lock_delete(&share->lock);
VOID(pthread_mutex_destroy(&share->intern_lock));
+ my_atomic_rwlock_destroy(&share->physical_logging_rwlock);
{
int i,keys;
keys = share->state.header.keys;
@@ -113,7 +123,7 @@ int mi_close(register MI_INFO *info)
if (info->dfile >= 0 && my_close(info->dfile,MYF(0)))
error = my_errno;
- myisam_log_command(MI_LOG_CLOSE,info,NULL,0,error);
+ myisam_log_command_logical(MI_LOG_CLOSE, info, NULL, 0, error);
my_free((uchar*) info,MYF(0));
if (error)
diff -Nrup a/storage/myisam/mi_create.c b/storage/myisam/mi_create.c
--- a/storage/myisam/mi_create.c 2007-10-09 19:55:16 +02:00
+++ b/storage/myisam/mi_create.c 2007-11-06 20:56:00 +01:00
@@ -636,6 +636,10 @@ int mi_create(const char *name,uint keys
MYF(0), name + dirname_length(name));
goto err;
}
+ /* TRUNCATE TABLE does not work with physical logging */
+ DBUG_ASSERT((options & HA_OPTION_TMP_TABLE) || !mi_log_tables_physical ||
+ !hash_search(mi_log_tables_physical, filename,
+ strlen(filename)));
if ((file= my_create_with_symlink(linkname_ptr, filename, 0, create_mode,
MYF(MY_WME | create_flag))) < 0)
@@ -701,7 +705,7 @@ int mi_create(const char *name,uint keys
}
DBUG_PRINT("info", ("write state info and base info"));
- if (mi_state_info_write(file, &share.state, 2) ||
+ if (mi_state_info_write(&share, file, &share.state, 2) ||
mi_base_info_write(file, &share.base))
goto err;
#ifndef DBUG_OFF
diff -Nrup a/storage/myisam/mi_delete.c b/storage/myisam/mi_delete.c
--- a/storage/myisam/mi_delete.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_delete.c 2007-11-06 20:56:00 +01:00
@@ -100,13 +100,15 @@ int mi_delete(MI_INFO *info,const uchar
info->state->records--;
mi_sizestore(lastpos,info->lastpos);
- myisam_log_command(MI_LOG_DELETE,info,(uchar*) lastpos,sizeof(lastpos),0);
+ myisam_log_command_logical(MI_LOG_DELETE, info,
+ (uchar*) lastpos, sizeof(lastpos), 0);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (delete)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (delete)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
DBUG_RETURN(0);
@@ -114,7 +116,8 @@ int mi_delete(MI_INFO *info,const uchar
err:
save_errno=my_errno;
mi_sizestore(lastpos,info->lastpos);
- myisam_log_command(MI_LOG_DELETE,info,(uchar*) lastpos, sizeof(lastpos),0);
+ myisam_log_command_logical(MI_LOG_DELETE, info,
+ (uchar*) lastpos, sizeof(lastpos), 0);
if (save_errno != HA_ERR_RECORD_CHANGED)
{
mi_print_error(info->s, HA_ERR_CRASHED);
diff -Nrup a/storage/myisam/mi_delete_all.c b/storage/myisam/mi_delete_all.c
--- a/storage/myisam/mi_delete_all.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_delete_all.c 2007-11-06 20:56:01 +01:00
@@ -47,7 +47,7 @@ int mi_delete_all_rows(MI_INFO *info)
for (i=0 ; i < share->base.keys ; i++)
state->key_root[i]= HA_OFFSET_ERROR;
- myisam_log_command(MI_LOG_DELETE_ALL,info,(uchar*) 0,0,0);
+ myisam_log_command_logical(MI_LOG_DELETE_ALL, info, (uchar*) 0, 0, 0);
/*
If we are using delayed keys or if the user has done changes to the tables
since it was locked then there may be key blocks in the key cache
@@ -56,6 +56,9 @@ int mi_delete_all_rows(MI_INFO *info)
if (my_chsize(info->dfile, 0, 0, MYF(MY_WME)) ||
my_chsize(share->kfile, share->base.keystart, 0, MYF(MY_WME)) )
goto err;
+ if (unlikely(mi_get_physical_logging_state(info->s)))
+ _myisam_log_command(&myisam_physical_log, MI_LOG_DELETE_ALL, share,
+ NULL, 0, 0);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
#ifdef HAVE_MMAP
/* Resize mmaped area */
diff -Nrup a/storage/myisam/mi_dynrec.c b/storage/myisam/mi_dynrec.c
--- a/storage/myisam/mi_dynrec.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_dynrec.c 2007-11-06 20:56:01 +01:00
@@ -193,9 +193,11 @@ size_t mi_nommap_pread(MI_INFO *info, uc
size_t mi_mmap_pwrite(MI_INFO *info, const uchar *Buffer,
size_t Count, my_off_t offset, myf MyFlags)
{
+ MYISAM_SHARE *share= info->s;
+ uint ret;
DBUG_PRINT("info", ("mi_write with mmap %d\n", info->dfile));
- if (info->s->concurrent_insert)
- rw_rdlock(&info->s->mmap_lock);
+ if (share->concurrent_insert)
+ rw_rdlock(&share->mmap_lock);
/*
The following test may fail in the following cases:
@@ -204,21 +206,24 @@ size_t mi_mmap_pwrite(MI_INFO *info, con
memory mapped area.
*/
- if (info->s->mmaped_length >= offset + Count)
+ if (share->mmaped_length >= offset + Count)
{
- memcpy(info->s->file_map + offset, Buffer, Count);
- if (info->s->concurrent_insert)
- rw_unlock(&info->s->mmap_lock);
- return 0;
+ memcpy(share->file_map + offset, Buffer, Count);
+ if (share->concurrent_insert)
+ rw_unlock(&share->mmap_lock);
+ ret= 0;
}
else
{
- info->s->nonmmaped_inserts++;
- if (info->s->concurrent_insert)
- rw_unlock(&info->s->mmap_lock);
- return my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
+ share->nonmmaped_inserts++;
+ if (share->concurrent_insert)
+ rw_unlock(&share->mmap_lock);
+ ret= my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
}
-
+ if (unlikely(mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYD,
+ share, Buffer, Count, offset);
+ return ret;
}
@@ -227,7 +232,12 @@ size_t mi_mmap_pwrite(MI_INFO *info, con
size_t mi_nommap_pwrite(MI_INFO *info, const uchar *Buffer,
size_t Count, my_off_t offset, myf MyFlags)
{
- return my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
+ MYISAM_SHARE *share= info->s;
+ uint ret= my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
+ if (unlikely(mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYD,
+ share, Buffer, Count, offset);
+ return ret;
}
diff -Nrup a/storage/myisam/mi_examine_log.c b/storage/myisam/mi_examine_log.c
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/storage/myisam/mi_examine_log.c 2007-11-06 20:56:01 +01:00
@@ -0,0 +1,868 @@
+/** Copyright (C) 2000-2007 MySQL AB
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; version 2 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+
+/**
+ @file
+ Function to display and apply a MyISAM logical or physical log to tables.
+
+ Prints what is in a MyISAM (logical or physical) log, optionally applies
+ the changes to tables (all tables or only a set specified on the command
+ line). Is used both by the standalone program myisamlog and by the restore
+ code of the MyISAM online backup driver.
+*/
+
+#ifndef USE_MY_FUNC
+#define USE_MY_FUNC
+#endif
+
+#include "myisamdef.h"
+#include <my_tree.h>
+#include <stdarg.h>
+#ifdef HAVE_GETRUSAGE
+#include <sys/resource.h>
+#endif
+
+/** Human-readable names of commands storable in MyISAM logs */
+const char *mi_log_command_name[]=
+{"open","write","update","delete","close","extra","lock",
+ "delete-all", "write-bytes-to-MYD", "write-bytes-to-MYI", "chsize-MYI",
+ /*
+ This one is special: it is never in log records, it's just used by
+ mi_examine_log() to tell the user that it failed when reopening a table. It
+ has to be last before NullS.
+ */
+ "re-open", NullS};
+
+#define FILENAME(A) (A ? A->show_name : "Unknown")
+
+/** In some cases we do not want to flush the index header in mi_close() */
+static my_bool update_index_on_close= TRUE;
+
+struct file_info {
+ long process;
+ /**
+ File descriptor of the table at time of logging: in logical log,
+ it's descriptor of data file; in physical log: of index file.
+ All log records contain a corresponding descriptor value to indicate the
+ table they are about.
+ */
+ int filenr;
+ int id;
+ uint rnd;
+ char *name, *show_name;
+ uchar *record;
+ MI_INFO *isam;
+ /**
+ If 'isam' is currently closed. A not 'used' file is always 'closed' (why
+ open it?). A 'used' file may temporarily be closed because of the max
+ open file descriptors limit (but if we later meet a command which wants
+ to use this file, we will re-open it).
+ */
+ bool closed;
+ /** If this table matches the inclusion rules (or has to be ignored) */
+ bool used;
+ ulong accessed;
+};
+
+struct test_if_open_param {
+ char * name;
+ int max_id;
+};
+
+struct st_access_param
+{
+ ulong min_accessed;
+ struct file_info *found;
+};
+
+#define NO_FILEPOS HA_OFFSET_ERROR
+
+void mi_examine_log_param_init(MI_EXAMINE_LOG_PARAM *param);
+int mi_examine_log(MI_EXAMINE_LOG_PARAM *param);
+static int read_string(IO_CACHE *file,uchar* *to,uint length);
+static int file_info_compare(void *cmp_arg, void *a,void *b);
+static int test_if_open(struct file_info *key,element_count count,
+ struct test_if_open_param *param);
+static void fix_blob_pointers(MI_INFO *isam,uchar *record);
+static int test_when_accessed(struct file_info *key,element_count count,
+ struct st_access_param *access_param);
+static void file_info_free(struct file_info *info);
+static int close_some_file(TREE *tree);
+static int reopen_closed_file(TREE *tree,struct file_info *file_info);
+static int find_record_with_key(struct file_info *file_info,uchar *record);
+static int mi_close_care_state(MI_INFO *info);
+static void printf_log(uint verbose, ulong isamlog_process,
+ my_off_t isamlog_filepos, const char *format,...);
+static bool cmp_filename(struct file_info *file_info, const char *name);
+
+
+void mi_examine_log_param_init(MI_EXAMINE_LOG_PARAM *mi_exl)
+{
+ bzero(mi_exl,sizeof(*mi_exl));
+ mi_exl->number_of_commands= (ulong) ~0L;
+ mi_exl->record_pos= NO_FILEPOS;
+}
+
+
+/**
+ Displays or applies the content of a MyISAM logical or physical log
+ to tables.
+
+ Applies either to all tables referenced by the log, or only to a subset
+ specified in mi_exl->table_selection_hook.
+
+ @param mi_exl Parameters of the applying
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+int mi_examine_log(MI_EXAMINE_LOG_PARAM *mi_exl)
+{
+ ulong isamlog_process;
+ my_off_t isamlog_filepos;
+ uint command, result, files_open, big_numbers;
+ ulong access_time,length;
+ my_off_t filepos;
+ int lock_command,mi_result;
+ char isam_file_name[FN_REFLEN], llbuff[21], llbuff2[21];
+ uchar head[20], *head_ptr;
+ uchar *buff;
+ struct test_if_open_param open_param;
+ IO_CACHE cache;
+ File log_file;
+ FILE *write_file;
+ enum ha_extra_function extra_command;
+ TREE tree;
+ struct file_info file_info,*curr_file_info;
+ uint head_len[][2]=
+ { { 11, 14 }, { 15, 22 }, { 15, 22 }, { 11, 14 }, { 11, 14 }, { 11, 14 },
+ { 11, 14 }, { 11, 14 }, { 9, 16 }, { 9, 16 }, { 7, 12 } };
+ uint has_pid_and_result[]= {1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0};
+ DBUG_ENTER("mi_examine_log");
+
+ compile_time_assert((sizeof(mi_log_command_name) /
+ sizeof(mi_log_command_name[0]) ==
+ (MI_LOG_END_SENTINEL + 2)) &&
+ (sizeof(has_pid_and_result) /
+ sizeof(has_pid_and_result[0]) ==
+ MI_LOG_END_SENTINEL) &&
+ (sizeof(head_len) / sizeof(head_len[0]) ==
+ MI_LOG_END_SENTINEL) &&
+ (MI_LOG_END_SENTINEL <= MI_LOG_BIG_NUMBERS) &&
+ (sizeof(mi_exl->com_count) /
+ sizeof(mi_exl->com_count[0]) == MI_LOG_END_SENTINEL));
+ if ((log_file=my_open(mi_exl->log_filename,O_RDONLY,MYF(MY_WME))) < 0)
+ DBUG_RETURN(1);
+ write_file=0;
+ if (mi_exl->write_filename)
+ {
+ if (!(write_file=my_fopen(mi_exl->write_filename,O_WRONLY,MYF(MY_WME))))
+ {
+ my_close(log_file,MYF(0));
+ DBUG_RETURN(1);
+ }
+ }
+
+ init_io_cache(&cache,log_file,0,READ_CACHE,mi_exl->start_offset,0,MYF(0));
+ bzero(mi_exl->com_count,sizeof(mi_exl->com_count));
+ init_tree(&tree,0,0,sizeof(file_info),(qsort_cmp2) file_info_compare,1,
+ (tree_element_free) file_info_free, NULL);
+ VOID(init_key_cache(dflt_key_cache,KEY_CACHE_BLOCK_SIZE,KEY_CACHE_SIZE,
+ 0, 0));
+
+ files_open=0; access_time=0;
+ while (access_time++ != mi_exl->number_of_commands &&
+ !my_b_read(&cache, head, 1))
+ {
+ isamlog_filepos=my_b_tell(&cache)-1L;
+ head_ptr= head;
+ command=(uint) head_ptr[0];
+ command-= (big_numbers= (command & MI_LOG_BIG_NUMBERS));
+ if (my_b_read(&cache, head, head_len[command][big_numbers] - 1))
+ goto err;
+ if (big_numbers)
+ {
+ file_info.filenr= mi_uint3korr(head);
+ head_ptr+= 3;
+ }
+ else
+ {
+ file_info.filenr= mi_uint2korr(head);
+ head_ptr+= 2;
+ }
+ if (has_pid_and_result[command])
+ {
+ isamlog_process= file_info.process= (long) mi_uint4korr(head_ptr);
+ head_ptr+= 4;
+ if (!mi_exl->opt_processes)
+ file_info.process=0;
+ result= mi_uint2korr(head_ptr);
+ head_ptr+= 2;
+ }
+ else
+ isamlog_process= file_info.process= result= 0;
+ if ((curr_file_info=(struct file_info*) tree_search(&tree, &file_info,
+ tree.custom_arg)))
+ {
+ curr_file_info->accessed=access_time;
+ if (mi_exl->update && curr_file_info->used && curr_file_info->closed)
+ {
+ if (reopen_closed_file(&tree,curr_file_info))
+ {
+ command=sizeof(mi_exl->com_count)/sizeof(mi_exl->com_count[0][0])/3;
+ result=0;
+ goto com_err;
+ }
+ mi_exl->re_open_count++;
+ }
+ }
+ DBUG_PRINT("info",("command: %u curr_file_info: 0x%lx used: %u",
+ command, (ulong)curr_file_info,
+ curr_file_info ? curr_file_info->used : 0));
+ /*
+ We update our statistic (how many commands issued, per command type),
+ if this is a valid command about a file we want to include.
+ For MI_LOG_OPEN decision must be postponed, as curr_file_info is
+ meaningless for it.
+ */
+ if ((command <
+ sizeof(mi_exl->com_count)/sizeof(mi_exl->com_count[0][0])/3) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)) &&
+ (((enum myisam_log_commands) command) != MI_LOG_OPEN))
+ {
+ mi_exl->com_count[command][0]++;
+ if (result)
+ mi_exl->com_count[command][1]++;
+ }
+ switch ((enum myisam_log_commands) command) {
+ case MI_LOG_OPEN:
+ if (curr_file_info)
+ printf("\nWarning: %s is opened with same process and filenumber\n"
+ "Maybe you should use the -P option ?\n",
+ curr_file_info->show_name);
+ file_info.name=0;
+ file_info.show_name=0;
+ file_info.record=0;
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ if (read_string(&cache, (uchar **)&file_info.name, length))
+ goto err;
+ {
+ uint i;
+ char *pos,*to;
+
+ /* Fix if old DOS files to new format */
+ for (pos=file_info.name; (pos=strchr(pos,'\\')) ; pos++)
+ *pos= '/';
+
+ pos=file_info.name;
+ for (i=0 ; i < mi_exl->prefix_remove ; i++)
+ {
+ char *next;
+ if (!(next=strchr(pos,'/')))
+ break;
+ pos=next+1;
+ }
+ to=isam_file_name;
+ if (mi_exl->filepath)
+ to=convert_dirname(isam_file_name,mi_exl->filepath,NullS);
+ strmov(to,pos);
+ fn_ext(isam_file_name)[0]=0; /* Remove extension */
+ }
+ open_param.name=file_info.name;
+ open_param.max_id=0;
+ VOID(tree_walk(&tree, (tree_walk_action)test_if_open,
+ (void*)&open_param, left_root_right));
+ file_info.id=open_param.max_id+1;
+ /*
+ * In the line below +10 is added to accomodate '<' and '>' chars
+ * plus '\0' at the end, so that there is place for 7 digits.
+ * It is improbable that same table can have that many entries in
+ * the table cache.
+ * The additional space is needed for the sprintf commands two lines
+ * below.
+ */
+ file_info.show_name=my_memdup(isam_file_name,
+ (uint) strlen(isam_file_name)+10,
+ MYF(MY_WME));
+ if (file_info.id > 1)
+ sprintf(strend(file_info.show_name),"<%d>",file_info.id);
+ file_info.closed=1;
+ file_info.accessed=access_time;
+ file_info.used= !mi_exl->table_selection_hook ||
+ ((*(mi_exl->table_selection_hook))(isam_file_name));
+ if (mi_exl->update && file_info.used)
+ {
+ if (files_open >= mi_exl->max_files)
+ {
+ if (close_some_file(&tree))
+ goto com_err;
+ files_open--;
+ }
+ /*
+ index may be truncated (if physical logging excluded its pages so
+ use HA_OPEN_FOR_REPAIR).
+ */
+ if (!(file_info.isam= mi_open(isam_file_name, O_RDWR,
+ HA_OPEN_FOR_REPAIR |
+ HA_OPEN_WAIT_IF_LOCKED)))
+ goto com_err;
+ if (!(file_info.record=my_malloc(file_info.isam->s->base.reclength,
+ MYF(MY_WME))))
+ goto end;
+ files_open++;
+ file_info.closed=0;
+ }
+ VOID(tree_insert(&tree, (uchar*) &file_info, 0, tree.custom_arg));
+ if (file_info.used)
+ {
+ if (mi_exl->verbose && !mi_exl->record_pos_file)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: open -> %d",file_info.show_name, file_info.filenr);
+ mi_exl->com_count[command][0]++;
+ /* given how we log MI_LOG_OPEN, "result" is always 0 here */
+ if (result)
+ mi_exl->com_count[command][1]++;
+ }
+ break;
+ case MI_LOG_CLOSE:
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],result);
+ if (curr_file_info)
+ {
+ if (!curr_file_info->closed)
+ files_open--;
+ VOID(tree_delete(&tree, (uchar*) curr_file_info, 0, tree.custom_arg));
+ }
+ break;
+ case MI_LOG_EXTRA:
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ DBUG_ASSERT(length == 1);
+ if (my_b_read(&cache, head, length))
+ goto err;
+ extra_command=(enum ha_extra_function) head[0];
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s(%d) -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command], (int) extra_command,result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_extra(curr_file_info->isam, extra_command, 0) != (int) result)
+ {
+ fflush(stdout);
+ VOID(fprintf(stderr,
+ "Warning: error %d, expected %d on command %s at %s\n",
+ my_errno,result,mi_log_command_name[command],
+ llstr(isamlog_filepos,llbuff)));
+ fflush(stderr);
+ }
+ }
+ break;
+ case MI_LOG_DELETE:
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ DBUG_ASSERT(length == 8);
+ if (my_b_read(&cache, head, length))
+ goto err;
+ filepos=mi_sizekorr(head);
+ if (mi_exl->verbose &&
+ (!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos ||
+ mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info,mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %ld -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],(long) filepos,result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Didn't find row to delete with mi_rrnd");
+ mi_exl->com_count[command][2]++; /* Mark error */
+ }
+ mi_result=mi_delete(curr_file_info->isam,curr_file_info->record);
+ if ((mi_result == 0 && result) ||
+ (mi_result && (uint) my_errno != result))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_result)
+ mi_exl->com_count[command][2]++; /* Mark error */
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Got result %d from mi_delete instead of %d",
+ mi_result, result);
+ }
+ }
+ break;
+ case MI_LOG_WRITE:
+ case MI_LOG_UPDATE:
+ if (big_numbers)
+ {
+ filepos= mi_sizekorr(head_ptr);
+ head_ptr+= 8;
+ length= mi_uint4korr(head_ptr);
+ }
+ else
+ {
+ filepos= mi_uint4korr(head_ptr);
+ head_ptr+= 4;
+ length= mi_uint2korr(head_ptr);
+ }
+ buff=0;
+ if (read_string(&cache,&buff,length))
+ goto err;
+ if ((!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos || mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info,mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ {
+ if (write_file &&
+ (my_fwrite(write_file,buff,length,MYF(MY_WAIT_IF_FULL | MY_NABP))))
+ goto end;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %ld, length=%ld -> %d",
+ FILENAME(curr_file_info),
+ mi_log_command_name[command], filepos,length,result);
+ }
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (curr_file_info->isam->s->base.blobs)
+ fix_blob_pointers(curr_file_info->isam,buff);
+ if ((enum myisam_log_commands) command == MI_LOG_UPDATE)
+ {
+ if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
+ {
+ if (!mi_exl->recover)
+ {
+ result=0;
+ goto com_err;
+ }
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Didn't find row to update with mi_rrnd");
+ if (mi_exl->recover == 1 || result ||
+ find_record_with_key(curr_file_info,buff))
+ {
+ mi_exl->com_count[command][2]++; /* Mark error */
+ break;
+ }
+ }
+ mi_result=mi_update(curr_file_info->isam,curr_file_info->record,
+ buff);
+ if ((mi_result == 0 && result) ||
+ (mi_result && (uint) my_errno != result))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Got result %d from mi_update instead of %d",
+ mi_result, result);
+ if (mi_result)
+ mi_exl->com_count[command][2]++; /* Mark error */
+ }
+ }
+ else
+ {
+ mi_result=mi_write(curr_file_info->isam,buff);
+ if ((mi_result == 0 && result) ||
+ (mi_result && (uint) my_errno != result))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Got result %d from mi_write instead of %d",
+ mi_result, result);
+ if (mi_result)
+ mi_exl->com_count[command][2]++; /* Mark error */
+ }
+ if (!mi_exl->recover && filepos != curr_file_info->isam->lastpos)
+ {
+ printf("error: Wrote at position: %s, should have been %s",
+ llstr(curr_file_info->isam->lastpos,llbuff),
+ llstr(filepos,llbuff2));
+ goto end;
+ }
+ }
+ }
+ my_free(buff,MYF(0));
+ break;
+ case MI_LOG_WRITE_BYTES_MYI:
+ case MI_LOG_WRITE_BYTES_MYD:
+ if (big_numbers)
+ {
+ filepos= mi_sizekorr(head_ptr);
+ head_ptr+= 8;
+ length= mi_uint4korr(head_ptr);
+ }
+ else
+ {
+ filepos= mi_uint4korr(head_ptr);
+ head_ptr+= 4;
+ length= mi_uint2korr(head_ptr);
+ }
+ buff=0;
+ if (read_string(&cache, &buff, length))
+ goto err;
+ if ((!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos ||
+ mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info,mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ {
+ if (write_file &&
+ (my_fwrite(write_file, buff, length,
+ MYF(MY_WAIT_IF_FULL | MY_NABP))))
+ goto end;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %s, length=%lu -> %d",
+ FILENAME(curr_file_info),
+ mi_log_command_name[command], llstr(filepos,llbuff),
+ length, result);
+ }
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ update_index_on_close= FALSE;
+ if (my_pwrite((command == MI_LOG_WRITE_BYTES_MYI) ?
+ curr_file_info->isam->s->kfile :
+ curr_file_info->isam->dfile,
+ buff,length,filepos,MYF(MY_NABP)))
+ goto com_err;
+ }
+ my_free(buff,MYF(0));
+ break;
+ case MI_LOG_CHSIZE_MYI:
+ /* here 'filepos' means new length of file */
+ if (big_numbers)
+ filepos= mi_sizekorr(head_ptr);
+ else
+ filepos= mi_uint4korr(head_ptr);
+ if ((!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos ||
+ mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info, mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ {
+ if (write_file &&
+ (my_fwrite(write_file, buff, length,
+ MYF(MY_WAIT_IF_FULL | MY_NABP))))
+ goto end;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %s -> %d", FILENAME(curr_file_info),
+ mi_log_command_name[command], llstr(filepos,llbuff),
+ result);
+ }
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ update_index_on_close= FALSE;
+ if (my_chsize(curr_file_info->isam->s->kfile, filepos,
+ 0, MYF(MY_WME)))
+ goto com_err;
+ }
+ break;
+ case MI_LOG_LOCK:
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ DBUG_ASSERT(length == 4);
+ if (my_b_read(&cache, head, length))
+ goto err;
+ /* remove 'flag' (see mi_lock_database() */
+ lock_command= (uchar)(uint4korr(head));
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s(%d) -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],lock_command,result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_lock_database(curr_file_info->isam,lock_command) !=
+ (int) result)
+ goto com_err;
+ }
+ break;
+ case MI_LOG_DELETE_ALL:
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_delete_all_rows(curr_file_info->isam) != (int) result)
+ goto com_err;
+ }
+ break;
+ default:
+ fflush(stdout);
+ VOID(fprintf(stderr,
+ "Error: found unknown command %d in logfile, aborted\n",
+ command));
+ fflush(stderr);
+ goto end;
+ }
+ }
+ end_key_cache(dflt_key_cache,1);
+ delete_tree(&tree);
+ VOID(end_io_cache(&cache));
+ VOID(my_close(log_file,MYF(0)));
+ if (write_file && my_fclose(write_file,MYF(MY_WME)))
+ DBUG_RETURN(1);
+ DBUG_RETURN(0);
+
+ err:
+ fflush(stdout);
+ VOID(fprintf(stderr,"Got error %d when reading from logfile\n",my_errno));
+ fflush(stderr);
+ goto end;
+ com_err:
+ fflush(stdout);
+ VOID(fprintf(stderr,"Got error %d, expected %d on command %s at %s\n",
+ my_errno,result,mi_log_command_name[command],
+ llstr(isamlog_filepos,llbuff)));
+ fflush(stderr);
+ end:
+ end_key_cache(dflt_key_cache, 1);
+ delete_tree(&tree);
+ VOID(end_io_cache(&cache));
+ VOID(my_close(log_file,MYF(0)));
+ if (write_file)
+ VOID(my_fclose(write_file,MYF(MY_WME)));
+ DBUG_RETURN(1);
+}
+
+
+static int read_string(IO_CACHE *file, register uchar* *to,
+ register uint length)
+{
+ DBUG_ENTER("read_string");
+
+ if (*to)
+ my_free((uchar*) *to,MYF(0));
+ if (!(*to= (uchar*) my_malloc(length+1,MYF(MY_WME))) ||
+ my_b_read(file, *to,length))
+ {
+ if (*to)
+ my_free(*to,MYF(0));
+ *to= 0;
+ DBUG_RETURN(1);
+ }
+ *((char*) *to+length)= '\0';
+ DBUG_RETURN (0);
+} /* read_string */
+
+
+static int file_info_compare(void* cmp_arg __attribute__((unused)),
+ void *a, void *b)
+{
+ long lint;
+
+ if ((lint=((struct file_info*) a)->process -
+ ((struct file_info*) b)->process))
+ return lint < 0L ? -1 : 1;
+ return ((struct file_info*) a)->filenr - ((struct file_info*) b)->filenr;
+}
+
+ /* ARGSUSED */
+
+static int test_if_open (struct file_info *key,
+ element_count count __attribute__((unused)),
+ struct test_if_open_param *param)
+{
+ if (!strcmp(key->name,param->name) && key->id > param->max_id)
+ param->max_id=key->id;
+ return 0;
+}
+
+
+static void fix_blob_pointers(MI_INFO *info, uchar *record)
+{
+ uchar *pos;
+ MI_BLOB *blob,*end;
+
+ pos=record+info->s->base.reclength;
+ for (end=info->blobs+info->s->base.blobs, blob= info->blobs;
+ blob != end ;
+ blob++)
+ {
+ memcpy_fixed(record+blob->offset+blob->pack_length,&pos,sizeof(char*));
+ pos+=_mi_calc_blob_length(blob->pack_length,record+blob->offset);
+ }
+}
+
+ /* close the file with hasn't been accessed for the longest time */
+ /* ARGSUSED */
+
+static int test_when_accessed (struct file_info *key,
+ element_count count __attribute__((unused)),
+ struct st_access_param *access_param)
+{
+ if (key->accessed < access_param->min_accessed && ! key->closed)
+ {
+ access_param->min_accessed=key->accessed;
+ access_param->found=key;
+ }
+ return 0;
+}
+
+
+static void file_info_free(struct file_info *fileinfo)
+{
+ DBUG_ENTER("file_info_free");
+ /* The 2 conditions below can be true only if 'update' */
+ if (!fileinfo->closed)
+ VOID(mi_close_care_state(fileinfo->isam));
+ if (fileinfo->record)
+ my_free(fileinfo->record,MYF(0));
+ my_free(fileinfo->name,MYF(0));
+ my_free(fileinfo->show_name,MYF(0));
+ DBUG_VOID_RETURN;
+}
+
+
+
+static int close_some_file(TREE *tree)
+{
+ struct st_access_param access_param;
+
+ access_param.min_accessed=LONG_MAX;
+ access_param.found=0;
+
+ VOID(tree_walk(tree,(tree_walk_action) test_when_accessed,
+ (void*) &access_param,left_root_right));
+ if (!access_param.found)
+ return 1; /* No open file that is possibly to close */
+ if (mi_close_care_state(access_param.found->isam))
+ return 1;
+ access_param.found->closed=1;
+ return 0;
+}
+
+
+static int reopen_closed_file(TREE *tree, struct file_info *fileinfo)
+{
+ char name[FN_REFLEN];
+ if (close_some_file(tree))
+ return 1; /* No file to close */
+ strmov(name,fileinfo->show_name);
+ if (fileinfo->id > 1)
+ *strrchr(name,'<')='\0'; /* Remove "<id>" */
+
+ if (!(fileinfo->isam= mi_open(name, O_RDWR,
+ HA_OPEN_FOR_REPAIR | HA_OPEN_WAIT_IF_LOCKED)))
+ return 1;
+ fileinfo->closed=0;
+ return 0;
+}
+
+ /* Try to find record with uniq key */
+
+static int find_record_with_key(struct file_info *file_info, uchar *record)
+{
+ uint key;
+ MI_INFO *info=file_info->isam;
+ uchar tmp_key[MI_MAX_KEY_BUFF];
+
+ for (key=0 ; key < info->s->base.keys ; key++)
+ {
+ if (mi_is_key_active(info->s->state.key_map, key) &&
+ info->s->keyinfo[key].flag & HA_NOSAME)
+ {
+ VOID(_mi_make_key(info,key,tmp_key,record,0L));
+ return mi_rkey(info,file_info->record,(int) key,tmp_key,0,
+ HA_READ_KEY_EXACT);
+ }
+ }
+ return 1;
+}
+
+
+/**
+ In practice this is only called if verbose>=1. When mi_examine_log() is
+ used in the server it is with verbose==0 so this is not called.
+*/
+static void printf_log(uint verbose, ulong isamlog_process,
+ my_off_t isamlog_filepos, const char *format,...)
+{
+ char llbuff[21];
+ va_list args;
+ va_start(args,format);
+ DBUG_ASSERT(verbose > 0);
+ if (verbose > 2)
+ printf("%9s:",llstr(isamlog_filepos,llbuff));
+ if (verbose > 1)
+ printf("%5ld ",isamlog_process); /* Write process number */
+ (void) vprintf((char*) format,args);
+ putchar('\n');
+ va_end(args);
+}
+
+
+static bool cmp_filename(struct file_info *file_info, const char *name)
+{
+ if (!file_info)
+ return 1;
+ return strcmp(file_info->name,name) ? 1 : 0;
+}
+
+
+/**
+ Closes a table but, if physical log, does not update its state on disk.
+
+ mi_close() calls mi_state_info_write() if the table is corrupted.
+ This can happen for example is the table is from an online backup which
+ made a copy of its data file and only its index' header.
+ But in that case, if we have executed some MI_LOG_WRITE_BYTES_MYI commands,
+ the state in memory is older than the state on disk, so we do not want to
+ call mi_state_info_write(), it would cancel what we have just done.
+ The solution is to mark the table as "read only" before mi_close().We have
+ no problem with updating the MYISAM_SHARE structure as we are not
+ multi-threaded (i.e. nobody uses the share while we are changing it to
+ read-only). It is also not a problem if this "read only" influences
+ next users of this same share, as a backup log contains all index header
+ writes logged, and so all next users can skip calling
+ mi_state_info_write() too.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+static int mi_close_care_state(MI_INFO *info)
+{
+ if (!update_index_on_close)
+ info->s->mode= O_RDONLY;
+ return mi_close(info);
+}
diff -Nrup a/storage/myisam/mi_extra.c b/storage/myisam/mi_extra.c
--- a/storage/myisam/mi_extra.c 2007-08-26 13:31:22 +02:00
+++ b/storage/myisam/mi_extra.c 2007-11-06 20:56:01 +01:00
@@ -19,7 +19,9 @@
#endif
static void mi_extra_keyflag(MI_INFO *info, enum ha_extra_function function);
-
+static int log_flushed_write_cache_physical(IO_CACHE *cache_to_table,
+ const uchar *buffert,
+ uint length, my_off_t offset);
/*
Set options and buffers to optimize table handling
@@ -144,6 +146,13 @@ int mi_extra(MI_INFO *info, enum ha_extr
HA_STATE_WRITE_AT_END |
HA_STATE_EXTEND_BLOCK);
}
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+ if (!share->temporary)
+ {
+ info->rec_cache.post_write= log_flushed_write_cache_physical;
+ info->rec_cache.arg= share;
+ }
+#endif
break;
case HA_EXTRA_PREPARE_FOR_UPDATE:
if (info->s->data_file_type != DYNAMIC_RECORD)
@@ -248,7 +257,7 @@ int mi_extra(MI_INFO *info, enum ha_extr
}
}
share->state.state= *info->state;
- error=mi_state_info_write(share->kfile,&share->state,1 | 2);
+ error= mi_state_info_write(share, share->kfile, &share->state, 1 | 2);
}
break;
case HA_EXTRA_FORCE_REOPEN:
@@ -386,7 +395,7 @@ int mi_extra(MI_INFO *info, enum ha_extr
{
char tmp[1];
tmp[0]=function;
- myisam_log_command(MI_LOG_EXTRA,info,(uchar*) tmp,1,error);
+ myisam_log_command_logical(MI_LOG_EXTRA, info, (uchar*)tmp, 1, error);
}
DBUG_RETURN(error);
} /* mi_extra */
@@ -436,6 +445,17 @@ int mi_reset(MI_INFO *info)
*/
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /*
+ Logically there should not be a WRITE_CACHE at this stage. It is
+ important: mi_log_stop_physical() may be flushing the write cache at
+ this same moment, if we do it here concurrently it is not
+ safe. Basically, a client thread can flush the WRITE_CACHE if and only
+ if it is not concurrent with mi_log_stop_physical() i.e. holds a
+ write-thr_lock or intern_lock. mi_reset() for example does not satisfy
+ this, because the SQL layer unlocks tables before calling
+ ha_myisam::reset().
+ */
+ DBUG_ASSERT(!(info->opt_flag & WRITE_CACHE_USED));
info->opt_flag&= ~(READ_CACHE_USED | WRITE_CACHE_USED);
error= end_io_cache(&info->rec_cache);
}
@@ -454,4 +474,31 @@ int mi_reset(MI_INFO *info)
info->update= ((info->update & HA_STATE_CHANGED) | HA_STATE_NEXT_FOUND |
HA_STATE_PREV_FOUND);
DBUG_RETURN(error);
+}
+
+
+/**
+ Logs when the WRITE_CACHE is flushed to the data file, to the physical
+ log.
+
+ @param cache_for_table pointer to the table's WRITE_CACHE IO_CACHE
+ @param buffert argument to the pwrite
+ @param length length of buffer
+ @param filepos offset in file where buffer was written
+
+ @return Operation status, always 0
+ @retval 0 ok. Yes, even if log write fails we return ok, don't want
+ to make the table writer believe its table is now
+ corrupted.
+*/
+static int log_flushed_write_cache_physical(IO_CACHE *cache_for_table,
+ const uchar *buffert,
+ uint length, my_off_t filepos)
+{
+ MYISAM_SHARE *share= (MYISAM_SHARE *)(cache_for_table->arg);
+ DBUG_ENTER("log_flushed_write_cache_physical");
+ if (unlikely(mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYD, share, buffert,
+ length, filepos);
+ DBUG_RETURN(0);
}
diff -Nrup a/storage/myisam/mi_locking.c b/storage/myisam/mi_locking.c
--- a/storage/myisam/mi_locking.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_locking.c 2007-11-06 20:56:01 +01:00
@@ -13,11 +13,13 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- locking of isam-tables.
- reads info from a isam-table. Must be first request before doing any furter
- calls to any isamfunktion. Is used to allow many process use the same
- isamdatabase.
+/**
+ @file
+ Locking of MyISAM tables.
+
+ Reads info from a isam table. Must be first request before doing any
+ further calls to any isam function. Is used to allow many processes to use
+ the same isam database.
*/
#include "ftdefs.h"
@@ -70,6 +72,8 @@ int mi_lock_database(MI_INFO *info, int
}
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /* Logically there should not be a WRITE_CACHE at this stage */
+ DBUG_ASSERT(!(info->opt_flag & WRITE_CACHE_USED));
if (end_io_cache(&info->rec_cache))
{
error=my_errno;
@@ -79,43 +83,16 @@ int mi_lock_database(MI_INFO *info, int
}
if (!count)
{
+ int local_error;
DBUG_PRINT("info",("changed: %u w_locks: %u",
(uint) share->changed, share->w_locks));
- if (share->changed && !share->w_locks)
- {
-#ifdef HAVE_MMAP
- if ((info->s->mmaped_length != info->s->state.state.data_file_length) &&
- (info->s->nonmmaped_inserts > MAX_NONMAPPED_INSERTS))
- {
- if (info->s->concurrent_insert)
- rw_wrlock(&info->s->mmap_lock);
- mi_remap_file(info, info->s->state.state.data_file_length);
- info->s->nonmmaped_inserts= 0;
- if (info->s->concurrent_insert)
- rw_unlock(&info->s->mmap_lock);
- }
-#endif
- share->state.process= share->last_process=share->this_process;
- share->state.unique= info->last_unique= info->this_unique;
- share->state.update_count= info->last_loop= ++info->this_loop;
- if (mi_state_info_write(share->kfile, &share->state, 1))
- error=my_errno;
- share->changed=0;
- if (myisam_flush)
- {
- if (my_sync(share->kfile, MYF(0)))
- error= my_errno;
- if (my_sync(info->dfile, MYF(0)))
- error= my_errno;
- }
- else
- share->not_flushed=1;
- if (error)
- {
- mi_print_error(info->s, HA_ERR_CRASHED);
- mi_mark_crashed(info);
- }
- }
+ if (share->changed && !share->w_locks &&
+ (local_error= mi_remap_file_and_write_state_for_unlock(info)))
+ {
+ error= local_error;
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info);
+ }
if (info->lock_type != F_EXTRA_LCK)
{
if (share->r_locks)
@@ -252,11 +229,14 @@ int mi_lock_database(MI_INFO *info, int
}
#endif
pthread_mutex_unlock(&share->intern_lock);
-#if defined(FULL_LOG) || defined(_lint)
- lock_type|=(int) (flag << 8); /* Set bit to set if real lock */
- myisam_log_command(MI_LOG_LOCK,info,(uchar*) &lock_type,sizeof(lock_type),
- error);
-#endif
+ if (my_b_inited(&myisam_logical_log))
+ {
+ uchar lock_type_buff[4];
+ lock_type|=(int) (flag << 8); /* Set bit to set if real lock */
+ int4store(lock_type_buff, lock_type);
+ myisam_log_command_logical(MI_LOG_LOCK, info,
+ lock_type_buff, sizeof(lock_type_buff), error);
+ }
DBUG_RETURN(error);
} /* mi_lock_database */
@@ -451,7 +431,7 @@ int _mi_writeinfo(register MI_INFO *info
share->state.process= share->last_process= share->this_process;
share->state.unique= info->last_unique= info->this_unique;
share->state.update_count= info->last_loop= ++info->this_loop;
- if ((error=mi_state_info_write(share->kfile, &share->state, 1)))
+ if ((error= mi_state_info_write(share, share->kfile, &share->state, 1)))
olderror=my_errno;
#ifdef __WIN__
if (myisam_flush)
@@ -537,6 +517,10 @@ int _mi_mark_file_changed(MI_INFO *info)
{
mi_int2store(buff,share->state.open_count);
buff[2]=1; /* Mark that it's changed */
+ /*
+ Don't need to log it to physical log, as online backup does dirty
+ copies anyway.
+ */
DBUG_RETURN(my_pwrite(share->kfile,buff,sizeof(buff),
sizeof(share->state.header),
MYF(MY_NABP)));
@@ -566,6 +550,10 @@ int _mi_decrement_open_count(MI_INFO *in
{
share->state.open_count--;
mi_int2store(buff,share->state.open_count);
+ /*
+ Don't need to log it to physical log, as online backup does dirty
+ copies anyway.
+ */
write_error=my_pwrite(share->kfile,buff,sizeof(buff),
sizeof(share->state.header),
MYF(MY_NABP));
@@ -574,4 +562,54 @@ int _mi_decrement_open_count(MI_INFO *in
lock_error=mi_lock_database(info,old_lock);
}
return test(lock_error || write_error);
+}
+
+
+/**
+ Remaps the data file, and writes state to index file.
+
+ When we unlock a table and no other thread has announced it is going to
+ write to it (w_locks==0), we want to flush some information to disk, so
+ that in case of crash the table is not too much corrupted. Physical
+ logging, when it is turning logging of for a table, needs to do this too,
+ so that this information reaches the log.
+
+ @param info table
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+int mi_remap_file_and_write_state_for_unlock(MI_INFO *info)
+{
+ MYISAM_SHARE *share= info->s;
+ int error= 0;
+#ifdef HAVE_MMAP
+ if ((info->s->mmaped_length != info->s->state.state.data_file_length) &&
+ (info->s->nonmmaped_inserts > MAX_NONMAPPED_INSERTS))
+ {
+ if (info->s->concurrent_insert)
+ rw_wrlock(&info->s->mmap_lock);
+ mi_remap_file(info, info->s->state.state.data_file_length);
+ info->s->nonmmaped_inserts= 0;
+ if (info->s->concurrent_insert)
+ rw_unlock(&info->s->mmap_lock);
+ }
+#endif
+ share->state.process= share->last_process=share->this_process;
+ share->state.unique= info->last_unique= info->this_unique;
+ share->state.update_count= info->last_loop= ++info->this_loop;
+ if (mi_state_info_write(share, share->kfile, &share->state, 1))
+ error=my_errno;
+ share->changed=0;
+ if (myisam_flush)
+ {
+ if (my_sync(share->kfile, MYF(0)))
+ error= my_errno;
+ if (my_sync(info->dfile, MYF(0)))
+ error= my_errno;
+ }
+ else
+ share->not_flushed=1;
+ return error;
}
diff -Nrup a/storage/myisam/mi_log.c b/storage/myisam/mi_log.c
--- a/storage/myisam/mi_log.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_log.c 2007-11-06 20:56:01 +01:00
@@ -13,9 +13,57 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Logging of MyISAM commands and records on logfile for debugging
- The log can be examined with help of the myisamlog command.
+/**
+ @file
+ Logging of MyISAM commands and records.
+
+ The physical log contains each call to OS write functions on the MyISAM
+ files. Most of its entries are physical for example "write these bytes at
+ this offset". For example, a mi_write() with lots of BLOBs in many places
+ will cause lots of entries in this log. It also contains some logical ones
+ like MI_LOG_DELETE_ALL (we wouldn't want to log the deletion of all rows
+ one by one).
+
+ The logical log contains each call to higher-level operations like
+ mi_write()/mi_update(). A sample entry is "write this record". For example,
+ a mi_write() with lots of BLOBs will cause one single entry in this log.
+
+ Writes to the logical log happen when the logical operation
+ happens. Entries in it refer to a table by file descriptor of its data
+ file.
+
+ Writes to the physical log happen when the physical operation happens,
+ i.e. when the file is written, which can be at three moments:
+ -# when the row write directly writes to the file (mi_[no]mmap_pwrite())
+ -# if the row write went to a WRITE_CACHE, when this cache gets written to
+ the file (post_write callback in that cache)
+ -# if the row write went to the key cache, when this key cache block gets
+ written ("flushed") to the file (post_write callback in that block)
+ Additionally, an entry for opening and an entry for closing the table, are
+ written to the physical log: the first "direct row write" or "WRITE_CACHE"
+ or "key cache block flush" log write for a certain MYISAM_SHARE, an entry
+ for opening (MI_LOG_OPEN) is written. All entries refer to the table by the
+ file descriptor of the index file; the MI_LOG_OPEN entry links this number
+ to a table name. The entry for closing is written by mi_close() if an entry
+ for opening had been written before and if the index file is being closed.
+
+ Physical log is used for online backup, because if applied to a dirtily
+ copied table it can make this table consistent. A dirty table
+ contains internally inconsistent records, so applying a logical log to it
+ (like mi_update()) is impossible).
+
+ Both logs:
+ - are idempotent
+ - can be used to debug MyISAM
+ - can be examined and applied to tables with the myisamlog utility.
+
+ Logical log is about all tables, is turned on before opening tables (for
+ example turned on at program's start and off at program's end).
+
+ Physical log is about to a set of tables, can be turned on and off at any
+ time.
+
+ mi_log() is the entry point.
*/
#include "myisamdef.h"
@@ -36,114 +84,381 @@
#define GETPID() myisam_pid
#endif
- /* Activate logging if flag is 1 and reset logging if flag is 0 */
-
+/** the log_type global variable is probably legacy, it's always 0 now */
static int log_type=0;
ulong myisam_pid=0;
-
-int mi_log(int activate_log)
+static int mi_log_open_cache(enum enum_mi_log_type type,
+ const char *log_filename);
+static int mi_log_close_cache(enum enum_mi_log_type type);
+static int mi_log_start_physical(const char *log_filename,
+ const HASH *tables);
+static int mi_log_stop_physical();
+
+
+/**
+ Starts MyISAM logical logging for all tables or physical logging for
+ a set of tables, or stops it.
+
+ @param action what to do (start, stop (in)consistently)
+ @param type physical or logical
+ @param log_filename name of the log file to create (only for physical
+ log, logical log has a static name)
+ @param tables hash of names of tables for which we want logging
+ (only for physical log)
+
+ @note For the logical log, MI_LOG_CLOSE_INCONSISTENT and
+ MI_LOG_CLOSE_CONSISTENT are identical: log will be consistent only if
+ tables not being written now.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error; then caller should call mi_log_stop_physical(TRUE)
+*/
+int mi_log(enum enum_mi_log_action action, enum enum_mi_log_type type,
+ const char *log_filename, const HASH *tables)
{
- int error=0;
- char buff[FN_REFLEN];
+ int error;
DBUG_ENTER("mi_log");
- log_type=activate_log;
- if (activate_log)
+ if (type == MI_LOG_LOGICAL)
{
- if (!myisam_pid)
- myisam_pid=(ulong) getpid();
- if (myisam_log_file < 0)
- {
- if ((myisam_log_file = my_create(fn_format(buff,myisam_log_filename,
- "",".log",4),
- 0,(O_RDWR | O_BINARY | O_APPEND),MYF(0)))
- < 0)
- DBUG_RETURN(my_errno);
- }
+ DBUG_ASSERT(log_filename == NULL);
+ DBUG_ASSERT(tables == NULL);
+ if (action == MI_LOG_ACTION_OPEN)
+ error= mi_log_open_cache(type, myisam_logical_log_filename);
+ else
+ error= mi_log_close_cache(type);
+ DBUG_RETURN(error);
}
- else if (myisam_log_file >= 0)
+
+#ifndef HAVE_MYISAM_PHYSICAL_LOGGING
+ DBUG_ASSERT(0);
+ DBUG_RETURN(1);
+#endif
+
+ /* starting/stopping are complex operations so split in functions */
+ switch (action)
{
- error=my_close(myisam_log_file,MYF(0)) ? my_errno : 0 ;
- myisam_log_file= -1;
+ case MI_LOG_ACTION_OPEN:
+ error= mi_log_start_physical(log_filename, tables);
+ break;
+ case MI_LOG_ACTION_CLOSE_CONSISTENT:
+ case MI_LOG_ACTION_CLOSE_INCONSISTENT:
+ error= mi_log_stop_physical(action);
+ break;
+ default:
+ DBUG_ASSERT(0);
}
DBUG_RETURN(error);
}
- /* Logging of records and commands on logfile */
- /* All logs starts with command(1) dfile(2) process(4) result(2) */
+/**
+ Sets up a log's IO_CACHE (for logical or physical log).
-void _myisam_log(enum myisam_log_commands command, MI_INFO *info,
- const uchar *buffert, uint length)
+ Both logs are IO_CACHE to be fast. But the logical log flushes it after
+ every operation, as it is share-able between different processes and so
+ another process may want to write to this log as soon as we unlock its
+ file. Also, as this log is used for debugging it must contain as much
+ information as possible in case of crash. For this log, using IO_CACHE
+ still makes sense as it decreases the number of my_write() calls.
+
+ @param type physical or logical
+ @param log_filename only for physical log (logical log has a static
+ name)
+
+ @note logs are not created with MY_WAIT_IF_FULL: a log can itself be the
+ cause of filling the disk, so better corrupt it (and make a backup
+ fail for example) than prevent other normal operations.
+
+ @todo A realistic benchmark to see if the size of the IO_CACHE makes any
+ speed difference.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+static int mi_log_open_cache(enum enum_mi_log_type type,
+ const char *log_filename)
{
- uchar buff[11];
- int error,old_errno;
- ulong pid=(ulong) GETPID();
- old_errno=my_errno;
- bzero(buff,sizeof(buff));
- buff[0]=(char) command;
- mi_int2store(buff+1,info->dfile);
- mi_int4store(buff+3,pid);
- mi_int2store(buff+9,length);
+ int error=0;
+ char buff[FN_REFLEN];
+ int access_flags;
+ File file;
+ IO_CACHE *log;
+ uint cache_size;
+ DBUG_ENTER("mi_log_open_cache");
+
+ DBUG_ASSERT(log_filename != NULL);
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (type == MI_LOG_LOGICAL)
+ {
+ log = &myisam_logical_log;
+ /* O_APPEND as file may exist and we want to keep it */
+ access_flags= O_WRONLY | O_BINARY | O_APPEND;
+ /* small cache size as frequent flushes */
+ cache_size= IO_SIZE;
+ }
+ else
+ {
+ DBUG_ASSERT(type == MI_LOG_PHYSICAL);
+ log = &myisam_physical_log;
+ /* We want to fail if file exists */
+ access_flags= O_WRONLY | O_BINARY | O_TRUNC | O_EXCL;
+ /*
+ We want a large IO_CACHE to have large contiguous disk writes.
+ In many systems this size is affordable. In small embedded ones it is
+ not, but would they use this log?
+ */
+ cache_size= IO_SIZE*256;
+ }
+ if (!myisam_pid)
+ myisam_pid=(ulong) getpid();
+ if (!my_b_inited(log))
+ {
+ DBUG_ASSERT(log_filename);
+ fn_format(buff, log_filename, "", "", MY_UNPACK_FILENAME);
+ if ((file= my_create(buff,
+ 0, access_flags,
+ MYF(MY_WME | ME_WAITTANG))) < 0)
+ error= my_errno;
+ else if (init_io_cache(log, file,
+ cache_size, WRITE_CACHE,
+ my_tell(file,MYF(MY_WME)), 0,
+ MYF(MY_WME | MY_NABP)))
+ {
+ error= my_errno;
+ my_close(file, MYF(MY_WME));
+ }
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ DBUG_RETURN(error);
+}
- pthread_mutex_lock(&THR_LOCK_myisam);
- error=my_lock(myisam_log_file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- VOID(my_write(myisam_log_file,buff,sizeof(buff),MYF(0)));
- VOID(my_write(myisam_log_file,buffert,length,MYF(0)));
- if (!error)
- error=my_lock(myisam_log_file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- pthread_mutex_unlock(&THR_LOCK_myisam);
- my_errno=old_errno;
+
+/**
+ Destroy's a log's IO_CACHE (for logical or physical log).
+
+ @param type physical or logical
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+static int mi_log_close_cache(enum enum_mi_log_type type)
+{
+ int error= 0;
+ IO_CACHE *log;
+ DBUG_ENTER("mi_log_close_cache");
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (type == MI_LOG_LOGICAL)
+ log = &myisam_logical_log;
+ else
+ {
+ DBUG_ASSERT(type == MI_LOG_PHYSICAL);
+ log = &myisam_physical_log;
+ }
+ if (my_b_inited(log))
+ {
+ if (end_io_cache(log) ||
+ my_close(log->file,MYF(MY_WME)))
+ error= my_errno;
+ log->file= -1;
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ DBUG_RETURN(error);
}
-void _myisam_log_command(enum myisam_log_commands command, MI_INFO *info,
- const uchar *buffert, uint length, int result)
+/**
+ Logs a MyISAM command and its return code to log.
+
+ If this is a physical log and MI_LOG_OPEN has not already been stored for
+ this MYISAM_SHARE in this log, also writes a MI_LOG_OPEN.
+
+ @param log pointer to the log's IO_CACHE
+ @param command MyISAM command (see code for allowed commands)
+ @param info_or_share MI_INFO if logical log, MYISAM_SHARE if physical
+ @param buffert usually argument to the command (e.g. name of file
+ to open for MI_LOG_OPEN), may be NULL
+ @param length length of buffert (0 if NULL)
+ @param result return code of the command
+*/
+void _myisam_log_command(IO_CACHE *log, enum myisam_log_commands command,
+ void *info_or_share,
+ const uchar *buffert, uint length, int result)
{
- uchar buff[9];
- int error,old_errno;
+ uchar buff[14];
+ int error, old_errno, bufflen;
ulong pid=(ulong) GETPID();
-
+ my_bool logical= (log == &myisam_logical_log);
+ MYISAM_SHARE *share;
+ File file= logical ? ((MI_INFO *)info_or_share)->dfile :
+ (share= (MYISAM_SHARE *)info_or_share)->kfile;
+ LINT_INIT(share);
+
+ DBUG_ASSERT(command == MI_LOG_OPEN || command == MI_LOG_DELETE ||
+ command == MI_LOG_CLOSE || command == MI_LOG_EXTRA ||
+ command == MI_LOG_LOCK || command == MI_LOG_DELETE_ALL);
old_errno=my_errno;
- buff[0]=(char) command;
- mi_int2store(buff+1,info->dfile);
- mi_int4store(buff+3,pid);
- mi_int2store(buff+7,result);
- pthread_mutex_lock(&THR_LOCK_myisam);
- error=my_lock(myisam_log_file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- VOID(my_write(myisam_log_file,buff,sizeof(buff),MYF(0)));
- if (buffert)
- VOID(my_write(myisam_log_file,buffert,length,MYF(0)));
- if (!error)
- error=my_lock(myisam_log_file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_ASSERT(((uint)result) <= UINT_MAX16);
+ if (file >= UINT_MAX16 || length >= UINT_MAX16)
+ {
+ buff[0]= ((uchar) command) | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(file < (2<<24));
+ mi_int3store(buff + 1, file);
+ mi_int4store(buff + 4, pid);
+ mi_int2store(buff + 8, result);
+ mi_int4store(buff + 10, length);
+ bufflen= 14;
+ }
+ else
+ {
+ buff[0]= (uchar) command;
+ mi_int2store(buff + 1, file);
+ mi_int4store(buff + 3, pid);
+ mi_int2store(buff + 7, result);
+ mi_int2store(buff + 9, length);
+ bufflen= 11;
+ }
+retry:
+ /*
+ Reasons to not use THR_LOCK_myisam to serialize log writes:
+ - better concurrency (not stealing THR_LOCK_myisam which is used for opens
+ and closes including long table flushes)
+ - mi_close() flushes indexes while holding THR_LOCK_myisam, and that flush
+ can cause log writes, so we would lock the mutex twice.
+ */
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ /*
+ We need to check that 'log' is not closed, this can happen for a physical
+ log. Indeed we do not have full control on the table from the thread doing
+ mi_log_stop_physical(); it could be an inconsistent logging stop (in
+ the middle of writes) or even a consistent one (table can be in
+ mi_lock_database(F_UNLCK) and thus want to flush its header)). Log might
+ just have been closed while the table still has physical_logging true.
+ */
+ if (likely(my_b_inited(log) != NULL))
+ {
+ if (!logical)
+ {
+ if (command == MI_LOG_OPEN)
+ {
+ /*
+ If there could be two concurrent writers on a MyISAM table, it could
+ be that they both do a myisam_log_command(c) where c!=MI_LOG_OPEN,
+ which both see MI_LOG_OPEN_stored_in_physical_log false, and both
+ call myisam_log_command(MI_LOG_OPEN); we would then have to make one
+ single winner: one will run before the other, the other should
+ notice MI_LOG_OPEN_stored_in_physical_log became true and back off.
+ But there is always at most one writer to a MyISAM table, so the
+ assertion below should always be ok
+ */
+ DBUG_ASSERT(!share->MI_LOG_OPEN_stored_in_physical_log);
+ share->MI_LOG_OPEN_stored_in_physical_log= TRUE;
+ /*
+ We must keep the mutex between setting the boolean above and writing
+ to the log ; one instant after unlocking the mutex, the log may be
+ closed and so it would be wrong to say that the MI_LOG_OPEN is in
+ the log (it would possibly influence a next physical log).
+ */
+ }
+ else if (unlikely(!share->MI_LOG_OPEN_stored_in_physical_log))
+ {
+ DBUG_ASSERT(command != MI_LOG_CLOSE);
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ _myisam_log_command(&myisam_physical_log, MI_LOG_OPEN, share,
+ share->unresolv_file_name,
+ strlen(share->unresolv_file_name), 0);
+ goto retry;
+ }
+ }
+ /*
+ Don't be fooled: physical logging does not work with --external-locking
+ (the physical log is private to the one mysqld). The my_lock() is
+ relevant only for the logical log.
+ */
+ error=my_lock(log->file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
+ /*
+ Any failure to write the log does not prevent the table write (table
+ should still be usable even though log breaks).
+ but sets up log->hard_write_error_in_the_past, which can be tested by
+ those who want to use this log.
+ */
+ VOID(my_b_write(log, buff, bufflen));
+ if (buffert)
+ VOID(my_b_write(log, buffert, length));
+ else
+ DBUG_ASSERT(length == 0);
+ /*
+ Another process (if external locking) may want to append to the logical
+ log as soon as we unlock its file, we must flush it. Physical log is not
+ share-able, does not need to flush.
+ */
+ if (logical)
+ flush_io_cache(log);
+ if (!error)
+ error=my_lock(log->file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
my_errno=old_errno;
}
-void _myisam_log_record(enum myisam_log_commands command, MI_INFO *info,
- const uchar *record, my_off_t filepos, int result)
+/**
+ Logs a MyISAM command (involving a record: MI_LOG_WRITE etc) and its
+ return code to the logical log.
+
+ @param command MyISAM command (MI_LOG_UPDATE|WRITE)
+ @param info MI_INFO
+ @param record record to write/update/etc, not NULL
+ @param filepos offset in data file where record starts
+ @param result return code of the command
+*/
+void _myisam_log_record_logical(enum myisam_log_commands command,
+ MI_INFO *info, const uchar *record,
+ my_off_t filepos, int result)
{
- uchar buff[21],*pos;
+ uchar buff[22],*pos;
int error,old_errno;
- uint length;
+ uint length, bufflen;
ulong pid=(ulong) GETPID();
+ DBUG_ASSERT(command == MI_LOG_UPDATE || command == MI_LOG_WRITE);
old_errno=my_errno;
if (!info->s->base.blobs)
length=info->s->base.reclength;
else
length=info->s->base.reclength+ _my_calc_total_blob_length(info,record);
- buff[0]=(uchar) command;
- mi_int2store(buff+1,info->dfile);
- mi_int4store(buff+3,pid);
- mi_int2store(buff+7,result);
- mi_sizestore(buff+9,filepos);
- mi_int4store(buff+17,length);
- pthread_mutex_lock(&THR_LOCK_myisam);
- error=my_lock(myisam_log_file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- VOID(my_write(myisam_log_file, buff,sizeof(buff),MYF(0)));
- VOID(my_write(myisam_log_file, record,info->s->base.reclength,MYF(0)));
+ DBUG_ASSERT(((uint)result) <= UINT_MAX16);
+ if (info->dfile >= UINT_MAX16 || filepos >= UINT_MAX32 ||
+ length >= UINT_MAX16)
+ {
+ buff[0]= ((uchar) command) | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(info->dfile < (2<<24));
+ mi_int3store(buff + 1, info->dfile);
+ mi_int4store(buff + 4, pid);
+ mi_int2store(buff + 8, result);
+ mi_sizestore(buff + 10, filepos);
+ mi_int4store(buff + 18, length);
+ bufflen= 22;
+ }
+ else
+ {
+ buff[0]= (uchar) command;
+ mi_int2store(buff + 1, info->dfile);
+ mi_int4store(buff + 3, pid);
+ mi_int2store(buff + 7, result);
+ mi_int4store(buff + 9, filepos);
+ mi_int2store(buff + 13, length);
+ bufflen= 15;
+ }
+
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ error= my_lock(myisam_logical_log.file, F_WRLCK, 0L, F_TO_EOF,
+ MYF(MY_SEEK_NOT_DONE));
+ VOID(my_b_write(&myisam_logical_log, buff, bufflen));
+ VOID(my_b_write(&myisam_logical_log, record, info->s->base.reclength));
if (info->s->base.blobs)
{
MI_BLOB *blob,*end;
@@ -152,13 +467,386 @@ void _myisam_log_record(enum myisam_log_
blob != end ;
blob++)
{
- memcpy_fixed((uchar*) &pos, record+blob->offset+blob->pack_length,
+ memcpy_fixed(&pos, record+blob->offset+blob->pack_length,
sizeof(char*));
- VOID(my_write(myisam_log_file,pos,blob->length,MYF(0)));
+ VOID(my_b_write(&myisam_logical_log, pos, blob->length));
}
}
+ flush_io_cache(&myisam_logical_log);
if (!error)
- error=my_lock(myisam_log_file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- pthread_mutex_unlock(&THR_LOCK_myisam);
+ error= my_lock(myisam_logical_log.file, F_UNLCK, 0L, F_TO_EOF,
+ MYF(MY_SEEK_NOT_DONE));
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
my_errno=old_errno;
+}
+
+
+/* THE FOLLOWING FUNCTIONS SERVE ONLY FOR PHYSICAL LOGGING */
+
+/**
+ Logs a my_pwrite() (done to data or index file) to the physical log.
+
+ Also logs MI_LOG_OPEN if first time. Thus, a MI_INFO will write MI_LOG_OPEN
+ to the log only if it is doing a write to the table: a table which does
+ only reads logs nothing to the physical log.
+
+ @param command MyISAM command (MI_LOG_WRITE_BYTES_TO_MYD|MYI)
+ @param share table's share
+ @param buffert argument to the pwrite
+ @param length length of buffer
+ @param filepos offset in file where buffer was written
+
+ @note length may be small (for example, if updating only a numeric field of
+ a record, it could be only a few bytes), so we try to minimize the header's
+ size of the log entry.
+*/
+void myisam_log_pwrite_physical(enum myisam_log_commands command,
+ MYISAM_SHARE *share, const uchar *buffert,
+ uint length, my_off_t filepos)
+{
+ uchar buff[21];
+ int old_errno, bufflen;
+ DBUG_ENTER("myisam_log_pwrite_physical");
+ DBUG_ASSERT(command == MI_LOG_WRITE_BYTES_MYD ||
+ command == MI_LOG_WRITE_BYTES_MYI);
+ DBUG_ASSERT(buffert != NULL && length > 0);
+ old_errno= my_errno;
+ if (share->kfile >= UINT_MAX16 || filepos >= UINT_MAX32 ||
+ length >= UINT_MAX16)
+ {
+ buff[0]= ((uchar) command) | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(share->kfile < (2<<24));
+ mi_int3store(buff + 1, share->kfile);
+ mi_sizestore(buff + 4, filepos);
+ mi_int4store(buff + 12, length);
+ bufflen= 16;
+ }
+ else
+ {
+ buff[0]= (uchar) command;
+ mi_int2store(buff + 1, share->kfile);
+ mi_int4store(buff + 3, filepos);
+ mi_int2store(buff + 7, length);
+ bufflen= 9;
+ }
+ /* pid and result are not needed */
+retry:
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (likely(my_b_inited(&myisam_physical_log) != NULL))
+ {
+ if (unlikely(!share->MI_LOG_OPEN_stored_in_physical_log))
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ _myisam_log_command(&myisam_physical_log, MI_LOG_OPEN, share,
+ share->unresolv_file_name,
+ strlen(share->unresolv_file_name), 0);
+ goto retry;
+ }
+ VOID(my_b_write(&myisam_physical_log, buff, bufflen));
+ VOID(my_b_write(&myisam_physical_log, buffert, length));
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ my_errno= old_errno;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Logs a my_chsize() done to the index file to the physical log.
+
+ Also logs MI_LOG_OPEN if first time.
+
+ @param share table's share
+ @param new_length new length of the table's index file
+*/
+void myisam_log_chsize_kfile_physical(MYISAM_SHARE *share,
+ my_off_t new_length)
+{
+ uchar buff[12];
+ int old_errno, bufflen;
+ DBUG_ENTER("myisam_log_chsize_kfile_physical");
+ old_errno= my_errno;
+ if (share->kfile >= UINT_MAX16 || new_length >= UINT_MAX32)
+ {
+ buff[0]= MI_LOG_CHSIZE_MYI | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(share->kfile < (2<<24));
+ mi_int3store(buff + 1, share->kfile);
+ mi_sizestore(buff + 4, new_length);
+ bufflen= 12;
+ }
+ else
+ {
+ buff[0]= MI_LOG_CHSIZE_MYI;
+ mi_int2store(buff + 1, share->kfile);
+ mi_int4store(buff + 3, new_length);
+ bufflen= 7;
+ }
+ /* pid and result are not needed */
+retry:
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (likely(my_b_inited(&myisam_physical_log) != NULL))
+ {
+ if (unlikely(!share->MI_LOG_OPEN_stored_in_physical_log))
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ _myisam_log_command(&myisam_physical_log, MI_LOG_OPEN, share,
+ share->unresolv_file_name,
+ strlen(share->unresolv_file_name), 0);
+ goto retry;
+ }
+ VOID(my_b_write(&myisam_physical_log, buff, bufflen));
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ my_errno= old_errno;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Starts MyISAM physical logging for a set of tables.
+
+ Physical logging is used for online backup.
+ A condition of correctness of online backup is that:
+ after the copy process has started (i.e. after the function below has
+ terminated), any update done to a table-to-back-up must be present in the
+ log. This guides the algorithm below.
+
+ All writes (my_write, my_pwrite, memcpy to mmap'ed area, my_chsize) to the
+ data or index file are done this way:
+ @code
+ {
+ write_to_data_or_index_file;
+ if ((atomic read of MYISAM_SHARE::physical_logging)==1)
+ write log record to physical log;
+ }
+ @endcode
+
+ The present function sets MYISAM_SHARE::physical_logging to 1 using an
+ atomic write. Atomic write happens before or after atomic read above, and
+ atomic read sees the latest value. If before, change will be in the log. If
+ after, it is also after the write_to_data_or_index_file and thus change
+ will be in the copy. So correctness is always guaranteed. Note the
+ importance of checking MYISAM_SHARE::logging always _after_
+ write_to_data_or_index_file for the reasoning to hold.
+
+ @param log_filename Name of the physical log file to create
+ @param tables Hash of names of tables for which we want logging
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+static int mi_log_start_physical(const char *log_filename, const HASH *tables)
+{
+ LIST *list_item;
+ int error;
+ DBUG_ENTER("mi_log_start_physical");
+ DBUG_ASSERT(log_filename != NULL);
+ DBUG_ASSERT(hash_inited(tables));
+
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (mi_log_tables_physical) /* physical logging already running */
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_ASSERT(0); /* because it should not happen */
+ DBUG_RETURN(1);
+ }
+ mi_log_tables_physical= tables;
+
+ if (unlikely(mi_log_open_cache(MI_LOG_PHYSICAL, log_filename)))
+ {
+ error= 1;
+ goto end;
+ }
+ /* Go through all open MyISAM tables (MI_INFOs) */
+ for (list_item= myisam_open_list; list_item; list_item= list_item->next)
+ {
+ MI_INFO *info= (MI_INFO*)list_item->data;
+ MYISAM_SHARE *share= info->s;
+ DBUG_PRINT("info",("table '%s' 0x%lx tested against hash",
+ share->unique_file_name, (ulong)info));
+ if (!hash_search(mi_log_tables_physical, share->unique_file_name,
+ share->unique_name_length))
+ continue;
+ /* Backup kernel shouldn't ask for temporary table's backup */
+ DBUG_ASSERT(!share->temporary);
+ /*
+ Some key blocks may be in cache and not on disk. Their post_write hook
+ is NULL, so they will not go into the log. We need to force them into
+ the copy by flushing them now.
+ On the other hand, we don't need to flush WRITE_CACHE or the state
+ because every time they are written to disk (at the latest in
+ mi_log_stop_physical()) they check for physical logging so any now
+ cached info will finally reach the log.
+ */
+ pthread_mutex_lock(&share->intern_lock);
+ if ((mi_log_index_pages_physical &&
+ (share->kfile >= 0) &&
+ flush_key_blocks(share->key_cache, share->kfile, FLUSH_KEEP)))
+ {
+ error= 1;
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info); /* Mark that table must be checked */
+ pthread_mutex_unlock(&share->intern_lock);
+ goto end;
+ }
+ pthread_mutex_unlock(&share->intern_lock);
+ mi_set_physical_logging_state(info->s, 1);
+ }
+ error= 0;
+end:
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ if (unlikely(error))
+ mi_log_stop_physical(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_PHYSICAL,
+ NULL, NULL);
+ DBUG_RETURN(error);
+}
+
+
+/**
+ Stops MyISAM physical logging.
+
+ As part of this stop operation, user can request that the physical log ends
+ in a consistent state, i.e. that it contains copies of the currently cached
+ key pages etc. To be consistent assumes that the caller has relevant tables
+ write-locked, indeed otherwise the log could end in the middle of a
+ statement, and applying it would produce a likely corrupted table.Online
+ backup needs such a consistent log to be able to create consistent table
+ copies from the log. If online backup is being cancelled, then there is no
+ need that the physical log be consistent.
+
+ @param action MI_LOG_ACTION_CLOSE_CONSISTENT or
+ MI_LOG_ACTION_CLOSE_INCONSISTENT.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+
+ @note Even if MI_LOG_ACTION_CLOSE_CONSISTENT, tables may be being written
+ now (in practice caller has read-locked tables, but those tables may be
+ just going out of a write (after thr_unlock(), before or inside
+ mi_lock_database(F_UNLCK) which may be flushing the index header or index
+ pages).
+*/
+static int mi_log_stop_physical(enum enum_mi_log_action action)
+{
+ int error= 0;
+ LIST *list_item;
+ DBUG_ENTER("mi_log_stop_physical");
+ DBUG_ASSERT(action == MI_LOG_ACTION_CLOSE_CONSISTENT ||
+ action == MI_LOG_ACTION_CLOSE_INCONSISTENT);
+
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (mi_log_tables_physical == NULL) /* no physical logging running */
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_RETURN(0); /* it's ok if it happens */
+ }
+ /*
+ This is a pointer to a object provided by the caller through
+ mi_log_start_physical(); such object is to be freed by the caller.
+ */
+ mi_log_tables_physical= NULL;
+
+ if (action == MI_LOG_ACTION_CLOSE_CONSISTENT)
+ {
+ for (list_item= myisam_open_list; list_item; list_item= list_item->next)
+ {
+ MI_INFO *info= (MI_INFO*)list_item->data;
+ MYISAM_SHARE *share= info->s;
+ /*
+ Setting of the variable below always happens under THR_LOCK_myisam,
+ which we have here, so we don't need atomic operations to read here.
+ */
+ if (!info->s->physical_logging)
+ continue;
+ /*
+ Must take intern_lock, at least because key cache isn't safe if two
+ calls to flush_key_blocks_int() run concurrently on the same file.
+ */
+ pthread_mutex_lock(&share->intern_lock);
+ /*
+ It is possible that some statement just finished, has not called
+ mi_lock_database(F_UNLCK) yet, and so some key blocks would still be
+ in memory even if !delay_key_write. So we have to flush below even
+ in this case, to put them into the log.
+
+ It is also possible (same scenario) that some WRITE_CACHE is not
+ flushed yet. This should not happen but it does (can just be a
+ forgotten mi_extra(HA_EXTRA_NO_CACHE)); so mi_close() and
+ mi_lock_database(F_UNLCK) flush the cache; so we have to do it here
+ too, to put the data into the log. Mutices in mi_close() and
+ mi_lock_database() ensure that they don't flush at the same time as us
+ (which could corrupt the cache).
+
+ It is also possible (same scenario) that the index's header has not
+ been written yet and nobody is going to do it for us; indeed this can
+ happen (two concurrent threads): thread1 has just done
+ mi_lock_database(F_WRLCK), is blocked by the thr_lock of our caller,
+ thread2 has finished its write statement and is going to execute
+ mi_lock_database(F_UNLCK); no index header flush will be done by the
+ mi_lock_database(F_UNLCK) of thread2 as w_locks is >0 (due to
+ thread1). And no index header flush will be done by thread1 as it is
+ blocked. So, we need to flush the index header here, to put it into
+ the log.
+
+ Of course, for the flushing above to reach the log, it has to be done
+ before setting share->physical_logging to false and before closing the
+ log.
+ */
+ if ((mi_log_index_pages_physical &&
+ (share->kfile >= 0) &&
+ flush_key_blocks(share->key_cache, share->kfile, FLUSH_KEEP)) ||
+ ((info->opt_flag & WRITE_CACHE_USED) &&
+ flush_io_cache(&info->rec_cache)) ||
+ (share->changed && mi_remap_file_and_write_state_for_unlock(info)))
+ {
+ error= 1; /* we continue, because log has to be closed anyway */
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info); /* Mark that table must be checked */
+ }
+ pthread_mutex_unlock(&share->intern_lock);
+ } /* ... for (list_item=...) */
+ } /* ... if (action == MI_LOG_ACTION_CLOSE_CONSISTENT) */
+
+ /*
+ Online backup wants to pick this log with my_read() calls, to send it to
+ the backup stream. So we don't delete log but close it now, so that its
+ IO_CACHE goes to disk (so that all log is visible to the my_read()
+ calls). Another reason related to concurrency is mentioned below.
+ */
+ if (mi_log_close_cache(MI_LOG_PHYSICAL))
+ error= 1;
+
+ for (list_item= myisam_open_list; list_item; list_item= list_item->next)
+ {
+ MYISAM_SHARE *share= ((MI_INFO*)list_item->data)->s;
+ /*
+ Setting of the variable below always happens under THR_LOCK_myisam,
+ which we have here, so we don't need atomic operations to read here.
+ */
+ if (!share->physical_logging)
+ continue;
+ mi_set_physical_logging_state(share, 0);
+ /*
+ We reset MI_LOG_OPEN_stored_in_physical_log. How is this safe with a
+ concurrent logging operation (like myisam_log_pwrite_physical()) which
+ may want to set it to TRUE at the same time?
+ The concurrent logging operation runs either before or after log closing
+ (serialization ensured by THR_LOCK_myisam_log). If before, it is before
+ us (us==resetter), because log closing is before us, so we win. If
+ after, the concurrent logging operation finds the log closed and so
+ will not change MI_LOG_OPEN_stored_in_physical_log (so we win again).
+ Note the importance of closing the log before, for the reasoning to
+ hold.
+ */
+ share->MI_LOG_OPEN_stored_in_physical_log= FALSE;
+ }
+
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ /*
+ From this moment on, from the point of view of MyISAM, a new physical log
+ (a new backup) can start (new log will use a different tmp name).
+ */
+ DBUG_RETURN(error);
}
diff -Nrup a/storage/myisam/mi_open.c b/storage/myisam/mi_open.c
--- a/storage/myisam/mi_open.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_open.c 2007-11-06 20:56:01 +01:00
@@ -94,6 +94,12 @@ MI_INFO *mi_open(const char *name, int m
head_length=sizeof(share_buff.state.header);
bzero((uchar*) &info,sizeof(info));
+ /*
+ 'name' is an unresolved name (no .sym or Unix symbolic link
+ resolution). Physical logging needs it. We resolve 'name' in 'org_name'
+ and 'name_buff'. Don't change how name_buff is built without updating
+ myisam_backup::Backup::begin().
+ */
my_realpath(name_buff, fn_format(org_name,name,"",MI_NAME_IEXT,
MY_UNPACK_FILENAME),MYF(0));
pthread_mutex_lock(&THR_LOCK_myisam);
@@ -292,6 +298,7 @@ MI_INFO *mi_open(const char *name, int m
&share->rec,
(share->base.fields+1)*sizeof(MI_COLUMNDEF),
&share->blobs,sizeof(MI_BLOB)*share->base.blobs,
+ &share->unresolv_file_name,strlen(name)+1,
&share->unique_file_name,strlen(name_buff)+1,
&share->index_file_name,strlen(index_name)+1,
&share->data_file_name,strlen(data_name)+1,
@@ -313,6 +320,7 @@ MI_INFO *mi_open(const char *name, int m
memcpy((char*) share->state.key_del,
(char*) key_del, (sizeof(my_off_t) *
share->state.header.max_block_size_index));
+ strmov(share->unresolv_file_name,name);
strmov(share->unique_file_name, name_buff);
share->unique_name_length= strlen(name_buff);
strmov(share->index_file_name, index_name);
@@ -510,6 +518,7 @@ MI_INFO *mi_open(const char *name, int m
#ifdef THREAD
thr_lock_init(&share->lock);
VOID(pthread_mutex_init(&share->intern_lock,MY_MUTEX_INIT_FAST));
+ my_atomic_rwlock_init(&share->physical_logging_rwlock);
for (i=0; i<keys; i++)
VOID(my_rwlock_init(&share->key_root_lock[i], NULL));
VOID(my_rwlock_init(&share->mmap_lock, NULL));
@@ -567,7 +576,6 @@ MI_INFO *mi_open(const char *name, int m
share->base.max_key_length),
&info.lastkey,share->base.max_key_length*3+1,
&info.first_mbr_key, share->base.max_key_length,
- &info.filename,strlen(name)+1,
&info.rtree_recursion_state,have_rtree ? 1024 : 0,
NullS))
goto err;
@@ -576,7 +584,6 @@ MI_INFO *mi_open(const char *name, int m
if (!have_rtree)
info.rtree_recursion_state= NULL;
- strmov(info.filename,name);
memcpy(info.blobs,share->blobs,sizeof(MI_BLOB)*share->base.blobs);
info.lastkey2=info.lastkey+share->base.max_key_length;
@@ -635,14 +642,19 @@ MI_INFO *mi_open(const char *name, int m
#ifdef THREAD
thr_lock_data_init(&share->lock,&m_info->lock,(void*) m_info);
#endif
+ if (mi_log_tables_physical &&
+ hash_search(mi_log_tables_physical, share->unique_file_name,
+ share->unique_name_length))
+ m_info->s->physical_logging= TRUE; /* set before publishing table */
m_info->open_list.data=(void*) m_info;
myisam_open_list=list_add(myisam_open_list,&m_info->open_list);
pthread_mutex_unlock(&THR_LOCK_myisam);
- if (myisam_log_file >= 0)
+ if (my_b_inited(&myisam_logical_log))
{
intern_filename(name_buff,share->index_file_name);
- _myisam_log(MI_LOG_OPEN, m_info, (uchar*) name_buff, strlen(name_buff));
+ _myisam_log_command(&myisam_logical_log, MI_LOG_OPEN, m_info, name_buff,
+ strlen(name_buff), 0);
}
DBUG_RETURN(m_info);
@@ -838,15 +850,25 @@ static void setup_key_functions(register
}
-/*
- Function to save and store the header in the index file (.MYI)
+/**
+ Function to store state into the index file's header.
+
+ @param share table's share
+ @param file file descriptor of the index file
+ @param state state of the table
+ @param pWrite if my_pwrite() or my_write() should be used
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
*/
-uint mi_state_info_write(File file, MI_STATE_INFO *state, uint pWrite)
+uint mi_state_info_write(MYISAM_SHARE *share, File file,
+ MI_STATE_INFO *state, uint pWrite)
{
uchar buff[MI_STATE_INFO_SIZE + MI_STATE_EXTRA_SIZE];
uchar *ptr=buff;
- uint i, keys= (uint) state->header.keys,
+ uint i, ret, keys= (uint) state->header.keys,
key_blocks=state->header.max_block_size_index;
DBUG_ENTER("mi_state_info_write");
@@ -898,11 +920,13 @@ uint mi_state_info_write(File file, MI_S
}
}
- if (pWrite & 1)
- DBUG_RETURN(my_pwrite(file, buff, (size_t) (ptr-buff), 0L,
- MYF(MY_NABP | MY_THREADSAFE)) != 0);
- DBUG_RETURN(my_write(file, buff, (size_t) (ptr-buff),
- MYF(MY_NABP)) != 0);
+ ret= (pWrite & 1) ? (my_pwrite(file,(char*) buff, (uint) (ptr-buff), 0L,
+ MYF(MY_NABP | MY_THREADSAFE)) != 0):
+ (my_write(file, (char*) buff, (uint) (ptr-buff), MYF(MY_NABP)) != 0);
+ if (mi_get_physical_logging_state(share))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI,
+ share, (char*) buff, (uint) (ptr-buff), 0L);
+ DBUG_RETURN(ret);
}
diff -Nrup a/storage/myisam/mi_page.c b/storage/myisam/mi_page.c
--- a/storage/myisam/mi_page.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_page.c 2007-11-06 20:56:01 +01:00
@@ -17,6 +17,8 @@
#include "myisamdef.h"
+static int log_key_page_flush_physical(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos);
/* Fetch a key-page in memory */
uchar *_mi_fetch_keypage(register MI_INFO *info, MI_KEYDEF *keyinfo,
@@ -65,15 +67,16 @@ int _mi_write_keypage(register MI_INFO *
my_off_t page, int level, uchar *buff)
{
reg3 uint length;
+ MYISAM_SHARE *share= info->s;
DBUG_ENTER("_mi_write_keypage");
#ifndef FAST /* Safety check */
- if (page < info->s->base.keystart ||
+ if (page < share->base.keystart ||
page+keyinfo->block_length > info->state->key_file_length ||
(page & (MI_MIN_KEY_BLOCK_LENGTH-1)))
{
DBUG_PRINT("error",("Trying to write inside key status region: key_start: %lu length: %lu page: %lu",
- (long) info->s->base.keystart,
+ (long) share->base.keystart,
(long) info->state->key_file_length,
(long) page));
my_errno=EINVAL;
@@ -93,11 +96,24 @@ int _mi_write_keypage(register MI_INFO *
length=keyinfo->block_length;
}
#endif
- DBUG_RETURN((key_cache_write(info->s->key_cache,
- info->s->kfile,page, level, (uchar*) buff,length,
- (uint) keyinfo->block_length,
- (int) ((info->lock_type != F_UNLCK) ||
- info->s->delay_key_write))));
+ /*
+ It's more efficient to have the key cache write to the physical log when
+ it flushes than write now (a mass insert may touch the same index page
+ again and again; if we write now we write one log record per row write; if
+ we write only at flush time we write one log record per touched page).
+ Temporary tables never do physical logging, it's safe to give them a NULL
+ hook.
+ */
+ DBUG_RETURN(key_cache_write(share->key_cache,
+ share->kfile, page, level, (uchar*)buff, length,
+ (uint) keyinfo->block_length,
+ (int) ((info->lock_type != F_UNLCK) ||
+ share->delay_key_write),
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+ !share->temporary ?
+ log_key_page_flush_physical :
+#endif
+ NULL, share));
} /* mi_write_keypage */
@@ -108,18 +124,24 @@ int _mi_dispose(register MI_INFO *info,
{
my_off_t old_link;
uchar buff[8];
+ MYISAM_SHARE *share= info->s;
DBUG_ENTER("_mi_dispose");
DBUG_PRINT("enter",("pos: %ld", (long) pos));
- old_link= info->s->state.key_del[keyinfo->block_size_index];
- info->s->state.key_del[keyinfo->block_size_index]= pos;
+ old_link= share->state.key_del[keyinfo->block_size_index];
+ share->state.key_del[keyinfo->block_size_index]= pos;
mi_sizestore(buff,old_link);
- info->s->state.changed|= STATE_NOT_SORTED_PAGES;
- DBUG_RETURN(key_cache_write(info->s->key_cache,
- info->s->kfile, pos , level, buff,
- sizeof(buff),
- (uint) keyinfo->block_length,
- (int) (info->lock_type != F_UNLCK)));
+ share->state.changed|= STATE_NOT_SORTED_PAGES;
+ DBUG_RETURN(key_cache_write(share->key_cache,
+ share->kfile, pos, level, buff,
+ sizeof(buff),
+ (uint) keyinfo->block_length,
+ (int) (info->lock_type != F_UNLCK),
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+ !share->temporary ?
+ log_key_page_flush_physical :
+#endif
+ NULL, share));
} /* _mi_dispose */
@@ -158,3 +180,34 @@ my_off_t _mi_new(register MI_INFO *info,
DBUG_PRINT("exit",("Pos: %ld",(long) pos));
DBUG_RETURN(pos);
} /* _mi_new */
+
+
+/**
+ Logs when the key cache flushes a page to the file (so far, always the
+ index file), to the physical log.
+
+ Argument cannot be a MI_INFO* (the MI_INFO which put the page in the key
+ cache may have been freed long ago when the page is finally flushed), it is
+ MYISAM_SHARE* which is sure to be valid.
+
+ @param arg MYISAM_SHARE* where the block belongs
+ @param buffert argument to the pwrite
+ @param length length of buffer
+ @param filepos offset in file where buffer was written
+
+ @return Operation status, always 0
+ @retval 0 ok. Yes, even if log write fails we return ok, don't want
+ to make the table writer believe its table is now
+ corrupted.
+*/
+static int log_key_page_flush_physical(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos)
+{
+ MYISAM_SHARE *share= (MYISAM_SHARE *)arg;
+ DBUG_ENTER("log_key_page_flush_physical");
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, share, buffert,
+ length, filepos);
+ DBUG_RETURN(0);
+}
diff -Nrup a/storage/myisam/mi_panic.c b/storage/myisam/mi_panic.c
--- a/storage/myisam/mi_panic.c 2006-12-31 01:06:40 +01:00
+++ b/storage/myisam/mi_panic.c 2007-11-06 20:56:01 +01:00
@@ -54,6 +54,7 @@ int mi_panic(enum ha_panic_function flag
error=my_errno;
if (info->opt_flag & READ_CACHE_USED)
{
+ /* QQ Why do we flush a READ_CACHE? it's a no-op */
if (flush_io_cache(&info->rec_cache))
error=my_errno;
reinit_io_cache(&info->rec_cache,READ_CACHE,0,
@@ -78,15 +79,17 @@ int mi_panic(enum ha_panic_function flag
{ /* Open closed files */
char name_buff[FN_REFLEN];
if (info->s->kfile < 0)
- if ((info->s->kfile= my_open(fn_format(name_buff,info->filename,"",
- N_NAME_IEXT,4),info->mode,
- MYF(MY_WME))) < 0)
+ if ((info->s->kfile= my_open(fn_format(name_buff,
+ info->s->unresolv_file_name,
+ "", N_NAME_IEXT, 4),
+ info->mode, MYF(MY_WME))) < 0)
error = my_errno;
if (info->dfile < 0)
{
- if ((info->dfile= my_open(fn_format(name_buff,info->filename,"",
- N_NAME_DEXT,4),info->mode,
- MYF(MY_WME))) < 0)
+ if ((info->dfile= my_open(fn_format(name_buff,
+ info->s->unresolv_file_name,
+ "", N_NAME_DEXT, 4),
+ info->mode, MYF(MY_WME))) < 0)
error = my_errno;
info->rec_cache.file=info->dfile;
}
@@ -103,7 +106,8 @@ int mi_panic(enum ha_panic_function flag
}
if (flag == HA_PANIC_CLOSE)
{
- VOID(mi_log(0)); /* Close log if neaded */
+ /* Close log if needed */
+ mi_log(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_LOGICAL, NULL, NULL);
ft_free_stopwords();
}
pthread_mutex_unlock(&THR_LOCK_myisam);
diff -Nrup a/storage/myisam/mi_rrnd.c b/storage/myisam/mi_rrnd.c
--- a/storage/myisam/mi_rrnd.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_rrnd.c 2007-11-06 20:56:01 +01:00
@@ -31,10 +31,8 @@
int mi_rrnd(MI_INFO *info, uchar *buf, register my_off_t filepos)
{
- my_bool skip_deleted_blocks;
+ my_bool skip_deleted_blocks= 0;
DBUG_ENTER("mi_rrnd");
-
- skip_deleted_blocks=0;
if (filepos == HA_OFFSET_ERROR)
{
diff -Nrup a/storage/myisam/mi_static.c b/storage/myisam/mi_static.c
--- a/storage/myisam/mi_static.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_static.c 2007-11-06 20:56:01 +01:00
@@ -13,9 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Static variables for MyISAM library. All definied here for easy making of
- a shared library
+/**
+ @file
+ Static variables for MyISAM library.
+ All defined here for easy making of a shared library
*/
#ifndef _global_h
@@ -27,8 +28,9 @@ uchar NEAR myisam_file_magic[]=
{ (uchar) 254, (uchar) 254,'\007', '\001', };
uchar NEAR myisam_pack_file_magic[]=
{ (uchar) 254, (uchar) 254,'\010', '\002', };
-char * myisam_log_filename=(char*) "myisam.log";
-File myisam_log_file= -1;
+char *myisam_logical_log_filename= (char*)"myisam.log";
+IO_CACHE myisam_physical_log; /**< Physical log (used by online backup) */
+IO_CACHE myisam_logical_log; /**< Logical log (used for debugging) */
uint myisam_quick_table_bits=9;
ulong myisam_block_size= MI_KEY_BLOCK_LENGTH; /* Best by test */
my_bool myisam_flush=0, myisam_delay_key_write=0, myisam_single_user=0;
@@ -59,3 +61,14 @@ uint NEAR myisam_readnext_vec[]=
SEARCH_BIGGER, SEARCH_BIGGER, SEARCH_SMALLER, SEARCH_BIGGER, SEARCH_SMALLER,
SEARCH_BIGGER, SEARCH_SMALLER, SEARCH_SMALLER
};
+
+/** Hash of all tables for which we want physical logging */
+const HASH *mi_log_tables_physical;
+/**
+ If page changes to the index file should be logged to the physical log.
+
+ @note Changes to the header of the index file of a table in physical
+ logging are always logged because the header is not redundant with the data
+ file.
+*/
+my_bool mi_log_index_pages_physical;
diff -Nrup a/storage/myisam/mi_test2.c b/storage/myisam/mi_test2.c
--- a/storage/myisam/mi_test2.c 2007-10-09 19:55:17 +02:00
+++ b/storage/myisam/mi_test2.c 2007-11-06 20:56:01 +01:00
@@ -215,7 +215,7 @@ int main(int argc, char *argv[])
&create_info,create_flag))
goto err;
if (use_log)
- mi_log(1);
+ mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
if (!(file=mi_open(filename,2,HA_OPEN_ABORT_IF_LOCKED)))
goto err;
if (!silent)
diff -Nrup a/storage/myisam/mi_test3.c b/storage/myisam/mi_test3.c
--- a/storage/myisam/mi_test3.c 2007-08-13 15:11:17 +02:00
+++ b/storage/myisam/mi_test3.c 2007-11-06 20:56:01 +01:00
@@ -169,7 +169,7 @@ void start_test(int id)
MI_INFO *file,*file1,*file2=0,*lock;
if (use_log)
- mi_log(1);
+ mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
if (!(file1=mi_open(filename,O_RDWR,HA_OPEN_WAIT_IF_LOCKED)) ||
!(file2=mi_open(filename,O_RDWR,HA_OPEN_WAIT_IF_LOCKED)))
{
@@ -214,7 +214,7 @@ void start_test(int id)
mi_close(file1);
mi_close(file2);
if (use_log)
- mi_log(0);
+ mi_log(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_LOGICAL, NULL, NULL);
if (error)
{
printf("%2d: Aborted\n",id); fflush(stdout);
diff -Nrup a/storage/myisam/mi_update.c b/storage/myisam/mi_update.c
--- a/storage/myisam/mi_update.c 2007-08-13 15:11:17 +02:00
+++ b/storage/myisam/mi_update.c 2007-11-06 20:56:01 +01:00
@@ -170,7 +170,7 @@ int mi_update(register MI_INFO *info, co
info->update= (HA_STATE_CHANGED | HA_STATE_ROW_CHANGED | HA_STATE_AKTIV |
key_changed);
- myisam_log_record(MI_LOG_UPDATE,info,newrec,info->lastpos,0);
+ myisam_log_record_logical(MI_LOG_UPDATE, info, newrec, info->lastpos, 0);
/*
Every myisam function that updates myisam table must end with
call to _mi_writeinfo(). If operation (second param of
@@ -185,8 +185,9 @@ int mi_update(register MI_INFO *info, co
allow_break(); /* Allow SIGHUP & SIGINT */
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (update)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (update)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
DBUG_RETURN(0);
@@ -231,7 +232,8 @@ err:
key_changed);
err_end:
- myisam_log_record(MI_LOG_UPDATE,info,newrec,info->lastpos,my_errno);
+ myisam_log_record_logical(MI_LOG_UPDATE, info, newrec,
+ info->lastpos, my_errno);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
if (save_errno == HA_ERR_KEY_NOT_FOUND)
diff -Nrup a/storage/myisam/mi_write.c b/storage/myisam/mi_write.c
--- a/storage/myisam/mi_write.c 2007-09-21 10:15:04 +02:00
+++ b/storage/myisam/mi_write.c 2007-11-06 20:56:01 +01:00
@@ -154,12 +154,13 @@ int mi_write(MI_INFO *info, uchar *recor
HA_STATE_ROW_CHANGED);
info->state->records++;
info->lastpos=filepos;
- myisam_log_record(MI_LOG_WRITE,info,record,filepos,0);
+ myisam_log_record_logical(MI_LOG_WRITE, info, record, filepos, 0);
VOID(_mi_writeinfo(info, WRITEINFO_UPDATE_KEYFILE));
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (update)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (update)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
@@ -231,7 +232,7 @@ err:
my_errno=save_errno;
err2:
save_errno=my_errno;
- myisam_log_record(MI_LOG_WRITE,info,record,filepos,my_errno);
+ myisam_log_record_logical(MI_LOG_WRITE, info, record, filepos, my_errno);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
DBUG_RETURN(my_errno=save_errno);
diff -Nrup a/storage/myisam/myisam_backup_driver.cc b/storage/myisam/myisam_backup_driver.cc
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/storage/myisam/myisam_backup_driver.cc 2007-11-06 20:56:01 +01:00
@@ -0,0 +1,1875 @@
+/** Copyright (C) 2007 MySQL AB
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; version 2 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+
+/**
+ @file
+ Online backup driver for the MyISAM storage engine.
+
+ @see myisam_backup
+*/
+
+#define MYSQL_SERVER 1 // need it to have mysql_tmpdir defined
+#include "mysql_priv.h"
+#include "ha_myisam.h"
+#include "myisamdef.h" // to access dfile and kfile
+#include "backup/backup_engine.h"
+#include "backup/backup_aux.h" // for build_table_list()
+#include <hash.h>
+
+/**
+ Online backup driver for the MyISAM storage engine.
+
+ Reference of the Online Backup API:
+ http://forge.mysql.com/source/OnlineBackup.
+
+ Here is how the MyISAM online backup works.
+ It is online because we dirtily copy the data and index files,
+ and the tables maintain a physical idempotent log of changes done to them
+ during the copy process, applying this log to the dirty copy yields a clean
+ table corresponding to how the original table was when logging ended.
+
+ A condition for this to work is that any update done to a table after the
+ copy process started must be present in the log. So if an update is
+ proceeding, we must wait for it to go to the log before starting the copy
+ process.
+
+ HOW THE BACKUP WORKS
+
+ In backup::begin(), we instruct all needed tables to do backup
+ logging; this does not have to wait for existing updates to complete,
+ neither does it stall new updates.
+
+ Then we dirtily copy them in Backup::get_data(). That copy is intensive on
+ the hard drive, so can be optionally throttled (via a configurable sleep).
+
+ When the copy process is done with tables, it signals the backup kernel
+ that it is ready to lock tables (to create a validity point).
+ To not waste its time until the Backup::prelock() request is sent by the
+ backup kernel, the copy process starts copying the log.
+
+ Now the Backup::prelock() request comes.
+ To finish the backup, we need to synchronize (=read-lock) all tables of the
+ backup (thus creating a consistent state accross them), stop logging for
+ all of them, and unlock tables. This lock can wait for a long time if there
+ is a long running update, so we do all the locking work before
+ Backup::lock(): Backup::prelock() sets up a separate thread which issues a
+ LOCK TABLES READ on our tables; without waiting for completion of that,
+ prelock() returns backup::OK which means "I have not completed my
+ preparations for locking".
+
+ In Backup::get_data(), the driver monitors the status of the locking
+ thread, and when finally that thread has managed to get its locks, we stop
+ logging and reply backup::READY.
+
+ So note the difference: this time, we have to wait for all updates to
+ finish, and stall new ones.
+
+ Next Backup::get_data() calls, if there are, send the final tail of the
+ log.
+
+ Backup::lock() comes, it's an empty operation for the driver.
+
+ Later we get a Backup::unlock() request. That kills the locking thread,
+ which thus unlocks tables. And Backup::end() cleans up memory.
+
+ HOW THE RESTORE WORKS
+
+ In Restore::send_data() we receive data which we write to tables (those
+ tables have just been created with their correct structure, but no data, by
+ the backup kernel). We similarly restore the log.
+
+ In Restore::end(), we apply the log to tables, making them clean.
+ If of the index file we backed up only the header (an option), we here do
+ an index rebuild.
+ Voila, the table is ready to work.
+
+ @todo if an index rebuild is needed, possibly do it at backup time.
+*/
+namespace myisam_backup {
+
+using backup::byte;
+using backup::result_t;
+using backup::version_t;
+using backup::Table_list;
+using backup::Table_ref;
+using backup::Buffer;
+
+
+/** Like Table_ref but with file name added */
+class Myisam_table_ref
+{
+public:
+ Myisam_table_ref(const Table_ref &);
+protected:
+ String db, name;
+ String file_name; ///< concatenation of db and table name
+};
+
+
+Myisam_table_ref::Myisam_table_ref(const Table_ref &tbl)
+{
+ const char *db_arg= tbl.db().name().ptr();
+ const char *name_arg= tbl.name().ptr();
+ /**
+ We keep local copies of the db and name. This is because during restore,
+ the Table_ref is apparently modified before the Table_restore is done
+ (symptom is that starting from second Table_restore::send_data() we see
+ Table_ref being garbage, and this is a problem in
+ Table_restore::post_restore()). Rafal suspects a bug.
+ @todo Once fixed, we can replace "String db,name" by "Table_ref &ref",
+ and this will save memory.
+ As Rafal is changing relevant code now, it may go away.
+ */
+ db.append(db_arg);
+ name.append(name_arg);
+ /*
+ Note: this way below will break with non-ASCII characters;
+ what driver should be passed is what engine used to create the table, ie
+ output of build_table_filename().
+ Just replace "building" with "bûilding" in backup.test to see issues.
+ This is remembered in WL#4060.
+
+ But note, when we repair the table, we use open_temporary_table() which
+ requires db and table name separated.
+ */
+ file_name.append("./");
+ file_name.append(db_arg);
+ file_name.append("/");
+ file_name.append(name_arg);
+}
+
+
+/**
+ Backup engine class. It is the master class: a Backup_engine creates a
+ Backup_driver and a corresponding Restore_driver. @see backup::Engine.
+*/
+class Engine: public Backup_engine
+{
+ public:
+ Engine() {}
+ virtual const version_t version() const { return 0; };
+ virtual result_t get_backup(const uint32, const Table_list &,
+ Backup_driver* &);
+ virtual result_t get_restore(version_t, const uint32,
+ const Table_list &,Restore_driver* &);
+ virtual void free() { delete this; }
+};
+
+/*************************
+ *
+ * BACKUP FUNCTIONALITY
+ *
+ *************************/
+
+class Object_backup;
+
+
+/**
+ Handles backup orders received from the backup kernel (implements the API).
+*/
+class Backup: public Backup_driver
+{
+public:
+ Backup(const Table_list &);
+ virtual ~Backup();
+ /** Estimates total size of backup. @todo improve it */
+ virtual size_t size() { return UNKNOWN_SIZE; };
+ /** Estimates size of backup before lock. @todo improve it */
+ virtual size_t init_size() { return UNKNOWN_SIZE; };
+ virtual result_t begin(const size_t);
+ virtual result_t end();
+ virtual result_t get_data(Buffer &);
+ virtual result_t prelock();
+ virtual result_t lock();
+ virtual result_t unlock();
+ virtual result_t cancel()
+ {
+ return backup::OK ; // free() will be called and suffice
+ };
+ virtual void free() { delete this; };
+ void lock_tables_TL_READ_NO_INSERT();
+
+private:
+ enum { DUMPING_DATA_INDEX_FILES,
+ DUMPING_LOG_FILE_BEFORE_TABLES_ARE_LOCKED,
+ DUMPING_LOG_FILE_AFTER_TABLES_ARE_LOCKED,
+ DONE, ERROR } state;
+ Object_backup *image; ///< object in backup currently
+ uint stream; ///< which stream we are currently writing to
+ char backup_log_name[FN_REFLEN];
+ /**
+ All db||table names in a HASH structure. Passed to MyISAM functions for
+ them to detect if a table is part of the backup (=> should do logging) or
+ not.
+ */
+ HASH *hash_of_tables;
+ /** Locking of tables goes through these states */
+ enum { LOCK_NOT_STARTED, LOCK_IN_PROGRESS, LOCK_ACQUIRED, LOCK_ERROR }
+ lock_state;
+ /**
+ The locking thread (so that we can kill it). Creating a validity point is
+ only possible by locking all tables (it is the only way to have tables
+ consistent with each other, as we have no UNDO log). But locking via
+ thr_lock() is blocking. So, to have a non-blocking prelock() call, this
+ locking is done in a separate thread (named "the locking thread").
+ */
+ THD *lock_thd;
+ bool cannot_delete_lock_thd;
+ pthread_cond_t COND_lock_state; ///< for communication with locking thread
+ void kill_locking_thread();
+ static const ulong bytes_between_sleeps= 10*1024*1024;
+ /** After copying bytes_between_sleeps we sleep sleep_time */
+ ulong sleep_time;
+ ulong bytes_since_last_sleep; ///< how many bytes sent since we last slept
+};
+
+/**
+ When we send a backup packet to the backup kernel, we prefix it with a code
+ which tells which type of file this packet belongs to. Starts at 1 because
+ garbage is often zeros and we want to spot it.
+*/
+enum enum_file_code { DATA_FILE_CODE= 1,
+ WHOLE_INDEX_FILE_CODE, HEADER_INDEX_FILE_CODE,
+ LOG_FILE_CODE };
+
+/** An object to backup; in practice, a table or the log */
+class Object_backup
+{
+public:
+ virtual result_t get_data(Buffer &)= 0;
+ virtual ~Object_backup() {};
+ bool internal_error() { return state == ERROR; }
+ /**
+ The only reason to have an end() and call it from the destructor, instead
+ of putting the code into the destructor, is that when the caller does a
+ "delete image", it cannot be told about errors, while if the caller does
+ "image->end()" (and then "delete image") it can see an error.
+ */
+ virtual result_t end()= 0; ///< cleanups
+protected:
+ enum { OK, ERROR } state; ///< serves to detect an error during construction
+};
+
+
+/**
+ An object to back up is made of one or more such files. This class does not
+ open the file, user has to open it. This class closes the file upon
+ termination.
+*/
+class File_backup
+{
+public:
+ result_t get_data(Buffer &);
+ File_backup() : fd(-1), backup_file_size(0) {}
+ /**
+ Initializes the object.
+
+ @param fd_arg file descriptor to attach to
+ @param file_size_arg copy should start after copying that many bytes
+ @param file_code_arg code to store at start of each sent data packet
+ */
+ void init(int fd_arg, my_off_t file_size_arg, enum_file_code file_code_arg)
+ { fd= fd_arg; file_size= file_size_arg; file_code= file_code_arg; }
+ result_t close_file();
+private:
+ int fd; ///< file descriptor
+ /**
+ After backing up that many bytes of the file, we can stop. In case of
+ ftruncate() happening to the file, we may even copy less than this size.
+ */
+ my_off_t file_size;
+ enum_file_code file_code; ///< code stored at start of each backup block
+ my_off_t backup_file_size; ///< how much of the file we already backed up
+};
+
+
+/** Handles backing up a single table */
+class Table_backup: public Myisam_table_ref, public Object_backup
+{
+public:
+ Table_backup(const backup::Table_ref &);
+ virtual ~Table_backup();
+ virtual result_t get_data(Buffer &);
+ virtual result_t end(); ///< cleanups
+private:
+ File_backup dfile_backup, kfile_backup;
+ enum { DATA_FILE, INDEX_FILE } in_file; ///< which file we are dumping now
+};
+
+
+/** Handles backing up the log */
+class Log_backup: public Object_backup
+{
+public:
+ Log_backup(const char *);
+ virtual ~Log_backup();
+ virtual result_t get_data(Buffer &);
+ virtual result_t end();
+private:
+ const char *log_name;
+ File_backup log_file_backup;
+ bool log_deleted; ///< if we have already deleted the log or not
+};
+
+
+/**
+ Creates a backup driver, per the backup API. @see backup::Engine.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+
+*/
+result_t Engine::get_backup(const uint32, const Table_list &tables,
+ Backup_driver* &drv)
+{
+ Backup *ptr= new Backup(tables);
+ if (unlikely(!ptr))
+ return backup::ERROR;
+ drv= ptr;
+ return backup::OK;
+}
+
+
+Backup::Backup(const Table_list &tables):
+ Backup_driver(tables), state(ERROR), image(NULL), stream(1),
+ hash_of_tables(NULL), lock_state(LOCK_NOT_STARTED), lock_thd(NULL),
+ cannot_delete_lock_thd(FALSE), bytes_since_last_sleep(0)
+{
+ /*
+ Driver is not ready at this point, so state is ERROR.
+ This constructor cannot fail, otherwise begin() would have to detect it.
+ */
+ pthread_cond_init(&COND_lock_state, NULL);
+}
+
+
+/** Kills the locking thread when it is time to unlock tables */
+void Backup::kill_locking_thread()
+{
+ /*
+ If everything worked well, when unlock() calls us we kill the thread and
+ so when free() calls us the locking thread is already dead here
+ (LOCK_ERROR).
+ */
+retry:
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (lock_state != LOCK_ERROR) // thread not already dead, kill it
+ {
+ /*
+ If the locking thread had no time to create THD (very unlikely), wait
+ for it.
+ */
+ if (unlikely(!lock_thd))
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ sleep(1);
+ goto retry;
+ }
+ /*
+ Locking thread had time to create its THD, may be inside table locking
+ (waiting for others to release locks etc), wake it up and kill it. Or it
+ may have locked tables successfully, and be waiting for us to kill it.
+ To do that we will use lock_thd, but how to be sure that lock_thd is not
+ being deleted now? One way would be to hold THR_LOCK_myisam but
+ THD::awake() can't bear it (same mutex locked twice).
+ Another way is to take lock_thd->LOCK_delete (THD::awake() requires it
+ anyway), but again that requires that lock_thd is not deleted while we
+ access the mutex. We cannot hold THR_LOCK_myisam to get LOCK_delete,
+ because that could deadlock if a some other thread is doing a KILL on
+ the locking thread (it would indeed take LOCK_delete and then
+ THR_LOCK_myisam to wake up the locking thread).
+ So So we set a flag:
+ */
+ cannot_delete_lock_thd= TRUE;
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ /*
+ So now lock_thd cannot be destroyed.
+ We kill the thread (which will in particular work if it is waiting for
+ some table locks).
+ */
+ pthread_mutex_lock(&lock_thd->LOCK_delete);
+ lock_thd->awake(THD::KILL_CONNECTION);
+ pthread_mutex_unlock(&lock_thd->LOCK_delete);
+ /* won't look at lock_thd anymore, allow its deletion */
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ cannot_delete_lock_thd= FALSE;
+ /* we wake up thread if it was blocked on the bool above */
+ pthread_cond_broadcast(&COND_lock_state);
+ /* And we wait for the thread to inform of its death */
+ while (lock_state != LOCK_ERROR)
+ pthread_cond_wait(&COND_lock_state, &THR_LOCK_myisam);
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+}
+
+
+/**
+ This destructor is only called by the class' free().
+ It cleans up any leftover the driver could have. It is safe to call it at
+ any point. In a normal (no error) situation, the hash freeing is the only
+ operation done here, all the rest should already have been done by earlier
+ stages.
+*/
+Backup::~Backup()
+{
+ /* If we had already started backup logging, we must dirtily stop it */
+ mi_log(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_PHYSICAL, NULL, NULL);
+ delete image;
+ if (hash_of_tables)
+ {
+ hash_free(hash_of_tables);
+ delete hash_of_tables;
+ hash_of_tables= NULL;
+ }
+ kill_locking_thread();
+ pthread_cond_destroy(&COND_lock_state);
+}
+
+
+/** Usual parameter to hash_init() */
+static uchar
+*backup_get_table_from_hash_key(const uchar *lsc, uint *length,
+ my_bool not_used __attribute__ ((unused)))
+{
+ const ::LEX_STRING *ls= reinterpret_cast<const ::LEX_STRING *>(lsc);
+ *length= ls->length;
+ return reinterpret_cast< uchar *>(ls->str);
+}
+
+
+/** Usual parameter to hash_init() */
+static void backup_free_hash_key(void *lsv)
+{
+ my_free(lsv, MYF(0));
+}
+
+
+#define SET_STATE_TO_ERROR_AND_DBUG_RETURN { \
+ state= ERROR; \
+ DBUG_PRINT("error",("driver got an error at %s:%d",__FILE__,__LINE__)); \
+ DBUG_RETURN(backup::ERROR); }
+
+/* use this one only in constructors */
+#define SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN { \
+ state= ERROR; \
+ DBUG_PRINT("error",("driver got an error at %s:%d",__FILE__,__LINE__)); \
+ DBUG_VOID_RETURN; }
+
+
+/**
+ Sets MyISAM in a state ready for the copy to start. I.e. builds
+ a hash of tables and starts MyISAM physical logging for those tables.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Backup::begin(const size_t)
+{
+ DBUG_ENTER("myisam_backup::Backup::begin");
+ DBUG_PRINT("info",("%d tables", m_tables.count()));
+
+ /*
+ per the API, all significant allocations (large mem, opening files) must
+ not be in the constructor but in begin() or later.
+ */
+ DBUG_ASSERT(!hash_of_tables); // no double call to begin()
+ hash_of_tables= new HASH;
+ if (!hash_of_tables ||
+ hash_init(hash_of_tables, &my_charset_bin, m_tables.count(), 0, 0,
+ backup_get_table_from_hash_key, backup_free_hash_key, 0))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ /* Build the hash of tables for the MyISAM layer (mi_backup_log.c etc) */
+ for (uint n=0 ; n < m_tables.count() ; n++ )
+ {
+ char unique_file_name[FN_REFLEN], *str;
+ uint str_len;
+ ::LEX_STRING *hash_key;
+
+ my_realpath(unique_file_name,
+ fn_format(unique_file_name,m_tables[n].name().ptr(),
+ m_tables[n].db().name().ptr(),
+ MI_NAME_IEXT,
+ MY_UNPACK_FILENAME), MYF(MY_WME));
+ str_len= strlen(unique_file_name);
+ my_multi_malloc(MYF(MY_WME),
+ &hash_key, sizeof(*hash_key),
+ &str, str_len, NullS);
+ if (!hash_key)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ memcpy(str, unique_file_name, str_len);
+ hash_key->length= str_len;
+ hash_key->str= str;
+ if (my_hash_insert(hash_of_tables,
+ reinterpret_cast< uchar *>(hash_key)))
+ {
+ my_free(hash_key, MYF(0));
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ DBUG_PRINT("info",("table '%.*s' inserted in hash",
+ hash_key->length, hash_key->str));
+ }
+
+ {
+ THD *thd= current_thd;
+ /*
+ If tmpdir is in RAM (/dev/shm etc), we may exhaust it if our log is big
+ */
+ my_snprintf(backup_log_name, sizeof(backup_log_name),
+ "%s/%s%lx_%lx_%x-backuplog", mysql_tmpdir,
+ tmp_file_prefix, current_pid, thd->thread_id,
+ thd->tmp_table++); // it's not a tmp table but what...
+ unpack_filename(backup_log_name, backup_log_name);
+#if 0
+ /*
+ Of course driver shouldn't pollute thd's state like this, it's just for
+ debugging now.
+ */
+ thd->proc_info="MyISAM dump going to start";
+#endif
+ }
+
+ {
+ /**
+ Until there exists a framework by which the user tells, via SQL,
+ indications on how it wants the backup, and by which the backup kernel
+ tells it to the driver (API), we have to resort to this or an option.
+ @todo choose between the two. GUILHEM_TODO.
+ */
+ char *env_arg= getenv("MYISAM_BACKUP_NO_INDEX");
+ /* By default we log index pages */
+ mi_log_index_pages_physical= !(env_arg && atoi(env_arg));
+ env_arg= getenv("MYISAM_BACKUP_SLEEP");
+ /*
+ By default we don't sleep at all; however, 500 ms every 10MB gives a
+ low penalty on clients, so it can be a good choice.
+ */
+ sleep_time= env_arg ? atoi(env_arg) : 0;
+ }
+
+ if (mi_log(MI_LOG_ACTION_OPEN, MI_LOG_PHYSICAL,
+ backup_log_name, hash_of_tables))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ state= DUMPING_DATA_INDEX_FILES;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ If some error happened, end() is not called but free() is. So we do all
+ cleanup in free() i.e. in the destructor, and nothing here.
+
+ @return Operation status
+ @retval backup::OK
+*/
+result_t Backup::end()
+{
+ DBUG_ENTER("myisam_backup::Backup::end");
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Sends backup data for tables and log to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status (see the API for when they are returned)
+ @retval backup::OK
+ @retval backup::DONE
+ @retval backup::READY
+ @retval backup::ERROR
+*/
+result_t Backup::get_data(Buffer &buf)
+{
+ result_t ret;
+ DBUG_ENTER("myisam::backup::Backup::get_data");
+ DBUG_PRINT("enter",("stream %d",stream));
+
+ /* we are currently on stream 'stream' */
+ buf.table_no= stream;
+
+ /*
+ Rafal and I agreed that one single ERROR from the driver will cause the
+ upper layer to not call the driver anymore except for free().
+ */
+ DBUG_ASSERT(state != ERROR);
+ DBUG_ASSERT(buf.data != NULL); // to check that caller gave room
+
+ if (state == DONE)
+ {
+ /*
+ We never come here, because after returning from the call where we sent
+ the last piece of the last stream (when we set our internal state to
+ DONE), all streams were closed, so the upper layer wouldn't call us
+ again. At least it was so during testing. But if it calls us, we do all
+ that the API expects us to do:
+ */
+ buf.size= buf.table_no= 0;
+ buf.last= TRUE;
+ DBUG_RETURN(backup::DONE);
+ }
+
+ buf.last= TRUE; // by default
+
+ if (unlikely(image == NULL))
+ {
+ /*
+ Let's create it.
+ Table 0 will be image 1 on stream 1. Table N will be image N+1 on stream
+ N+1. Log will be image 0 on stream 0.
+ */
+ if (stream >= 1)
+ image= new Table_backup(m_tables[stream-1]);
+ else
+ image= new Log_backup(backup_log_name);
+ if (image == NULL || image->internal_error())
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+
+ if ((ret= image->get_data(buf)) != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ if (sleep_time)
+ {
+ bytes_since_last_sleep+= buf.size;
+ /* sched_yield() is not as flexible (higher penalty) as sleep() */
+ if (bytes_since_last_sleep > bytes_between_sleeps)
+ {
+ my_sleep(sleep_time * 1000UL);
+ bytes_since_last_sleep= 0;
+ }
+ }
+
+ if (state == DUMPING_LOG_FILE_BEFORE_TABLES_ARE_LOCKED)
+ {
+ DBUG_ASSERT(stream == 0);
+ /*
+ We are sending the log; even if reached its EOF, some more may be
+ appended to it before prelock() ends, so this is not the stream's end.
+ */
+ buf.last= FALSE;
+ /*
+ API docs say we should return READY, but Rafal says OK is better (one
+ READY to signal end of initial phase; then OKs; one READY to signal end
+ of prelock(); then OKs).
+ */
+ if (lock_state == LOCK_NOT_STARTED)
+ DBUG_RETURN(backup::OK);
+ /* Let's see if the locking thread has finished locking all tables */
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (lock_state == LOCK_IN_PROGRESS) // not yet
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_RETURN(backup::OK);
+ }
+ if (lock_state != LOCK_ACQUIRED) // it failed, so do we
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ DBUG_PRINT("info",("locking thread acquired locks on tables"));
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ if (mi_log(MI_LOG_ACTION_CLOSE_CONSISTENT, MI_LOG_PHYSICAL, NULL, NULL))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ state= DUMPING_LOG_FILE_AFTER_TABLES_ARE_LOCKED;
+ /* signal "end of prepare-for-lock, ready for lock()" */
+ DBUG_RETURN(backup::READY);
+ }
+ else if (buf.last)
+ {
+ /*
+ we are sending the last chunk of the image, next call will be about the
+ next image:
+ */
+ if (image->end() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ delete image;
+ image= NULL; /* next call of this function should open the next object */
+ stream++; /* and send it on the next stream */
+ if (state == DUMPING_DATA_INDEX_FILES && stream > m_tables.count())
+ {
+ /* all tables done */
+ stream= 0; // send the log on stream 0
+ state= DUMPING_LOG_FILE_BEFORE_TABLES_ARE_LOCKED;
+ ret= backup::READY; // end of initial phase
+ }
+ else if (state == DUMPING_LOG_FILE_AFTER_TABLES_ARE_LOCKED) // log done
+ state= DONE;
+ }
+
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Creates a validity point by locking all tables. This is the only job of the
+ locking thread: call this function which locks tables, then wait for being
+ killed (which will unlock tables).
+
+ @todo make a function which does all needed initializations, for those
+ threads (like this thread, the slave SQL thread, the NDB binlog thread)
+ which open/lock tables without going through the usual client thread calls
+ (do_command()/dispatch_command()/mysql_parse()/mysql_execute_command().
+ That will probably happen when Chuck's patch is pushed, as it does similar
+ thread creation: http://lists.mysql.com/commits/36809 (be_thread.cc)
+ GUILHEM_TODO.
+ @todo use a method which does not open closed tables. This will be needed
+ when backing up lots of tables (more than the limit of open file
+ descriptors). Monty suggests calling thr_multi_lock()+mi_open() on already
+ open tables (so, like open_and_lock_tables() on them), plus, for
+ closed tables, a thread which opens a closed table would acquire a read
+ thr_lock lock on it and give this lock to the backup thread...
+*/
+void Backup::lock_tables_TL_READ_NO_INSERT()
+{
+ THD *thd;
+ TABLE_LIST *tables_in_TABLE_LIST_form=NULL ; ///< for open_and_lock_tables()
+ DBUG_ENTER("myisam::backup::Backup::lock_tables_TL_READ_NO_INSERT");
+
+ thd= new THD;
+ if (unlikely(!thd))
+ goto end2;
+ thd->thread_stack = reinterpret_cast< char *>(&thd);
+ pthread_mutex_lock(&LOCK_thread_count);
+ thd->thread_id= thread_id++;
+ pthread_mutex_unlock(&LOCK_thread_count);
+ if (unlikely(thd->store_globals())) // for a proper MEM_ROOT
+ goto end2;
+ thd->init_for_queries(); // opening tables needs a proper LEX
+ thd->command= COM_DAEMON;
+ thd->system_thread= SYSTEM_THREAD_BACKUP;
+ thd->version= refresh_version;
+ thd->set_time();
+ thd->main_security_ctx.host_or_ip= "";
+ thd->client_capabilities= 0;
+ my_net_init(&thd->net, 0);
+ thd->main_security_ctx.master_access= ~0;
+ thd->main_security_ctx.priv_user= 0;
+ thd->real_id= pthread_self();
+ /*
+ Making this thread visible to SHOW PROCESSLIST is useful for
+ troubleshooting a backup job (why does it stall etc).
+ */
+ pthread_mutex_lock(&LOCK_thread_count);
+ threads.append(thd);
+ pthread_mutex_unlock(&LOCK_thread_count);
+ lex_start(thd);
+ mysql_reset_thd_for_next_command(thd);
+ /*
+ As locking tables can be a long operation, we need to support
+ cancellability during that time. So we publish our THD now to the thread
+ which created us (the "master" thread), so that it can kill us early if
+ needed.
+ */
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ lock_thd= thd;
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ /*
+ We need TL_READ_NO_INSERT (and not TL_READ) because we want to prevent
+ concurrent inserts (we indeed need to freeze the tables to correspond to
+ a position in the binlog).
+ */
+ tables_in_TABLE_LIST_form=
+ backup::build_table_list(m_tables, TL_READ_NO_INSERT);
+ if (!tables_in_TABLE_LIST_form)
+ goto end2;
+ if (open_and_lock_tables(thd, tables_in_TABLE_LIST_form))
+ goto end;
+
+ DBUG_PRINT("info",("MyISAM backup locking thread got locks"));
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ thd->enter_cond(&COND_lock_state, &THR_LOCK_myisam,
+ "MyISAM backup: holding table locks");
+ /* show master thread that we got locks */
+ lock_state= LOCK_ACQUIRED;
+ /* and wait for it to kill us */
+ while (!thd->killed)
+ pthread_cond_wait(&COND_lock_state, &THR_LOCK_myisam);
+ thd->exit_cond("MyISAM backup: terminating");
+
+end:
+ DBUG_PRINT("info",("MyISAM backup locking thread dying"));
+ close_thread_tables(thd);
+end2:
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ while (cannot_delete_lock_thd)
+ {
+ /* master thread is looking at our THD; wait for authorization */
+ pthread_cond_wait(&COND_lock_state, &THR_LOCK_myisam);
+ }
+ lock_state= LOCK_ERROR;
+ pthread_cond_broadcast(&COND_lock_state);
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ backup::free_table_list(tables_in_TABLE_LIST_form);
+ net_end(&thd->net);
+ delete thd;
+ DBUG_VOID_RETURN;
+}
+
+
+/** Entry point for the locking thread */
+pthread_handler_t separate_thread_for_locking(void *arg)
+{
+ my_thread_init();
+ DBUG_PRINT("info", ("myisam_backup::separate_thread_for_locking"));
+ pthread_detach_this_thread();
+ (static_cast<Backup *>(arg))->lock_tables_TL_READ_NO_INSERT();
+ my_thread_end();
+ pthread_exit(0);
+ return 0;
+}
+
+
+/**
+ Launches a separate thread ("locking thread") which will lock
+ tables. Locking in a separate thread is needed to have a non-blocking
+ prelock() (given that thr_lock() is blocking).
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Backup::prelock()
+{
+ DBUG_ENTER("myisam_backup::Backup::prelock");
+ lock_state= LOCK_IN_PROGRESS;
+ {
+ pthread_t th;
+ if (pthread_create(&th, &connection_attrib,
+ separate_thread_for_locking, this))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ DBUG_RETURN(backup::OK);
+}
+
+
+result_t Backup::lock()
+{
+ DBUG_ENTER("myisam_backup::Backup::lock");
+ /* locking was done in prelock() already, nothing to do */
+ DBUG_RETURN(backup::OK);
+}
+
+
+result_t Backup::unlock()
+{
+ DBUG_ENTER("myisam_backup::Backup::unlock");
+ /* kill the locking thread which owns table locks, it will unlock them */
+ kill_locking_thread();
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Backs up the log.
+
+ @todo For now we read the log file from disk. We could instead try to
+ "steal" it from its IO_CACHE; that might reduce the log portion which goes
+ to disk, if the backup thread is fast enough to catch up on client threads
+ filling the log.
+*/
+
+Log_backup::Log_backup(const char *log_name_arg) : log_name(log_name_arg),
+ log_deleted(FALSE)
+{
+ DBUG_ENTER("myisam_backup::Log_backup::Log_backup");
+ int fd= my_open(log_name, O_RDONLY, MYF(MY_WME));
+ if (fd < 0)
+ SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN;
+ /*
+ Log is alone on the shared stream for now, so LOG_FILE_CODE is useless,
+ except that it allows us to verify that what restore sends us is really a
+ log.
+ */
+ log_file_backup.init(fd, ~(ULL(0)), LOG_FILE_CODE);
+ state= OK;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Closes and deletes the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Log_backup::end()
+{
+ DBUG_ENTER("myisam_backup::Log_backup::end");
+ /*
+ Log is safe in the stream, or backup is cancelled, so we don't need it
+ anymore.
+ */
+ if (log_file_backup.close_file() != backup::OK ||
+ (!log_deleted && my_delete(log_name, MYF(MY_WME))))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ log_deleted= TRUE;
+ DBUG_RETURN(backup::OK);
+}
+
+
+Log_backup::~Log_backup()
+{
+ /* If all went well, we don't do anything here. */
+ if (end() != backup::OK)
+ {
+ /*
+ Repetition of failing deletes could fill disk, warn.
+ It's a destructor, no other way to report an error than print.
+ */
+ sql_print_error("MyISAM backup driver: could not close or delete log");
+ }
+}
+
+
+/** The header of a MYI index file always fits in this size */
+#define MAX_INDEX_HEADER_SIZE (64*1024)
+
+
+/**
+ Opens a MyISAM table for backing it up.
+
+ @param tbl The table to open
+*/
+Table_backup::Table_backup(const backup::Table_ref &tbl) :
+ Myisam_table_ref(tbl)
+{
+ MI_INFO *mi_info;
+ File dfiledes= -1, kfiledes= -1;
+ my_off_t file_size;
+ DBUG_ENTER("myisam_backup::Table_backup::Table_backup");
+ DBUG_PRINT("info",("Initializing backup image for table %s",
+ file_name.ptr()));
+ /*
+ Here we use low-level mi_* functions as all we want is a pair of file
+ descriptors.
+ O_RDONLY is not ok, as it forces all instances of the table to be
+ read-only (sets HA_OPTION_READ_ONLY_DATA of share->options).
+ HA_OPEN_FOR_REPAIR so that we can back it up even if it is corrupted (we
+ don't do that now).
+ */
+ mi_info= mi_open(file_name.ptr(), O_RDWR, HA_OPEN_FOR_REPAIR);
+ if (!mi_info) // table does not exist? not normal
+ goto err;
+ if (mi_is_crashed(mi_info))
+ {
+ /*
+ For some users, a backup has to be fully correct to be considered
+ usable, so a corrupted table should stop the backup. However sometimes,
+ user knows a table is corrupted but still wants to back it up (it could
+ be that the machine's hardware is failing, currently only the index file
+ is bad, then it makes sense to back it up before the table is completely
+ destroyed). Or it could be 10000 tables to back up with a single one
+ corrupted, then it makes sense to do the backup and later repair the
+ single one.
+ For now we abort the backup. Optionally we should not.
+ */
+ goto err;
+ }
+ /*
+ we create our own descriptors, to use my_read() (faster than my_pread()
+ which may use mutex).
+ */
+ dfiledes= my_open(mi_info->s->data_file_name, O_RDONLY, MYF(MY_WME));
+ kfiledes= my_open(mi_info->s->unique_file_name, O_RDONLY, MYF(MY_WME));
+ if ((dfiledes < 0) || (kfiledes < 0))
+ goto err;
+ mi_close(mi_info);
+ mi_info= NULL;
+ file_size= my_seek(dfiledes, 0, SEEK_END, MYF(MY_WME));
+ if (file_size == MY_FILEPOS_ERROR ||
+ my_seek(dfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR)
+ goto err;
+ dfile_backup.init(dfiledes, file_size, DATA_FILE_CODE);
+ if (mi_log_index_pages_physical)
+ {
+ file_size= my_seek(kfiledes, 0, SEEK_END, MYF(MY_WME));
+ if (file_size == MY_FILEPOS_ERROR ||
+ my_seek(kfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR)
+ goto err;
+ kfile_backup.init(kfiledes, file_size, WHOLE_INDEX_FILE_CODE);
+ }
+ else
+ kfile_backup.init(kfiledes,
+ MAX_INDEX_HEADER_SIZE /* upper limit */ ,
+ HEADER_INDEX_FILE_CODE);
+ in_file= DATA_FILE; // dump the data file first (no specific reason)
+ state= OK;
+ DBUG_VOID_RETURN;
+ /*
+ Note: we are copying an index file of a table, which may have instances in
+ the MySQL table cache, so after restore it will show up as
+ "warning: 1 client is using or hasn't closed the table properly".
+ Maybe do a quick index update on the table at the end of restore to
+ remove this warning. But how to know if the problem pre-dates backup ?
+ */
+err:
+ if (dfiledes > 0)
+ my_close(dfiledes, MYF(0));
+ if (kfiledes > 0)
+ my_close(kfiledes, MYF(0));
+ if (mi_info != NULL)
+ mi_close(mi_info);
+ SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN;
+}
+
+
+/**
+ Closes the MyISAM table.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Table_backup::end()
+{
+ DBUG_ENTER("myisam_backup::Table_backup::end");
+ /* even if one close fails we still want to try the other one */
+ if ((dfile_backup.close_file() != backup::OK) |
+ (kfile_backup.close_file() != backup::OK))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+Table_backup::~Table_backup()
+{
+ /* If all went well, we don't do anything here. */
+ if (end() != backup::OK)
+ {
+ /*
+ Close failure is important, warn.
+ It's a destructor, no other way to report an error than print.
+ */
+ sql_print_error("MyISAM backup driver: could not close table '%s'",
+ file_name.ptr());
+ }
+}
+
+
+/**
+ Sends backup data for one table to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Table_backup::get_data(Buffer &buf)
+{
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Table_backup::get_data");
+ switch (in_file)
+ {
+ case DATA_FILE:
+ ret= dfile_backup.get_data(buf);
+ if (buf.last) // move to dumping the index file...
+ {
+ in_file= INDEX_FILE;
+ buf.last= FALSE; // ... so this is not the last buffer on this stream
+ }
+ break;
+ case INDEX_FILE:
+ ret= kfile_backup.get_data(buf);
+ break;
+ default:
+ DBUG_ASSERT(0);
+ };
+ if (ret != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Sends backup data for the log to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Log_backup::get_data(Buffer &buf)
+{
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Log_backup::get_data");
+ /*
+ See, we detect a log write error encountered by the MyISAM myisam_log*
+ and mi_log* functions, every time we read a packet from the log file.
+ */
+ if (((ret= log_file_backup.get_data(buf)) != backup::OK) ||
+ (myisam_physical_log.hard_write_error_in_the_past == -1))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Closes a file in backup.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t File_backup::close_file()
+{
+ int ret;
+ if (fd < 0)
+ return backup::OK;
+ ret= my_close(fd, MYF(MY_WME));
+ fd= -1;
+ return ret ? backup::ERROR : backup::OK;
+}
+
+
+/**
+ Sends backup data for a single file to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t File_backup::get_data(Buffer &buf)
+{
+ int res;
+ uint howmuch= buf.size;
+ result_t ret= backup::OK;
+
+ DBUG_ENTER("myisam_backup::File_backup::get_data");
+
+ buf.size= 1;
+ *buf.data= static_cast<uchar>(file_code);
+ howmuch--;
+ DBUG_ASSERT(howmuch > 0);
+
+ if (backup_file_size >= file_size)
+ res= 0; // we don't have to read/send the rest of file
+ else
+ {
+ res= my_read(fd, buf.data + 1, howmuch, MYF(MY_WME));
+ // DBUG_DUMP("sending",buf_ptr-1, 16);
+ }
+ if (res < 0)
+ {
+ ret= backup::ERROR;
+ goto end;
+ }
+ backup_file_size+= res;
+ if (res == 0) // end of file
+ {
+ buf.size= 0; // don't even send a packet
+ goto end;
+ }
+ buf.size+= res;
+ buf.last= FALSE;
+end:
+ DBUG_PRINT("info",("ret %d buf.last %d buf.size %d",
+ ret, buf.last, buf.size));
+ DBUG_RETURN(ret);
+}
+
+
+/**************************************
+ *
+ * RESTORE FUNCTIONALITY
+ *
+ **************************************/
+
+class Object_restore;
+
+/**
+ Handles restore orders received from the backup kernel (implements the
+ API).
+*/
+class Restore: public Restore_driver
+{
+public:
+ Restore(const Table_list &tables);
+ virtual ~Restore();
+ virtual result_t begin(const size_t);
+ virtual result_t end();
+ virtual result_t send_data(Buffer &buf);
+ virtual result_t cancel()
+ {
+ /* Nothing to do in cancel(); free() will suffice */
+ return backup::OK;
+ };
+ virtual void free() { delete this; };
+
+private:
+ enum { PUMPING, DONE, ERROR } state;
+ uint images_left; ///< how many images left to restore
+ Object_restore **images; ///< one for the log and one per table
+ char restore_log_name[FN_REFLEN];
+};
+
+
+/** An object to restore; in practice, a table or the log */
+class Object_restore
+{
+public:
+ virtual result_t send_data(const Buffer &buf)= 0;
+ virtual ~Object_restore() {};
+ /**
+ Closes the object, post_restore() can later be called. Whereas in
+ Object_backup, closing is done in end() (there is no close()), here we
+ have a dedicated close() method. This is because we must close tables and
+ the log then apply the log then repair indices: we need to close way
+ before end()).
+ */
+ virtual result_t close()= 0;
+ /** Does additional restore operations between close() and end() */
+ virtual result_t post_restore()= 0;
+ bool internal_error() { return state == ERROR; }
+ virtual result_t end()= 0; ///< cleanups
+protected:
+ enum { OK, ERROR } state;
+};
+
+
+/**
+ An object to restore is made of one or more such files. This class does not
+ open the file, user has to open it. This class does not close the file by
+ default, but can do so if requested.
+*/
+class File_restore
+{
+public:
+ File_restore() : fd(-1) {}
+ void init(int fd_arg) { fd= fd_arg; }
+ result_t send_data(const Buffer &);
+ result_t close_file();
+private:
+ int fd; ///< file descriptor
+};
+
+
+/** Handles restoring a single table */
+class Table_restore: public Object_restore, public Myisam_table_ref
+{
+public:
+ Table_restore(const Table_ref &tbl);
+ virtual result_t send_data(const Buffer &buf);
+ virtual ~Table_restore();
+ virtual result_t close();
+ virtual result_t post_restore();
+ virtual result_t end(); ///< cleanups
+ private:
+ File_restore dfile_restore, kfile_restore;
+ bool rebuild_index; ///< if we have to rebuild index or not
+ THD *thd; ///< rebuilding index requires a THD
+};
+
+
+/** Handles restoring the log */
+class Log_restore: public Object_restore
+{
+public:
+ Log_restore(const char *log_name_arg);
+ virtual result_t send_data(const Buffer &buf);
+ virtual ~Log_restore();
+ virtual result_t close();
+ virtual result_t post_restore();
+ virtual result_t end();
+private:
+ const char *log_name;
+ File_restore log_file_restore;
+ bool log_deleted; ///< if we have already deleted the log or not
+};
+
+
+/**
+ Creates a restore driver, per the backup API. @see backup::Engine.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Engine::get_restore(version_t ver, const uint32,
+ const Table_list &tables, Restore_driver* &drv)
+{
+ Restore *ptr= new Restore(tables);
+ if (unlikely(!ptr))
+ return backup::ERROR;
+ drv= ptr;
+ return backup::OK;
+}
+
+
+Restore::Restore(const Table_list &tables):
+ Restore_driver(tables), state(ERROR), images_left(0), images(NULL)
+{
+ /* This constructor cannot fail otherwise begin() would have to detect it */
+}
+
+
+/**
+ This destructor is only called by the class' free(). It cleans up any
+ leftover the driver could have. It is safe to call it at any point. In a
+ normal (no error) situation, it does nothing, all should already have been
+ done by earlier stages.
+*/
+Restore::~Restore()
+{
+ if (images)
+ {
+ for (uint n= 0; n <= m_tables.count(); ++n)
+ delete images[n];
+ delete[] images;
+ }
+}
+
+
+/**
+ Sets MyISAM in a state ready for us to restore. I.e. creates a temporary
+ file to host the log's restored copy.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Restore::begin(const size_t)
+{
+ THD *thd= current_thd;
+ DBUG_ENTER("myisam_backup::Restore::begin");
+ my_snprintf(restore_log_name, sizeof(restore_log_name),
+ "%s/%s%lx_%lx_%x-restorelog", mysql_tmpdir,
+ tmp_file_prefix, current_pid, thd->thread_id,
+ thd->tmp_table++);
+ unpack_filename(restore_log_name, restore_log_name);
+
+ images_left= 1 + m_tables.count();
+ images= new Object_restore*[images_left];
+ if (unlikely(!images))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ bzero(images, images_left * sizeof(*images));
+ state= PUMPING;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ If no error happened, we have to apply the log and possibly repair
+ indexes; this has to be done here and not in the destructor (as it has to
+ be done only in case of success, while a destructor runs in all cases).
+ Because we have no "end of stream" notifications yet, when we come here all
+ our tables/logs are opened. and log is not applied (both things which could
+ be done in send_data() if we knew end-of-stream). Repairing indexes, on the
+ other hand, really has to be done here.
+
+ @todo selective restore (this is just passing a proper function which
+ checks if the table is in a hash of tables).
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Restore::end()
+{
+ DBUG_ENTER("myisam_backup::Restore::end");
+ /*
+ Rafal said currently end() is called in case of error but said he'll fix
+ that (only free() will be called)
+ */
+ DBUG_ASSERT(state != ERROR);
+ if (images)
+ {
+ for (uint n=0; n <= m_tables.count(); ++n)
+ if (images[n] && images[n]->close() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ /*
+ Tables are closed. Apply backup log if it exists (it does not exist if
+ it was empty at backup time), this is post_restore() of images[0]. Then
+ repair indices if needed (post_restore() of other images).
+ */
+ for (uint n=0; n <= m_tables.count(); ++n)
+ if (images[n] && images[n]->post_restore() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ /*
+ By doing here the work of the destructor we can test the return code of
+ end(). We don't do it for tables as they will do nothing in end()
+ (except freeing their memory) so that can be left to the destructor.
+ */
+ if (images[0] && images[0]->end() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Receives and restores data for tables and log from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Restore::send_data(Buffer &buf)
+{
+ result_t ret;
+ uint stream= buf.table_no;
+ DBUG_ENTER("myisam_backup::Restore::send_data");
+ DBUG_PRINT("enter",("Got packet with %d bytes from stream %d",
+ buf.size, buf.table_no));
+
+ if (state == DONE)
+ {
+ /* we never come here */
+ DBUG_PRINT("info",("Ignoring the packet (all objects already restored)"));
+ DBUG_RETURN(backup::DONE);
+ }
+
+ Object_restore *image= images[stream];
+
+ /*
+ We create an image when we see a new stream.
+ Still we have N open tables during the last table's restore.
+ But when Rafal implements that the last buffer of a stream has
+ buf.last==TRUE (soon), we can close tables earlier.
+ */
+ if (!image)
+ {
+ if (stream >= 1)
+ image= new Table_restore(m_tables[stream-1]);
+ else
+ image= new Log_restore(restore_log_name);
+ images[stream]= image;
+ if (unlikely(!image || image->internal_error()))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+
+ if ((ret= image->send_data(buf)) != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ // for when we have "end of stream" notifications:
+#if 0
+ if (buf.last)
+ {
+ if (image->close() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ images_left--;
+ if (images_left == 0)
+ {
+ state= DONE;
+ /* DONE means done with all send_data() calls, but we have more work */
+ DBUG_RETURN(backup::DONE);
+ }
+ }
+#endif
+
+ DBUG_RETURN(backup::OK);
+}
+
+
+
+/**
+ Restores the log.
+
+ @param log_name_arg Name under which log should be created
+*/
+Log_restore::Log_restore(const char *log_name_arg) : log_name(log_name_arg)
+{
+ DBUG_ENTER("myisam_backup::Log_restore::Log_restore");
+ int fd= my_create(log_name, 0, O_WRONLY, MYF(MY_WME));
+ if (fd < 0)
+ {
+ log_deleted= TRUE;
+ SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN;
+ }
+ log_deleted= FALSE;
+ log_file_restore.init(fd);
+ state= OK;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Closes the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Log_restore::close()
+{
+ DBUG_ENTER("myisam_backup::Log_restore::close");
+ if (log_file_restore.close_file() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Applies the log to restored tables, to make them consistent.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Log_restore::post_restore()
+{
+ MI_EXAMINE_LOG_PARAM mi_exl;
+ DBUG_ENTER("myisam_backup::Log_restore::post_restore");
+ mi_examine_log_param_init(&mi_exl);
+ mi_exl.log_filename= log_name;
+ mi_exl.update= 1;
+ /*
+ For max_files, the assumption is that at backup time the server had
+ enough file descriptors and so should have that many now.
+ */
+ mi_exl.max_files= open_files_limit;
+ if (mi_examine_log(&mi_exl))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Closes and deletes the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Log_restore::end()
+{
+ DBUG_ENTER("myisam_backup::Log_restore::end");
+ /* log is applied so we don't need it anymore */
+ if (close() != backup::OK ||
+ (!log_deleted && my_delete(log_name, MYF(MY_WME))))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ log_deleted= TRUE;
+ DBUG_RETURN(backup::OK);
+}
+
+
+Log_restore::~Log_restore()
+{
+ /* If all went well, we don't do anything here. */
+ if (end() != backup::OK)
+ {
+ /*
+ Repetition of failing deletes could fill disk, warn.
+ It's a destructor, no other way to report an error than print.
+ */
+ sql_print_error("MyISAM restore driver: could not close or delete log");
+ }
+}
+
+
+/** Opens a MyISAM table for restoring it */
+Table_restore::Table_restore(const Table_ref &tbl):
+ Myisam_table_ref(tbl), rebuild_index(FALSE)
+{
+ MI_INFO *mi_info;
+ File dfiledes= -1, kfiledes= -1;
+ DBUG_ENTER("myisam_backup::Table_restore::Table_restore");
+ DBUG_PRINT("enter",("Initializing backup image for table %s",
+ file_name.ptr()));
+ /*
+ Here we use low-level mi_* functions as all we want is a pair of file
+ descriptors.
+ Though we only want to write (O_WRONLY), the SQL layer uses only O_RDONLY
+ and O_RDWR, so here we don't try to be original.
+ */
+ mi_info= mi_open(file_name.ptr(), O_RDWR, 0);
+ if (!mi_info)
+ {
+ /* table does not exist or is corrupted? not normal, it's just created */
+ goto err;
+ }
+ /*
+ It's ok to copy the kfile descriptor and write() to it as the upper layers
+ guarantee that we are the only user of the brand new table (nobody will
+ lseek() under our feet).
+ */
+ if (((dfiledes= my_dup(mi_info->dfile, MYF(MY_WME))) < 0) ||
+ ((kfiledes= my_dup(mi_info->s->kfile, MYF(MY_WME))) < 0))
+ goto err;
+ /*
+ We are going to my_write() to the files without updating the table's
+ state (mi_info->state). If we called mi_close() only at end of restore,
+ that function may write its out-of-date state on the table.
+ */
+ mi_close(mi_info);
+ mi_info= NULL;
+ /* seek them at start, because we use my_write() */
+ if ((my_seek(dfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR) ||
+ (my_seek(kfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR))
+ goto err;
+ dfile_restore.init(dfiledes);
+ kfile_restore.init(kfiledes);
+ thd= current_thd;
+ state= OK;
+ DBUG_VOID_RETURN;
+err:
+ if (dfiledes > 0)
+ my_close(dfiledes, MYF(0));
+ if (kfiledes > 0)
+ my_close(kfiledes, MYF(0));
+ if (mi_info != NULL)
+ mi_close(mi_info);
+}
+
+
+/**
+ Closes a table.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Table_restore::close()
+{
+ DBUG_ENTER("myisam_backup::Table_restore::close");
+ DBUG_PRINT("info",("table: %s", file_name.ptr()));
+ if ((dfile_restore.close_file() != backup::OK) |
+ (kfile_restore.close_file() != backup::OK))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Closes a table.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Table_restore::end()
+{
+ return close();
+}
+
+
+Table_restore::~Table_restore()
+{
+ if (end() != backup::OK)
+ {
+ /*
+ Close failure is important, warn.
+ It's a destructor, no other way to report an error than print.
+ */
+ sql_print_error("MyISAM restore driver: could not close table");
+ }
+}
+
+
+/**
+ Repairs table's index if needed. Has to be done after applying the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Table_restore::post_restore()
+{
+ HA_CHECK_OPT check_opt;
+ int error;
+ Vio* save_vio;
+ DBUG_ENTER("myisam_backup::Table_restore::post_restore");
+
+ if (!rebuild_index)
+ DBUG_RETURN(backup::OK); // nothing to do
+
+ /*
+ myisamchk() as well as ha_myisam::repair() do a lot of operations before
+ and after mi_repair(); to not duplicate code we reuse one of them.
+ As we are in the server here, we use the one of the server.
+ A "new ha_myisam + ha_open()" is not sufficient as TABLE and TABLE_SHARE
+ are needed for ha_myisam::open(). So we use open_temporary_table() which
+ sets up all fine without touching thread's structure (and so, without
+ causing problems to locks, without interfering with close_thread_tables()
+ which would be done by another driver in the same thread etc).
+ Note that as the table has just been created, and in theory is protected
+ from any usage, by the upper backup layer, opening it with
+ open_temporary_table() is correct.
+ */
+ TABLE *table;
+ char path[FN_REFLEN];
+ build_table_filename(path, sizeof(path), db.ptr(), name.ptr(), "", 0);
+ table= open_temporary_table(thd, path, db.ptr(), name.ptr(),
+ false, OTM_OPEN);
+
+ if ((error= (!table || !table->file)))
+ goto err;
+
+ check_opt.init();
+ check_opt.flags|= T_VERY_SILENT | T_CALC_CHECKSUM | T_QUICK;
+ /*
+ We do not want repair() to spam us with messages (protocol->store() etc).
+ Just send them to the error log, and report the failure in case of
+ problems.
+ Note that ha_myisam::restore() does not do that (merely uses the same
+ check_opt.flags as us), as it is allowed to return an array of errors.
+ */
+ save_vio= thd->net.vio;
+ thd->net.vio= NULL;
+ error= table->file->ha_repair(thd,&check_opt) != 0;
+ thd->net.vio= save_vio;
+
+err:
+ if (table)
+ {
+ intern_close_table(table);
+ my_free(table, MYF(0));
+ }
+ if (error)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Receives and restores data for one table from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Table_restore::send_data(const Buffer &buf)
+{
+ enum enum_file_code file_code= static_cast<enum enum_file_code>(*buf.data);
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Table_restore::send_data");
+
+ switch (file_code)
+ {
+ case DATA_FILE_CODE:
+ ret= dfile_restore.send_data(buf);
+ break;
+ case HEADER_INDEX_FILE_CODE:
+ rebuild_index= TRUE; // because we are given only the index's header
+ // fall through
+ case WHOLE_INDEX_FILE_CODE:
+ ret= kfile_restore.send_data(buf);
+ break;
+ default:
+ DBUG_PRINT("info",("packet with code %d I didn't expect", file_code));
+ DBUG_ASSERT(0);
+ ret= backup::ERROR;
+ }
+ if (ret != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Receives and restores data for the log from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t Log_restore::send_data(const Buffer &buf)
+{
+ enum enum_file_code file_code= static_cast<enum enum_file_code>(*buf.data);
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Log_restore::send_data");
+
+ ret= (file_code == LOG_FILE_CODE) ? log_file_restore.send_data(buf) :
+ backup::ERROR;
+ if (ret != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Receives and restores data for one single file from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t File_restore::send_data(const Buffer &buf)
+{
+ uint howmuch= buf.size;
+
+ DBUG_ENTER("myisam_backup::File_restore::send_data");
+ //DBUG_DUMP("receiving",buf.data + 1, 16);
+
+ howmuch--; // skip the first byte which contains the code
+
+ uint res= my_write(fd, buf.data +1, howmuch, MYF(MY_WME));
+
+ DBUG_RETURN((res != howmuch) ? backup::ERROR : backup::OK);
+}
+
+
+/**
+ Closes a file in restore.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+result_t File_restore::close_file()
+{
+ int ret;
+ if (fd < 0)
+ return backup::OK;
+ ret= my_close(fd, MYF(MY_WME));
+ fd= -1;
+ return ret ? backup::ERROR : backup::OK;
+}
+
+
+} // myisam_backup namespace
+
+
+/*
+ Backup kernel guarantees one single backup at a time, thus can use a static
+ object.
+*/
+static myisam_backup::Engine single_backup_engine;
+/**
+ Returns the backup Engine used by this storage engine, per the API.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+Backup_result_t myisam_backup_engine(handlerton *self, Backup_engine* &be)
+{
+ be= new myisam_backup::Engine();
+
+ if (unlikely(!be))
+ return backup::ERROR;
+
+ return backup::OK;
+}
diff -Nrup a/storage/myisam/myisam_ftdump.c b/storage/myisam/myisam_ftdump.c
--- a/storage/myisam/myisam_ftdump.c 2007-05-10 11:59:33 +02:00
+++ b/storage/myisam/myisam_ftdump.c 2007-11-06 20:56:01 +01:00
@@ -98,7 +98,8 @@ int main(int argc,char *argv[])
if ((inx >= info->s->base.keys) ||
!(info->s->keyinfo[inx].flag & HA_FULLTEXT))
{
- printf("Key %d in table %s is not a FULLTEXT key\n", inx, info->filename);
+ printf("Key %d in table %s is not a FULLTEXT key\n", inx,
+ info->s->unresolv_file_name);
goto err;
}
diff -Nrup a/storage/myisam/myisamdef.h b/storage/myisam/myisamdef.h
--- a/storage/myisam/myisamdef.h 2007-08-26 13:31:22 +02:00
+++ b/storage/myisam/myisamdef.h 2007-11-06 20:56:01 +01:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* This file is included by all internal myisam files */
+/**
+ @file
+ This file is included by all internal myisam files
+*/
#include "myisam.h" /* Structs & some defines */
#include "myisampack.h" /* packing of keys */
@@ -21,6 +24,7 @@
#ifdef THREAD
#include <my_pthread.h>
#include <thr_lock.h>
+#include <my_atomic.h>
#else
#include <my_no_pthread.h>
#endif
@@ -154,6 +158,7 @@ typedef struct st_mi_isam_pack {
#define MAX_NONMAPPED_INSERTS 1000
+/** Information shared by all open instances of the same table */
typedef struct st_mi_isam_share { /* Shared between opens */
MI_STATE_INFO state;
MI_BASE_INFO base;
@@ -219,6 +224,27 @@ typedef struct st_mi_isam_share { /* Sha
uint nonmmaped_inserts; /* counter of writing in non-mmaped
area */
rw_lock_t mmap_lock;
+ /**
+ If this table is doing physical logging (1) or not (0).
+ Set under MI_INFO::physical_logging_rwlock and THR_LOCK_myisam.
+ Read under either one of the two locks above.
+ */
+ volatile int32 physical_logging;
+ /** For protecting MYISAM_SHARE::physical_logging */
+ my_atomic_rwlock_t physical_logging_rwlock;
+ /**
+ File name before resolving any symlink or expanding directory.
+ Used by physical logging. In order to restore to a data directory with a
+ different path from the original one, we need unresolved relative names.
+ */
+ char *unresolv_file_name;
+ /**
+ If we already stored MI_LOG_OPEN in physical log for this share.
+ Set to TRUE only by writer thread under THR_LOCK_myisam_log atomically
+ with logging the MI_LOG_OPEN; set to FALSE only by mi_log_stop_physical()
+ after closing the log.
+ */
+ my_bool MI_LOG_OPEN_stored_in_physical_log;
} MYISAM_SHARE;
@@ -234,6 +260,7 @@ typedef struct st_mi_bit_buff { /* Used
typedef my_bool (*index_cond_func_t)(void *param);
+/** Information local to the table's instance */
struct st_myisam_info {
MYISAM_SHARE *s; /* Shared between open:s */
MI_STATUS_INFO *state,save_state;
@@ -244,7 +271,6 @@ struct st_myisam_info {
DYNAMIC_ARRAY *ft1_to_ft2; /* used only in ft1->ft2 conversion */
MEM_ROOT ft_memroot; /* used by the parser */
MYSQL_FTPARSER_PARAM *ftparser_param; /* share info between init/deinit */
- char *filename; /* parameter to open filename */
uchar *buff, /* Temp area for key */
*lastkey,*lastkey2; /* Last used search key */
uchar *first_mbr_key; /* Searhed spatial key */
@@ -472,7 +498,7 @@ typedef struct st_mi_sort_param
#define mi_unique_store(A,B) mi_int4store((A),(B))
#ifdef THREAD
-extern pthread_mutex_t THR_LOCK_myisam;
+extern pthread_mutex_t THR_LOCK_myisam, THR_LOCK_myisam_log;
#endif
#if !defined(THREAD) || defined(DONT_USE_RW_LOCKS)
#define rw_wrlock(A) {}
@@ -486,8 +512,10 @@ extern LIST *myisam_open_list;
extern uchar NEAR myisam_file_magic[],NEAR myisam_pack_file_magic[];
extern uint NEAR myisam_read_vec[],NEAR myisam_readnext_vec[];
extern uint myisam_quick_table_bits;
-extern File myisam_log_file;
+extern IO_CACHE myisam_logical_log, myisam_physical_log;
extern ulong myisam_pid;
+extern const HASH *mi_log_tables_physical;
+extern my_bool mi_log_index_pages_physical;
/* This is used by _mi_calc_xxx_key_length och _mi_store_key */
@@ -688,13 +716,33 @@ typedef struct st_mi_block_info { /* Par
#define SORT_BUFFER_INIT (2048L*1024L-MALLOC_OVERHEAD)
#define MIN_SORT_BUFFER (4096-MALLOC_OVERHEAD)
+/** Commands storable in MyISAM logs (L/P= in logical/physical log) */
enum myisam_log_commands {
- MI_LOG_OPEN,MI_LOG_WRITE,MI_LOG_UPDATE,MI_LOG_DELETE,MI_LOG_CLOSE,MI_LOG_EXTRA,MI_LOG_LOCK,MI_LOG_DELETE_ALL
+ MI_LOG_OPEN, /**< when mi_open() LP */
+ MI_LOG_WRITE, /**< when mi_write() L */
+ MI_LOG_UPDATE, /**< when mi_update() L */
+ MI_LOG_DELETE, /**< when mi_delete() L */
+ MI_LOG_CLOSE, /**< when mi_close() LP */
+ MI_LOG_EXTRA, /**< when mi_extra() L */
+ MI_LOG_LOCK, /**< when mi_lock_database() L */
+ MI_LOG_DELETE_ALL, /**< when mi_delete_all LP */
+ MI_LOG_WRITE_BYTES_MYD, /**< when MyISAM writes to the data file P */
+ MI_LOG_WRITE_BYTES_MYI, /**< when MyISAM writes to the index file P */
+ MI_LOG_CHSIZE_MYI, /**< when MyISAM changes size of index file P */
+ MI_LOG_END_SENTINEL /**< keep this one unused and last */
};
-
-#define myisam_log(a,b,c,d) if (myisam_log_file >= 0) _myisam_log(a,b,c,d)
-#define myisam_log_command(a,b,c,d,e) if (myisam_log_file >= 0) _myisam_log_command(a,b,c,d,e)
-#define myisam_log_record(a,b,c,d,e) if (myisam_log_file >= 0) _myisam_log_record(a,b,c,d,e)
+extern const char *mi_log_command_name[];
+/** Logs a command if this log is open */
+#define myisam_log_command_logical(command, info_or_share, buffert, length, result) \
+ if (my_b_inited(&myisam_logical_log)) \
+ _myisam_log_command(&myisam_logical_log, command, info_or_share, \
+ buffert, length, result)
+/** Logs a command involving a record if this log is open */
+#define myisam_log_record_logical(command, info, record, filepos, result) \
+ if (my_b_inited(&myisam_logical_log)) \
+ _myisam_log_record_logical(command, info, record, filepos, result)
+/** If log record stores numerical info in long format */
+#define MI_LOG_BIG_NUMBERS 128
#define fast_mi_writeinfo(INFO) if (!(INFO)->s->tot_locks) (void) _mi_writeinfo((INFO),0)
#define fast_mi_readinfo(INFO) ((INFO)->lock_type == F_UNLCK) && _mi_readinfo((INFO),F_RDLCK,1)
@@ -709,14 +757,57 @@ extern uint _mi_pack_get_block_info(MI_I
MI_BLOCK_INFO *info, uchar **rec_buff_p,
File file, my_off_t filepos);
extern void _my_store_blob_length(uchar *pos,uint pack_length,uint length);
-extern void _myisam_log(enum myisam_log_commands command,MI_INFO *info,
- const uchar *buffert,uint length);
-extern void _myisam_log_command(enum myisam_log_commands command,
- MI_INFO *info, const uchar *buffert,
- uint length, int result);
-extern void _myisam_log_record(enum myisam_log_commands command,MI_INFO *info,
- const uchar *record,my_off_t filepos,
- int result);
+extern void _myisam_log_command(IO_CACHE *log,
+ enum myisam_log_commands command,
+ void *info_or_share, const uchar *buffert,
+ uint length, int result);
+extern void _myisam_log_record_logical(enum myisam_log_commands command,
+ MI_INFO *info, const uchar *record,
+ my_off_t filepos, int result);
+extern void myisam_log_pwrite_physical(enum myisam_log_commands command,
+ MYISAM_SHARE *share,
+ const uchar *buffert, uint length,
+ my_off_t filepos);
+extern void myisam_log_chsize_kfile_physical(MYISAM_SHARE *share,
+ my_off_t new_length);
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+static inline int32 mi_get_physical_logging_state(const MYISAM_SHARE *share)
+{
+ int32 ret;
+ my_atomic_rwlock_rdlock(&share->physical_logging_rwlock);
+ ret= my_atomic_load32(&(((MYISAM_SHARE *)share)->physical_logging));
+ my_atomic_rwlock_rdunlock(&share->physical_logging_rwlock);
+ return ret;
+}
+static inline void
+mi_set_physical_logging_state(MYISAM_SHARE *share, int32 new_state)
+{
+ my_atomic_rwlock_wrlock(&share->physical_logging_rwlock);
+ my_atomic_store32(&share->physical_logging, new_state);
+ my_atomic_rwlock_wrunlock(&share->physical_logging_rwlock);
+}
+#else
+#define mi_get_physical_logging_state(share) 0
+#define mi_set_physical_logging_state(share, new_state)
+#endif
+/**
+ IN and OUT structure for instructing how to apply a MyISAM log and later
+ getting statistics about this log.
+*/
+typedef struct mi_examine_log_param
+{
+ uint verbose, update, max_files, re_open_count, recover, prefix_remove,
+ opt_processes;
+ ulong number_of_commands;
+ my_off_t start_offset,record_pos;
+ const char *log_filename, *filepath, *write_filename, *record_pos_file;
+ /** Count of commands found in log and their errors */
+ ulong com_count[MI_LOG_END_SENTINEL][3];
+ my_bool (*table_selection_hook)(const char *); /**< to filter tables */
+} MI_EXAMINE_LOG_PARAM;
+extern void mi_examine_log_param_init(MI_EXAMINE_LOG_PARAM *param);
+extern int mi_examine_log(MI_EXAMINE_LOG_PARAM *param);
+
extern void mi_report_error(int errcode, const char *file_name);
extern my_bool _mi_memmap_file(MI_INFO *info);
extern void _mi_unmap_file(MI_INFO *info);
@@ -732,8 +823,10 @@ extern size_t mi_nommap_pread(MI_INFO *i
extern size_t mi_nommap_pwrite(MI_INFO *info, const uchar *Buffer,
size_t Count, my_off_t offset, myf MyFlags);
-uint mi_state_info_write(File file, MI_STATE_INFO *state, uint pWrite);
+uint mi_state_info_write(MYISAM_SHARE *share, File file,
+ MI_STATE_INFO *state, uint pWrite);
uchar *mi_state_info_read(uchar *ptr, MI_STATE_INFO *state);
+int mi_remap_file_and_write_state_for_unlock(MI_INFO *info);
uint mi_state_info_read_dsk(File file, MI_STATE_INFO *state, my_bool pRead);
uint mi_base_info_write(File file, MI_BASE_INFO *base);
uchar *my_n_base_info_read(uchar *ptr, MI_BASE_INFO *base);
diff -Nrup a/storage/myisam/myisamlog.c b/storage/myisam/myisamlog.c
--- a/storage/myisam/myisamlog.c 2007-08-13 15:11:17 +02:00
+++ b/storage/myisam/myisamlog.c 2007-11-06 20:56:01 +01:00
@@ -13,7 +13,15 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* write whats in isam.log */
+/**
+ @file
+ Utility to display and apply a MyISAM logical or physical log to tables.
+
+ Prints what is in a MyISAM (logical or physical/backup) log, optionally
+ applies the changes to tables (all tables or only a set specified on the
+ command line). Works standalone (tables must not be modified by the
+ server during this).
+*/
#ifndef USE_MY_FUNC
#define USE_MY_FUNC
@@ -26,60 +34,15 @@
#include <sys/resource.h>
#endif
-#define FILENAME(A) (A ? A->show_name : "Unknown")
-
-struct file_info {
- long process;
- int filenr,id;
- uint rnd;
- char *name, *show_name;
- uchar *record;
- MI_INFO *isam;
- bool closed,used;
- ulong accessed;
-};
-
-struct test_if_open_param {
- char * name;
- int max_id;
-};
-
-struct st_access_param
-{
- ulong min_accessed;
- struct file_info *found;
-};
-
#define NO_FILEPOS (ulong) ~0L
-extern int main(int argc,char * *argv);
static void get_options(int *argc,char ***argv);
-static int examine_log(char * file_name,char **table_names);
-static int read_string(IO_CACHE *file,uchar* *to,uint length);
-static int file_info_compare(void *cmp_arg, void *a,void *b);
-static int test_if_open(struct file_info *key,element_count count,
- struct test_if_open_param *param);
-static void fix_blob_pointers(MI_INFO *isam,uchar *record);
-static int test_when_accessed(struct file_info *key,element_count count,
- struct st_access_param *access_param);
-static void file_info_free(struct file_info *info);
-static int close_some_file(TREE *tree);
-static int reopen_closed_file(TREE *tree,struct file_info *file_info);
-static int find_record_with_key(struct file_info *file_info,uchar *record);
-static void printf_log(const char *str,...);
-static bool cmp_filename(struct file_info *file_info,char * name);
-
-static uint verbose=0,update=0,test_info=0,max_files=0,re_open_count=0,
- recover=0,prefix_remove=0,opt_processes=0;
-static char *log_filename=0, *filepath=0, *write_filename=0;
-static char *record_pos_file= 0;
-static ulong com_count[10][3],number_of_commands=(ulong) ~0L,
- isamlog_process;
-static my_off_t isamlog_filepos,start_offset=0,record_pos= HA_OFFSET_ERROR;
-static const char *command_name[]=
-{"open","write","update","delete","close","extra","lock","re-open",
- "delete-all", NullS};
+static my_bool matches_list_of_tables(const char *isam_file_name);
+
+static MI_EXAMINE_LOG_PARAM mi_exl;
+static char **table_names;
+static uint test_info=0;
int main(int argc, char **argv)
{
@@ -87,40 +50,60 @@ int main(int argc, char **argv)
ulong total_count,total_error,total_recover;
MY_INIT(argv[0]);
- log_filename=myisam_log_filename;
+ mi_examine_log_param_init(&mi_exl);
+ mi_exl.log_filename= myisam_logical_log_filename; /* the default */
get_options(&argc,&argv);
+ if (argv[0]) /* some table names passed on command line */
+ {
+ table_names= argv;
+ mi_exl.table_selection_hook= matches_list_of_tables;
+ }
+
/* Number of MyISAM files we can have open at one time */
- max_files= (my_set_max_open_files(min(max_files,8))-6)/2;
- if (update)
+ mi_exl.max_files= (my_set_max_open_files(max(mi_exl.max_files,8))-6)/2;
+
+ /*
+ Program must work in all conditions: support symbolic links.
+ It should not be a security risk.
+ */
+#ifdef USE_SYMDIR
+ my_use_symdir= 1;
+#endif
+
+ if (mi_exl.update)
printf("Trying to %s MyISAM files according to log '%s'\n",
- (recover ? "recover" : "update"),log_filename);
- error= examine_log(log_filename,argv);
- if (update && ! error)
+ (mi_exl.recover ? "recover" : "update"),mi_exl.log_filename);
+
+ error= mi_examine_log(&mi_exl);
+
+ if (mi_exl.update && ! error)
puts("Tables updated successfully");
total_count=total_error=total_recover=0;
- for (i=first=0 ; command_name[i] ; i++)
+ for (i=first=0 ; mi_log_command_name[i] ; i++)
{
- if (com_count[i][0])
+ if (mi_exl.com_count[i][0])
{
if (!first++)
{
- if (verbose || update)
+ if (mi_exl.verbose || mi_exl.update)
puts("");
- puts("Commands Used count Errors Recover errors");
+ puts("Commands Used count Errors"
+ " Recover errors");
}
- printf("%-12s%9ld%10ld%17ld\n",command_name[i],com_count[i][0],
- com_count[i][1],com_count[i][2]);
- total_count+=com_count[i][0];
- total_error+=com_count[i][1];
- total_recover+=com_count[i][2];
+ printf("%-20s%9ld%10ld%15ld\n", mi_log_command_name[i],
+ mi_exl.com_count[i][0],
+ mi_exl.com_count[i][1],mi_exl.com_count[i][2]);
+ total_count+=mi_exl.com_count[i][0];
+ total_error+=mi_exl.com_count[i][1];
+ total_recover+=mi_exl.com_count[i][2];
}
}
if (total_count)
printf("%-12s%9ld%10ld%17ld\n","Total",total_count,total_error,
total_recover);
- if (re_open_count)
+ if (mi_exl.re_open_count)
printf("Had to do %d re-open because of too few possibly open files\n",
- re_open_count);
+ mi_exl.re_open_count);
VOID(mi_panic(HA_PANIC_CLOSE));
my_free_open_file_info();
my_end(test_info ? MY_CHECK_ERROR | MY_GIVE_INFO : MY_CHECK_ERROR);
@@ -156,11 +139,11 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- number_of_commands=(ulong) atol(pos);
+ mi_exl.number_of_commands= (ulong) atol(pos);
pos=" ";
break;
case 'u':
- update=1;
+ mi_exl.update=1;
break;
case 'f':
if (! *++pos)
@@ -170,7 +153,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- max_files=(uint) atoi(pos);
+ mi_exl.max_files=(uint) atoi(pos);
pos=" ";
break;
case 'i':
@@ -184,7 +167,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- start_offset=(my_off_t) strtoll(pos,NULL,10);
+ mi_exl.start_offset=(my_off_t) strtoll(pos,NULL,10);
pos=" ";
break;
case 'p':
@@ -195,14 +178,14 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- prefix_remove=atoi(pos);
+ mi_exl.prefix_remove=atoi(pos);
break;
case 'r':
- update=1;
- recover++;
+ mi_exl.update=1;
+ mi_exl.recover++;
break;
case 'P':
- opt_processes=1;
+ mi_exl.opt_processes=1;
break;
case 'R':
if (! *++pos)
@@ -212,14 +195,14 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- record_pos_file=(char*) pos;
+ mi_exl.record_pos_file=(char*) pos;
if (!--*argc)
goto err;
- record_pos=(my_off_t) strtoll(*(++*argv),NULL,10);
+ mi_exl.record_pos=(my_off_t) strtoll(*(++*argv),NULL,10);
pos=" ";
break;
case 'v':
- verbose++;
+ mi_exl.verbose++;
break;
case 'w':
if (! *++pos)
@@ -229,7 +212,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- write_filename=(char*) pos;
+ mi_exl.write_filename=(char*) pos;
pos=" ";
break;
case 'F':
@@ -240,7 +223,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- filepath= (char*) pos;
+ mi_exl.filepath= (char*) pos;
pos=" ";
break;
case 'V':
@@ -249,13 +232,13 @@ static void get_options(register int *ar
case 'I':
case '?':
#include <help_start.h>
- printf("%s Ver 1.4 for %s at %s\n",my_progname,SYSTEM_TYPE,
+ printf("%s Ver 2.0 for %s at %s\n",my_progname,SYSTEM_TYPE,
MACHINE_TYPE);
puts("By Monty, for your professional use\n");
if (version)
break;
puts("Write info about whats in a MyISAM log file.");
- printf("If no file name is given %s is used\n",log_filename);
+ printf("If no file name is given %s is used\n",mi_exl.log_filename);
puts("");
printf(usage,my_progname);
puts("");
@@ -286,7 +269,7 @@ static void get_options(register int *ar
}
if (*argc >= 1)
{
- log_filename=(char*) pos;
+ mi_exl.log_filename=(char*) pos;
(*argc)--;
(*argv)++;
}
@@ -298,550 +281,17 @@ static void get_options(register int *ar
}
-static int examine_log(char * file_name, char **table_names)
-{
- uint command,result,files_open;
- ulong access_time,length;
- my_off_t filepos;
- int lock_command,mi_result;
- char isam_file_name[FN_REFLEN],llbuff[21],llbuff2[21];
- uchar head[20];
- uchar* buff;
- struct test_if_open_param open_param;
- IO_CACHE cache;
- File file;
- FILE *write_file;
- enum ha_extra_function extra_command;
- TREE tree;
- struct file_info file_info,*curr_file_info;
- DBUG_ENTER("examine_log");
-
- if ((file=my_open(file_name,O_RDONLY,MYF(MY_WME))) < 0)
- DBUG_RETURN(1);
- write_file=0;
- if (write_filename)
- {
- if (!(write_file=my_fopen(write_filename,O_WRONLY,MYF(MY_WME))))
- {
- my_close(file,MYF(0));
- DBUG_RETURN(1);
- }
- }
-
- init_io_cache(&cache,file,0,READ_CACHE,start_offset,0,MYF(0));
- bzero((uchar*) com_count,sizeof(com_count));
- init_tree(&tree,0,0,sizeof(file_info),(qsort_cmp2) file_info_compare,1,
- (tree_element_free) file_info_free, NULL);
- VOID(init_key_cache(dflt_key_cache,KEY_CACHE_BLOCK_SIZE,KEY_CACHE_SIZE,
- 0, 0));
-
- files_open=0; access_time=0;
- while (access_time++ != number_of_commands &&
- !my_b_read(&cache,(uchar*) head,9))
- {
- isamlog_filepos=my_b_tell(&cache)-9L;
- file_info.filenr= mi_uint2korr(head+1);
- isamlog_process=file_info.process=(long) mi_uint4korr(head+3);
- if (!opt_processes)
- file_info.process=0;
- result= mi_uint2korr(head+7);
- if ((curr_file_info=(struct file_info*) tree_search(&tree, &file_info,
- tree.custom_arg)))
- {
- curr_file_info->accessed=access_time;
- if (update && curr_file_info->used && curr_file_info->closed)
- {
- if (reopen_closed_file(&tree,curr_file_info))
- {
- command=sizeof(com_count)/sizeof(com_count[0][0])/3;
- result=0;
- goto com_err;
- }
- }
- }
- command=(uint) head[0];
- if (command < sizeof(com_count)/sizeof(com_count[0][0])/3 &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- {
- com_count[command][0]++;
- if (result)
- com_count[command][1]++;
- }
- switch ((enum myisam_log_commands) command) {
- case MI_LOG_OPEN:
- if (!table_names[0])
- {
- com_count[command][0]--; /* Must be counted explicite */
- if (result)
- com_count[command][1]--;
- }
-
- if (curr_file_info)
- printf("\nWarning: %s is opened with same process and filenumber\n"
- "Maybe you should use the -P option ?\n",
- curr_file_info->show_name);
- if (my_b_read(&cache,(uchar*) head,2))
- goto err;
- file_info.name=0;
- file_info.show_name=0;
- file_info.record=0;
- if (read_string(&cache,(uchar**) &file_info.name,
- (uint) mi_uint2korr(head)))
- goto err;
- {
- uint i;
- char *pos,*to;
-
- /* Fix if old DOS files to new format */
- for (pos=file_info.name; (pos=strchr(pos,'\\')) ; pos++)
- *pos= '/';
-
- pos=file_info.name;
- for (i=0 ; i < prefix_remove ; i++)
- {
- char *next;
- if (!(next=strchr(pos,'/')))
- break;
- pos=next+1;
- }
- to=isam_file_name;
- if (filepath)
- to=convert_dirname(isam_file_name,filepath,NullS);
- strmov(to,pos);
- fn_ext(isam_file_name)[0]=0; /* Remove extension */
- }
- open_param.name=file_info.name;
- open_param.max_id=0;
- VOID(tree_walk(&tree,(tree_walk_action) test_if_open,(void*) &open_param,
- left_root_right));
- file_info.id=open_param.max_id+1;
- /*
- * In the line below +10 is added to accomodate '<' and '>' chars
- * plus '\0' at the end, so that there is place for 7 digits.
- * It is improbable that same table can have that many entries in
- * the table cache.
- * The additional space is needed for the sprintf commands two lines
- * below.
- */
- file_info.show_name=my_memdup(isam_file_name,
- (uint) strlen(isam_file_name)+10,
- MYF(MY_WME));
- if (file_info.id > 1)
- sprintf(strend(file_info.show_name),"<%d>",file_info.id);
- file_info.closed=1;
- file_info.accessed=access_time;
- file_info.used=1;
- if (table_names[0])
- {
- char **name;
- file_info.used=0;
- for (name=table_names ; *name ; name++)
- {
- if (!strcmp(*name,isam_file_name))
- file_info.used=1; /* Update/log only this */
- }
- }
- if (update && file_info.used)
- {
- if (files_open >= max_files)
- {
- if (close_some_file(&tree))
- goto com_err;
- files_open--;
- }
- if (!(file_info.isam= mi_open(isam_file_name,O_RDWR,
- HA_OPEN_WAIT_IF_LOCKED)))
- goto com_err;
- if (!(file_info.record=my_malloc(file_info.isam->s->base.reclength,
- MYF(MY_WME))))
- goto end;
- files_open++;
- file_info.closed=0;
- }
- VOID(tree_insert(&tree, (uchar*) &file_info, 0, tree.custom_arg));
- if (file_info.used)
- {
- if (verbose && !record_pos_file)
- printf_log("%s: open -> %d",file_info.show_name, file_info.filenr);
- com_count[command][0]++;
- if (result)
- com_count[command][1]++;
- }
- break;
- case MI_LOG_CLOSE:
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s -> %d",FILENAME(curr_file_info),
- command_name[command],result);
- if (curr_file_info)
- {
- if (!curr_file_info->closed)
- files_open--;
- VOID(tree_delete(&tree, (uchar*) curr_file_info, 0, tree.custom_arg));
- }
- break;
- case MI_LOG_EXTRA:
- if (my_b_read(&cache,(uchar*) head,1))
- goto err;
- extra_command=(enum ha_extra_function) head[0];
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s(%d) -> %d",FILENAME(curr_file_info),
- command_name[command], (int) extra_command,result);
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (mi_extra(curr_file_info->isam, extra_command, 0) != (int) result)
- {
- fflush(stdout);
- VOID(fprintf(stderr,
- "Warning: error %d, expected %d on command %s at %s\n",
- my_errno,result,command_name[command],
- llstr(isamlog_filepos,llbuff)));
- fflush(stderr);
- }
- }
- break;
- case MI_LOG_DELETE:
- if (my_b_read(&cache,(uchar*) head,8))
- goto err;
- filepos=mi_sizekorr(head);
- if (verbose && (!record_pos_file ||
- ((record_pos == filepos || record_pos == NO_FILEPOS) &&
- !cmp_filename(curr_file_info,record_pos_file))) &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s at %ld -> %d",FILENAME(curr_file_info),
- command_name[command],(long) filepos,result);
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
- {
- if (!recover)
- goto com_err;
- if (verbose)
- printf_log("error: Didn't find row to delete with mi_rrnd");
- com_count[command][2]++; /* Mark error */
- }
- mi_result=mi_delete(curr_file_info->isam,curr_file_info->record);
- if ((mi_result == 0 && result) ||
- (mi_result && (uint) my_errno != result))
- {
- if (!recover)
- goto com_err;
- if (mi_result)
- com_count[command][2]++; /* Mark error */
- if (verbose)
- printf_log("error: Got result %d from mi_delete instead of %d",
- mi_result, result);
- }
- }
- break;
- case MI_LOG_WRITE:
- case MI_LOG_UPDATE:
- if (my_b_read(&cache,(uchar*) head,12))
- goto err;
- filepos=mi_sizekorr(head);
- length=mi_uint4korr(head+8);
- buff=0;
- if (read_string(&cache,&buff,(uint) length))
- goto err;
- if ((!record_pos_file ||
- ((record_pos == filepos || record_pos == NO_FILEPOS) &&
- !cmp_filename(curr_file_info,record_pos_file))) &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- {
- if (write_file &&
- (my_fwrite(write_file,buff,length,MYF(MY_WAIT_IF_FULL | MY_NABP))))
- goto end;
- if (verbose)
- printf_log("%s: %s at %ld, length=%ld -> %d",
- FILENAME(curr_file_info),
- command_name[command], filepos,length,result);
- }
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (curr_file_info->isam->s->base.blobs)
- fix_blob_pointers(curr_file_info->isam,buff);
- if ((enum myisam_log_commands) command == MI_LOG_UPDATE)
- {
- if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
- {
- if (!recover)
- {
- result=0;
- goto com_err;
- }
- if (verbose)
- printf_log("error: Didn't find row to update with mi_rrnd");
- if (recover == 1 || result ||
- find_record_with_key(curr_file_info,buff))
- {
- com_count[command][2]++; /* Mark error */
- break;
- }
- }
- mi_result=mi_update(curr_file_info->isam,curr_file_info->record,
- buff);
- if ((mi_result == 0 && result) ||
- (mi_result && (uint) my_errno != result))
- {
- if (!recover)
- goto com_err;
- if (verbose)
- printf_log("error: Got result %d from mi_update instead of %d",
- mi_result, result);
- if (mi_result)
- com_count[command][2]++; /* Mark error */
- }
- }
- else
- {
- mi_result=mi_write(curr_file_info->isam,buff);
- if ((mi_result == 0 && result) ||
- (mi_result && (uint) my_errno != result))
- {
- if (!recover)
- goto com_err;
- if (verbose)
- printf_log("error: Got result %d from mi_write instead of %d",
- mi_result, result);
- if (mi_result)
- com_count[command][2]++; /* Mark error */
- }
- if (!recover && filepos != curr_file_info->isam->lastpos)
- {
- printf("error: Wrote at position: %s, should have been %s",
- llstr(curr_file_info->isam->lastpos,llbuff),
- llstr(filepos,llbuff2));
- goto end;
- }
- }
- }
- my_free(buff,MYF(0));
- break;
- case MI_LOG_LOCK:
- if (my_b_read(&cache,(uchar*) head,sizeof(lock_command)))
- goto err;
- memcpy_fixed(&lock_command,head,sizeof(lock_command));
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s(%d) -> %d\n",FILENAME(curr_file_info),
- command_name[command],lock_command,result);
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (mi_lock_database(curr_file_info->isam,lock_command) !=
- (int) result)
- goto com_err;
- }
- break;
- case MI_LOG_DELETE_ALL:
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s -> %d\n",FILENAME(curr_file_info),
- command_name[command],result);
- break;
- default:
- fflush(stdout);
- VOID(fprintf(stderr,
- "Error: found unknown command %d in logfile, aborted\n",
- command));
- fflush(stderr);
- goto end;
- }
- }
- end_key_cache(dflt_key_cache,1);
- delete_tree(&tree);
- VOID(end_io_cache(&cache));
- VOID(my_close(file,MYF(0)));
- if (write_file && my_fclose(write_file,MYF(MY_WME)))
- DBUG_RETURN(1);
- DBUG_RETURN(0);
-
- err:
- fflush(stdout);
- VOID(fprintf(stderr,"Got error %d when reading from logfile\n",my_errno));
- fflush(stderr);
- goto end;
- com_err:
- fflush(stdout);
- VOID(fprintf(stderr,"Got error %d, expected %d on command %s at %s\n",
- my_errno,result,command_name[command],
- llstr(isamlog_filepos,llbuff)));
- fflush(stderr);
- end:
- end_key_cache(dflt_key_cache, 1);
- delete_tree(&tree);
- VOID(end_io_cache(&cache));
- VOID(my_close(file,MYF(0)));
- if (write_file)
- VOID(my_fclose(write_file,MYF(MY_WME)));
- DBUG_RETURN(1);
-}
-
-
-static int read_string(IO_CACHE *file, register uchar* *to, register uint length)
-{
- DBUG_ENTER("read_string");
-
- if (*to)
- my_free((uchar*) *to,MYF(0));
- if (!(*to= (uchar*) my_malloc(length+1,MYF(MY_WME))) ||
- my_b_read(file,(uchar*) *to,length))
- {
- if (*to)
- my_free(*to,MYF(0));
- *to= 0;
- DBUG_RETURN(1);
- }
- *((char*) *to+length)= '\0';
- DBUG_RETURN (0);
-} /* read_string */
-
-
-static int file_info_compare(void* cmp_arg __attribute__((unused)),
- void *a, void *b)
-{
- long lint;
-
- if ((lint=((struct file_info*) a)->process -
- ((struct file_info*) b)->process))
- return lint < 0L ? -1 : 1;
- return ((struct file_info*) a)->filenr - ((struct file_info*) b)->filenr;
-}
-
- /* ARGSUSED */
-
-static int test_if_open (struct file_info *key,
- element_count count __attribute__((unused)),
- struct test_if_open_param *param)
-{
- if (!strcmp(key->name,param->name) && key->id > param->max_id)
- param->max_id=key->id;
- return 0;
-}
-
-
-static void fix_blob_pointers(MI_INFO *info, uchar *record)
-{
- uchar *pos;
- MI_BLOB *blob,*end;
-
- pos=record+info->s->base.reclength;
- for (end=info->blobs+info->s->base.blobs, blob= info->blobs;
- blob != end ;
- blob++)
- {
- memcpy_fixed(record+blob->offset+blob->pack_length,&pos,sizeof(char*));
- pos+=_mi_calc_blob_length(blob->pack_length,record+blob->offset);
- }
-}
-
- /* close the file with hasn't been accessed for the longest time */
- /* ARGSUSED */
-
-static int test_when_accessed (struct file_info *key,
- element_count count __attribute__((unused)),
- struct st_access_param *access_param)
+static my_bool matches_list_of_tables(const char *isam_file_name)
{
- if (key->accessed < access_param->min_accessed && ! key->closed)
+ if (table_names && table_names[0])
{
- access_param->min_accessed=key->accessed;
- access_param->found=key;
- }
- return 0;
-}
-
-
-static void file_info_free(struct file_info *fileinfo)
-{
- DBUG_ENTER("file_info_free");
- if (update)
- {
- if (!fileinfo->closed)
- VOID(mi_close(fileinfo->isam));
- if (fileinfo->record)
- my_free(fileinfo->record,MYF(0));
- }
- my_free(fileinfo->name,MYF(0));
- my_free(fileinfo->show_name,MYF(0));
- DBUG_VOID_RETURN;
-}
-
-
-
-static int close_some_file(TREE *tree)
-{
- struct st_access_param access_param;
-
- access_param.min_accessed=LONG_MAX;
- access_param.found=0;
-
- VOID(tree_walk(tree,(tree_walk_action) test_when_accessed,
- (void*) &access_param,left_root_right));
- if (!access_param.found)
- return 1; /* No open file that is possibly to close */
- if (mi_close(access_param.found->isam))
- return 1;
- access_param.found->closed=1;
- return 0;
-}
-
-
-static int reopen_closed_file(TREE *tree, struct file_info *fileinfo)
-{
- char name[FN_REFLEN];
- if (close_some_file(tree))
- return 1; /* No file to close */
- strmov(name,fileinfo->show_name);
- if (fileinfo->id > 1)
- *strrchr(name,'<')='\0'; /* Remove "<id>" */
-
- if (!(fileinfo->isam= mi_open(name,O_RDWR,HA_OPEN_WAIT_IF_LOCKED)))
- return 1;
- fileinfo->closed=0;
- re_open_count++;
- return 0;
-}
-
- /* Try to find record with uniq key */
-
-static int find_record_with_key(struct file_info *file_info, uchar *record)
-{
- uint key;
- MI_INFO *info=file_info->isam;
- uchar tmp_key[MI_MAX_KEY_BUFF];
-
- for (key=0 ; key < info->s->base.keys ; key++)
- {
- if (mi_is_key_active(info->s->state.key_map, key) &&
- info->s->keyinfo[key].flag & HA_NOSAME)
+ char **name;
+ for (name= table_names ; *name ; name++)
{
- VOID(_mi_make_key(info,key,tmp_key,record,0L));
- return mi_rkey(info,file_info->record,(int) key,tmp_key,0,
- HA_READ_KEY_EXACT);
+ if (!strcmp(*name, isam_file_name))
+ return 1;
}
+ return 0;
}
return 1;
-}
-
-
-static void printf_log(const char *format,...)
-{
- char llbuff[21];
- va_list args;
- va_start(args,format);
- if (verbose > 2)
- printf("%9s:",llstr(isamlog_filepos,llbuff));
- if (verbose > 1)
- printf("%5ld ",isamlog_process); /* Write process number */
- (void) vprintf((char*) format,args);
- putchar('\n');
- va_end(args);
-}
-
-
-static bool cmp_filename(struct file_info *file_info, char * name)
-{
- if (!file_info)
- return 1;
- return strcmp(file_info->name,name) ? 1 : 0;
}
diff -Nrup a/storage/myisam/myisampack.c b/storage/myisam/myisampack.c
--- a/storage/myisam/myisampack.c 2007-08-13 15:11:17 +02:00
+++ b/storage/myisam/myisampack.c 2007-11-06 20:56:01 +01:00
@@ -506,9 +506,12 @@ static int compress(PACK_MRG_INFO *mrg,c
/* Create temporary or join file */
if (backup)
- VOID(fn_format(org_name,isam_file->filename,"",MI_NAME_DEXT,2));
+ VOID(fn_format(org_name,isam_file->s->unresolv_file_name,
+ "", MI_NAME_DEXT, MY_REPLACE_EXT));
else
- VOID(fn_format(org_name,isam_file->filename,"",MI_NAME_DEXT,2+4+16));
+ VOID(fn_format(org_name,isam_file->s->unresolv_file_name,
+ "", MI_NAME_DEXT,
+ MY_REPLACE_EXT|MY_UNPACK_FILENAME|MY_RESOLVE_SYMLINKS));
if (!test_only && result_table)
{
/* Make a new indexfile based on first file in list */
@@ -693,7 +696,9 @@ static int compress(PACK_MRG_INFO *mrg,c
{
if (backup)
{
- if (my_rename(org_name,make_old_name(temp_name,isam_file->filename),
+ if (my_rename(org_name,
+ make_old_name(temp_name,
+ isam_file->s->unresolv_file_name),
MYF(MY_WME)))
error=1;
else
@@ -2977,7 +2982,7 @@ static int save_state(MI_INFO *isam_file
VOID(my_chsize(share->kfile, share->base.keystart, 0, MYF(0)));
if (share->base.keys)
isamchk_neaded=1;
- DBUG_RETURN(mi_state_info_write(share->kfile,&share->state,1+2));
+ DBUG_RETURN(mi_state_info_write(share, share->kfile, &share->state, 1+2));
}
@@ -3010,7 +3015,7 @@ static int save_state_mrg(File file,PACK
if (isam_file->s->base.keys)
isamchk_neaded=1;
state.changed=STATE_CHANGED | STATE_NOT_ANALYZED; /* Force check of table */
- DBUG_RETURN (mi_state_info_write(file,&state,1+2));
+ DBUG_RETURN (mi_state_info_write(isam_file->s, file, &state, 1+2));
}
diff -Nrup a/storage/myisammrg/ha_myisammrg.cc b/storage/myisammrg/ha_myisammrg.cc
--- a/storage/myisammrg/ha_myisammrg.cc 2007-08-13 15:11:17 +02:00
+++ b/storage/myisammrg/ha_myisammrg.cc 2007-11-06 20:56:01 +01:00
@@ -121,7 +121,7 @@ int ha_myisammrg::open(const char *name,
DBUG_PRINT("error",("reclength: %lu mean_rec_length: %lu",
table->s->reclength, stats.mean_rec_length));
if (test_if_locked & HA_OPEN_FOR_REPAIR)
- myrg_print_wrong_table(file->open_tables->table->filename);
+ myrg_print_wrong_table(file->open_tables->table->s->unresolv_file_name);
error= HA_ERR_WRONG_MRG_TABLE_DEF;
goto err;
}
@@ -142,7 +142,7 @@ int ha_myisammrg::open(const char *name,
{
error= HA_ERR_WRONG_MRG_TABLE_DEF;
if (test_if_locked & HA_OPEN_FOR_REPAIR)
- myrg_print_wrong_table(u_table->table->filename);
+ myrg_print_wrong_table(u_table->table->s->unresolv_file_name);
else
{
my_free((uchar*) recinfo, MYF(0));
@@ -484,7 +484,7 @@ void ha_myisammrg::update_create_info(HA
if (!(ptr = (TABLE_LIST *) thd->calloc(sizeof(TABLE_LIST))))
goto err;
- split_file_name(open_table->table->filename, &db, &name);
+ split_file_name(open_table->table->s->unresolv_file_name, &db, &name);
if (!(ptr->table_name= thd->strmake(name.str, name.length)))
goto err;
if (db.length && !(ptr->db= thd->strmake(db.str, db.length)))
@@ -593,7 +593,7 @@ void ha_myisammrg::append_create_info(St
LEX_STRING db, name;
LINT_INIT(db.str);
- split_file_name(open_table->table->filename, &db, &name);
+ split_file_name(open_table->table->s->unresolv_file_name, &db, &name);
if (open_table != first)
packet->append(',');
/* Report database for mapped table if it isn't in current database */
diff -Nrup a/storage/myisammrg/myrg_info.c b/storage/myisammrg/myrg_info.c
--- a/storage/myisammrg/myrg_info.c 2006-12-31 01:06:41 +01:00
+++ b/storage/myisammrg/myrg_info.c 2007-11-06 20:56:01 +01:00
@@ -50,7 +50,8 @@ int myrg_status(MYRG_INFO *info,register
info->records+=file->table->s->state.state.records;
info->del+=file->table->s->state.state.del;
DBUG_PRINT("info2",("table: %s, offset: %lu",
- file->table->filename,(ulong) file->file_offset));
+ file->table->s->unresolv_file_name,
+ (ulong) file->file_offset));
}
x->records= info->records;
x->deleted= info->del;
diff -Nrup a/storage/myisammrg/myrg_rrnd.c b/storage/myisammrg/myrg_rrnd.c
--- a/storage/myisammrg/myrg_rrnd.c 2007-05-10 11:59:33 +02:00
+++ b/storage/myisammrg/myrg_rrnd.c 2007-11-06 20:56:01 +01:00
@@ -111,6 +111,6 @@ static MYRG_TABLE *find_table(MYRG_TABLE
start=mid;
}
DBUG_PRINT("info",("offset: %lu, table: %s",
- (ulong) pos, start->table->filename));
+ (ulong) pos, start->table->s->unresolv_file_name));
DBUG_RETURN(start);
}
