Below is the list of changes that have just been committed into a local
6.0 repository of istruewing. When istruewing does a push these changes
will be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html
ChangeSet@stripped, 2008-05-07 17:31:01+02:00, istruewing@stripped +69 -0
WL#866 - Online backup driver for MyISAM
Not to be pushed. This is just an intermediate commit.
Patch from Guilhem Bichot ported to a current backup kernel.
When this driver (file myisam_backup_engine.cc) is to do a backup,
it starts physical logging of changes for target tables (file
mi_log.c), dirtily copies tables, then the physical log.
When this driver does a restore, it copies back tables and the log,
applies the log (file mi_examine_log.c).
The data file is always backed up; for the index file two methods are
available: either only its 64KB header is copied and then restore will
repair indexes, or the whole index file is copied.
Backup starts and runs without disturbing any running or new update
except at the very end: when creating the validity point it locks all
backed-up MyISAM tables with a read lock (so, stalls new update
statements and waits for running update statements to finish).
I recommend to readers to read in order myisam_backup_engine.cc,
mi_log.c then other files.
GUILHEM_TODO tag identifies the most urgent todo.
Fixes for misc bugs found along the way of development:
missing initialization in Bitmap<64>, issues with the dbug library.
---
WL#866 "myisam online backup driver" fix for compiler warnings/errors
---
WL#866 "online backup for MyISAM": Windows compiler/test fixes
---
WL#866 "online backup driver for MyISAM" gcc warnings fixes
---
WL#866 "MyISAM online backup driver": give error if the backup archive
which kernel asks us to restore, is more a too recent version.
This prepares for the future.
---
Merge stella.local:/home2/mydev/mysql-6.0-wl4279
into stella.local:/home2/mydev/mysql-6.0-backup-wl866
dbug/dbug.c@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +1 -0
bugfix provided by Serg, for when doing "SET DEBUG=-d,etc"
(process hung).
include/atomic/generic-msvc.h@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +11 -6
Interlocked* functions use LONG as 32-bit integer type; my_atomic.h
rather provides int32 (=int on Windows), int* does not naturally
cast to LONG* and build fails; adding explicit casts.
include/atomic/nolock.h@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +4 -1
comment
include/keycache.h@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +11 -2
A block now has an optional post_write_arg. A key cache has a global
post_write function which is called after any block is written to its file.
The block's post_write_arg is set through key_cache_write(). The
key cache's post_write is set when opening MyISAM tables.
This is used by MyISAM's physical logging: when a key page
is flushed to the file, the write is also recorded in the physical log,
thanks to such callback.
include/my_global.h@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +5 -3
When testing a C++ file against "gcc -Wold-style-cast", these casts
surfaced. STATIC_CAST means "a static cast which works in
C and in C++, without -Wold-style-cast warnings".
include/my_sys.h@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +27 -11
New members of IO_CACHE:
* post_write (called by my_b_flush_io_cache()),
* hard_write_error_in_the_past: like the existing 'error' except that
once set it is not reset until cache is reinitialized; 'error' is
reset at next _my_b_write() call for example. Is useful when we want
to lazily (once in a while) monitor if an IO_CACHE got a write error
(and so, file is not usable) instead of monitoring each IO_CACHE write.
These are used by MyISAM's online backup (myisam_backup_engine.cc)
to test if the physical log got a write error.
include/myisam.h@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +17 -3
declarations' update.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +1 -0
don't pollute rest of the calling test by keeping a lock.
mysql-test/mysql-test-run.pl@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +22 -0
ability to call myisamlog from tests (for new test t/myisamlog.test)
---
look for myisamlog in storage/myisam/(debug|release)
mysql-test/r/backup_myisam1.result@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +7 -0
result
mysql-test/r/backup_myisam1.result@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/r/backup_myisam2.result@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +61 -0
result is ok: checksums match.
mysql-test/r/backup_myisam2.result@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/r/backup_no_be.result@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +1 -1
We are using the native MyISAM backup driver now.
mysql-test/r/myisamlog.result@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +48 -0
result
---
independent of directory
mysql-test/r/myisamlog.result@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +1 -0
result update
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +1 -0
result update
mysql-test/t/backup_myisam1-master.opt@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +1 -0
test with external locking
mysql-test/t/backup_myisam1-master.opt@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/t/backup_myisam1.test@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +14 -0
test that backup fails with external locking
mysql-test/t/backup_myisam1.test@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/t/backup_myisam2.test@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +66 -0
test of records >64k (see MI_LOG_BIG_NUMBERS in mi_log.c)
mysql-test/t/backup_myisam2.test@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/t/key_cache.test@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +3 -3
As the size of the BLOCK_LINK structure has grown (to accomodate
the new post_write_arg member), the number
of available blocks for a fixed key cache size has decreased
(a similar fix will be needed for 64-bit systems too)
---
fix for 64-bit pointers (one pointer was added to the key cache
block structure, so less blocks)
mysql-test/t/myisamlog-master.opt@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +1 -0
To test the MyISAM logical log, which cannot be enabled on the fly.
mysql-test/t/myisamlog-master.opt@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysql-test/t/myisamlog.test@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +37 -0
Test of MyISAM logical logging and the myisamlog utility:
see if they can repopulate a table.
---
independent of directory
mysql-test/t/myisamlog.test@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
mysys/CMakeLists.txt@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +1 -1
include my_dup.c in mysys under Windows
mysys/mf_iocache.c@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +53 -19
Calling the new post_write hook. Note that it does not work with
SEQ_READ_APPEND IO_CACHEs (we don't need a post_write for them now).
Setting IO_CACHE::hard_write_error_in_the_past at the same time we
set IO_CACHE::error to -1 in functions which do writes.
IO_CACHE::hard_write_error_in_the_past is however not reset for
each write, that's its difference from IO_CACHE::error.
mysys/mf_keycache.c@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +60 -17
A block now has an optional post_write_arg which is used by
keycache->post_write after flushing the block.
We make sure that all pwrite() call keycache->post_write, by replacing
them by a new key_cache_pwrite().
This is used by MyISAM's physical logging.
mysys/my_thr_init.c@stripped, 2008-05-07 17:30:57+02:00, istruewing@stripped +15 -4
A dedicated mutex for the MyISAM log (better not use THR_LOCK_myisam
which is already used at every mi_open()/mi_close(), and also it made
coding easier, see comments in _myisam_log_command()).
In my_thread_end(), do a DBUG_POP() so that if the thread has used
SET DEBUG=, the memory allocated by that SET is freed.
sql/backup/stream_v1_transport.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +2 -1
Fix an inifinite loop. EOS went undetected before.
sql/item_func.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +22 -7
debug_sync_point() implicitely releases any user-level lock held
by the current thread (like GET_LOCK()) ones; this is incorrect for
testing: assert that it does not hold a lock, and a comment to suggest
proper usage.
sql/mysqld.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +7 -7
emphasizing that the MyISAM log set from the command line is the
logical one.
sql/sql_bitmap.h@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +2 -2
Fix for valgrind error (unrelated to MyISAM online backup driver):
Bitmap<64> needs to initialize in its constructor, like the any-other-size
Bitmap does. Bug specific of 6.0.
sql/sql_cache.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +3 -2
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
sql/sql_class.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +3 -2
For the "locking threads" created by some online backup drivers
(MyISAM one and default lock-based one), THD::awake() must abort locks.
That if() dates from a time where the only possible system threads
were the delayed insert system thread and the slave threads.
sql/sql_load.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +1 -2
cast not needed
sql/sql_repl.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +4 -1
new prototype
sql/sql_repl.h@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +1 -1
new prototype
sql/sql_table.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +3 -0
As start_bulk_insert() has been called, end_bulk_insert() must be
called (bug found thanks to new assertion in mi_locking.c)
storage/myisam/CMakeLists.txt@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +1 -1
include new files in build
storage/myisam/Makefile.am@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +1 -1
include new files in build
storage/myisam/ha_myisam.cc@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +12 -7
MyISAM now has an online backup driver, see myisam_backup_engine.cc.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name.
Unset STATE_BAD_OPEN_COUNT after a successful check or repair.
Fix for doxygen.err warnings.
storage/myisam/ha_myisam.h@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +4 -0
MyISAM now has an online backup driver
storage/myisam/mi_check.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +83 -11
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging.
Note: REPAIR/OPTIMIZE TABLE are not handled by
physical logging; they should be blocked by the MySQL online backup
kernel: when they delete/create/rename files, they are as much
a problem as ALTER TABLE. So we assert for these cases.
At end of INSERT SELECT, if the table was originally empty,
an index rebuild is done. This uses my_pwrite() and my_chsize(),
which we thus have to write to the physical log.
storage/myisam/mi_close.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +12 -2
If we failed to flush the key cache, we need to mark the table as
corrupted (fix for an unlikely bug).
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging.
If the table has logged a MI_LOG_OPEN to the physical log (because it
has logged some writes there), it needs to log a MI_LOG_CLOSE too.
Regarding the added assertion, see revision comment in mi_locking.c.
storage/myisam/mi_create.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +9 -1
Ensure that no table is truncated if it is doing physical
logging. We *could* support TRUNCATE during an online backup of
MyISAM (switch off HTON_CAN_RECREATE during backup) but:
- it would require some synchro (no backup can start while recreate-based
truncate is running, etc)
- consistent-snapshot driver can't bear TRUNCATE anyway.
Another possibility is also to always switch off HTON_CAN_RECREATE.
storage/myisam/mi_delete.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +7 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_delete_all.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +4 -1
Log to the physical log the MI_DELETE_ALL operation
storage/myisam/mi_dynrec.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +23 -13
Log to the physical log the operation of writing bytes to the data file.
Always log _after_ the write (see comment at start of
mi_log_start_physical()).
storage/myisam/mi_examine_log.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +871 -0
This is the examine_log() function taken out of myisamlog.c, as we
now need it in the server too, to restore from an online backup.
It is renamed to mi_examine_log().
examine_log() used global variables of myisamlog.c, that is not possible
now so mi_examine_log() takes in parameter a new structure
MI_EXAMINE_LOG_PARAM whose content corresponds to the old global
variables.
The function is extended to understand new log commands found in
physical logs: MI_LOG_WRITE_BYTES_MYI, MI_LOG_WRITE_BYTES_MYD,
MI_LOG_CHSIZE_MYI, as well as MI_LOG_DELETE_ALL which wasn't replayed
properly (a bug).
Other bugfix: NO_FILEPOS was too small for 64-bit machines.
Other bugfix: MI_LOG_LOCK applying could not work because it didn't
remove the 'flag' added at the end of mi_lock_database().
To see the real code changes I made (because, most of lines are just
moved from myisamlog.c), you should diff mi_examine_log() with
examine_log() from the old myisamlog.c (I recommend it).
storage/myisam/mi_examine_log.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
storage/myisam/mi_extra.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +59 -3
When we set up a write cache (HA_EXTRA_WRITE_CACHE) we give the cache
a post_write call, which will make sure that writes to the data file
will go to the physical log if needed.
It is more efficient that logging all my_b_write()s as it generates
less log records, and anyway we needed the post_write callback for
when logging is enabled while the table is in already the middle of
using a write cache, when non-logged my_b_write()s have passed already.
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging
storage/myisam/mi_locking.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +89 -46
Comments explaining why certain writes to the index file needn't go into
the physical log.
New function mi_remap_file_and_write_state_for_unlock(), used when
unlocking a table and in mi_backup_stop_logging_for_tables().
If logical log, we now always log MI_LOG_LOCK. Indeed, without it,
the MI_LOG_EXTRA failed to apply when command was HA_EXTRA_CACHE (see
the EACCES error in mi_extra():
missing MI_LOG_LOCK => info->lock_type==F_UNLCK => applying of
MI_LOG_EXTRA failed (so myisamlog.test got such warnings in
mysqltest.log: "Warning: error 13, expected 0 on command extra at 211").
We also make the storage of lock_type portable (independent of
endianess).
storage/myisam/mi_log.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +792 -79
mi_log() can now open a logical (==debug)or physical (==backup) log:
it thus receives new parameters (list of tables...); it can
close logs too. Opening and closing a physical log is more work
than a logical one, so additional functions are called.
Logging functions now use IO_CACHE instead of my_write(), to be
faster.
_myisam_log() merged into _myisam_log_command() (they were very
similar).
If _myisam_log_command() uses small numbers (file descriptor,
file offset buffer length) it stores them in few bytes; if they are big
it stores them in more bytes; how many bytes are used is denoted
by the highest bit of the 'command' (MI_LOG_BIG_NUMBERS).
That and the merge of _myisam_log() and _myisam_log_command()
imply that old myisamlog won't read new logical log, but it's ok: the
MyISAM log was so far only for debugging, advertised as such,
apparently used only by MySQL devs, and had a bug (MI_LOG_DELETE_ALL
was not replayed by myisamlog) which nobody reported.
_myisam_log_record() renamed to _myisam_log_record_logical()
(used only by logical logging).
Logging functions now need to check if the log is opened _inside the
log's mutex_, because the physical log can close at any time (while
the old logical log, was either open or closed for the
lifetime of the MySQL server process).
_myisam_log_command() may log MI_LOG_OPEN on top of its command
(needed for physical logging), if this is a physical log and
the first time this table's share writes to it.
_myisam_log_record() is used only by logical logging, renamed it.
---
compiler warnings
---
fix for compiler warnings -Wuninitialized
storage/myisam/mi_open.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +114 -25
When opening a table, detect if it must do physical logging;
and remember with STATE_BAD_OPEN_COUNT if open_count>0 at first
open (this way, online backup can know if the open_count>0 which it
sees is a real problem or not); set key cache's post_write.
mi_state_info_write() needs the MYISAM_SHARE now, as it needs to
record the my_pwrite() done to the index file, into the physical log.
Note, for now all calls to mi_state_info_write() have the state
equal to the share's state but future projects may not have that,
that's why the two arguments stay.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name,
indeed physical logging needs this unresolved name, and
sometimes cannot have MI_INFO available, but always has MYISAM_SHARE
(see log_key_cache_flush_physical()). It is also more
efficient (why duplicate the same string in all MI_INFOs for a same
table?).
---
compiler warnings
storage/myisam/mi_page.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +17 -15
passing "share" as post_write_arg to key_cache_write()
storage/myisam/mi_panic.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +11 -7
update for new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_rrnd.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +1 -3
cosmetic change
storage/myisam/mi_static.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +38 -5
New physical log in MyISAM. MyISAM-specific error messages.
---
error message for when backup archive's format is too recent
storage/myisam/mi_test2.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +3 -3
update for new prototype. Complement to the fix of BUG#30094
(this complement has been approved by Serg and pushed in another
tree months ago); the bug caused random assertion failures.
storage/myisam/mi_test3.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +2 -2
update for new prototype
storage/myisam/mi_test_all.sh@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +2 -0
We need to remove logs from previous "mi_test_all" runs, or a bug
in writing myisam.log would continue to make the test fail even
though bug is fixed (mi_test2 would just append to the bad log).
storage/myisam/mi_update.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +6 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_write.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +5 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/myisam_backup_engine.cc@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +1934 -0
MyISAM online backup engine. See this file's first lines (namespace
myisam_backup) for a description of how it works.
This is the file which really drives the backup/restore inside MyISAM
(see it as the master which calls functions from other files to
perform some subtasks).
Note that Chuck has taken some code from this file and it is now
duplicated into sql/backup/ files, I should merge the two
to avoid duplication, when Chuck's work is pushed
---
compiler warnings
---
Give error message if the backup archive which the kernel asks
us to restore is of a more recent version that ourselves (we cannot
parse it).
storage/myisam/myisam_backup_engine.cc@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +0 -0
storage/myisam/myisam_ftdump.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +2 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/myisamchk.c@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +4 -2
STATE_BAD_OPEN_COUNT reset on repair
storage/myisam/myisamdef.h@stripped, 2008-05-07 17:30:58+02:00, istruewing@stripped +135 -19
Adding member "physical_logging" to MYISAM_SHARE, tells if the table
is currently doing physical logging or not.
It is read and set with atomic operations: this provides the needed
synchronization between the table-writer thread (if the table is being
written now) (which is the reader of "physical_logging") and the thread
turning physical logging on or off (which is the writer of
"physical_logging").
Adding member "MI_LOG_OPEN_stored_in_physical_log" to MYISAM_SHARE, to
remember if we already stored MI_LOG_OPEN in this physical log for this
table.
A new mutex THR_LOCK_myisam_log to protect MyISAM logs (instead of
THR_LOCK_myisam, for concurrency and coding reasons (explained
in _myisam_log_command()).
New commands which can be stored only in the physical log:
MI_LOG_WRITE_BYTES_MYD, MI_LOG_WRITE_BYTES_MYI, MI_LOG_CHSIZE_MYI.
Updates to myisam_log_* macros to adapt to the fact that logs are now
an IO_CACHE and not a plain OS file.
As most of the logic of myisamlog moved out of myisamlog.c into
mi_examine_log.c, mi_examine_log() cannot use globals of myisamlog.c
anymore; a new structure MI_EXAMINE_LOG_PARAM is introduced instead
(like MI_CHECK). mi_state_info_write() needs the MYISAM_SHARE now, as
it is used by physical logging.
Flag STATE_BAD_OPEN_COUNT.
Note: all extern vars touched in this file have their documentation
in the file where they are defined.
---
compiler warnings
---
'extern "C"' needed for what is used inside myisam_backup_engine.cc.
---
error for when backup archive's format is too recent
storage/myisam/myisamlog.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +74 -624
The main function of this file, examine_log(), is now needed in
the server to be able to do a restore from an online backup,
so moves to mi_examine_log.c.
I changed a meaningless min() to max().
New log commands (MI_LOG_etc) have long names so I widened the
space for displaying their name to 20 characters.
Due to backward-incompatibilities (see mi_log.c) we bump version
number to 2.0.
---
Manual merge
storage/myisam/myisampack.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +10 -5
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name.
mi_state_info_write() needs MYISAM_SHARE as physical logging does.
storage/myisammrg/ha_myisammrg.cc@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +5 -5
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_info.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +2 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_open.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +3 -3
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_rrnd.c@stripped, 2008-05-07 17:30:59+02:00, istruewing@stripped +1 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
diff -Nrup a/dbug/dbug.c b/dbug/dbug.c
--- a/dbug/dbug.c 2008-03-28 19:03:59 +01:00
+++ b/dbug/dbug.c 2008-05-07 17:30:57 +02:00
@@ -1321,6 +1321,7 @@ static struct link *ListDel(struct link
free((void*) delme);
}
} while (*cur && *(cur=&((*cur)->next_link)));
+ ctlp++;
}
return head;
}
diff -Nrup a/include/atomic/generic-msvc.h b/include/atomic/generic-msvc.h
--- a/include/atomic/generic-msvc.h 2007-10-09 19:55:11 +02:00
+++ b/include/atomic/generic-msvc.h 2008-05-07 17:30:57 +02:00
@@ -21,25 +21,30 @@
intrinsics.
*/
#define MY_ATOMIC_MODE "msvc-intrinsics"
+/* the 32-bit type for Interlocked functions is LONG */
+#define IL_EXCHG_TYPE32 LONG
+#define IL_EXCHG_TYPEptr PVOID
#define IL_EXCHG_ADD32 InterlockedExchangeAdd
#define IL_COMP_EXCHG32 InterlockedCompareExchange
#define IL_COMP_EXCHGptr InterlockedCompareExchangePointer
#define IL_EXCHG32 InterlockedExchange
#define IL_EXCHGptr InterlockedExchangePointer
#define make_atomic_add_body(S) \
- v= IL_EXCHG_ADD ## S (a, v)
+ v= (int ## S)IL_EXCHG_ADD ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ## S)(v))
#define make_atomic_cas_body(S) \
- int ## S initial_cmp= *cmp; \
- int ## S initial_a= IL_COMP_EXCHG ## S (a, set, initial_cmp); \
- if (!(ret= (initial_a == initial_cmp))) *cmp= initial_a;
+ IL_EXCHG_TYPE ## S initial_cmp= *cmp; \
+ IL_EXCHG_TYPE ## S initial_a= IL_COMP_EXCHG ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ## S)(set), initial_cmp); \
+ if (!(ret= (initial_a == initial_cmp))) *cmp= (int ## S)initial_a;
#define make_atomic_swap_body(S) \
- v= IL_EXCHG ## S (a, v)
+ v= IL_EXCHG ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ## S)(v))
#define make_atomic_load_body(S) \
ret= 0; /* avoid compiler warning */ \
- ret= IL_COMP_EXCHG ## S (a, ret, ret);
+ ret= IL_COMP_EXCHG ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ## S)(ret), (IL_EXCHG_TYPE ## S)(ret));
#else /* cleanup */
+#undef IL_EXCHG_TYPE32
+#undef IL_EXCHG_TYPEptr
#undef IL_EXCHG_ADD32
#undef IL_COMP_EXCHG32
#undef IL_COMP_EXCHGptr
diff -Nrup a/include/atomic/nolock.h b/include/atomic/nolock.h
--- a/include/atomic/nolock.h 2008-01-12 02:45:42 +01:00
+++ b/include/atomic/nolock.h 2008-05-07 17:30:57 +02:00
@@ -32,7 +32,10 @@
#endif
#ifdef make_atomic_cas_body
-
+/*
+ Type not used so minimal size (emptry struct has different size between C
+ and C++, zero-length array is gcc-specific).
+*/
typedef char my_atomic_rwlock_t __attribute__ ((unused));
#define my_atomic_rwlock_destroy(name)
#define my_atomic_rwlock_init(name)
diff -Nrup a/include/keycache.h b/include/keycache.h
--- a/include/keycache.h 2007-10-02 09:32:29 +02:00
+++ b/include/keycache.h 2008-05-07 17:30:57 +02:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* Key cache variable structures */
+/**
+ @file
+ Key cache API
+*/
#ifndef _keycache_h
#define _keycache_h
@@ -34,6 +37,10 @@ typedef struct st_keycache_wqueue
struct st_my_thread_var *last_thread; /* circular list of waiting threads */
} KEYCACHE_WQUEUE;
+/** Callback called when any block is flushed */
+typedef int (*KEYCACHE_POST_WRITE_CALLBACK)(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos);
+
#define CHANGED_BLOCKS_HASH 128 /* must be power of 2 */
/*
@@ -81,6 +88,7 @@ typedef struct st_key_cache
KEYCACHE_WQUEUE waiting_for_block; /* requests waiting for a free block */
BLOCK_LINK *changed_blocks[CHANGED_BLOCKS_HASH]; /* hash for dirty file bl.*/
BLOCK_LINK *file_blocks[CHANGED_BLOCKS_HASH]; /* hash for other file bl.*/
+ KEYCACHE_POST_WRITE_CALLBACK post_write;/**< Called when flushing any block*/
/*
The following variables are and variables used to hold parameters for
@@ -124,7 +132,8 @@ extern int key_cache_insert(KEY_CACHE *k
extern int key_cache_write(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length,
- uint block_length,int force_write);
+ uint block_length, int force_write,
+ void *post_write_arg);
extern int flush_key_blocks(KEY_CACHE *keycache,
int file, enum flush_type type);
extern void end_key_cache(KEY_CACHE *keycache, my_bool cleanup);
diff -Nrup a/include/my_global.h b/include/my_global.h
--- a/include/my_global.h 2008-04-14 12:09:56 +02:00
+++ b/include/my_global.h 2008-05-07 17:30:57 +02:00
@@ -68,9 +68,11 @@
#ifdef __cplusplus
#define C_MODE_START extern "C" {
#define C_MODE_END }
+#define STATIC_CAST(TYPE) static_cast<TYPE>
#else
#define C_MODE_START
#define C_MODE_END
+#define STATIC_CAST(TYPE) (TYPE)
#endif
#if defined(_WIN32) || defined(_WIN64) || defined(__WIN32__) || defined(WIN32)
@@ -924,7 +926,7 @@ typedef long long my_ptrdiff_t;
#define my_offsetof(TYPE, MEMBER) \
((size_t)((char *)&(((TYPE *)0x10)->MEMBER) - (char*)0x10))
-#define NullS (char *) 0
+#define NullS STATIC_CAST(char *)(0)
/* Nowdays we do not support MessyDos */
#ifndef NEAR
#define NEAR /* Who needs segments ? */
@@ -1041,7 +1043,7 @@ typedef ulonglong my_off_t;
#else
typedef unsigned long my_off_t;
#endif
-#define MY_FILEPOS_ERROR (~(my_off_t) 0)
+#define MY_FILEPOS_ERROR (~STATIC_CAST(my_off_t)(0))
#if !defined(__WIN__)
typedef off_t os_off_t;
#endif
@@ -1078,7 +1080,7 @@ typedef char bool; /* Ordinary boolean
#define INT8(v) (int8) (v)
#define INT16(v) (int16) (v)
#define INT32(v) (int32) (v)
-#define MYF(v) (myf) (v)
+#define MYF(v) STATIC_CAST(myf)(v)
#ifndef LL
#ifdef HAVE_LONG_LONG
diff -Nrup a/include/my_sys.h b/include/my_sys.h
--- a/include/my_sys.h 2008-04-14 12:11:45 +02:00
+++ b/include/my_sys.h 2008-05-07 17:30:57 +02:00
@@ -13,6 +13,11 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+/**
+ @file
+ mysys library API
+*/
+
#ifndef _my_sys_h
#define _my_sys_h
C_MODE_START
@@ -344,7 +349,10 @@ typedef struct st_dynamic_string
} DYNAMIC_STRING;
struct st_io_cache;
-typedef int (*IO_CACHE_CALLBACK)(struct st_io_cache*);
+/** Function called when certain events happen to an IO_CACHE */
+typedef int (*IO_CACHE_CALLBACK)(struct st_io_cache *cache,
+ const uchar *buffert, uint length,
+ my_off_t filepos);
#ifdef THREAD
typedef struct st_io_cache_share
@@ -443,21 +451,24 @@ typedef struct st_io_cache /* Used when
*/
enum cache_type type;
/*
- Callbacks when the actual read I/O happens. These were added and
- are currently used for binary logging of LOAD DATA INFILE - when a
- block is read from the file, we create a block create/append event, and
- when IO_CACHE is closed, we create an end event. These functions could,
- of course be used for other things
- */
- IO_CACHE_CALLBACK pre_read;
- IO_CACHE_CALLBACK post_read;
- IO_CACHE_CALLBACK pre_close;
+ Callbacks were added and are currently used for binary logging of LOAD
+ DATA INFILE - when a block is read from the file, we create a block
+ create/append event, and when IO_CACHE is closed, we create an end event;
+ also used to write the MyISAM WRITE_CACHE blocks to the MyISAM physical
+ log. These functions could, of course be used for other things. Note: some
+ callbacks share the same argument ("arg").
+ */
+ IO_CACHE_CALLBACK pre_read; /**< called before reading from disk */
+ IO_CACHE_CALLBACK post_read; /**< called after reading from disk */
+ IO_CACHE_CALLBACK pre_close; /**< called before ending the cache */
+ /** Called _after_ writing to disk; not honoured by SEQ_READ_APPEND */
+ IO_CACHE_CALLBACK post_write;
/*
Counts the number of times, when we were forced to use disk. We use it to
increase the binlog_cache_disk_use status variable.
*/
ulong disk_writes;
- void* arg; /* for use by pre/post_read */
+ void *arg; /**< used by pre/post_read,post_write */
char *file_name; /* if used with 'open_cached_file' */
char *dir,*prefix;
File file; /* file descriptor */
@@ -469,6 +480,11 @@ typedef struct st_io_cache /* Used when
partial.
*/
int seek_not_done,error;
+ /**
+ Cumulative 'error' since last [re]init_io_cache(). Useful if cache's user
+ polls for errors only once in a while.
+ */
+ int hard_write_error_in_the_past;
/* buffer_length is memory size allocated for buffer or write_buffer */
size_t buffer_length;
/* read_length is the same as buffer_length except when we use async io */
diff -Nrup a/include/myisam.h b/include/myisam.h
--- a/include/myisam.h 2008-04-01 17:13:53 +02:00
+++ b/include/myisam.h 2008-05-07 17:30:57 +02:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* This file should be included when using myisam_funktions */
+/**
+ @file
+ This file should be included when using MyISAM functions.
+*/
#ifndef _myisam_h
#define _myisam_h
@@ -32,6 +35,7 @@ extern "C" {
#endif
#include "my_handler.h"
#include <mysql/plugin.h>
+#include <hash.h>
/*
Limit max keys according to HA_MAX_POSSIBLE_KEY
@@ -249,8 +253,10 @@ typedef struct st_columndef /* column i
#endif
} MI_COLUMNDEF;
+/** Physical logging is always compiled in. Undefine if want to benchmark */
+#define HAVE_MYISAM_PHYSICAL_LOGGING 1
-extern char * myisam_log_filename; /* Name of logfile */
+extern char * myisam_logical_log_filename;
extern ulong myisam_block_size;
extern ulong myisam_concurrent_insert;
extern my_bool myisam_flush,myisam_delay_key_write,myisam_single_user;
@@ -295,7 +301,15 @@ extern int mi_extra(struct st_myisam_inf
extern int mi_reset(struct st_myisam_info *file);
extern ha_rows mi_records_in_range(MI_INFO *info, int inx,
key_range *min_key, key_range *max_key);
-extern int mi_log(int activate_log);
+/** Open/close actions allowed on a MyISAM log */
+enum enum_mi_log_action
+{
+ MI_LOG_ACTION_OPEN,
+ MI_LOG_ACTION_CLOSE_CONSISTENT, MI_LOG_ACTION_CLOSE_INCONSISTENT
+};
+enum enum_mi_log_type { MI_LOG_PHYSICAL, MI_LOG_LOGICAL };
+extern int mi_log(enum enum_mi_log_action action, enum enum_mi_log_type type,
+ const char *log_filename, const HASH *tables);
extern int mi_is_changed(struct st_myisam_info *info);
extern int mi_delete_all_rows(struct st_myisam_info *info);
extern ulong _mi_calc_blob_length(uint length , const uchar *pos);
diff -Nrup a/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test b/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test
--- a/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test 2008-02-27 15:50:42 +01:00
+++ b/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test 2008-05-07 17:30:57 +02:00
@@ -315,6 +315,7 @@ disconnect con3;
connection con4;
select get_lock("a",10); # wait for rollback to finish
+do release_lock("a");
flush logs;
# we check that the error code of the "ROLLBACK" event is 0 and not
diff -Nrup a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl
--- a/mysql-test/mysql-test-run.pl 2008-04-29 11:27:22 +02:00
+++ b/mysql-test/mysql-test-run.pl 2008-05-07 17:30:57 +02:00
@@ -142,6 +142,7 @@ our $exe_mysql;
our $exe_mysqladmin;
our $exe_mysql_upgrade;
our $exe_mysqlbinlog;
+our $exe_myisamlog;
our $exe_mysql_client_test;
our $exe_bug25714;
our $exe_mysqld;
@@ -1955,6 +1956,27 @@ sub environment_setup () {
" --debug=d:t:A,$path_vardir_trace/log/mysqlbinlog.trace";
}
$ENV{'MYSQL_BINLOG'}= $cmdline_mysqlbinlog;
+
+ # ----------------------------------------------------
+ # Setup env so childs can execute myisamlog
+ # ----------------------------------------------------
+
+ my $myisam_path= mtr_file_exists(vs_config_dirs("storage/myisam", ""),
+ "$glob_basedir/storage/myisam",
+ "$glob_basedir/bin");
+
+ $exe_myisamlog=
+ mtr_exe_exists("$myisam_path/myisamlog");
+
+ my $cmdline_myisamlog=
+ mtr_native_path($exe_myisamlog);
+
+ if ( $opt_debug )
+ {
+ $cmdline_myisamlog .=
+ " -#d:t:A,$path_vardir_trace/log/myisamlog.trace";
+ }
+ $ENV{'MYISAMLOG'}= $cmdline_myisamlog;
# ----------------------------------------------------
# Setup env so childs can execute mysql
diff -Nrup a/mysql-test/r/backup_myisam1.result b/mysql-test/r/backup_myisam1.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/backup_myisam1.result 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,7 @@
+drop database if exists mysqltest;
+create database mysqltest;
+use mysqltest;
+CREATE TABLE t1 (a int) engine=myisam;
+BACKUP DATABASE mysqltest TO 'test.ba';
+ERROR HY000: Got error -1 'online backup impossible with --external-locking' from MyISAM
+DROP DATABASE mysqltest;
diff -Nrup a/mysql-test/r/backup_myisam2.result b/mysql-test/r/backup_myisam2.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/backup_myisam2.result 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,61 @@
+drop database if exists mysqltest;
+create database mysqltest;
+use mysqltest;
+CREATE TABLE t1 (a longtext) engine=myisam;
+SELECT get_lock("data_prepare", 100);
+get_lock("data_prepare", 100)
+1
+SET SESSION debug="+d,enter,info,query,backup_debug";
+BACKUP DATABASE mysqltest TO 'test.ba';
+use mysqltest;
+insert into t1 values ("text");
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+select length(a) from t1;
+length(a)
+262144
+checksum table t1;
+Table Checksum
+mysqltest.t1 1728069308
+SELECT release_lock("data_prepare");
+release_lock("data_prepare")
+1
+Backup Summary
+ header = 26 bytes
+ meta-data = 79 bytes
+ data = 528071 bytes
+ --------------
+ total 528176 bytes
+repair table t1 quick;
+Table Op Msg_type Msg_text
+mysqltest.t1 repair status OK
+DROP DATABASE mysqltest;
+RESTORE FROM 'test.ba';
+Restore Summary
+ header = 26 bytes
+ meta-data = 79 bytes
+ data = 528071 bytes
+ --------------
+ total 528176 bytes
+select length(a) from t1;
+length(a)
+262144
+checksum table t1;
+Table Checksum
+mysqltest.t1 1728069308
+drop database mysqltest;
+SET SESSION debug="-d,enter,info,query,backup_debug";
diff -Nrup a/mysql-test/r/backup_no_be.result b/mysql-test/r/backup_no_be.result
--- a/mysql-test/r/backup_no_be.result 2008-04-16 20:23:01 +02:00
+++ b/mysql-test/r/backup_no_be.result 2008-05-07 17:30:57 +02:00
@@ -16,7 +16,7 @@ SELECT max(backup_id) INTO @id FROM mysq
WHERE command LIKE 'BACKUP DATABASE db1 %';
SELECT engines FROM mysql.online_backup WHERE backup_id=@id;
engines
-Default
+MyISAM
SET SESSION debug="d,backup_test_dummy_be_factory";
SELECT @@debug;
@@debug
diff -Nrup a/mysql-test/r/myisamlog.result b/mysql-test/r/myisamlog.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/myisamlog.result 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,48 @@
+create database if not exists mysqltest;
+use mysqltest;
+drop table if exists t1;
+create table t1 (a int, b varchar(100)) engine=myisam;
+insert into t1 values(1,'life'), (2,'file');
+insert into t1 select a*5, concat("A ",b) from t1;
+update t1 set b="A knife" where a=5;
+delete from t1 where a=10;
+select * from t1;
+a b
+1 life
+2 file
+5 A knife
+truncate table t1;
+flush table t1;
+Commands Used count Errors Recover errors
+open 16 0 0
+write 4 0 0
+update 1 0 0
+delete 1 0 0
+close 8 0 0
+extra 72 0 0
+lock 48 0 0
+Total 150 0 0
+select * from t1;
+a b
+flush table t1;
+Trying to update MyISAM files according to log 'VARDIR/master-data/myisam.log'
+Tables updated successfully
+
+Commands Used count Errors Recover errors
+open 17 0 0
+write 4 0 0
+update 1 0 0
+delete 1 0 0
+close 9 0 0
+extra 75 0 0
+lock 50 0 0
+Total 157 0 0
+select * from t1;
+a b
+1 life
+2 file
+5 A knife
+check table t1 extended;
+Table Op Msg_type Msg_text
+mysqltest.t1 check status OK
+drop database mysqltest;
diff -Nrup a/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result b/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
--- a/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result 2008-04-01 15:44:50 +02:00
+++ b/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result 2008-05-07 17:30:57 +02:00
@@ -433,6 +433,7 @@ insert into t2 select * from t1;
select get_lock("a",10);
get_lock("a",10)
1
+do release_lock("a");
flush logs;
select
(@a:=load_file("MYSQLTEST_VARDIR/tmp/mix_innodb_myisam_binlog.output"))
diff -Nrup a/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result b/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
--- a/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2008-04-01 15:44:50 +02:00
+++ b/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2008-05-07 17:30:57 +02:00
@@ -376,6 +376,7 @@ insert into t2 select * from t1;
select get_lock("a",10);
get_lock("a",10)
1
+do release_lock("a");
flush logs;
select
(@a:=load_file("MYSQLTEST_VARDIR/tmp/mix_innodb_myisam_binlog.output"))
diff -Nrup a/mysql-test/t/backup_myisam1-master.opt b/mysql-test/t/backup_myisam1-master.opt
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/backup_myisam1-master.opt 2008-05-07 17:30:59 +02:00
@@ -0,0 +1 @@
+--external-locking=1
diff -Nrup a/mysql-test/t/backup_myisam1.test b/mysql-test/t/backup_myisam1.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/backup_myisam1.test 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,14 @@
+# Test specific of MyISAM's online backup:
+# see if --external-locking=1 causes backup to fail as expected
+
+--disable_warnings
+drop database if exists mysqltest;
+--enable_warnings
+create database mysqltest;
+use mysqltest;
+CREATE TABLE t1 (a int) engine=myisam;
+
+--error ER_GET_ERRMSG
+BACKUP DATABASE mysqltest TO 'test.ba';
+
+DROP DATABASE mysqltest;
diff -Nrup a/mysql-test/t/backup_myisam2.test b/mysql-test/t/backup_myisam2.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/backup_myisam2.test 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,66 @@
+# Tests specific of MyISAM's online backup
+
+--source include/have_debug.inc
+
+connect (backup,localhost,root,,);
+connect (restore,localhost,root,,);
+
+# Dedicated connection for GET_LOCK(). BACKUP_BREAKPOINT, present in
+# various places like mysql_delete(), causes current lock to be
+# released, that's why GET_LOCK() and DELETE must not be made by the
+# same connection.
+connect (syncer,localhost,root,,);
+
+connection backup;
+
+--disable_warnings
+drop database if exists mysqltest;
+--enable_warnings
+create database mysqltest;
+use mysqltest;
+
+# test of long records (causing records' length to be stored in a long
+# format in the backup log)
+
+CREATE TABLE t1 (a longtext) engine=myisam;
+
+connection syncer;
+# make sure we can update MyISAM tables before the backup finishes,
+# to test backup in online conditions
+SELECT get_lock("data_prepare", 100);
+
+connection backup;
+# prepare to block:
+SET SESSION debug="+d,query,enter,info,query,backup_debug";
+send BACKUP DATABASE mysqltest TO 'test.ba';
+
+connection restore;
+# Must wait to know when backup has entered lock.
+let $wait_condition = SELECT state = "debug_sync_point: data_prepare"
+ FROM INFORMATION_SCHEMA.PROCESSLIST
+ WHERE info LIKE "backup database %";
+--source include/wait_condition.inc
+
+use mysqltest;
+insert into t1 values ("text");
+let $1=16;
+while ($1)
+{
+ update t1 set a=concat(a,a);
+ dec $1;
+}
+select length(a) from t1;
+checksum table t1;
+
+connection syncer;
+SELECT release_lock("data_prepare");
+
+connection backup;
+reap;
+repair table t1 quick;
+DROP DATABASE mysqltest;
+RESTORE FROM 'test.ba';
+select length(a) from t1;
+checksum table t1;
+drop database mysqltest;
+SET SESSION debug="-d,query,enter,info,query,backup_debug";
diff -Nrup a/mysql-test/t/key_cache.test b/mysql-test/t/key_cache.test
--- a/mysql-test/t/key_cache.test 2008-03-27 17:43:16 +01:00
+++ b/mysql-test/t/key_cache.test 2008-05-07 17:30:57 +02:00
@@ -71,7 +71,7 @@ show status like 'key_blocks_used';
# Following results differs on 64 and 32 bit systems because of different
# pointer sizes, which takes up different amount of space in key cache
---replace_result 1812 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
+--replace_result 1805 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1663 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
insert into t1 values (1, 'qqqq'), (11, 'yyyy');
@@ -84,7 +84,7 @@ update t1 set p=2 where p=1;
update t2 set i=2 where i=1;
show status like 'key_blocks_used';
---replace_result 1808 KEY_BLOCKS_UNUSED 1789 KEY_BLOCKS_UNUSED 1670 KEY_BLOCKS_UNUSED 1814 KEY_BLOCKS_UNUSED 1820 KEY_BLOCKS_UNUSED
+--replace_result 1801 KEY_BLOCKS_UNUSED 1789 KEY_BLOCKS_UNUSED 1659 KEY_BLOCKS_UNUSED 1814 KEY_BLOCKS_UNUSED 1820 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
cache index t1 key (`primary`) in keycache1;
@@ -146,7 +146,7 @@ cache index t1,t2 in default;
drop table t1,t2,t3;
show status like 'key_blocks_used';
---replace_result 1812 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
+--replace_result 1805 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1663 KEY_BLOCKS_UNUSED 1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
diff -Nrup a/mysql-test/t/myisamlog-master.opt b/mysql-test/t/myisamlog-master.opt
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/myisamlog-master.opt 2008-05-07 17:30:59 +02:00
@@ -0,0 +1 @@
+--log-isam
diff -Nrup a/mysql-test/t/myisamlog.test b/mysql-test/t/myisamlog.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/myisamlog.test 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,37 @@
+# Test of MyISAM logical logging and the myisamlog utility:
+# see if they can repopulate a table.
+
+disable_warnings;
+create database if not exists mysqltest;
+use mysqltest;
+drop table if exists t1;
+enable_warnings;
+# CREATE generates no entry in the logical log
+create table t1 (a int, b varchar(100)) engine=myisam;
+
+# we put some data into the table
+insert into t1 values(1,'life'), (2,'file');
+insert into t1 select a*5, concat("A ",b) from t1;
+update t1 set b="A knife" where a=5;
+delete from t1 where a=10;
+select * from t1;
+
+# wipe it out (TRUNCATE generates no entry in the logical log)
+truncate table t1;
+# close the table (as we are going to change it out of the server process)
+flush table t1;
+
+# look at log
+exec $MYISAMLOG $MYSQLTEST_VARDIR/master-data/myisam.log ;
+# should not have changed the table
+select * from t1;
+flush table t1;
+
+# apply log to empty table
+--replace_result $MYSQLTEST_VARDIR VARDIR
+exec $MYISAMLOG -F $MYSQLTEST_VARDIR/master-data -u $MYSQLTEST_VARDIR/master-data/myisam.log ;
+
+# reopen the table and verify that content is back
+select * from t1;
+check table t1 extended;
+drop database mysqltest;
diff -Nrup a/mysys/CMakeLists.txt b/mysys/CMakeLists.txt
--- a/mysys/CMakeLists.txt 2007-11-08 00:18:26 +01:00
+++ b/mysys/CMakeLists.txt 2008-05-07 17:30:57 +02:00
@@ -35,7 +35,7 @@ SET(MYSYS_SOURCES array.c charset-def.c
my_clock.c my_compress.c my_conio.c my_copy.c my_crc32.c my_create.c my_delete.c
my_div.c my_error.c my_file.c my_fopen.c my_fstream.c my_gethostbyname.c
my_gethwaddr.c my_getopt.c my_getsystime.c my_getwd.c my_handler.c my_init.c
- my_lib.c my_lock.c my_lockmem.c my_malloc.c my_messnc.c
+ my_lib.c my_lock.c my_lockmem.c my_malloc.c my_messnc.c my_dup.c
my_mkdir.c my_mmap.c my_net.c my_once.c my_open.c my_pread.c my_pthread.c
my_quick.c my_read.c my_realloc.c my_redel.c my_rename.c my_seek.c my_sleep.c
my_static.c my_symlink.c my_symlink2.c my_sync.c my_thr_init.c my_wincond.c
diff -Nrup a/mysys/mf_iocache.c b/mysys/mf_iocache.c
--- a/mysys/mf_iocache.c 2007-10-11 17:07:33 +02:00
+++ b/mysys/mf_iocache.c 2008-05-07 17:30:57 +02:00
@@ -129,6 +129,9 @@ init_functions(IO_CACHE* info)
}
+/* FUNCTIONS TO SET UP OR RESET A CACHE */
+
+
/*
Initialize an IO_CACHE object
@@ -166,7 +169,7 @@ int init_io_cache(IO_CACHE *info, File f
info->file= file;
info->type= TYPE_NOT_SET; /* Don't set it until mutex are created */
info->pos_in_file= seek_offset;
- info->pre_close = info->pre_read = info->post_read = 0;
+ info->pre_close= info->pre_read= info->post_read= info->post_write= NULL;
info->arg = 0;
info->alloced_buffer = 0;
info->buffer=0;
@@ -278,7 +281,7 @@ int init_io_cache(IO_CACHE *info, File f
/* End_of_file may be changed by user later */
info->end_of_file= end_of_file;
- info->error=0;
+ info->error= info->hard_write_error_in_the_past= 0;
info->type= type;
init_functions(info);
#ifdef HAVE_AIOWAIT
@@ -405,7 +408,7 @@ my_bool reinit_io_cache(IO_CACHE *info,
}
}
info->type=type;
- info->error=0;
+ info->error= info->hard_write_error_in_the_past= 0;
init_functions(info);
#ifdef HAVE_AIOWAIT
@@ -422,6 +425,8 @@ my_bool reinit_io_cache(IO_CACHE *info,
} /* reinit_io_cache */
+/* FUNCTIONS TO DO READS FROM THE CACHE */
+
/*
Read buffered.
@@ -1471,14 +1476,19 @@ int _my_b_get(IO_CACHE *info)
uchar buff;
IO_CACHE_CALLBACK pre_read,post_read;
if ((pre_read = info->pre_read))
- (*pre_read)(info);
+ (*pre_read)(info, NULL, 0, 0);
if ((*(info)->read_function)(info,&buff,1))
return my_b_EOF;
if ((post_read = info->post_read))
- (*post_read)(info);
+ (*post_read)(info, NULL, 0, 0);
return (int) (uchar) buff;
}
+/* FUNCTIONS TO DO WRITES TO THE CACHE */
+
+#define set_hard_write_error(CACHE) \
+ ((CACHE)->error= (CACHE)->hard_write_error_in_the_past= -1)
+
/*
Write a byte buffer to IO_CACHE and flush to disk
if IO_CACHE is full.
@@ -1496,7 +1506,7 @@ int _my_b_write(register IO_CACHE *info,
if (info->pos_in_file+info->buffer_length > info->end_of_file)
{
my_errno=errno=EFBIG;
- return info->error = -1;
+ return set_hard_write_error(info);
}
rest_length= (size_t) (info->write_end - info->write_pos);
@@ -1520,13 +1530,15 @@ int _my_b_write(register IO_CACHE *info,
*/
if (my_seek(info->file,info->pos_in_file,MY_SEEK_SET,MYF(0)))
{
- info->error= -1;
+ set_hard_write_error(info);
return (1);
}
info->seek_not_done=0;
}
if (my_write(info->file, Buffer, length, info->myflags | MY_NABP))
- return info->error= -1;
+ return set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, length, info->pos_in_file);
#ifdef THREAD
/*
@@ -1571,7 +1583,7 @@ int my_b_append(register IO_CACHE *info,
*/
DBUG_ASSERT(!info->share);
#endif
-
+ DBUG_ASSERT(info->post_write == NULL); /* unsupported */
lock_append_buffer(info);
rest_length= (size_t) (info->write_end - info->write_pos);
if (Count <= rest_length)
@@ -1591,7 +1603,7 @@ int my_b_append(register IO_CACHE *info,
if (my_write(info->file,Buffer, length, info->myflags | MY_NABP))
{
unlock_append_buffer(info);
- return info->error= -1;
+ return set_hard_write_error(info);
}
Count-=length;
Buffer+=length;
@@ -1644,12 +1656,21 @@ int my_block_write(register IO_CACHE *in
{
/* Of no overlap, write everything without buffering */
if (pos + Count <= info->pos_in_file)
- return my_pwrite(info->file, Buffer, Count, pos,
- info->myflags | MY_NABP);
+ {
+ int ret= my_pwrite(info->file, Buffer, Count, pos,
+ info->myflags | MY_NABP);
+ if (unlikely(ret))
+ set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, Count, pos);
+ return ret;
+ }
/* Write the part of the block that is before buffer */
length= (uint) (info->pos_in_file - pos);
if (my_pwrite(info->file, Buffer, length, pos, info->myflags | MY_NABP))
- info->error= error= -1;
+ error= set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, length, pos);
Buffer+=length;
pos+= length;
Count-= length;
@@ -1711,7 +1732,7 @@ int my_b_flush_io_cache(IO_CACHE *info,
if (info->file == -1)
{
if (real_open_cached_file(info))
- DBUG_RETURN((info->error= -1));
+ DBUG_RETURN(set_hard_write_error(info));
}
LOCK_APPEND_BUFFER;
@@ -1739,29 +1760,42 @@ int my_b_flush_io_cache(IO_CACHE *info,
MY_FILEPOS_ERROR)
{
UNLOCK_APPEND_BUFFER;
- DBUG_RETURN((info->error= -1));
+ DBUG_RETURN(set_hard_write_error(info));
}
if (!append_cache)
info->seek_not_done=0;
}
- if (!append_cache)
- info->pos_in_file+=length;
info->write_end= (info->write_buffer+info->buffer_length-
((pos_in_file+length) & (IO_SIZE-1)));
if (my_write(info->file,info->write_buffer,length,
info->myflags | MY_NABP))
- info->error= -1;
+ set_hard_write_error(info);
else
info->error= 0;
if (!append_cache)
{
+ /*
+ This post_write is really POST-write; callers depend on this! So
+ always call it after writing to the file, not before.
+ */
+ if (info->post_write)
+ (*(info->post_write))(info, info->write_buffer,
+ length, info->pos_in_file);
+ /*
+ The addition below will make the info->pos_in_file be the end of
+ written block; whereas the value we needed in post_write is the
+ value before the addition. That's why we called post_write before
+ this.
+ */
+ info->pos_in_file+=length;
set_if_bigger(info->end_of_file,(pos_in_file+length));
}
else
{
info->end_of_file+=(info->write_pos-info->append_read_pos);
DBUG_ASSERT(info->end_of_file == my_tell(info->file,MYF(0)));
+ DBUG_ASSERT(info->post_write == NULL); /* unsupported */
}
info->append_read_pos=info->write_pos=info->write_buffer;
@@ -1815,7 +1849,7 @@ int end_io_cache(IO_CACHE *info)
if ((pre_close=info->pre_close))
{
- (*pre_close)(info);
+ (*pre_close)(info, NULL, 0, 0);
info->pre_close= 0;
}
if (info->alloced_buffer)
diff -Nrup a/mysys/mf_keycache.c b/mysys/mf_keycache.c
--- a/mysys/mf_keycache.c 2008-04-01 15:44:51 +02:00
+++ b/mysys/mf_keycache.c 2008-05-07 17:30:57 +02:00
@@ -13,7 +13,8 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
+/**
+ @file
These functions handle keyblock cacheing for ISAM and MyISAM tables.
One cache can handle many files.
@@ -36,7 +37,9 @@
blocks_unused is the sum of never used blocks in the pool and of currently
free blocks. blocks_used is the number of blocks fetched from the pool and
as such gives the maximum number of in-use blocks at any time.
+*/
+/*
Key Cache Locking
=================
@@ -213,6 +216,7 @@ struct st_block_link
uint hits_left; /* number of hits left until promotion */
ulonglong last_hit_time; /* timestamp of the last hit */
KEYCACHE_CONDVAR *condvar; /* condition variable for 'no readers' event */
+ void *post_write_arg; /**< post_write's argument*/
};
KEY_CACHE dflt_key_cache_var;
@@ -401,6 +405,7 @@ int init_key_cache(KEY_CACHE *keycache,
keycache->in_init= 0;
pthread_mutex_init(&keycache->cache_lock, MY_MUTEX_INIT_FAST);
keycache->resize_queue.last_thread= NULL;
+ keycache->post_write= NULL;
}
keycache->key_cache_mem_size= use_mem;
@@ -764,12 +769,37 @@ void end_key_cache(KEY_CACHE *keycache,
{
pthread_mutex_destroy(&keycache->cache_lock);
keycache->key_cache_inited= keycache->can_be_used= 0;
+ keycache->post_write= NULL;
KEYCACHE_DEBUG_CLOSE;
}
DBUG_VOID_RETURN;
} /* end_key_cache */
+/**
+ Does a my_pwrite() to the file and then calls callback. Arguments are those
+ of my_pwrite() plus the callback and its argument.
+
+ @note The callback is really POST-write; callers depend on this! So always
+ call it after writing to the file, not before.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 write or callback failed
+*/
+
+static inline int key_cache_pwrite(int Filedes, const uchar *Buffer,
+ uint Count, my_off_t offset, myf MyFlags,
+ KEYCACHE_POST_WRITE_CALLBACK callback,
+ void *callback_arg)
+{
+ int ret= my_pwrite(Filedes, Buffer, Count, offset, MyFlags);
+ if (callback)
+ ret|= (*callback)(callback_arg, Buffer, Count, offset);
+ return ret;
+}
+
+
#ifdef THREAD
/*
@@ -2207,11 +2237,14 @@ restart:
The call is thread safe because only the current
thread might change the block->hash_link value
*/
- error= my_pwrite(block->hash_link->file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
+ error= key_cache_pwrite(block->hash_link->file,
+ block->buffer + block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos +
+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write,
+ block->post_write_arg);
keycache_pthread_mutex_lock(&keycache->cache_lock);
/* Block status must not have changed. */
@@ -2954,6 +2987,8 @@ int key_cache_insert(KEY_CACHE *keycache
length length of the buffer
dont_write if is 0 then all dirty pages involved in writing
should have been flushed from key cache
+ post_write_arg argument which will be passed to key cache's
+ post_write callback
RETURN VALUE
0 if a success, 1 - otherwise.
@@ -2973,7 +3008,8 @@ int key_cache_write(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length,
uint block_length __attribute__((unused)),
- int dont_write)
+ int dont_write,
+ void *post_write_arg)
{
my_bool locked_and_incremented= FALSE;
int error=0;
@@ -2991,7 +3027,9 @@ int key_cache_write(KEY_CACHE *keycache,
/* Force writing from buff into disk. */
keycache->global_cache_w_requests++;
keycache->global_cache_write++;
- if (my_pwrite(file, buff, length, filepos, MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, buff, length, filepos,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, post_write_arg))
DBUG_RETURN(1);
/* purecov: end */
}
@@ -3066,8 +3104,10 @@ int key_cache_write(KEY_CACHE *keycache,
/* Used in the server. */
keycache->global_cache_write++;
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- if (my_pwrite(file, (uchar*) buff, read_length, filepos + offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, (uchar*) buff, read_length,
+ filepos + offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, post_write_arg))
error=1;
keycache_pthread_mutex_lock(&keycache->cache_lock);
}
@@ -3159,6 +3199,7 @@ int key_cache_write(KEY_CACHE *keycache,
*/
if (!(block->status & BLOCK_ERROR))
{
+ block->post_write_arg= post_write_arg;
#if !defined(SERIALIZED_READ_FROM_CACHE)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
#endif
@@ -3234,8 +3275,9 @@ no_key_cache:
keycache->global_cache_write++;
if (locked_and_incremented)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- if (my_pwrite(file, (uchar*) buff, length, filepos,
- MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, (uchar*) buff, length, filepos,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, post_write_arg))
error=1;
if (locked_and_incremented)
keycache_pthread_mutex_lock(&keycache->cache_lock);
@@ -3461,11 +3503,12 @@ static int flush_cached_blocks(KEY_CACHE
(BLOCK_READ | BLOCK_IN_FLUSH | BLOCK_CHANGED | BLOCK_IN_USE));
block->status|= BLOCK_IN_FLUSHWRITE;
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- error= my_pwrite(file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
+ error= key_cache_pwrite(file,
+ block->buffer+block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, block->post_write_arg);
keycache_pthread_mutex_lock(&keycache->cache_lock);
keycache->global_cache_write++;
if (error)
diff -Nrup a/mysys/my_thr_init.c b/mysys/my_thr_init.c
--- a/mysys/my_thr_init.c 2008-04-01 11:20:59 +02:00
+++ b/mysys/my_thr_init.c 2008-05-07 17:30:57 +02:00
@@ -13,8 +13,9 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Functions to handle initializating and allocationg of all mysys & debug
+/**
+ @file
+ Functions for initialization and allocation of all mysys & debug
thread variables.
*/
@@ -29,8 +30,12 @@ pthread_key(struct st_my_thread_var*, TH
pthread_key(struct st_my_thread_var, THR_KEY_mysys);
#endif /* USE_TLS */
pthread_mutex_t THR_LOCK_malloc,THR_LOCK_open,
- THR_LOCK_lock,THR_LOCK_isam,THR_LOCK_myisam,THR_LOCK_heap,
- THR_LOCK_net, THR_LOCK_charset, THR_LOCK_threads, THR_LOCK_time;
+ THR_LOCK_lock,THR_LOCK_isam,THR_LOCK_heap, THR_LOCK_net,
+ THR_LOCK_charset, THR_LOCK_threads, THR_LOCK_time;
+/** For insert/delete in the list of MyISAM open tables */
+pthread_mutex_t THR_LOCK_myisam;
+/** For writing to the MyISAM logs */
+pthread_mutex_t THR_LOCK_myisam_log;
pthread_cond_t THR_COND_threads;
uint THR_thread_count= 0;
uint my_thread_end_wait_time= 5;
@@ -143,6 +148,7 @@ my_bool my_thread_global_init(void)
pthread_mutex_init(&THR_LOCK_lock,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_isam,MY_MUTEX_INIT_SLOW);
pthread_mutex_init(&THR_LOCK_myisam,MY_MUTEX_INIT_SLOW);
+ pthread_mutex_init(&THR_LOCK_myisam_log,MY_MUTEX_INIT_SLOW);
pthread_mutex_init(&THR_LOCK_heap,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_net,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_charset,MY_MUTEX_INIT_FAST);
@@ -209,6 +215,7 @@ void my_thread_global_end(void)
pthread_mutex_destroy(&THR_LOCK_lock);
pthread_mutex_destroy(&THR_LOCK_isam);
pthread_mutex_destroy(&THR_LOCK_myisam);
+ pthread_mutex_destroy(&THR_LOCK_myisam_log);
pthread_mutex_destroy(&THR_LOCK_heap);
pthread_mutex_destroy(&THR_LOCK_net);
pthread_mutex_destroy(&THR_LOCK_time);
@@ -332,6 +339,10 @@ void my_thread_end(void)
/* tmp->dbug is allocated inside DBUG library */
if (tmp->dbug)
{
+ /*
+ Frees memory allocated in SET DEBUG=...; does nothing if there were no
+ SET DEBUG in a thread.
+ */
DBUG_POP();
free(tmp->dbug);
tmp->dbug=0;
diff -Nrup a/sql/backup/stream_v1_transport.c b/sql/backup/stream_v1_transport.c
--- a/sql/backup/stream_v1_transport.c 2007-12-03 21:28:17 +01:00
+++ b/sql/backup/stream_v1_transport.c 2008-05-07 17:30:58 +02:00
@@ -1480,7 +1480,8 @@ int bstream_read_part(backup_stream *s,
saved= *data;
data->end= data->begin + howmuch;
- as_read(&s->stream,data,buf);
+ if (as_read(&s->stream,data,buf) == BSTREAM_EOS)
+ s->state= EOS;
s->buf.begin += data->begin - saved.begin;
s->buf.pos= s->buf.begin;
diff -Nrup a/sql/item_func.cc b/sql/item_func.cc
--- a/sql/item_func.cc 2008-04-29 11:27:22 +02:00
+++ b/sql/item_func.cc 2008-05-07 17:30:58 +02:00
@@ -3375,21 +3375,35 @@ longlong Item_master_pos_wait::val_int()
}
#ifdef EXTRA_DEBUG
+/**
+ When a connection con1 does a GET_LOCK() to synchronize with a another
+ connection con2 doing debug_sync_point(), con1 should not run any statements
+ after that until RELEASE_LOCK(). I.e.: con1 should dedicate itself to
+ synchronization only.
+ */
void debug_sync_point(const char* lock_name, uint lock_timeout)
{
THD* thd=current_thd;
User_level_lock* ull;
struct timespec abstime;
size_t lock_name_len;
+ DBUG_ENTER("debug_sync_point");
+ DBUG_PRINT("enter", ("lock_name: '%s'", lock_name));
lock_name_len= strlen(lock_name);
- pthread_mutex_lock(&LOCK_user_locks);
- if (thd->ull)
- {
- item_user_lock_release(thd->ull);
- thd->ull=0;
- }
+ /*
+ If thread already has a lock:
+ - we cannot automatically release it (it would likely cause
+ synchronization problems in tests, the test writer probably didn't want
+ this implicit release, as it cannot have inspected all code files to see
+ if one of them calls debug_sync_point())
+ - we cannot keep it either, because general user-level lock management
+ relies on one lock max per thread (THD::ull is not a list, GET_LOCK()
+ does an implicit release), and overwriting thd->ull is not ok.
+ */
+ DBUG_ASSERT(thd->ull == NULL);
+ pthread_mutex_lock(&LOCK_user_locks);
/*
If the lock has not been aquired by some client, we do not want to
create an entry for it, since we immediately release the lock. In
@@ -3401,7 +3415,7 @@ void debug_sync_point(const char* lock_n
lock_name_len))))
{
pthread_mutex_unlock(&LOCK_user_locks);
- return;
+ DBUG_VOID_RETURN;
}
ull->count++;
@@ -3453,6 +3467,7 @@ void debug_sync_point(const char* lock_n
thd->ull=0;
}
pthread_mutex_unlock(&LOCK_user_locks);
+ DBUG_VOID_RETURN;
}
#endif
diff -Nrup a/sql/mysqld.cc b/sql/mysqld.cc
--- a/sql/mysqld.cc 2008-05-05 21:37:26 +02:00
+++ b/sql/mysqld.cc 2008-05-07 17:30:58 +02:00
@@ -711,7 +711,7 @@ static bool kill_in_progress, segfaulted
#ifdef HAVE_STACK_TRACE_ON_SEGV
static my_bool opt_do_pstack;
#endif /* HAVE_STACK_TRACE_ON_SEGV */
-static my_bool opt_bootstrap, opt_myisam_log;
+static my_bool opt_bootstrap, opt_myisam_logical_log;
static int cleanup_done;
static ulong opt_specialflag, opt_myisam_block_size;
static char *opt_update_logname, *opt_binlog_index_name;
@@ -4155,8 +4155,8 @@ server.");
pthread_attr_setstacksize(&connection_attrib, NW_THD_STACKSIZE);
#endif
- if (opt_myisam_log)
- (void) mi_log(1);
+ if (opt_myisam_logical_log)
+ (void) mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
#if defined(HAVE_MLOCKALL) && defined(MCL_CURRENT) && !defined(EMBEDDED_LIBRARY)
if (locked_in_memory && !getuid())
@@ -5963,8 +5963,8 @@ Disable with --skip-large-pages.",
(uchar**) &log_error_file_ptr, (uchar**) &log_error_file_ptr, 0, GET_STR,
OPT_ARG, 0, 0, 0, 0, 0, 0},
{"log-isam", OPT_ISAM_LOG, "Log all MyISAM changes to file.",
- (uchar**) &myisam_log_filename, (uchar**) &myisam_log_filename, 0, GET_STR,
- OPT_ARG, 0, 0, 0, 0, 0, 0},
+ (uchar**) &myisam_logical_log_filename, (uchar**)
+ &myisam_logical_log_filename, 0, GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
{"log-long-format", '0',
"Log some extra information to update log. Please note that this option is deprecated; see --log-short-format option.",
0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0},
@@ -7598,7 +7598,7 @@ static void mysql_init_variables(void)
opt_tc_log_file= (char *)"tc.log"; // no hostname in tc_log file name !
opt_secure_auth= 0;
opt_secure_file_priv= 0;
- opt_bootstrap= opt_myisam_log= 0;
+ opt_bootstrap= opt_myisam_logical_log= 0;
mqh_used= 0;
segfaulted= kill_in_progress= 0;
cleanup_done= 0;
@@ -7885,7 +7885,7 @@ mysqld_get_one_option(int optid,
thd_startup_options|=OPTION_BIG_TABLES;
break;
case (int) OPT_ISAM_LOG:
- opt_myisam_log=1;
+ opt_myisam_logical_log=1;
break;
case (int) OPT_UPDATE_LOG:
opt_update_log=1;
diff -Nrup a/sql/sql_bitmap.h b/sql/sql_bitmap.h
--- a/sql/sql_bitmap.h 2008-03-27 18:44:47 +01:00
+++ b/sql/sql_bitmap.h 2008-05-07 17:30:58 +02:00
@@ -156,7 +156,7 @@ template <> class Bitmap<64>
{
ulonglong map;
public:
- Bitmap<64>() { map= 0; }
+ Bitmap<64>() { init(); }
#if defined(__NETWARE__) || defined(__MWERKS__)
/*
Metwork compiler gives error on Bitmap<64>
@@ -167,7 +167,7 @@ public:
#else
explicit Bitmap<64>(uint prefix_to_set) { set_prefix(prefix_to_set); }
#endif
- void init() { }
+ void init() { clear_all(); }
void init(uint prefix_to_set) { set_prefix(prefix_to_set); }
uint length() const { return 64; }
void set_bit(uint n) { map|= ((ulonglong)1) << n; }
diff -Nrup a/sql/sql_cache.cc b/sql/sql_cache.cc
--- a/sql/sql_cache.cc 2008-04-14 12:10:00 +02:00
+++ b/sql/sql_cache.cc 2008-05-07 17:30:58 +02:00
@@ -2705,8 +2705,9 @@ Query_cache::register_tables_from_list(T
{
char key[MAX_DBKEY_LENGTH];
uint32 db_length;
- uint key_length= filename_2_table_key(key, table->table->filename,
- &db_length);
+ uint key_length=
+ filename_2_table_key(key, table->table->s->unresolv_file_name,
+ &db_length);
(++block_table)->n= ++n;
/*
There are not callback function for for MyISAM, and engine data
diff -Nrup a/sql/sql_class.cc b/sql/sql_class.cc
--- a/sql/sql_class.cc 2008-04-29 11:30:03 +02:00
+++ b/sql/sql_class.cc 2008-05-07 17:30:58 +02:00
@@ -1034,8 +1034,9 @@ void THD::awake(THD::killed_state state_
if (mysys_var)
{
pthread_mutex_lock(&mysys_var->mutex);
- if (!system_thread) // Don't abort locks
- mysys_var->abort=1;
+ if (system_thread == NON_SYSTEM_THREAD ||
+ system_thread == SYSTEM_THREAD_BACKUP)
+ mysys_var->abort= 1; // abort locks
/*
This broadcast could be up in the air if the victim thread
exits the cond in the time between read and broadcast, but that is
diff -Nrup a/sql/sql_load.cc b/sql/sql_load.cc
--- a/sql/sql_load.cc 2008-04-14 12:10:02 +02:00
+++ b/sql/sql_load.cc 2008-05-07 17:30:58 +02:00
@@ -1170,8 +1170,7 @@ READ_INFO::READ_INFO(File file_par, uint
cache.read_function = _my_b_net_read;
if (mysql_bin_log.is_open())
- cache.pre_read = cache.pre_close =
- (IO_CACHE_CALLBACK) log_loaded_block;
+ cache.pre_read= cache.pre_close = log_loaded_block;
#endif
}
}
diff -Nrup a/sql/sql_repl.cc b/sql/sql_repl.cc
--- a/sql/sql_repl.cc 2008-03-08 11:14:45 +01:00
+++ b/sql/sql_repl.cc 2008-05-07 17:30:58 +02:00
@@ -1714,7 +1714,10 @@ err:
@param file pointer to io-cache
@return 0
*/
-int log_loaded_block(IO_CACHE* file)
+int log_loaded_block(IO_CACHE* file,
+ const uchar *buffert __attribute__((unused)),
+ uint count __attribute__((unused)),
+ my_off_t filepos __attribute__((unused)))
{
DBUG_ENTER("log_loaded_block");
LOAD_FILE_INFO *lf_info;
diff -Nrup a/sql/sql_repl.h b/sql/sql_repl.h
--- a/sql/sql_repl.h 2008-03-14 18:38:48 +01:00
+++ b/sql/sql_repl.h 2008-05-07 17:30:58 +02:00
@@ -60,7 +60,7 @@ typedef struct st_load_file_info
bool wrote_create_file, log_delayed;
} LOAD_FILE_INFO;
-int log_loaded_block(IO_CACHE* file);
+int log_loaded_block(IO_CACHE *, const uchar *, uint, my_off_t);
int init_replication_sys_vars();
#endif /* HAVE_REPLICATION */
diff -Nrup a/sql/sql_table.cc b/sql/sql_table.cc
--- a/sql/sql_table.cc 2008-04-29 11:27:24 +02:00
+++ b/sql/sql_table.cc 2008-05-07 17:30:58 +02:00
@@ -7245,7 +7245,10 @@ copy_data_between_tables(TABLE *from,TAB
(SQL_SELECT *) 0, HA_POS_ERROR,
1, &examined_rows)) ==
HA_POS_ERROR)
+ {
+ to->file->ha_end_bulk_insert();
goto err;
+ }
}
};
diff -Nrup a/storage/myisam/CMakeLists.txt b/storage/myisam/CMakeLists.txt
--- a/storage/myisam/CMakeLists.txt 2008-02-22 15:51:22 +01:00
+++ b/storage/myisam/CMakeLists.txt 2008-05-07 17:30:58 +02:00
@@ -33,7 +33,7 @@ SET(MYISAM_SOURCES ft_boolean_search.c
mi_rsame.c mi_rsamepos.c mi_scan.c mi_search.c mi_static.c mi_statrec.c
mi_unique.c mi_update.c mi_write.c rt_index.c rt_key.c rt_mbr.c
rt_split.c sort.c sp_key.c ft_eval.h mi_extrafunc.h myisamdef.h
- rt_index.h mi_rkey.c)
+ rt_index.h mi_rkey.c mi_examine_log.c myisam_backup_engine.cc)
IF(NOT SOURCE_SUBLIBS)
diff -Nrup a/storage/myisam/Makefile.am b/storage/myisam/Makefile.am
--- a/storage/myisam/Makefile.am 2007-10-12 18:03:01 +02:00
+++ b/storage/myisam/Makefile.am 2008-05-07 17:30:58 +02:00
@@ -98,7 +98,7 @@ libmyisam_a_SOURCES = mi_open.c mi_extra
mi_keycache.c mi_preload.c \
ft_parser.c ft_stopwords.c ft_static.c \
ft_update.c ft_boolean_search.c ft_nlq_search.c sort.c \
- ha_myisam.cc \
+ mi_examine_log.c ha_myisam.cc myisam_backup_engine.cc \
rt_index.c rt_key.c rt_mbr.c rt_split.c sp_key.c
CLEANFILES = test?.MY? FT?.MY? isam.log mi_test_all rt_test.MY? sp_test.MY?
diff -Nrup a/storage/myisam/ha_myisam.cc b/storage/myisam/ha_myisam.cc
--- a/storage/myisam/ha_myisam.cc 2008-04-14 12:10:05 +02:00
+++ b/storage/myisam/ha_myisam.cc 2008-05-07 17:30:58 +02:00
@@ -775,7 +775,8 @@ int ha_myisam::check(THD* thd, HA_CHECK_
file->update|=HA_STATE_CHANGED | HA_STATE_ROW_CHANGED;
pthread_mutex_lock(&share->intern_lock);
share->state.changed&= ~(STATE_CHANGED | STATE_CRASHED |
- STATE_CRASHED_ON_REPAIR);
+ STATE_CRASHED_ON_REPAIR |
+ STATE_BAD_OPEN_COUNT);
if (!(table->db_stat & HA_READ_ONLY))
error=update_state_info(¶m,file,UPDATE_TIME | UPDATE_OPEN_COUNT |
UPDATE_STAT);
@@ -939,7 +940,7 @@ int ha_myisam::repair(THD *thd, MI_CHECK
param.thd= thd;
param.tmpdir= &mysql_tmpdir_list;
param.out_flag= 0;
- strmov(fixed_name,file->filename);
+ strmov(fixed_name,file->s->unresolv_file_name);
// Don't lock tables if we have used LOCK TABLE
if (!thd->locked_tables &&
@@ -1019,7 +1020,8 @@ int ha_myisam::repair(THD *thd, MI_CHECK
if ((share->state.changed & STATE_CHANGED) || mi_is_crashed(file))
{
share->state.changed&= ~(STATE_CHANGED | STATE_CRASHED |
- STATE_CRASHED_ON_REPAIR);
+ STATE_CRASHED_ON_REPAIR |
+ STATE_BAD_OPEN_COUNT);
file->update|=HA_STATE_CHANGED | HA_STATE_ROW_CHANGED;
}
/*
@@ -1686,11 +1688,11 @@ int ha_myisam::info(uint flag)
if table is symlinked (Ie; Real name is not same as generated name)
*/
data_file_name= index_file_name= 0;
- fn_format(name_buff, file->filename, "", MI_NAME_DEXT,
+ fn_format(name_buff, file->s->unresolv_file_name, "", MI_NAME_DEXT,
MY_APPEND_EXT | MY_UNPACK_FILENAME);
if (strcmp(name_buff, misam_info.data_file_name))
data_file_name=misam_info.data_file_name;
- fn_format(name_buff, file->filename, "", MI_NAME_IEXT,
+ fn_format(name_buff, file->s->unresolv_file_name, "", MI_NAME_IEXT,
MY_APPEND_EXT | MY_UNPACK_FILENAME);
if (strcmp(name_buff, misam_info.index_file_name))
index_file_name=misam_info.index_file_name;
@@ -1977,6 +1979,9 @@ static int myisam_init(void *p)
myisam_hton->create= myisam_create_handler;
myisam_hton->panic= myisam_panic;
myisam_hton->flags= HTON_CAN_RECREATE | HTON_SUPPORT_LOG_TABLES;
+#if !defined(EMBEDDED_LIBRARY) && defined(HAVE_MYISAM_PHYSICAL_LOGGING)
+ myisam_hton->get_backup_engine= myisam_backup_engine;
+#endif
return 0;
}
@@ -2064,8 +2069,8 @@ mysql_declare_plugin_end;
@brief Register a named table with a call back function to the query cache.
@param thd The thread handle
- @param table_key A pointer to the table name in the table cache
- @param key_length The length of the table name
+ @param table_name A pointer to the table name in the table cache
+ @param table_name_len The length of the table name
@param[out] engine_callback The pointer to the storage engine call back
function, currently 0
@param[out] engine_data Engine data will be set to 0.
diff -Nrup a/storage/myisam/ha_myisam.h b/storage/myisam/ha_myisam.h
--- a/storage/myisam/ha_myisam.h 2008-01-10 23:04:54 +01:00
+++ b/storage/myisam/ha_myisam.h 2008-05-07 17:30:58 +02:00
@@ -170,3 +170,7 @@ private:
friend my_bool index_cond_func_myisam(void *arg);
};
+#if !defined(EMBEDDED_LIBRARY) && defined(HAVE_MYISAM_PHYSICAL_LOGGING)
+// If embedded, there is no online backup
+Backup_result_t myisam_backup_engine(handlerton *self, Backup_engine* &be);
+#endif
diff -Nrup a/storage/myisam/mi_check.c b/storage/myisam/mi_check.c
--- a/storage/myisam/mi_check.c 2008-04-01 15:44:56 +02:00
+++ b/storage/myisam/mi_check.c 2008-05-07 17:30:58 +02:00
@@ -85,6 +85,7 @@ static SORT_KEY_BLOCKS *alloc_key_blocks
uint buffer_length);
static ha_checksum mi_byte_checksum(const uchar *buf, uint length);
static void set_data_file_type(SORT_INFO *sort_info, MYISAM_SHARE *share);
+static int chsize_kfile(MI_INFO *info);
void myisamchk_init(MI_CHECK *param)
{
@@ -1667,7 +1668,7 @@ int mi_repair(MI_CHECK *param, register
{
VOID(fputs(" \r",stdout)); VOID(fflush(stdout));
}
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
{
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d",
@@ -1732,6 +1733,11 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ /*
+ File change like this is not handled in physical log. filecopy() above
+ is also not handled.
+ */
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -1991,6 +1997,7 @@ int mi_sort_index(MI_CHECK *param, regis
VOID(my_close(share->kfile,MYF(MY_WME)));
share->kfile = -1;
VOID(my_close(new_file,MYF(MY_WME)));
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->index_file_name,MI_NAME_IEXT,INDEX_TMP_EXT,0,
MYF(0)) ||
mi_open_keyfile(share))
@@ -2096,6 +2103,7 @@ static int sort_one_index(MI_CHECK *para
/* Fill block with zero and write it to the new index file */
length=mi_getint(buff);
bzero((uchar*) buff+length,keyinfo->block_length-length);
+ DBUG_ASSERT(!info->s->physical_logging);
if (my_pwrite(new_file,(uchar*) buff,(uint) keyinfo->block_length,
new_page_pos,MYF(MY_NABP | MY_WAIT_IF_FULL)))
{
@@ -2158,7 +2166,7 @@ int lock_file(MI_CHECK *param, File file
} /* lock_file */
- /* Copy a block between two files */
+ /* Copy a block between two files. Not handled by physical logging. */
int filecopy(MI_CHECK *param, File to,File from,my_off_t start,
my_off_t length, const char *type)
@@ -2508,15 +2516,18 @@ int mi_repair_by_sort(MI_CHECK *param, r
skr=share->base.reloc*share->base.min_pack_length;
#endif
if (skr != sort_info.filelength && !info->s->base.raid_type)
+ {
+ DBUG_ASSERT(!share->physical_logging);
if (my_chsize(info->dfile,skr,0,MYF(0)))
mi_check_print_warning(param,
"Can't change size of datafile, error: %d",
my_errno);
+ }
}
if (param->testflag & T_CALC_CHECKSUM)
info->state->checksum=param->glob_crc;
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d",
my_errno);
@@ -2545,6 +2556,7 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -3028,15 +3040,18 @@ int mi_repair_parallel(MI_CHECK *param,
skr=share->base.reloc*share->base.min_pack_length;
#endif
if (skr != sort_info.filelength && !info->s->base.raid_type)
+ {
+ DBUG_ASSERT(!share->physical_logging);
if (my_chsize(info->dfile,skr,0,MYF(0)))
mi_check_print_warning(param,
"Can't change size of datafile, error: %d",
my_errno);
+ }
}
if (param->testflag & T_CALC_CHECKSUM)
info->state->checksum=param->glob_crc;
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d", my_errno);
@@ -3077,6 +3092,7 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -4057,9 +4073,16 @@ static int sort_insert_key(MI_SORT_PARAM
if (_mi_write_keypage(info, keyinfo, filepos, DFLT_INIT_HITS, anc_buff))
DBUG_RETURN(1);
}
- else if (my_pwrite(info->s->kfile,(uchar*) anc_buff,
- (uint) keyinfo->block_length,filepos, param->myf_rw))
- DBUG_RETURN(1);
+ else
+ {
+ if (my_pwrite(info->s->kfile, anc_buff,
+ keyinfo->block_length, filepos, param->myf_rw))
+ DBUG_RETURN(1);
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(info->s)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, info->s, anc_buff,
+ keyinfo->block_length, filepos);
+ }
DBUG_DUMP("buff",(uchar*) anc_buff,mi_getint(anc_buff));
/* Write separator-key to block in next level */
@@ -4162,9 +4185,17 @@ int flush_pending_blocks(MI_SORT_PARAM *
DFLT_INIT_HITS, key_block->buff))
DBUG_RETURN(1);
}
- else if (my_pwrite(info->s->kfile,(uchar*) key_block->buff,
- (uint) keyinfo->block_length,filepos, myf_rw))
- DBUG_RETURN(1);
+ else
+ {
+ if (my_pwrite(info->s->kfile, key_block->buff,
+ keyinfo->block_length,filepos, myf_rw))
+ DBUG_RETURN(1);
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(info->s)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, info->s,
+ key_block->buff, keyinfo->block_length,
+ filepos);
+ }
DBUG_DUMP("buff",(uchar*) key_block->buff,length);
nod_flag=1;
}
@@ -4444,7 +4475,7 @@ int update_state_info(MI_CHECK *param, M
*/
if (info->lock_type == F_WRLCK)
share->state.state= *info->state;
- if (mi_state_info_write(share->kfile,&share->state,1+2))
+ if (mi_state_info_write(share, share->kfile, &share->state, 1+2))
goto err;
share->changed=0;
}
@@ -4727,4 +4758,45 @@ set_data_file_type(SORT_INFO *sort_info,
mi_setup_functions(&tmp);
share->delete_record=tmp.delete_record;
}
+}
+
+/**
+ Changes the size of an index file, and logs the operation to the physical
+ log if needed.
+
+ The only known case when my_chsize(kfile) can happen on a table doing
+ physical logging, is when the table was empty, bulk insert on it has been
+ done, it's the end of bulk insert: we re-enable indices (mi_repair*()): thus
+ my_chsize() is in fact a void operation (file already has grown, starting
+ from empty, info->state->key_file_length is up-to-date and so file already
+ has the requested size). We however log the operation, in case there are
+ unknown cases.
+
+ @param info table
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+static int chsize_kfile(MI_INFO *info)
+{
+ MYISAM_SHARE *share= info->s;
+ my_off_t new_length= info->state->key_file_length;
+ int ret;
+#ifndef DBUG_OFF
+ my_bool no_length_change=
+ (my_seek(share->kfile, 0L, MY_SEEK_END, MYF(0)) == new_length);
+#endif
+
+ ret= my_chsize(share->kfile, new_length, 0, MYF(0));
+
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(share)))
+ {
+ DBUG_ASSERT(no_length_change);
+ myisam_log_chsize_kfile_physical(share, new_length);
+ }
+
+ return ret;
}
diff -Nrup a/storage/myisam/mi_close.c b/storage/myisam/mi_close.c
--- a/storage/myisam/mi_close.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_close.c 2008-05-07 17:30:58 +02:00
@@ -52,6 +52,8 @@ int mi_close(register MI_INFO *info)
}
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /* Logically there should not be a WRITE_CACHE at this stage */
+ DBUG_ASSERT(!(info->opt_flag & WRITE_CACHE_USED));
if (end_io_cache(&info->rec_cache))
error=my_errno;
info->opt_flag&= ~(READ_CACHE_USED | WRITE_CACHE_USED);
@@ -67,7 +69,11 @@ int mi_close(register MI_INFO *info)
flush_key_blocks(share->key_cache, share->kfile,
share->temporary ? FLUSH_IGNORE_CHANGED :
FLUSH_RELEASE))
+ {
error=my_errno;
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info); /* Mark that table must be checked */
+ }
if (share->kfile >= 0)
{
/*
@@ -77,7 +83,10 @@ int mi_close(register MI_INFO *info)
may be using the file at this point
*/
if (share->mode != O_RDONLY && mi_is_crashed(info))
- mi_state_info_write(share->kfile, &share->state, 1);
+ mi_state_info_write(share, share->kfile, &share->state, 1);
+ if (share->MI_LOG_OPEN_stored_in_physical_log)
+ _myisam_log_command(&myisam_physical_log, MI_LOG_CLOSE, share,
+ NULL, 0, error);
if (my_close(share->kfile,MYF(0)))
error = my_errno;
}
@@ -93,6 +102,7 @@ int mi_close(register MI_INFO *info)
#ifdef THREAD
thr_lock_delete(&share->lock);
VOID(pthread_mutex_destroy(&share->intern_lock));
+ my_atomic_rwlock_destroy(&share->physical_logging_rwlock);
{
int i,keys;
keys = share->state.header.keys;
@@ -113,7 +123,7 @@ int mi_close(register MI_INFO *info)
if (info->dfile >= 0 && my_close(info->dfile,MYF(0)))
error = my_errno;
- myisam_log_command(MI_LOG_CLOSE,info,NULL,0,error);
+ myisam_log_command_logical(MI_LOG_CLOSE, info, NULL, 0, error);
my_free((uchar*) info,MYF(0));
if (error)
diff -Nrup a/storage/myisam/mi_create.c b/storage/myisam/mi_create.c
--- a/storage/myisam/mi_create.c 2008-04-14 13:30:05 +02:00
+++ b/storage/myisam/mi_create.c 2008-05-07 17:30:58 +02:00
@@ -637,6 +637,14 @@ int mi_create(const char *name,uint keys
my_errno= HA_ERR_TABLE_EXIST;
goto err;
}
+ /*
+ TRUNCATE TABLE does not work with physical logging. If we changed TRUNCATE
+ to always use mi_delete_all_rows() (remove HTON_CAN_RECREATE from MyISAM)
+ this would solve the problem.
+ */
+ DBUG_ASSERT((options & HA_OPTION_TMP_TABLE) || !mi_log_tables_physical ||
+ !hash_search(mi_log_tables_physical, filename,
+ strlen(filename)));
if ((file= my_create_with_symlink(linkname_ptr, filename, 0, create_mode,
MYF(MY_WME | create_flag))) < 0)
@@ -702,7 +710,7 @@ int mi_create(const char *name,uint keys
}
DBUG_PRINT("info", ("write state info and base info"));
- if (mi_state_info_write(file, &share.state, 2) ||
+ if (mi_state_info_write(&share, file, &share.state, 2) ||
mi_base_info_write(file, &share.base))
goto err;
#ifndef DBUG_OFF
diff -Nrup a/storage/myisam/mi_delete.c b/storage/myisam/mi_delete.c
--- a/storage/myisam/mi_delete.c 2008-04-09 07:41:37 +02:00
+++ b/storage/myisam/mi_delete.c 2008-05-07 17:30:58 +02:00
@@ -100,13 +100,15 @@ int mi_delete(MI_INFO *info,const uchar
info->state->records--;
mi_sizestore(lastpos,info->lastpos);
- myisam_log_command(MI_LOG_DELETE,info,(uchar*) lastpos,sizeof(lastpos),0);
+ myisam_log_command_logical(MI_LOG_DELETE, info,
+ (uchar*) lastpos, sizeof(lastpos), 0);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (delete)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (delete)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
DBUG_RETURN(0);
@@ -114,7 +116,8 @@ int mi_delete(MI_INFO *info,const uchar
err:
save_errno=my_errno;
mi_sizestore(lastpos,info->lastpos);
- myisam_log_command(MI_LOG_DELETE,info,(uchar*) lastpos, sizeof(lastpos),0);
+ myisam_log_command_logical(MI_LOG_DELETE, info,
+ (uchar*) lastpos, sizeof(lastpos), 0);
if (save_errno != HA_ERR_RECORD_CHANGED)
{
mi_print_error(info->s, HA_ERR_CRASHED);
diff -Nrup a/storage/myisam/mi_delete_all.c b/storage/myisam/mi_delete_all.c
--- a/storage/myisam/mi_delete_all.c 2008-03-28 16:16:49 +01:00
+++ b/storage/myisam/mi_delete_all.c 2008-05-07 17:30:58 +02:00
@@ -47,7 +47,7 @@ int mi_delete_all_rows(MI_INFO *info)
for (i=0 ; i < share->base.keys ; i++)
state->key_root[i]= HA_OFFSET_ERROR;
- myisam_log_command(MI_LOG_DELETE_ALL,info,(uchar*) 0,0,0);
+ myisam_log_command_logical(MI_LOG_DELETE_ALL, info, (uchar*) 0, 0, 0);
/*
If we are using delayed keys or if the user has done changes to the tables
since it was locked then there may be key blocks in the key cache
@@ -60,6 +60,9 @@ int mi_delete_all_rows(MI_INFO *info)
if (my_chsize(info->dfile, 0, 0, MYF(MY_WME)) ||
my_chsize(share->kfile, share->base.keystart, 0, MYF(MY_WME)) )
goto err;
+ if (unlikely(mi_get_physical_logging_state(info->s)))
+ _myisam_log_command(&myisam_physical_log, MI_LOG_DELETE_ALL, share,
+ NULL, 0, 0);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
#ifdef HAVE_MMAP
/* Map again */
diff -Nrup a/storage/myisam/mi_dynrec.c b/storage/myisam/mi_dynrec.c
--- a/storage/myisam/mi_dynrec.c 2008-04-01 15:44:56 +02:00
+++ b/storage/myisam/mi_dynrec.c 2008-05-07 17:30:58 +02:00
@@ -193,9 +193,11 @@ size_t mi_nommap_pread(MI_INFO *info, uc
size_t mi_mmap_pwrite(MI_INFO *info, const uchar *Buffer,
size_t Count, my_off_t offset, myf MyFlags)
{
+ MYISAM_SHARE *share= info->s;
+ uint ret;
DBUG_PRINT("info", ("mi_write with mmap %d\n", info->dfile));
- if (info->s->concurrent_insert)
- rw_rdlock(&info->s->mmap_lock);
+ if (share->concurrent_insert)
+ rw_rdlock(&share->mmap_lock);
/*
The following test may fail in the following cases:
@@ -204,21 +206,24 @@ size_t mi_mmap_pwrite(MI_INFO *info, con
memory mapped area.
*/
- if (info->s->mmaped_length >= offset + Count)
+ if (share->mmaped_length >= offset + Count)
{
- memcpy(info->s->file_map + offset, Buffer, Count);
- if (info->s->concurrent_insert)
- rw_unlock(&info->s->mmap_lock);
- return 0;
+ memcpy(share->file_map + offset, Buffer, Count);
+ if (share->concurrent_insert)
+ rw_unlock(&share->mmap_lock);
+ ret= 0;
}
else
{
- info->s->nonmmaped_inserts++;
- if (info->s->concurrent_insert)
- rw_unlock(&info->s->mmap_lock);
- return my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
+ share->nonmmaped_inserts++;
+ if (share->concurrent_insert)
+ rw_unlock(&share->mmap_lock);
+ ret= my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
}
-
+ if (unlikely(mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYD,
+ share, Buffer, Count, offset);
+ return ret;
}
@@ -227,7 +232,12 @@ size_t mi_mmap_pwrite(MI_INFO *info, con
size_t mi_nommap_pwrite(MI_INFO *info, const uchar *Buffer,
size_t Count, my_off_t offset, myf MyFlags)
{
- return my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
+ MYISAM_SHARE *share= info->s;
+ uint ret= my_pwrite(info->dfile, Buffer, Count, offset, MyFlags);
+ if (unlikely(mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYD,
+ share, Buffer, Count, offset);
+ return ret;
}
diff -Nrup a/storage/myisam/mi_examine_log.c b/storage/myisam/mi_examine_log.c
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/storage/myisam/mi_examine_log.c 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,871 @@
+/* Copyright (C) 2000-2007 MySQL AB
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; version 2 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+
+/**
+ @file
+ Function to display and apply a MyISAM logical or physical log to tables.
+*/
+
+#ifndef USE_MY_FUNC
+#define USE_MY_FUNC
+#endif
+
+#include "myisamdef.h"
+#include <my_tree.h>
+#include <stdarg.h>
+#ifdef HAVE_GETRUSAGE
+#include <sys/resource.h>
+#endif
+
+/** Human-readable names of commands storable in MyISAM logs */
+const char *mi_log_command_name[]=
+{"open","write","update","delete","close","extra","lock",
+ "delete-all", "write-bytes-to-MYD", "write-bytes-to-MYI", "chsize-MYI",
+ /*
+ This one is special: it is never in log records, it's just used by
+ mi_examine_log() to tell the user that it failed when reopening a table. It
+ has to be last before NullS.
+ */
+ "re-open", NullS};
+
+#define FILENAME(A) (A ? A->show_name : "Unknown")
+
+/** In some cases we do not want to flush the index header in mi_close() */
+static my_bool update_index_on_close= TRUE;
+
+struct file_info {
+ long process;
+ /**
+ File descriptor of the table at time of logging: in logical log,
+ it's descriptor of data file; in physical log: of index file.
+ All log records contain a corresponding descriptor value to indicate the
+ table they are about.
+ */
+ int filenr;
+ int id;
+ uint rnd;
+ char *name, *show_name;
+ uchar *record;
+ MI_INFO *isam;
+ /**
+ If 'isam' is currently closed. A not 'used' file is always 'closed' (why
+ open it?). A 'used' file may temporarily be closed because of the max
+ open file descriptors limit (but if we later meet a command which wants
+ to use this file, we will re-open it).
+ */
+ my_bool closed;
+ /** If this table matches the inclusion rules (or has to be ignored) */
+ my_bool used;
+ ulong accessed;
+};
+
+struct test_if_open_param {
+ char * name;
+ int max_id;
+};
+
+struct st_access_param
+{
+ ulong min_accessed;
+ struct file_info *found;
+};
+
+#define NO_FILEPOS HA_OFFSET_ERROR
+
+void mi_examine_log_param_init(MI_EXAMINE_LOG_PARAM *param);
+int mi_examine_log(MI_EXAMINE_LOG_PARAM *param);
+static int read_string(IO_CACHE *file,uchar* *to,uint length);
+static int file_info_compare(void *cmp_arg, void *a,void *b);
+static int test_if_open(struct file_info *key,element_count count,
+ struct test_if_open_param *param);
+static void fix_blob_pointers(MI_INFO *isam,uchar *record);
+static int test_when_accessed(struct file_info *key,element_count count,
+ struct st_access_param *access_param);
+static void file_info_free(struct file_info *info);
+static int close_some_file(TREE *tree);
+static int reopen_closed_file(TREE *tree,struct file_info *file_info);
+static int find_record_with_key(struct file_info *file_info,uchar *record);
+static int mi_close_care_state(MI_INFO *info);
+static void printf_log(uint verbose, ulong isamlog_process,
+ my_off_t isamlog_filepos, const char *format,...);
+static my_bool cmp_filename(struct file_info *file_info, const char *name);
+
+
+void mi_examine_log_param_init(MI_EXAMINE_LOG_PARAM *mi_exl)
+{
+ bzero(mi_exl,sizeof(*mi_exl));
+ mi_exl->number_of_commands= (ulong) ~0L;
+ mi_exl->record_pos= NO_FILEPOS;
+}
+
+
+/**
+ Displays or applies the content of a MyISAM logical or physical log
+ to tables.
+
+ Applies either to all tables referenced by the log, or only to a subset
+ specified in mi_exl->table_selection_hook.
+ If applying the content of the log, this function should be called only
+ when all involved tables are closed and cannot be opened by any concurrent
+ thread/program. It indeed opens tables and modifies them without locking
+ them.
+ Is used both by the standalone program myisamlog and by the restore
+ code of the MyISAM online backup driver.
+
+ @param mi_exl Parameters of the applying
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+int mi_examine_log(MI_EXAMINE_LOG_PARAM *mi_exl)
+{
+ ulong isamlog_process;
+ my_off_t isamlog_filepos;
+ uint command, result, files_open, big_numbers;
+ ulong access_time,length;
+ my_off_t filepos;
+ int lock_command,mi_result;
+ char isam_file_name[FN_REFLEN], llbuff[21], llbuff2[21];
+ uchar head[20], *head_ptr;
+ uchar *buff;
+ struct test_if_open_param open_param;
+ IO_CACHE cache;
+ File log_file;
+ FILE *write_file;
+ enum ha_extra_function extra_command;
+ TREE tree;
+ struct file_info file_info,*curr_file_info;
+ uint head_len[][2]=
+ { { 11, 14 }, { 15, 22 }, { 15, 22 }, { 11, 14 }, { 11, 14 }, { 11, 14 },
+ { 11, 14 }, { 11, 14 }, { 9, 16 }, { 9, 16 }, { 7, 12 } };
+ uint has_pid_and_result[]= {1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0};
+ DBUG_ENTER("mi_examine_log");
+
+ compile_time_assert((sizeof(mi_log_command_name) /
+ sizeof(mi_log_command_name[0]) ==
+ (MI_LOG_END_SENTINEL + 2)) &&
+ (sizeof(has_pid_and_result) /
+ sizeof(has_pid_and_result[0]) ==
+ MI_LOG_END_SENTINEL) &&
+ (sizeof(head_len) / sizeof(head_len[0]) ==
+ MI_LOG_END_SENTINEL) &&
+ (MI_LOG_END_SENTINEL <= MI_LOG_BIG_NUMBERS) &&
+ (sizeof(mi_exl->com_count) /
+ sizeof(mi_exl->com_count[0]) == MI_LOG_END_SENTINEL));
+ if ((log_file=my_open(mi_exl->log_filename,O_RDONLY,MYF(MY_WME))) < 0)
+ DBUG_RETURN(1);
+ write_file=0;
+ if (mi_exl->write_filename)
+ {
+ if (!(write_file=my_fopen(mi_exl->write_filename,O_WRONLY,MYF(MY_WME))))
+ {
+ my_close(log_file,MYF(0));
+ DBUG_RETURN(1);
+ }
+ }
+
+ init_io_cache(&cache,log_file,0,READ_CACHE,mi_exl->start_offset,0,MYF(0));
+ bzero(mi_exl->com_count,sizeof(mi_exl->com_count));
+ init_tree(&tree,0,0,sizeof(file_info),(qsort_cmp2) file_info_compare,1,
+ (tree_element_free) file_info_free, NULL);
+ VOID(init_key_cache(dflt_key_cache,KEY_CACHE_BLOCK_SIZE,KEY_CACHE_SIZE,
+ 0, 0));
+
+ files_open=0; access_time=0;
+ while (access_time++ != mi_exl->number_of_commands &&
+ !my_b_read(&cache, head, 1))
+ {
+ isamlog_filepos=my_b_tell(&cache)-1L;
+ head_ptr= head;
+ command=(uint) head_ptr[0];
+ command-= (big_numbers= (command & MI_LOG_BIG_NUMBERS));
+ if (big_numbers != 0)
+ big_numbers= 1;
+ if (my_b_read(&cache, head, head_len[command][big_numbers] - 1))
+ goto err;
+ if (big_numbers)
+ {
+ file_info.filenr= mi_uint3korr(head);
+ head_ptr+= 3;
+ }
+ else
+ {
+ file_info.filenr= mi_uint2korr(head);
+ head_ptr+= 2;
+ }
+ if (has_pid_and_result[command])
+ {
+ isamlog_process= file_info.process= (long) mi_uint4korr(head_ptr);
+ head_ptr+= 4;
+ if (!mi_exl->opt_processes)
+ file_info.process=0;
+ result= mi_uint2korr(head_ptr);
+ head_ptr+= 2;
+ }
+ else
+ isamlog_process= file_info.process= result= 0;
+ if ((curr_file_info=(struct file_info*) tree_search(&tree, &file_info,
+ tree.custom_arg)))
+ {
+ curr_file_info->accessed=access_time;
+ if (mi_exl->update && curr_file_info->used && curr_file_info->closed)
+ {
+ if (reopen_closed_file(&tree,curr_file_info))
+ {
+ command=sizeof(mi_exl->com_count)/sizeof(mi_exl->com_count[0][0])/3;
+ result=0;
+ goto com_err;
+ }
+ mi_exl->re_open_count++;
+ }
+ }
+ DBUG_PRINT("info",("command: %u curr_file_info: 0x%lx used: %u",
+ command, (ulong)curr_file_info,
+ curr_file_info ? curr_file_info->used : 0));
+ /*
+ We update our statistic (how many commands issued, per command type),
+ if this is a valid command about a file we want to include.
+ For MI_LOG_OPEN decision must be postponed, as curr_file_info is
+ meaningless for it.
+ */
+ if ((command <
+ sizeof(mi_exl->com_count)/sizeof(mi_exl->com_count[0][0])/3) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)) &&
+ (((enum myisam_log_commands) command) != MI_LOG_OPEN))
+ {
+ mi_exl->com_count[command][0]++;
+ if (result)
+ mi_exl->com_count[command][1]++;
+ }
+ switch ((enum myisam_log_commands) command) {
+ case MI_LOG_OPEN:
+ if (curr_file_info)
+ printf("\nWarning: %s is opened with same process and filenumber\n"
+ "Maybe you should use the -P option ?\n",
+ curr_file_info->show_name);
+ file_info.name=0;
+ file_info.show_name=0;
+ file_info.record=0;
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ if (read_string(&cache, (uchar **)&file_info.name, length))
+ goto err;
+ {
+ uint i;
+ char *pos,*to;
+
+ /* Fix if old DOS files to new format */
+ for (pos=file_info.name; (pos=strchr(pos,'\\')) ; pos++)
+ *pos= '/';
+
+ pos=file_info.name;
+ for (i=0 ; i < mi_exl->prefix_remove ; i++)
+ {
+ char *next;
+ if (!(next=strchr(pos,'/')))
+ break;
+ pos=next+1;
+ }
+ to=isam_file_name;
+ if (mi_exl->filepath)
+ to=convert_dirname(isam_file_name,mi_exl->filepath,NullS);
+ strmov(to,pos);
+ fn_ext(isam_file_name)[0]=0; /* Remove extension */
+ }
+ open_param.name=file_info.name;
+ open_param.max_id=0;
+ VOID(tree_walk(&tree, (tree_walk_action)test_if_open,
+ (void*)&open_param, left_root_right));
+ file_info.id=open_param.max_id+1;
+ /*
+ * In the line below +10 is added to accomodate '<' and '>' chars
+ * plus '\0' at the end, so that there is place for 7 digits.
+ * It is improbable that same table can have that many entries in
+ * the table cache.
+ * The additional space is needed for the sprintf commands two lines
+ * below.
+ */
+ file_info.show_name=my_memdup(isam_file_name,
+ (uint) strlen(isam_file_name)+10,
+ MYF(MY_WME));
+ if (file_info.id > 1)
+ sprintf(strend(file_info.show_name),"<%d>",file_info.id);
+ file_info.closed=1;
+ file_info.accessed=access_time;
+ file_info.used= !mi_exl->table_selection_hook ||
+ ((*(mi_exl->table_selection_hook))(isam_file_name));
+ if (mi_exl->update && file_info.used)
+ {
+ if (files_open >= mi_exl->max_files)
+ {
+ if (close_some_file(&tree))
+ goto com_err;
+ files_open--;
+ }
+ /*
+ index may be truncated (if physical logging excluded its pages so
+ use HA_OPEN_FOR_REPAIR).
+ */
+ if (!(file_info.isam= mi_open(isam_file_name, O_RDWR,
+ HA_OPEN_FOR_REPAIR |
+ HA_OPEN_WAIT_IF_LOCKED)))
+ goto com_err;
+ if (!(file_info.record=my_malloc(file_info.isam->s->base.reclength,
+ MYF(MY_WME))))
+ goto end;
+ files_open++;
+ file_info.closed=0;
+ }
+ VOID(tree_insert(&tree, (uchar*) &file_info, 0, tree.custom_arg));
+ if (file_info.used)
+ {
+ if (mi_exl->verbose && !mi_exl->record_pos_file)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: open -> %d",file_info.show_name, file_info.filenr);
+ mi_exl->com_count[command][0]++;
+ /* given how we log MI_LOG_OPEN, "result" is always 0 here */
+ if (result)
+ mi_exl->com_count[command][1]++;
+ }
+ break;
+ case MI_LOG_CLOSE:
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],result);
+ if (curr_file_info)
+ {
+ if (!curr_file_info->closed)
+ files_open--;
+ VOID(tree_delete(&tree, (uchar*) curr_file_info, 0, tree.custom_arg));
+ }
+ break;
+ case MI_LOG_EXTRA:
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ DBUG_ASSERT(length == 1);
+ if (my_b_read(&cache, head, length))
+ goto err;
+ extra_command=(enum ha_extra_function) head[0];
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s(%d) -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command], (int) extra_command,result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_extra(curr_file_info->isam, extra_command, 0) != (int) result)
+ {
+ fflush(stdout);
+ VOID(fprintf(stderr,
+ "Warning: error %d, expected %d on command %s at %s\n",
+ my_errno,result,mi_log_command_name[command],
+ llstr(isamlog_filepos,llbuff)));
+ fflush(stderr);
+ }
+ }
+ break;
+ case MI_LOG_DELETE:
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ DBUG_ASSERT(length == 8);
+ if (my_b_read(&cache, head, length))
+ goto err;
+ filepos=mi_sizekorr(head);
+ if (mi_exl->verbose &&
+ (!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos ||
+ mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info,mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %ld -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],(long) filepos,result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Didn't find row to delete with mi_rrnd");
+ mi_exl->com_count[command][2]++; /* Mark error */
+ }
+ mi_result=mi_delete(curr_file_info->isam,curr_file_info->record);
+ if ((mi_result == 0 && result) ||
+ (mi_result && (uint) my_errno != result))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_result)
+ mi_exl->com_count[command][2]++; /* Mark error */
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Got result %d from mi_delete instead of %d",
+ mi_result, result);
+ }
+ }
+ break;
+ case MI_LOG_WRITE:
+ case MI_LOG_UPDATE:
+ if (big_numbers)
+ {
+ filepos= mi_sizekorr(head_ptr);
+ head_ptr+= 8;
+ length= mi_uint4korr(head_ptr);
+ }
+ else
+ {
+ filepos= mi_uint4korr(head_ptr);
+ head_ptr+= 4;
+ length= mi_uint2korr(head_ptr);
+ }
+ buff=0;
+ if (read_string(&cache,&buff,length))
+ goto err;
+ if ((!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos || mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info,mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ {
+ if (write_file &&
+ (my_fwrite(write_file,buff,length,MYF(MY_WAIT_IF_FULL | MY_NABP))))
+ goto end;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %ld, length=%ld -> %d",
+ FILENAME(curr_file_info),
+ mi_log_command_name[command], filepos,length,result);
+ }
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (curr_file_info->isam->s->base.blobs)
+ fix_blob_pointers(curr_file_info->isam,buff);
+ if ((enum myisam_log_commands) command == MI_LOG_UPDATE)
+ {
+ if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
+ {
+ if (!mi_exl->recover)
+ {
+ result=0;
+ goto com_err;
+ }
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Didn't find row to update with mi_rrnd");
+ if (mi_exl->recover == 1 || result ||
+ find_record_with_key(curr_file_info,buff))
+ {
+ mi_exl->com_count[command][2]++; /* Mark error */
+ break;
+ }
+ }
+ mi_result=mi_update(curr_file_info->isam,curr_file_info->record,
+ buff);
+ if ((mi_result == 0 && result) ||
+ (mi_result && (uint) my_errno != result))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Got result %d from mi_update instead of %d",
+ mi_result, result);
+ if (mi_result)
+ mi_exl->com_count[command][2]++; /* Mark error */
+ }
+ }
+ else
+ {
+ mi_result=mi_write(curr_file_info->isam,buff);
+ if ((mi_result == 0 && result) ||
+ (mi_result && (uint) my_errno != result))
+ {
+ if (!mi_exl->recover)
+ goto com_err;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "error: Got result %d from mi_write instead of %d",
+ mi_result, result);
+ if (mi_result)
+ mi_exl->com_count[command][2]++; /* Mark error */
+ }
+ if (!mi_exl->recover && filepos != curr_file_info->isam->lastpos)
+ {
+ printf("error: Wrote at position: %s, should have been %s",
+ llstr(curr_file_info->isam->lastpos,llbuff),
+ llstr(filepos,llbuff2));
+ goto end;
+ }
+ }
+ }
+ my_free(buff,MYF(0));
+ break;
+ case MI_LOG_WRITE_BYTES_MYI:
+ case MI_LOG_WRITE_BYTES_MYD:
+ if (big_numbers)
+ {
+ filepos= mi_sizekorr(head_ptr);
+ head_ptr+= 8;
+ length= mi_uint4korr(head_ptr);
+ }
+ else
+ {
+ filepos= mi_uint4korr(head_ptr);
+ head_ptr+= 4;
+ length= mi_uint2korr(head_ptr);
+ }
+ buff=0;
+ if (read_string(&cache, &buff, length))
+ goto err;
+ if ((!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos ||
+ mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info,mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ {
+ if (write_file &&
+ (my_fwrite(write_file, buff, length,
+ MYF(MY_WAIT_IF_FULL | MY_NABP))))
+ goto end;
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %s, length=%lu -> %d",
+ FILENAME(curr_file_info),
+ mi_log_command_name[command], llstr(filepos,llbuff),
+ length, result);
+ }
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ update_index_on_close= FALSE;
+ if (my_pwrite((command == MI_LOG_WRITE_BYTES_MYI) ?
+ curr_file_info->isam->s->kfile :
+ curr_file_info->isam->dfile,
+ buff,length,filepos,MYF(MY_NABP)))
+ goto com_err;
+ }
+ my_free(buff,MYF(0));
+ break;
+ case MI_LOG_CHSIZE_MYI:
+ /* here 'filepos' means new length of file */
+ if (big_numbers)
+ filepos= mi_sizekorr(head_ptr);
+ else
+ filepos= mi_uint4korr(head_ptr);
+ if ((!mi_exl->record_pos_file ||
+ ((mi_exl->record_pos == filepos ||
+ mi_exl->record_pos == NO_FILEPOS) &&
+ !cmp_filename(curr_file_info, mi_exl->record_pos_file))) &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ {
+ /* nothing to write to write_file ("length" is 0) */
+ if (mi_exl->verbose)
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s at %s -> %d", FILENAME(curr_file_info),
+ mi_log_command_name[command], llstr(filepos,llbuff),
+ result);
+ }
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ update_index_on_close= FALSE;
+ if (my_chsize(curr_file_info->isam->s->kfile, filepos,
+ 0, MYF(MY_WME)))
+ goto com_err;
+ }
+ break;
+ case MI_LOG_LOCK:
+ length= big_numbers ? mi_uint4korr(head_ptr) : mi_uint2korr(head_ptr);
+ DBUG_ASSERT(length == 4);
+ if (my_b_read(&cache, head, length))
+ goto err;
+ /* remove 'flag' (see mi_lock_database() */
+ lock_command= (uchar)(uint4korr(head));
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s(%d) -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],lock_command,result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_lock_database(curr_file_info->isam,lock_command) !=
+ (int) result)
+ goto com_err;
+ }
+ break;
+ case MI_LOG_DELETE_ALL:
+ if (mi_exl->verbose && !mi_exl->record_pos_file &&
+ (!mi_exl->table_selection_hook ||
+ (curr_file_info && curr_file_info->used)))
+ printf_log(mi_exl->verbose, isamlog_process, isamlog_filepos,
+ "%s: %s -> %d",FILENAME(curr_file_info),
+ mi_log_command_name[command],result);
+ if (mi_exl->update && curr_file_info && !curr_file_info->closed)
+ {
+ if (mi_delete_all_rows(curr_file_info->isam) != (int) result)
+ goto com_err;
+ }
+ break;
+ default:
+ fflush(stdout);
+ VOID(fprintf(stderr,
+ "Error: found unknown command %d in logfile, aborted\n",
+ command));
+ fflush(stderr);
+ goto end;
+ }
+ }
+ end_key_cache(dflt_key_cache,1);
+ delete_tree(&tree);
+ VOID(end_io_cache(&cache));
+ VOID(my_close(log_file,MYF(0)));
+ if (write_file && my_fclose(write_file,MYF(MY_WME)))
+ DBUG_RETURN(1);
+ DBUG_RETURN(0);
+
+ err:
+ fflush(stdout);
+ VOID(fprintf(stderr,"Got error %d when reading from logfile\n",my_errno));
+ fflush(stderr);
+ goto end;
+ com_err:
+ fflush(stdout);
+ VOID(fprintf(stderr,"Got error %d, expected %d on command %s at %s\n",
+ my_errno,result,mi_log_command_name[command],
+ llstr(isamlog_filepos,llbuff)));
+ fflush(stderr);
+ end:
+ end_key_cache(dflt_key_cache, 1);
+ delete_tree(&tree);
+ VOID(end_io_cache(&cache));
+ VOID(my_close(log_file,MYF(0)));
+ if (write_file)
+ VOID(my_fclose(write_file,MYF(MY_WME)));
+ DBUG_RETURN(1);
+}
+
+
+static int read_string(IO_CACHE *file, register uchar* *to,
+ register uint length)
+{
+ DBUG_ENTER("read_string");
+
+ if (*to)
+ my_free((uchar*) *to,MYF(0));
+ if (!(*to= (uchar*) my_malloc(length+1,MYF(MY_WME))) ||
+ my_b_read(file, *to,length))
+ {
+ if (*to)
+ my_free(*to,MYF(0));
+ *to= 0;
+ DBUG_RETURN(1);
+ }
+ *((char*) *to+length)= '\0';
+ DBUG_RETURN (0);
+} /* read_string */
+
+
+static int file_info_compare(void* cmp_arg __attribute__((unused)),
+ void *a, void *b)
+{
+ long lint;
+
+ if ((lint=((struct file_info*) a)->process -
+ ((struct file_info*) b)->process))
+ return lint < 0L ? -1 : 1;
+ return ((struct file_info*) a)->filenr - ((struct file_info*) b)->filenr;
+}
+
+ /* ARGSUSED */
+
+static int test_if_open (struct file_info *key,
+ element_count count __attribute__((unused)),
+ struct test_if_open_param *param)
+{
+ if (!strcmp(key->name,param->name) && key->id > param->max_id)
+ param->max_id=key->id;
+ return 0;
+}
+
+
+static void fix_blob_pointers(MI_INFO *info, uchar *record)
+{
+ uchar *pos;
+ MI_BLOB *blob,*end;
+
+ pos=record+info->s->base.reclength;
+ for (end=info->blobs+info->s->base.blobs, blob= info->blobs;
+ blob != end ;
+ blob++)
+ {
+ memcpy_fixed(record+blob->offset+blob->pack_length,&pos,sizeof(char*));
+ pos+=_mi_calc_blob_length(blob->pack_length,record+blob->offset);
+ }
+}
+
+ /* close the file with hasn't been accessed for the longest time */
+ /* ARGSUSED */
+
+static int test_when_accessed (struct file_info *key,
+ element_count count __attribute__((unused)),
+ struct st_access_param *access_param)
+{
+ if (key->accessed < access_param->min_accessed && ! key->closed)
+ {
+ access_param->min_accessed=key->accessed;
+ access_param->found=key;
+ }
+ return 0;
+}
+
+
+static void file_info_free(struct file_info *fileinfo)
+{
+ DBUG_ENTER("file_info_free");
+ /* The 2 conditions below can be true only if 'update' */
+ if (!fileinfo->closed)
+ VOID(mi_close_care_state(fileinfo->isam));
+ if (fileinfo->record)
+ my_free(fileinfo->record,MYF(0));
+ my_free(fileinfo->name,MYF(0));
+ my_free(fileinfo->show_name,MYF(0));
+ DBUG_VOID_RETURN;
+}
+
+
+
+static int close_some_file(TREE *tree)
+{
+ struct st_access_param access_param;
+
+ access_param.min_accessed=LONG_MAX;
+ access_param.found=0;
+
+ VOID(tree_walk(tree,(tree_walk_action) test_when_accessed,
+ (void*) &access_param,left_root_right));
+ if (!access_param.found)
+ return 1; /* No open file that is possibly to close */
+ if (mi_close_care_state(access_param.found->isam))
+ return 1;
+ access_param.found->closed=1;
+ return 0;
+}
+
+
+static int reopen_closed_file(TREE *tree, struct file_info *fileinfo)
+{
+ char name[FN_REFLEN];
+ if (close_some_file(tree))
+ return 1; /* No file to close */
+ strmov(name,fileinfo->show_name);
+ if (fileinfo->id > 1)
+ *strrchr(name,'<')='\0'; /* Remove "<id>" */
+
+ if (!(fileinfo->isam= mi_open(name, O_RDWR,
+ HA_OPEN_FOR_REPAIR | HA_OPEN_WAIT_IF_LOCKED)))
+ return 1;
+ fileinfo->closed=0;
+ return 0;
+}
+
+ /* Try to find record with uniq key */
+
+static int find_record_with_key(struct file_info *file_info, uchar *record)
+{
+ uint key;
+ MI_INFO *info=file_info->isam;
+ uchar tmp_key[MI_MAX_KEY_BUFF];
+
+ for (key=0 ; key < info->s->base.keys ; key++)
+ {
+ if (mi_is_key_active(info->s->state.key_map, key) &&
+ info->s->keyinfo[key].flag & HA_NOSAME)
+ {
+ VOID(_mi_make_key(info,key,tmp_key,record,0L));
+ return mi_rkey(info,file_info->record,(int) key,tmp_key,0,
+ HA_READ_KEY_EXACT);
+ }
+ }
+ return 1;
+}
+
+
+/**
+ In practice this is only called if verbose>=1. When mi_examine_log() is
+ used in the server it is with verbose==0 so this is not called.
+*/
+
+static void printf_log(uint verbose, ulong isamlog_process,
+ my_off_t isamlog_filepos, const char *format,...)
+{
+ char llbuff[21];
+ va_list args;
+ va_start(args,format);
+ DBUG_ASSERT(verbose > 0);
+ if (verbose > 2)
+ printf("%9s:",llstr(isamlog_filepos,llbuff));
+ if (verbose > 1)
+ printf("%5ld ",isamlog_process); /* Write process number */
+ (void) vprintf((char*) format,args);
+ putchar('\n');
+ va_end(args);
+}
+
+
+static my_bool cmp_filename(struct file_info *file_info, const char *name)
+{
+ if (!file_info)
+ return 1;
+ return strcmp(file_info->name,name) ? 1 : 0;
+}
+
+
+/**
+ Closes a table but, if physical log, does not update its state on disk.
+
+ mi_close() calls mi_state_info_write() if the table is corrupted.
+ This can happen for example is the table is from an online backup which
+ made a copy of its data file and only its index' header.
+ But in that case, if we have executed some MI_LOG_WRITE_BYTES_MYI commands,
+ the state in memory is older than the state on disk, so we do not want to
+ call mi_state_info_write(), it would cancel what we have just done.
+ The solution is to mark the table as "read only" before mi_close().We have
+ no problem with updating the MYISAM_SHARE structure as we are not
+ multi-threaded (i.e. nobody uses the share while we are changing it to
+ read-only). It is also not a problem if this "read only" influences
+ next users of this same share, as a backup log contains all index header
+ writes logged, and so all next users can skip calling
+ mi_state_info_write() too.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+static int mi_close_care_state(MI_INFO *info)
+{
+ if (!update_index_on_close)
+ info->s->mode= O_RDONLY;
+ return mi_close(info);
+}
diff -Nrup a/storage/myisam/mi_extra.c b/storage/myisam/mi_extra.c
--- a/storage/myisam/mi_extra.c 2008-04-01 15:44:56 +02:00
+++ b/storage/myisam/mi_extra.c 2008-05-07 17:30:58 +02:00
@@ -19,7 +19,9 @@
#endif
static void mi_extra_keyflag(MI_INFO *info, enum ha_extra_function function);
-
+static int log_flushed_write_cache_physical(IO_CACHE *cache_to_table,
+ const uchar *buffert,
+ uint length, my_off_t offset);
/*
Set options and buffers to optimize table handling
@@ -144,6 +146,19 @@ int mi_extra(MI_INFO *info, enum ha_extr
HA_STATE_WRITE_AT_END |
HA_STATE_EXTEND_BLOCK);
}
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+ if (!share->temporary)
+ {
+ /*
+ This is a post_write: physical_logging_state has to be checked after
+ doing the table write (see mi_log_start_physical()).
+ We set it now as physical logging may be requested later when the
+ cache has started being used.
+ */
+ info->rec_cache.post_write= log_flushed_write_cache_physical;
+ info->rec_cache.arg= share;
+ }
+#endif
break;
case HA_EXTRA_PREPARE_FOR_UPDATE:
if (info->s->data_file_type != DYNAMIC_RECORD)
@@ -248,7 +263,7 @@ int mi_extra(MI_INFO *info, enum ha_extr
}
}
share->state.state= *info->state;
- error=mi_state_info_write(share->kfile,&share->state,1 | 2);
+ error= mi_state_info_write(share, share->kfile, &share->state, 1 | 2);
}
break;
case HA_EXTRA_FORCE_REOPEN:
@@ -386,7 +401,7 @@ int mi_extra(MI_INFO *info, enum ha_extr
{
char tmp[1];
tmp[0]=function;
- myisam_log_command(MI_LOG_EXTRA,info,(uchar*) tmp,1,error);
+ myisam_log_command_logical(MI_LOG_EXTRA, info, (uchar*)tmp, 1, error);
}
DBUG_RETURN(error);
} /* mi_extra */
@@ -436,6 +451,19 @@ int mi_reset(MI_INFO *info)
*/
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /*
+ If there is a WRITE_CACHE here and we don't hold a write-lock or
+ intern_lock on the table, then mi_log_stop_physical() may be running
+ now in another thread and may be flushing the write cache now (and two
+ concurrent end_io_cache() will cause problems). For example when the
+ SQL layer unlocks tables and then calls ha_myisam::reset() we must not
+ come here. Temp tables are not concerned.
+ */
+ if (!share->temporary && (info->opt_flag & WRITE_CACHE_USED) &&
+ (info->lock.type <= TL_READ_NO_INSERT))
+ {
+ safe_mutex_assert_owner(&share->intern_lock);
+ }
info->opt_flag&= ~(READ_CACHE_USED | WRITE_CACHE_USED);
error= end_io_cache(&info->rec_cache);
}
@@ -454,4 +482,32 @@ int mi_reset(MI_INFO *info)
info->update= ((info->update & HA_STATE_CHANGED) | HA_STATE_NEXT_FOUND |
HA_STATE_PREV_FOUND);
DBUG_RETURN(error);
+}
+
+
+/**
+ Logs when the WRITE_CACHE is flushed to the data file, to the physical
+ log.
+
+ @param cache_for_table pointer to the table's WRITE_CACHE IO_CACHE
+ @param buffert argument to the pwrite
+ @param length length of buffer
+ @param filepos offset in file where buffer was written
+
+ @return Operation status, always 0
+ @retval 0 ok. Yes, even if log write fails we return ok, don't want
+ to make the table writer believe its table is now
+ corrupted.
+*/
+
+static int log_flushed_write_cache_physical(IO_CACHE *cache_for_table,
+ const uchar *buffert,
+ uint length, my_off_t filepos)
+{
+ MYISAM_SHARE *share= (MYISAM_SHARE *)(cache_for_table->arg);
+ DBUG_ENTER("log_flushed_write_cache_physical");
+ if (unlikely(mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYD, share, buffert,
+ length, filepos);
+ DBUG_RETURN(0);
}
diff -Nrup a/storage/myisam/mi_locking.c b/storage/myisam/mi_locking.c
--- a/storage/myisam/mi_locking.c 2007-10-12 18:03:01 +02:00
+++ b/storage/myisam/mi_locking.c 2008-05-07 17:30:58 +02:00
@@ -13,11 +13,13 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- locking of isam-tables.
- reads info from a isam-table. Must be first request before doing any furter
- calls to any isamfunktion. Is used to allow many process use the same
- isamdatabase.
+/**
+ @file
+ Locking of MyISAM tables.
+
+ Reads info from a isam table. Must be first request before doing any
+ further calls to any isam function. Is used to allow many processes to use
+ the same isam database.
*/
#include "ftdefs.h"
@@ -71,6 +73,12 @@ int mi_lock_database(MI_INFO *info, int
}
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /*
+ Logically there should not be a WRITE_CACHE at this stage, except
+ maybe for temporary tables.
+ */
+ DBUG_ASSERT(info->s->temporary ||
+ !(info->opt_flag & WRITE_CACHE_USED));
if (end_io_cache(&info->rec_cache))
{
error=my_errno;
@@ -80,43 +88,16 @@ int mi_lock_database(MI_INFO *info, int
}
if (!count)
{
+ int local_error;
DBUG_PRINT("info",("changed: %u w_locks: %u",
(uint) share->changed, share->w_locks));
- if (share->changed && !share->w_locks)
- {
-#ifdef HAVE_MMAP
- if ((info->s->mmaped_length != info->s->state.state.data_file_length) &&
- (info->s->nonmmaped_inserts > MAX_NONMAPPED_INSERTS))
- {
- if (info->s->concurrent_insert)
- rw_wrlock(&info->s->mmap_lock);
- mi_remap_file(info, info->s->state.state.data_file_length);
- info->s->nonmmaped_inserts= 0;
- if (info->s->concurrent_insert)
- rw_unlock(&info->s->mmap_lock);
- }
-#endif
- share->state.process= share->last_process=share->this_process;
- share->state.unique= info->last_unique= info->this_unique;
- share->state.update_count= info->last_loop= ++info->this_loop;
- if (mi_state_info_write(share->kfile, &share->state, 1))
- error=my_errno;
- share->changed=0;
- if (myisam_flush)
- {
- if (my_sync(share->kfile, MYF(0)))
- error= my_errno;
- if (my_sync(info->dfile, MYF(0)))
- error= my_errno;
- }
- else
- share->not_flushed=1;
- if (error)
- {
- mi_print_error(info->s, HA_ERR_CRASHED);
- mi_mark_crashed(info);
- }
- }
+ if (share->changed && !share->w_locks &&
+ (local_error= mi_remap_file_and_write_state_for_unlock(info)))
+ {
+ error= local_error;
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info);
+ }
if (info->lock_type != F_EXTRA_LCK)
{
if (share->r_locks)
@@ -256,11 +237,14 @@ int mi_lock_database(MI_INFO *info, int
}
#endif
pthread_mutex_unlock(&share->intern_lock);
-#if defined(FULL_LOG) || defined(_lint)
- lock_type|=(int) (flag << 8); /* Set bit to set if real lock */
- myisam_log_command(MI_LOG_LOCK,info,(uchar*) &lock_type,sizeof(lock_type),
- error);
-#endif
+ if (my_b_inited(&myisam_logical_log))
+ {
+ uchar lock_type_buff[4];
+ lock_type|=(int) (flag << 8); /* Set bit to set if real lock */
+ int4store(lock_type_buff, lock_type);
+ myisam_log_command_logical(MI_LOG_LOCK, info,
+ lock_type_buff, sizeof(lock_type_buff), error);
+ }
DBUG_RETURN(error);
} /* mi_lock_database */
@@ -455,7 +439,7 @@ int _mi_writeinfo(register MI_INFO *info
share->state.process= share->last_process= share->this_process;
share->state.unique= info->last_unique= info->this_unique;
share->state.update_count= info->last_loop= ++info->this_loop;
- if ((error=mi_state_info_write(share->kfile, &share->state, 1)))
+ if ((error= mi_state_info_write(share, share->kfile, &share->state, 1)))
olderror=my_errno;
#ifdef __WIN__
if (myisam_flush)
@@ -541,6 +525,10 @@ int _mi_mark_file_changed(MI_INFO *info)
{
mi_int2store(buff,share->state.open_count);
buff[2]=1; /* Mark that it's changed */
+ /*
+ Don't need to log it to physical log, as online backup does dirty
+ copies anyway.
+ */
DBUG_RETURN(my_pwrite(share->kfile,buff,sizeof(buff),
sizeof(share->state.header),
MYF(MY_NABP)));
@@ -570,6 +558,10 @@ int _mi_decrement_open_count(MI_INFO *in
{
share->state.open_count--;
mi_int2store(buff,share->state.open_count);
+ /*
+ Don't need to log it to physical log, as online backup does dirty
+ copies anyway.
+ */
write_error=my_pwrite(share->kfile,buff,sizeof(buff),
sizeof(share->state.header),
MYF(MY_NABP));
@@ -578,4 +570,55 @@ int _mi_decrement_open_count(MI_INFO *in
lock_error=mi_lock_database(info,old_lock);
}
return test(lock_error || write_error);
+}
+
+
+/**
+ Remaps the data file, and writes state to index file.
+
+ When we unlock a table and no other thread has announced it is going to
+ write to it (w_locks==0), we want to flush some information to disk, so
+ that in case of crash the table is not too much corrupted. Physical
+ logging, when it is turning logging of for a table, needs to do this too,
+ so that this information reaches the log.
+
+ @param info table
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+int mi_remap_file_and_write_state_for_unlock(MI_INFO *info)
+{
+ MYISAM_SHARE *share= info->s;
+ int error= 0;
+#ifdef HAVE_MMAP
+ if ((info->s->mmaped_length != info->s->state.state.data_file_length) &&
+ (info->s->nonmmaped_inserts > MAX_NONMAPPED_INSERTS))
+ {
+ if (info->s->concurrent_insert)
+ rw_wrlock(&info->s->mmap_lock);
+ mi_remap_file(info, info->s->state.state.data_file_length);
+ info->s->nonmmaped_inserts= 0;
+ if (info->s->concurrent_insert)
+ rw_unlock(&info->s->mmap_lock);
+ }
+#endif
+ share->state.process= share->last_process=share->this_process;
+ share->state.unique= info->last_unique= info->this_unique;
+ share->state.update_count= info->last_loop= ++info->this_loop;
+ if (mi_state_info_write(share, share->kfile, &share->state, 1))
+ error=my_errno;
+ share->changed=0;
+ if (myisam_flush)
+ {
+ if (my_sync(share->kfile, MYF(0)))
+ error= my_errno;
+ if (my_sync(info->dfile, MYF(0)))
+ error= my_errno;
+ }
+ else
+ share->not_flushed=1;
+ return error;
}
diff -Nrup a/storage/myisam/mi_log.c b/storage/myisam/mi_log.c
--- a/storage/myisam/mi_log.c 2007-08-13 15:11:16 +02:00
+++ b/storage/myisam/mi_log.c 2008-05-07 17:30:58 +02:00
@@ -13,9 +13,58 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Logging of MyISAM commands and records on logfile for debugging
- The log can be examined with help of the myisamlog command.
+/**
+ @file
+ Logging of MyISAM commands and records.
+
+ The physical log contains each call to OS write functions on the MyISAM
+ files. Most of its entries are physical for example "write these bytes at
+ this offset". For example, a mi_write() with lots of BLOBs in many places
+ will cause lots of entries in this log. It also contains some logical ones
+ like MI_LOG_DELETE_ALL (we wouldn't want to log the deletion of all rows
+ one by one).
+
+ The logical log contains each call to higher-level operations like
+ mi_write()/mi_update(). A sample entry is "write this record". For example,
+ a mi_write() with lots of BLOBs will cause one single entry in this log.
+
+ Writes to the logical log happen when the logical operation
+ happens. Entries in it refer to a table by file descriptor of its data
+ file.
+
+ Writes to the physical log happen when the physical operation happens,
+ i.e. when the file is written, which can be at three moments:
+ -# when the row write directly writes to the file (mi_[no]mmap_pwrite())
+ -# if the row write went to a WRITE_CACHE, when this cache gets written to
+ the file (post_write callback in that cache)
+ -# if the row write went to the key cache, when this key cache block gets
+ written ("flushed") to the file (post_write callback in that cache)
+ Additionally, an entry for opening and an entry for closing the table, are
+ written to the physical log: the first "direct row write" or "WRITE_CACHE"
+ or "key cache block flush" log write for a certain MYISAM_SHARE, an entry
+ for opening (MI_LOG_OPEN) is written. All entries refer to the table by the
+ file descriptor of the index file; the MI_LOG_OPEN entry links this number
+ to a table name. The entry for closing is written by mi_close() if an entry
+ for opening had been written before and if the index file is being closed.
+
+ Physical log is used for online backup, because if applied to a dirtily
+ copied table it can make this table consistent. A dirty table
+ contains internally inconsistent records, so applying a logical log to it
+ (like mi_update()) is impossible).
+
+ Both logs:
+ - are idempotent (if you apply such log to a table, then applying it a
+ second time has no effect).
+ - can be used to debug MyISAM
+ - can be examined and applied to tables with the myisamlog utility.
+
+ Logical log is about all tables, is turned on before opening tables (for
+ example turned on at program's start and off at program's end).
+
+ Physical log is about to a set of tables, can be turned on and off at any
+ time.
+
+ mi_log() is the entry point.
*/
#include "myisamdef.h"
@@ -36,114 +85,401 @@
#define GETPID() myisam_pid
#endif
- /* Activate logging if flag is 1 and reset logging if flag is 0 */
-
-static int log_type=0;
+/** the log_type global variable is probably obsolete, it's always 0 now */
+static const int log_type=0;
ulong myisam_pid=0;
+static int mi_log_open_cache(enum enum_mi_log_type type,
+ const char *log_filename);
+static int mi_log_close_cache(enum enum_mi_log_type type);
+static int mi_log_start_physical(const char *log_filename,
+ const HASH *tables);
+static int mi_log_stop_physical();
+
+
+/**
+ Starts MyISAM logical logging for all tables or physical logging for
+ a set of tables, or stops it.
+
+ @param action what to do (start, stop (in)consistently)
+ @param type physical or logical
+ @param log_filename name of the log file to create (only for physical
+ log, logical log has a static name)
+ @param tables hash of names of tables for which we want logging
+ (only for physical log)
+
+ @note For the logical log, MI_LOG_CLOSE_INCONSISTENT and
+ MI_LOG_CLOSE_CONSISTENT are identical: log will be consistent only if
+ tables not being written now.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error; then caller should call mi_log_stop_physical(TRUE)
+*/
-int mi_log(int activate_log)
+int mi_log(enum enum_mi_log_action action, enum enum_mi_log_type type,
+ const char *log_filename, const HASH *tables)
{
- int error=0;
- char buff[FN_REFLEN];
+ int error;
DBUG_ENTER("mi_log");
- log_type=activate_log;
- if (activate_log)
+ if (type == MI_LOG_LOGICAL)
{
- if (!myisam_pid)
- myisam_pid=(ulong) getpid();
- if (myisam_log_file < 0)
- {
- if ((myisam_log_file = my_create(fn_format(buff,myisam_log_filename,
- "",".log",4),
- 0,(O_RDWR | O_BINARY | O_APPEND),MYF(0)))
- < 0)
- DBUG_RETURN(my_errno);
- }
+ DBUG_ASSERT(log_filename == NULL);
+ DBUG_ASSERT(tables == NULL);
+ if (action == MI_LOG_ACTION_OPEN)
+ error= mi_log_open_cache(type, myisam_logical_log_filename);
+ else
+ error= mi_log_close_cache(type);
+ DBUG_RETURN(error);
}
- else if (myisam_log_file >= 0)
+
+#ifndef HAVE_MYISAM_PHYSICAL_LOGGING
+ DBUG_ASSERT(0);
+ DBUG_RETURN(1);
+#endif
+
+ /* starting/stopping are complex operations so split in functions */
+ switch (action)
{
- error=my_close(myisam_log_file,MYF(0)) ? my_errno : 0 ;
- myisam_log_file= -1;
+ case MI_LOG_ACTION_OPEN:
+ error= mi_log_start_physical(log_filename, tables);
+ break;
+ case MI_LOG_ACTION_CLOSE_CONSISTENT:
+ case MI_LOG_ACTION_CLOSE_INCONSISTENT:
+ error= mi_log_stop_physical(action);
+ break;
+ default:
+ DBUG_ASSERT(0);
+ error= 1;
}
DBUG_RETURN(error);
}
- /* Logging of records and commands on logfile */
- /* All logs starts with command(1) dfile(2) process(4) result(2) */
+/**
+ Sets up a log's IO_CACHE (for logical or physical log).
-void _myisam_log(enum myisam_log_commands command, MI_INFO *info,
- const uchar *buffert, uint length)
+ Both logs are IO_CACHE to be fast. But the logical log flushes it after
+ every operation, as it is share-able between different processes and so
+ another process may want to write to this log as soon as we unlock its
+ file. Also, as this log is used for debugging it must contain as much
+ information as possible in case of crash. For this log, using IO_CACHE
+ still makes sense as it decreases the number of my_write() calls.
+
+ @param type physical or logical
+ @param log_filename only for physical log (logical log has a static
+ name)
+
+ @note logs are not created with MY_WAIT_IF_FULL: a log can itself be the
+ cause of filling the disk, so better corrupt it (and make a backup
+ fail for example) than prevent other normal operations.
+
+ @todo A realistic benchmark to see if the size of the IO_CACHE makes any
+ speed difference.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+static int mi_log_open_cache(enum enum_mi_log_type type,
+ const char *log_filename)
{
- uchar buff[11];
- int error,old_errno;
- ulong pid=(ulong) GETPID();
- old_errno=my_errno;
- bzero(buff,sizeof(buff));
- buff[0]=(char) command;
- mi_int2store(buff+1,info->dfile);
- mi_int4store(buff+3,pid);
- mi_int2store(buff+9,length);
+ int error=0;
+ char buff[FN_REFLEN];
+ int access_flags;
+ File file;
+ IO_CACHE *log;
+ uint cache_size;
+ DBUG_ENTER("mi_log_open_cache");
+
+ DBUG_ASSERT(log_filename != NULL);
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (type == MI_LOG_LOGICAL)
+ {
+ log= &myisam_logical_log;
+ /* O_APPEND as file may exist and we want to keep it */
+ access_flags= O_WRONLY | O_BINARY | O_APPEND;
+ /* small cache size as frequent flushes */
+ cache_size= IO_SIZE;
+ }
+ else
+ {
+ DBUG_ASSERT(type == MI_LOG_PHYSICAL);
+ log= &myisam_physical_log;
+ /* We want to fail if file exists */
+ access_flags= O_WRONLY | O_BINARY | O_TRUNC | O_EXCL;
+ /*
+ We want a large IO_CACHE to have large contiguous disk writes.
+ In many systems this size is affordable. In small embedded ones it is
+ not, but would they use this log?
+ */
+ cache_size= IO_SIZE*256;
+ }
+ if (!myisam_pid)
+ myisam_pid=(ulong) getpid();
+ if (!my_b_inited(log))
+ {
+ DBUG_ASSERT(log_filename);
+ fn_format(buff, log_filename, "", "", MY_UNPACK_FILENAME);
+ if ((file= my_create(buff,
+ 0, access_flags,
+ MYF(MY_WME | ME_WAITTANG))) < 0)
+ error= my_errno;
+ else if (init_io_cache(log, file,
+ cache_size, WRITE_CACHE,
+ my_tell(file,MYF(MY_WME)), 0,
+ MYF(MY_WME | MY_NABP)))
+ {
+ error= my_errno;
+ my_close(file, MYF(MY_WME));
+ }
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ DBUG_RETURN(error);
+}
- pthread_mutex_lock(&THR_LOCK_myisam);
- error=my_lock(myisam_log_file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- VOID(my_write(myisam_log_file,buff,sizeof(buff),MYF(0)));
- VOID(my_write(myisam_log_file,buffert,length,MYF(0)));
- if (!error)
- error=my_lock(myisam_log_file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- pthread_mutex_unlock(&THR_LOCK_myisam);
- my_errno=old_errno;
+
+/**
+ Destroy's a log's IO_CACHE (for logical or physical log).
+
+ @param type physical or logical
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+static int mi_log_close_cache(enum enum_mi_log_type type)
+{
+ int error= 0;
+ IO_CACHE *log;
+ DBUG_ENTER("mi_log_close_cache");
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (type == MI_LOG_LOGICAL)
+ log = &myisam_logical_log;
+ else
+ {
+ DBUG_ASSERT(type == MI_LOG_PHYSICAL);
+ log = &myisam_physical_log;
+ }
+ if (my_b_inited(log))
+ {
+ if (end_io_cache(log) ||
+ my_close(log->file,MYF(MY_WME)))
+ error= my_errno;
+ log->file= -1;
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ DBUG_RETURN(error);
}
-void _myisam_log_command(enum myisam_log_commands command, MI_INFO *info,
- const uchar *buffert, uint length, int result)
+/**
+ Logs a MyISAM command and its return code to log.
+
+ If this is a physical log and MI_LOG_OPEN has not already been stored for
+ this MYISAM_SHARE in this log, also writes a MI_LOG_OPEN.
+
+ @param log pointer to the log's IO_CACHE
+ @param command MyISAM command (see code for allowed commands)
+ @param info_or_share MI_INFO if logical log, MYISAM_SHARE if physical
+ @param buffert usually argument to the command (e.g. name of file
+ to open for MI_LOG_OPEN), may be NULL
+ @param length length of buffert (0 if NULL)
+ @param result return code of the command
+*/
+
+void _myisam_log_command(IO_CACHE *log, enum myisam_log_commands command,
+ void *info_or_share,
+ const uchar *buffert, uint length, int result)
{
- uchar buff[9];
- int error,old_errno;
+ uchar header[14];
+ int error, old_errno, headerlen;
ulong pid=(ulong) GETPID();
+ my_bool logical= (log == &myisam_logical_log);
+ MYISAM_SHARE *share;
+ File file;
+
+ /*
+ Speed in online backup (physical log) matters more than in debugging
+ (logical log) so we use unlikely().
+ */
+ if (unlikely(logical))
+ {
+ file= ((MI_INFO *)info_or_share)->dfile;
+ LINT_INIT(share);
+ }
+ else
+ {
+ file= (share= (MYISAM_SHARE *)info_or_share)->kfile;
+ LINT_INIT(error);
+ }
+ DBUG_ASSERT(command == MI_LOG_OPEN || command == MI_LOG_DELETE ||
+ command == MI_LOG_CLOSE || command == MI_LOG_EXTRA ||
+ command == MI_LOG_LOCK || command == MI_LOG_DELETE_ALL);
old_errno=my_errno;
- buff[0]=(char) command;
- mi_int2store(buff+1,info->dfile);
- mi_int4store(buff+3,pid);
- mi_int2store(buff+7,result);
- pthread_mutex_lock(&THR_LOCK_myisam);
- error=my_lock(myisam_log_file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- VOID(my_write(myisam_log_file,buff,sizeof(buff),MYF(0)));
- if (buffert)
- VOID(my_write(myisam_log_file,buffert,length,MYF(0)));
- if (!error)
- error=my_lock(myisam_log_file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_ASSERT(((uint)result) <= UINT_MAX16);
+ if (file >= UINT_MAX16 || length >= UINT_MAX16)
+ {
+ header[0]= ((uchar) command) | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(file < (2<<24));
+ mi_int3store(header + 1, file);
+ mi_int4store(header + 4, pid);
+ mi_int2store(header + 8, result);
+ mi_int4store(header + 10, length);
+ headerlen= 14;
+ }
+ else
+ {
+ /* use a compact encoding for all these small numbers */
+ header[0]= (uchar) command;
+ mi_int2store(header + 1, file);
+ mi_int4store(header + 3, pid);
+ mi_int2store(header + 7, result);
+ mi_int2store(header + 9, length);
+ headerlen= 11;
+ }
+retry:
+ /*
+ Reasons to not use THR_LOCK_myisam to serialize log writes:
+ - better concurrency (not stealing THR_LOCK_myisam which is used for opens
+ and closes including long table flushes)
+ - mi_close() flushes indexes while holding THR_LOCK_myisam, and that flush
+ can cause log writes, so we would lock the mutex twice.
+ */
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ /*
+ We need to check that 'log' is not closed, this can happen for a physical
+ log. Indeed we do not have full control on the table from the thread doing
+ mi_log_stop_physical(); it could be an inconsistent logging stop (in
+ the middle of writes) or even a consistent one (table can be in
+ mi_lock_database(F_UNLCK) and thus want to flush its header)). Log might
+ just have been closed while the table still has physical_logging true.
+ */
+ if (likely(my_b_inited(log) != NULL))
+ {
+ if (!logical)
+ {
+ if (command == MI_LOG_OPEN)
+ {
+ /*
+ If there could be two concurrent writers on a MyISAM table, it could
+ be that they both do a myisam_log_command(c) where c!=MI_LOG_OPEN,
+ which both see MI_LOG_OPEN_stored_in_physical_log false, and both
+ call myisam_log_command(MI_LOG_OPEN); we would then have to make one
+ single winner: one will run before the other, the other should
+ notice MI_LOG_OPEN_stored_in_physical_log became true and back off.
+ But there is always at most one writer to a MyISAM table, so the
+ assertion below should always be ok
+ */
+ DBUG_ASSERT(!share->MI_LOG_OPEN_stored_in_physical_log);
+ share->MI_LOG_OPEN_stored_in_physical_log= TRUE;
+ /*
+ We must keep the mutex between setting the boolean above and writing
+ to the log ; one instant after unlocking the mutex, the log may be
+ closed and so it would be wrong to say that the MI_LOG_OPEN is in
+ the log (it would possibly influence a next physical log).
+ */
+ }
+ else if (unlikely(!share->MI_LOG_OPEN_stored_in_physical_log))
+ {
+ DBUG_ASSERT(command != MI_LOG_CLOSE);
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ _myisam_log_command(&myisam_physical_log, MI_LOG_OPEN, share,
+ (uchar *)share->unresolv_file_name,
+ strlen(share->unresolv_file_name), 0);
+ goto retry;
+ }
+ }
+ else
+ error=my_lock(log->file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
+ /*
+ Any failure to write the log does not prevent the table write (table
+ should still be usable even though log breaks).
+ but sets up log->hard_write_error_in_the_past, which can be tested by
+ those who want to use this log.
+ */
+ VOID(my_b_write(log, header, headerlen));
+ if (buffert)
+ VOID(my_b_write(log, buffert, length));
+ else
+ {
+ DBUG_ASSERT(length == 0);
+ }
+ /*
+ Another process (if external locking) may want to append to the logical
+ log as soon as we unlock its file, we must flush it. Physical log is not
+ share-able, does not need to flush.
+ */
+ if (logical)
+ {
+ flush_io_cache(log);
+ if (!error)
+ error=my_lock(log->file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
+ }
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
my_errno=old_errno;
}
-void _myisam_log_record(enum myisam_log_commands command, MI_INFO *info,
- const uchar *record, my_off_t filepos, int result)
+/**
+ Logs a MyISAM command (involving a record: MI_LOG_WRITE etc) and its
+ return code to the logical log.
+
+ @param command MyISAM command (MI_LOG_UPDATE|WRITE)
+ @param info MI_INFO
+ @param record record to write/update/etc, not NULL
+ @param filepos offset in data file where record starts
+ @param result return code of the command
+*/
+
+void _myisam_log_record_logical(enum myisam_log_commands command,
+ MI_INFO *info, const uchar *record,
+ my_off_t filepos, int result)
{
- uchar buff[21],*pos;
+ uchar header[22],*pos;
int error,old_errno;
- uint length;
+ uint length, headerlen;
ulong pid=(ulong) GETPID();
+ DBUG_ASSERT(command == MI_LOG_UPDATE || command == MI_LOG_WRITE);
old_errno=my_errno;
if (!info->s->base.blobs)
length=info->s->base.reclength;
else
length=info->s->base.reclength+ _my_calc_total_blob_length(info,record);
- buff[0]=(uchar) command;
- mi_int2store(buff+1,info->dfile);
- mi_int4store(buff+3,pid);
- mi_int2store(buff+7,result);
- mi_sizestore(buff+9,filepos);
- mi_int4store(buff+17,length);
- pthread_mutex_lock(&THR_LOCK_myisam);
- error=my_lock(myisam_log_file,F_WRLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- VOID(my_write(myisam_log_file, buff,sizeof(buff),MYF(0)));
- VOID(my_write(myisam_log_file, record,info->s->base.reclength,MYF(0)));
+ DBUG_ASSERT(((uint)result) <= UINT_MAX16);
+ if (info->dfile >= UINT_MAX16 || filepos >= UINT_MAX32 ||
+ length >= UINT_MAX16)
+ {
+ header[0]= ((uchar) command) | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(info->dfile < (2<<24));
+ mi_int3store(header + 1, info->dfile);
+ mi_int4store(header + 4, pid);
+ mi_int2store(header + 8, result);
+ mi_sizestore(header + 10, filepos);
+ mi_int4store(header + 18, length);
+ headerlen= 22;
+ }
+ else
+ {
+ header[0]= (uchar) command;
+ mi_int2store(header + 1, info->dfile);
+ mi_int4store(header + 3, pid);
+ mi_int2store(header + 7, result);
+ mi_int4store(header + 9, filepos);
+ mi_int2store(header + 13, length);
+ headerlen= 15;
+ }
+
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ error= my_lock(myisam_logical_log.file, F_WRLCK, 0L, F_TO_EOF,
+ MYF(MY_SEEK_NOT_DONE));
+ VOID(my_b_write(&myisam_logical_log, header, headerlen));
+ VOID(my_b_write(&myisam_logical_log, record, info->s->base.reclength));
if (info->s->base.blobs)
{
MI_BLOB *blob,*end;
@@ -152,13 +488,390 @@ void _myisam_log_record(enum myisam_log_
blob != end ;
blob++)
{
- memcpy_fixed((uchar*) &pos, record+blob->offset+blob->pack_length,
+ memcpy_fixed(&pos, record+blob->offset+blob->pack_length,
sizeof(char*));
- VOID(my_write(myisam_log_file,pos,blob->length,MYF(0)));
+ VOID(my_b_write(&myisam_logical_log, pos, blob->length));
}
}
+ flush_io_cache(&myisam_logical_log);
if (!error)
- error=my_lock(myisam_log_file,F_UNLCK,0L,F_TO_EOF,MYF(MY_SEEK_NOT_DONE));
- pthread_mutex_unlock(&THR_LOCK_myisam);
+ error= my_lock(myisam_logical_log.file, F_UNLCK, 0L, F_TO_EOF,
+ MYF(MY_SEEK_NOT_DONE));
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
my_errno=old_errno;
+}
+
+
+/* THE FOLLOWING FUNCTIONS SERVE ONLY FOR PHYSICAL LOGGING */
+
+/**
+ Logs a my_pwrite() (done to data or index file) to the physical log.
+
+ Also logs MI_LOG_OPEN if first time. Thus, a MI_INFO will write MI_LOG_OPEN
+ to the log only if it is doing a write to the table: a table which does
+ only reads logs nothing to the physical log.
+
+ @param command MyISAM command (MI_LOG_WRITE_BYTES_TO_MYD|MYI)
+ @param share table's share
+ @param buffert argument to the pwrite
+ @param length length of buffer
+ @param filepos offset in file where buffer was written
+
+ @note length may be small (for example, if updating only a numeric field of
+ a record, it could be only a few bytes), so we try to minimize the header's
+ size of the log entry (no 'pid', no 'result').
+*/
+
+void myisam_log_pwrite_physical(enum myisam_log_commands command,
+ MYISAM_SHARE *share, const uchar *buffert,
+ uint length, my_off_t filepos)
+{
+ uchar header[21];
+ int old_errno, headerlen;
+ DBUG_ENTER("myisam_log_pwrite_physical");
+ DBUG_ASSERT(command == MI_LOG_WRITE_BYTES_MYD ||
+ command == MI_LOG_WRITE_BYTES_MYI);
+ DBUG_ASSERT(buffert != NULL && length > 0);
+ old_errno= my_errno;
+ if (share->kfile >= UINT_MAX16 || filepos >= UINT_MAX32 ||
+ length >= UINT_MAX16)
+ {
+ header[0]= ((uchar) command) | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(share->kfile < (2<<24));
+ mi_int3store(header + 1, share->kfile);
+ mi_sizestore(header + 4, filepos);
+ mi_int4store(header + 12, length);
+ headerlen= 16;
+ }
+ else
+ {
+ header[0]= (uchar) command;
+ mi_int2store(header + 1, share->kfile);
+ mi_int4store(header + 3, filepos);
+ mi_int2store(header + 7, length);
+ headerlen= 9;
+ }
+ /* pid and result are not needed */
+retry:
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (likely(my_b_inited(&myisam_physical_log) != NULL))
+ {
+ if (unlikely(!share->MI_LOG_OPEN_stored_in_physical_log))
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ _myisam_log_command(&myisam_physical_log, MI_LOG_OPEN, share,
+ (uchar *)share->unresolv_file_name,
+ strlen(share->unresolv_file_name), 0);
+ goto retry;
+ }
+ VOID(my_b_write(&myisam_physical_log, header, headerlen));
+ VOID(my_b_write(&myisam_physical_log, buffert, length));
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ my_errno= old_errno;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Logs a my_chsize() done to the index file to the physical log.
+
+ Also logs MI_LOG_OPEN if first time.
+
+ @param share table's share
+ @param new_length new length of the table's index file
+*/
+
+void myisam_log_chsize_kfile_physical(MYISAM_SHARE *share,
+ my_off_t new_length)
+{
+ uchar header[12];
+ int old_errno, headerlen;
+ DBUG_ENTER("myisam_log_chsize_kfile_physical");
+ old_errno= my_errno;
+ if (share->kfile >= UINT_MAX16 || new_length >= UINT_MAX32)
+ {
+ header[0]= MI_LOG_CHSIZE_MYI | MI_LOG_BIG_NUMBERS;
+ DBUG_ASSERT(share->kfile < (2<<24));
+ mi_int3store(header + 1, share->kfile);
+ mi_sizestore(header + 4, new_length);
+ headerlen= 12;
+ }
+ else
+ {
+ header[0]= MI_LOG_CHSIZE_MYI;
+ mi_int2store(header + 1, share->kfile);
+ mi_int4store(header + 3, new_length);
+ headerlen= 7;
+ }
+ /* pid and result are not needed */
+retry:
+ pthread_mutex_lock(&THR_LOCK_myisam_log);
+ if (likely(my_b_inited(&myisam_physical_log) != NULL))
+ {
+ if (unlikely(!share->MI_LOG_OPEN_stored_in_physical_log))
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ _myisam_log_command(&myisam_physical_log, MI_LOG_OPEN, share,
+ (uchar *)share->unresolv_file_name,
+ strlen(share->unresolv_file_name), 0);
+ goto retry;
+ }
+ VOID(my_b_write(&myisam_physical_log, header, headerlen));
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam_log);
+ my_errno= old_errno;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Starts MyISAM physical logging for a set of tables.
+
+ Physical logging is used for online backup.
+ A condition of correctness of online backup is that:
+ after the copy process has started (i.e. after the function below has
+ terminated), any update done to a table-to-back-up must be present in the
+ log. This guides the algorithm below.
+
+ All writes (my_write, my_pwrite, memcpy to mmap'ed area, my_chsize) to the
+ data or index file are done this way:
+ @code
+ {
+ write_to_data_or_index_file;
+ if ((atomic read of MYISAM_SHARE::physical_logging) != 0)
+ write log record to physical log;
+ }
+ @endcode
+
+ The present function sets MYISAM_SHARE::physical_logging to 1 using an
+ atomic write. Atomic write happens before or after atomic read above, and
+ atomic read sees the latest value. If before, change will be in the log. If
+ after, it is also after the write_to_data_or_index_file and thus change
+ will be in the copy. So correctness is always guaranteed. Note the
+ importance of checking MYISAM_SHARE::logging always _after_
+ write_to_data_or_index_file, with an _atomic_read_ for the reasoning to
+ hold.
+
+ @param log_filename Name of the physical log file to create
+ @param tables Hash of names of tables for which we want logging
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+static int mi_log_start_physical(const char *log_filename, const HASH *tables)
+{
+ LIST *list_item;
+ int error;
+ DBUG_ENTER("mi_log_start_physical");
+ DBUG_ASSERT(log_filename != NULL);
+ DBUG_ASSERT(hash_inited(tables));
+
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (mi_log_tables_physical) /* physical logging already running */
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_ASSERT(0); /* because it should not happen */
+ DBUG_RETURN(1);
+ }
+ mi_log_tables_physical= tables;
+
+ if (unlikely(mi_log_open_cache(MI_LOG_PHYSICAL, log_filename)))
+ {
+ error= 1;
+ goto end;
+ }
+ /* Go through all open MyISAM tables */
+ for (list_item= myisam_open_list; list_item; list_item= list_item->next)
+ {
+ MI_INFO *info= (MI_INFO*)list_item->data;
+ MYISAM_SHARE *share= info->s;
+ DBUG_PRINT("info",("table '%s' 0x%lx tested against hash",
+ share->unique_file_name, (ulong)info));
+ if (!hash_search(mi_log_tables_physical, (uchar *)share->unique_file_name,
+ share->unique_name_length))
+ continue;
+ /* Backup kernel shouldn't ask for temporary table's backup */
+ DBUG_ASSERT(!share->temporary);
+ /*
+ We don't need to flush key blocks, WRITE_CACHE or the state
+ because every time they are written to disk (at the latest in
+ mi_log_stop_physical()) they check for physical logging
+ (key cache always has log_key_page_flush_physical() as
+ post_write, WRITE_CACHE always has log_flushed_write_cache_physical()
+ has post_write, even when _not_ in backup), so any now cached info will
+ finally reach the log.
+ Conversely, if we wanted to register no callback in key cache and
+ WRITE_CACHE when no backup is running (to save function calls
+ and atomic reads when no backup is running), we would have to
+ flush key cache and WRITE_CACHE here.
+ */
+ mi_set_physical_logging_state(info->s, 1);
+ }
+ error= 0;
+end:
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ if (unlikely(error))
+ mi_log_stop_physical(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_PHYSICAL,
+ NULL, NULL);
+ DBUG_RETURN(error);
+}
+
+
+/**
+ Stops MyISAM physical logging.
+
+ As part of this stop operation, user can request that the physical log ends
+ in a consistent state, i.e. that it contains copies of the currently cached
+ key pages etc. To be consistent assumes that the caller has relevant tables
+ write-locked, indeed otherwise the log could end in the middle of a
+ statement, and applying it would produce a likely corrupted table.Online
+ backup needs such a consistent log to be able to create consistent table
+ copies from the log. If online backup is being cancelled, then there is no
+ need that the physical log be consistent.
+
+ @param action MI_LOG_ACTION_CLOSE_CONSISTENT or
+ MI_LOG_ACTION_CLOSE_INCONSISTENT.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+
+ @note Even if MI_LOG_ACTION_CLOSE_CONSISTENT, tables may be being written
+ now (in practice caller has read-locked tables, but those tables may be
+ just going out of a write (after thr_unlock(), before or inside
+ mi_lock_database(F_UNLCK) which may be flushing the index header or index
+ pages).
+*/
+
+static int mi_log_stop_physical(enum enum_mi_log_action action)
+{
+ int error= 0;
+ LIST *list_item;
+ DBUG_ENTER("mi_log_stop_physical");
+ DBUG_ASSERT(action == MI_LOG_ACTION_CLOSE_CONSISTENT ||
+ action == MI_LOG_ACTION_CLOSE_INCONSISTENT);
+
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (mi_log_tables_physical == NULL) /* no physical logging running */
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_RETURN(0); /* it's ok if it happens */
+ }
+ /*
+ This is a pointer to a object provided by the caller through
+ mi_log_start_physical(); such object is to be freed by the caller.
+ */
+ mi_log_tables_physical= NULL;
+
+ if (action == MI_LOG_ACTION_CLOSE_CONSISTENT)
+ {
+ /**
+ @todo consider an algorithm which would not keep THR_LOCK_myisam for the
+ time of flushing all these tables' indices; we could do a first loop
+ with THR_LOCK_myisam to collect shares and "pin" them; then a second
+ loop without THR_LOCK_myisam, flushing and unpinning them.
+ */
+ for (list_item= myisam_open_list; list_item; list_item= list_item->next)
+ {
+ MI_INFO *info= (MI_INFO*)list_item->data;
+ MYISAM_SHARE *share= info->s;
+ /*
+ Setting of the variable below always happens under THR_LOCK_myisam,
+ which we have here, so we don't need atomic operations to read here.
+ */
+ if (!share->physical_logging)
+ continue;
+ /*
+ Must take intern_lock, at least because key cache isn't safe if two
+ calls to flush_key_blocks_int() run concurrently on the same file.
+ */
+ pthread_mutex_lock(&share->intern_lock);
+ /*
+ It is possible that some statement just finished, has not called
+ mi_lock_database(F_UNLCK) yet, and so some key blocks would still be
+ in memory even if !delay_key_write. So we have to flush below even
+ in this case, to put them into the log.
+
+ It is also possible (same scenario) that some WRITE_CACHE is not
+ flushed yet. This should not happen but it does (can just be a
+ forgotten mi_extra(HA_EXTRA_NO_CACHE)); so mi_close() and
+ mi_lock_database(F_UNLCK) flush the cache; so we have to do it here
+ too, to put the data into the log. Mutices in mi_close() and
+ mi_lock_database() ensure that they don't flush at the same time as us
+ (which could corrupt the cache). Nobody should flush the WRITE_CACHE
+ without a write-lock or intern_lock (see assertion in mi_reset()).
+
+ It is also possible (same scenario) that the index's header has not
+ been written yet and nobody is going to do it for us; indeed this can
+ happen (two concurrent threads): thread1 has just done
+ mi_lock_database(F_WRLCK), is blocked by the thr_lock of our caller,
+ thread2 has finished its write statement and is going to execute
+ mi_lock_database(F_UNLCK); no index header flush will be done by the
+ mi_lock_database(F_UNLCK) of thread2 as w_locks is >0 (due to
+ thread1). And no index header flush will be done by thread1 as it is
+ blocked. So, we need to flush the index header here, to put it into
+ the log.
+
+ Of course, for the flushing above to reach the log, it has to be done
+ before setting share->physical_logging to false and before closing the
+ log.
+ */
+ if ((mi_log_index_pages_physical &&
+ (share->kfile >= 0) &&
+ flush_key_blocks(share->key_cache, share->kfile, FLUSH_KEEP)) ||
+ ((info->opt_flag & WRITE_CACHE_USED) &&
+ flush_io_cache(&info->rec_cache)) ||
+ (share->changed && mi_remap_file_and_write_state_for_unlock(info)))
+ {
+ error= 1; /* we continue, because log has to be closed anyway */
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info); /* Mark that table must be checked */
+ }
+ pthread_mutex_unlock(&share->intern_lock);
+ } /* ... for (list_item=...) */
+ } /* ... if (action == MI_LOG_ACTION_CLOSE_CONSISTENT) */
+
+ /*
+ Online backup wants to pick this log with my_read() calls, to send it to
+ the backup stream. So we don't delete log but close it now, so that its
+ IO_CACHE goes to disk (so that all log is visible to the my_read()
+ calls). Another reason related to concurrency is mentioned below.
+ */
+ if (mi_log_close_cache(MI_LOG_PHYSICAL))
+ error= 1;
+
+ for (list_item= myisam_open_list; list_item; list_item= list_item->next)
+ {
+ MYISAM_SHARE *share= ((MI_INFO*)list_item->data)->s;
+ /*
+ Setting of the variable below always happens under THR_LOCK_myisam,
+ which we have here, so we don't need atomic operations to read here.
+ */
+ if (!share->physical_logging)
+ continue;
+ mi_set_physical_logging_state(share, 0);
+ /*
+ We reset MI_LOG_OPEN_stored_in_physical_log. How is this safe with a
+ concurrent logging operation (like myisam_log_pwrite_physical()) which
+ may want to set it to TRUE at the same time?
+ The concurrent logging operation runs either before or after log closing
+ (serialization ensured by THR_LOCK_myisam_log). If before, it is before
+ us (us==resetter), because log closing is before us, so we win. If
+ after, the concurrent logging operation finds the log closed and so
+ will not change MI_LOG_OPEN_stored_in_physical_log (so we win again).
+ Note the importance of closing the log before, for the reasoning to
+ hold.
+ */
+ share->MI_LOG_OPEN_stored_in_physical_log= FALSE;
+ }
+
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ /*
+ From this moment on, from the point of view of MyISAM, a new physical log
+ (a new backup) can start (new log will use a different tmp name).
+ */
+ DBUG_RETURN(error);
}
diff -Nrup a/storage/myisam/mi_open.c b/storage/myisam/mi_open.c
--- a/storage/myisam/mi_open.c 2008-04-14 12:10:05 +02:00
+++ b/storage/myisam/mi_open.c 2008-05-07 17:30:58 +02:00
@@ -32,6 +32,8 @@
#endif
static void setup_key_functions(MI_KEYDEF *keyinfo);
+static int log_key_page_flush_physical(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos);
#define get_next_element(to,pos,size) { memcpy((char*) to,pos,(size_t) size); \
pos+=size;}
@@ -94,6 +96,12 @@ MI_INFO *mi_open(const char *name, int m
head_length=sizeof(share_buff.state.header);
bzero((uchar*) &info,sizeof(info));
+ /*
+ 'name' is an unresolved name (no .sym or Unix symbolic link
+ resolution). Physical logging needs it. We resolve 'name' in 'org_name'
+ and 'name_buff'. Don't change how name_buff is built without updating
+ myisam_backup::Backup::begin().
+ */
my_realpath(name_buff, fn_format(org_name,name,"",MI_NAME_IEXT,
MY_UNPACK_FILENAME),MYF(0));
pthread_mutex_lock(&THR_LOCK_myisam);
@@ -106,6 +114,20 @@ MI_INFO *mi_open(const char *name, int m
share_buff.state.key_del=key_del;
share_buff.key_cache= multi_key_cache_search((uchar*) name_buff,
strlen(name_buff));
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+ if (unlikely((share_buff.key_cache->post_write == NULL) &&
+ (open_flags & HA_OPEN_FROM_SQL_LAYER)))
+
+ {
+ /*
+ This is a post_write: physical_logging_state has to be checked after
+ doing the table write (see mi_log_start_physical()).
+ We set it now as physical logging may be requested later when the
+ cache has started being used and blocks are cached.
+ */
+ share_buff.key_cache->post_write= log_key_page_flush_physical;
+ }
+#endif
DBUG_EXECUTE_IF("myisam_pretend_crashed_table_on_open",
if (strstr(name, "/t1"))
@@ -213,18 +235,29 @@ MI_INFO *mi_open(const char *name, int m
disk_pos= my_n_base_info_read(disk_cache + base_pos, &share->base);
share->state.state_length=base_pos;
- if (!(open_flags & HA_OPEN_FOR_REPAIR) &&
- ((share->state.changed & STATE_CRASHED) ||
- ((open_flags & HA_OPEN_ABORT_IF_CRASHED) &&
- (my_disable_locking && share->state.open_count))))
- {
- DBUG_PRINT("error",("Table is marked as crashed. open_flags: %u "
- "changed: %u open_count: %u !locking: %d",
- open_flags, share->state.changed,
- share->state.open_count, my_disable_locking));
- my_errno=((share->state.changed & STATE_CRASHED_ON_REPAIR) ?
- HA_ERR_CRASHED_ON_REPAIR : HA_ERR_CRASHED_ON_USAGE);
- goto err;
+ if (!(open_flags & HA_OPEN_FOR_REPAIR))
+ {
+ if ((share->state.changed & STATE_CRASHED) ||
+ ((open_flags & HA_OPEN_ABORT_IF_CRASHED) &&
+ (my_disable_locking && share->state.open_count)))
+ {
+ DBUG_PRINT("error",("Table is marked as crashed. open_flags: %u "
+ "changed: %u open_count: %u !locking: %d",
+ open_flags, share->state.changed,
+ share->state.open_count, my_disable_locking));
+ my_errno=((share->state.changed & STATE_CRASHED_ON_REPAIR) ?
+ HA_ERR_CRASHED_ON_REPAIR : HA_ERR_CRASHED_ON_USAGE);
+ goto err;
+ }
+ /*
+ Tell future openers that open_count was positive at first open (sign
+ of a problem). See myisam_backup_engine.cc.
+ */
+ if (my_disable_locking && share->state.open_count)
+ {
+ DBUG_PRINT("info", ("STATE_BAD_OPEN_COUNT set on"));
+ share->state.changed|= STATE_BAD_OPEN_COUNT;
+ }
}
/* sanity check */
@@ -295,6 +328,7 @@ MI_INFO *mi_open(const char *name, int m
&share->rec,
(share->base.fields+1)*sizeof(MI_COLUMNDEF),
&share->blobs,sizeof(MI_BLOB)*share->base.blobs,
+ &share->unresolv_file_name,strlen(name)+1,
&share->unique_file_name,strlen(name_buff)+1,
&share->index_file_name,strlen(index_name)+1,
&share->data_file_name,strlen(data_name)+1,
@@ -316,6 +350,7 @@ MI_INFO *mi_open(const char *name, int m
memcpy((char*) share->state.key_del,
(char*) key_del, (sizeof(my_off_t) *
share->state.header.max_block_size_index));
+ strmov(share->unresolv_file_name,name);
strmov(share->unique_file_name, name_buff);
share->unique_name_length= strlen(name_buff);
strmov(share->index_file_name, index_name);
@@ -513,6 +548,7 @@ MI_INFO *mi_open(const char *name, int m
#ifdef THREAD
thr_lock_init(&share->lock);
VOID(pthread_mutex_init(&share->intern_lock,MY_MUTEX_INIT_FAST));
+ my_atomic_rwlock_init(&share->physical_logging_rwlock);
for (i=0; i<keys; i++)
VOID(my_rwlock_init(&share->key_root_lock[i], NULL));
VOID(my_rwlock_init(&share->mmap_lock, NULL));
@@ -570,7 +606,6 @@ MI_INFO *mi_open(const char *name, int m
share->base.max_key_length),
&info.lastkey,share->base.max_key_length*3+1,
&info.first_mbr_key, share->base.max_key_length,
- &info.filename,strlen(name)+1,
&info.rtree_recursion_state,have_rtree ? 1024 : 0,
NullS))
goto err;
@@ -579,7 +614,6 @@ MI_INFO *mi_open(const char *name, int m
if (!have_rtree)
info.rtree_recursion_state= NULL;
- strmov(info.filename,name);
memcpy(info.blobs,share->blobs,sizeof(MI_BLOB)*share->base.blobs);
info.lastkey2=info.lastkey+share->base.max_key_length;
@@ -625,6 +659,13 @@ MI_INFO *mi_open(const char *name, int m
myisam_delay_key_write)
share->delay_key_write=1;
info.state= &share->state.state; /* Change global values by default */
+
+ if (unlikely(share->state.changed & STATE_BAD_OPEN_COUNT))
+ {
+ /* client may be a reader: ensure new state's flag not lost */
+ mi_state_info_write(share, share->kfile, &share->state, 1);
+ }
+
pthread_mutex_unlock(&share->intern_lock);
/* Allocate buffer for one record */
@@ -638,14 +679,19 @@ MI_INFO *mi_open(const char *name, int m
#ifdef THREAD
thr_lock_data_init(&share->lock,&m_info->lock,(void*) m_info);
#endif
+ if (mi_log_tables_physical &&
+ hash_search(mi_log_tables_physical, (uchar *)share->unique_file_name,
+ share->unique_name_length))
+ m_info->s->physical_logging= TRUE; /* set before publishing table */
m_info->open_list.data=(void*) m_info;
myisam_open_list=list_add(myisam_open_list,&m_info->open_list);
pthread_mutex_unlock(&THR_LOCK_myisam);
- if (myisam_log_file >= 0)
+ if (my_b_inited(&myisam_logical_log))
{
intern_filename(name_buff,share->index_file_name);
- _myisam_log(MI_LOG_OPEN, m_info, (uchar*) name_buff, strlen(name_buff));
+ _myisam_log_command(&myisam_logical_log, MI_LOG_OPEN, m_info,
+ (uchar *)name_buff, strlen(name_buff), 0);
}
DBUG_RETURN(m_info);
@@ -853,15 +899,25 @@ static void setup_key_functions(register
}
-/*
- Function to save and store the header in the index file (.MYI)
+/**
+ Function to store state into the index file's header.
+
+ @param share table's share
+ @param file file descriptor of the index file
+ @param state state of the table
+ @param pWrite if my_pwrite() or my_write() should be used
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
*/
-uint mi_state_info_write(File file, MI_STATE_INFO *state, uint pWrite)
+uint mi_state_info_write(MYISAM_SHARE *share, File file,
+ MI_STATE_INFO *state, uint pWrite)
{
uchar buff[MI_STATE_INFO_SIZE + MI_STATE_EXTRA_SIZE];
uchar *ptr=buff;
- uint i, keys= (uint) state->header.keys,
+ uint i, ret, keys= (uint) state->header.keys,
key_blocks=state->header.max_block_size_index;
DBUG_ENTER("mi_state_info_write");
@@ -913,11 +969,13 @@ uint mi_state_info_write(File file, MI_S
}
}
- if (pWrite & 1)
- DBUG_RETURN(my_pwrite(file, buff, (size_t) (ptr-buff), 0L,
- MYF(MY_NABP | MY_THREADSAFE)) != 0);
- DBUG_RETURN(my_write(file, buff, (size_t) (ptr-buff),
- MYF(MY_NABP)) != 0);
+ ret= (pWrite & 1) ? (my_pwrite(file, buff, (uint) (ptr-buff), 0L,
+ MYF(MY_NABP | MY_THREADSAFE)) != 0):
+ (my_write(file, buff, (uint) (ptr-buff), MYF(MY_NABP)) != 0);
+ if (mi_get_physical_logging_state(share))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI,
+ share, buff, (uint) (ptr-buff), 0L);
+ DBUG_RETURN(ret);
}
@@ -1346,3 +1404,34 @@ int mi_indexes_are_disabled(MI_INFO *inf
return 2;
}
+
+/**
+ Logs when the key cache flushes a page to the file (so far, always the
+ index file), to the physical log.
+
+ Argument cannot be a MI_INFO* (the MI_INFO which put the page in the key
+ cache may have been freed long ago when the page is finally flushed), it is
+ MYISAM_SHARE* which is sure to be valid.
+
+ @param arg MYISAM_SHARE* where the block belongs
+ @param buffert argument to the pwrite
+ @param length length of buffer
+ @param filepos offset in file where buffer was written
+
+ @return Operation status, always 0
+ @retval 0 ok. Yes, even if log write fails we return ok, don't want
+ to make the table writer believe its table is now
+ corrupted.
+*/
+
+static int log_key_page_flush_physical(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos)
+{
+ MYISAM_SHARE *share= (MYISAM_SHARE *)arg;
+ DBUG_ENTER("log_key_page_flush_physical");
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(share)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, share, buffert,
+ length, filepos);
+ DBUG_RETURN(0);
+}
diff -Nrup a/storage/myisam/mi_page.c b/storage/myisam/mi_page.c
--- a/storage/myisam/mi_page.c 2008-04-01 15:44:56 +02:00
+++ b/storage/myisam/mi_page.c 2008-05-07 17:30:58 +02:00
@@ -65,15 +65,16 @@ int _mi_write_keypage(register MI_INFO *
my_off_t page, int level, uchar *buff)
{
reg3 uint length;
+ MYISAM_SHARE *share= info->s;
DBUG_ENTER("_mi_write_keypage");
#ifndef FAST /* Safety check */
- if (page < info->s->base.keystart ||
+ if (page < share->base.keystart ||
page+keyinfo->block_length > info->state->key_file_length ||
(page & (MI_MIN_KEY_BLOCK_LENGTH-1)))
{
DBUG_PRINT("error",("Trying to write inside key status region: key_start: %lu length: %lu page: %lu",
- (long) info->s->base.keystart,
+ (long) share->base.keystart,
(long) info->state->key_file_length,
(long) page));
my_errno=EINVAL;
@@ -93,11 +94,11 @@ int _mi_write_keypage(register MI_INFO *
length=keyinfo->block_length;
}
#endif
- DBUG_RETURN((key_cache_write(info->s->key_cache,
- info->s->kfile,page, level, (uchar*) buff,length,
- (uint) keyinfo->block_length,
- (int) ((info->lock_type != F_UNLCK) ||
- info->s->delay_key_write))));
+ DBUG_RETURN(key_cache_write(share->key_cache,
+ share->kfile, page, level, (uchar*)buff, length,
+ (uint) keyinfo->block_length,
+ (int) ((info->lock_type != F_UNLCK) ||
+ share->delay_key_write), share));
} /* mi_write_keypage */
@@ -108,18 +109,19 @@ int _mi_dispose(register MI_INFO *info,
{
my_off_t old_link;
uchar buff[8];
+ MYISAM_SHARE *share= info->s;
DBUG_ENTER("_mi_dispose");
DBUG_PRINT("enter",("pos: %ld", (long) pos));
- old_link= info->s->state.key_del[keyinfo->block_size_index];
- info->s->state.key_del[keyinfo->block_size_index]= pos;
+ old_link= share->state.key_del[keyinfo->block_size_index];
+ share->state.key_del[keyinfo->block_size_index]= pos;
mi_sizestore(buff,old_link);
- info->s->state.changed|= STATE_NOT_SORTED_PAGES;
- DBUG_RETURN(key_cache_write(info->s->key_cache,
- info->s->kfile, pos , level, buff,
- sizeof(buff),
- (uint) keyinfo->block_length,
- (int) (info->lock_type != F_UNLCK)));
+ share->state.changed|= STATE_NOT_SORTED_PAGES;
+ DBUG_RETURN(key_cache_write(share->key_cache,
+ share->kfile, pos, level, buff,
+ sizeof(buff),
+ (uint) keyinfo->block_length,
+ (int) (info->lock_type != F_UNLCK), share));
} /* _mi_dispose */
diff -Nrup a/storage/myisam/mi_panic.c b/storage/myisam/mi_panic.c
--- a/storage/myisam/mi_panic.c 2006-12-31 01:06:40 +01:00
+++ b/storage/myisam/mi_panic.c 2008-05-07 17:30:58 +02:00
@@ -54,6 +54,7 @@ int mi_panic(enum ha_panic_function flag
error=my_errno;
if (info->opt_flag & READ_CACHE_USED)
{
+ /* QQ Why do we flush a READ_CACHE? it's a no-op */
if (flush_io_cache(&info->rec_cache))
error=my_errno;
reinit_io_cache(&info->rec_cache,READ_CACHE,0,
@@ -78,15 +79,17 @@ int mi_panic(enum ha_panic_function flag
{ /* Open closed files */
char name_buff[FN_REFLEN];
if (info->s->kfile < 0)
- if ((info->s->kfile= my_open(fn_format(name_buff,info->filename,"",
- N_NAME_IEXT,4),info->mode,
- MYF(MY_WME))) < 0)
+ if ((info->s->kfile= my_open(fn_format(name_buff,
+ info->s->unresolv_file_name,
+ "", N_NAME_IEXT, 4),
+ info->mode, MYF(MY_WME))) < 0)
error = my_errno;
if (info->dfile < 0)
{
- if ((info->dfile= my_open(fn_format(name_buff,info->filename,"",
- N_NAME_DEXT,4),info->mode,
- MYF(MY_WME))) < 0)
+ if ((info->dfile= my_open(fn_format(name_buff,
+ info->s->unresolv_file_name,
+ "", N_NAME_DEXT, 4),
+ info->mode, MYF(MY_WME))) < 0)
error = my_errno;
info->rec_cache.file=info->dfile;
}
@@ -103,7 +106,8 @@ int mi_panic(enum ha_panic_function flag
}
if (flag == HA_PANIC_CLOSE)
{
- VOID(mi_log(0)); /* Close log if neaded */
+ /* Close log if needed */
+ mi_log(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_LOGICAL, NULL, NULL);
ft_free_stopwords();
}
pthread_mutex_unlock(&THR_LOCK_myisam);
diff -Nrup a/storage/myisam/mi_rrnd.c b/storage/myisam/mi_rrnd.c
--- a/storage/myisam/mi_rrnd.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_rrnd.c 2008-05-07 17:30:58 +02:00
@@ -31,10 +31,8 @@
int mi_rrnd(MI_INFO *info, uchar *buf, register my_off_t filepos)
{
- my_bool skip_deleted_blocks;
+ my_bool skip_deleted_blocks= 0;
DBUG_ENTER("mi_rrnd");
-
- skip_deleted_blocks=0;
if (filepos == HA_OFFSET_ERROR)
{
diff -Nrup a/storage/myisam/mi_static.c b/storage/myisam/mi_static.c
--- a/storage/myisam/mi_static.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_static.c 2008-05-07 17:30:58 +02:00
@@ -13,9 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Static variables for MyISAM library. All definied here for easy making of
- a shared library
+/**
+ @file
+ Static variables for MyISAM library.
+ All defined here for easy making of a shared library
*/
#ifndef _global_h
@@ -27,8 +28,9 @@ uchar NEAR myisam_file_magic[]=
{ (uchar) 254, (uchar) 254,'\007', '\001', };
uchar NEAR myisam_pack_file_magic[]=
{ (uchar) 254, (uchar) 254,'\010', '\002', };
-char * myisam_log_filename=(char*) "myisam.log";
-File myisam_log_file= -1;
+char *myisam_logical_log_filename= (char*)"myisam.log";
+IO_CACHE myisam_physical_log; /**< Physical log (used by online backup) */
+IO_CACHE myisam_logical_log; /**< Logical log (used for debugging) */
uint myisam_quick_table_bits=9;
ulong myisam_block_size= MI_KEY_BLOCK_LENGTH; /* Best by test */
my_bool myisam_flush=0, myisam_delay_key_write=0, myisam_single_user=0;
@@ -59,3 +61,34 @@ uint NEAR myisam_readnext_vec[]=
SEARCH_BIGGER, SEARCH_BIGGER, SEARCH_SMALLER, SEARCH_BIGGER, SEARCH_SMALLER,
SEARCH_BIGGER, SEARCH_SMALLER, SEARCH_SMALLER
};
+
+/** Hash of all tables for which we want physical logging */
+const HASH *mi_log_tables_physical;
+/**
+ If page changes to the index file should be logged to the physical log.
+
+ @note Changes to the header of the index file of a table in physical
+ logging are always logged because the header is not redundant with the data
+ file.
+*/
+my_bool mi_log_index_pages_physical;
+
+/**
+ All MyISAM-specific error messages which may be sent to the user.
+ They will be localized (translated) as part of
+ http://forge.mysql.com/worklog/task.php?id=2940
+ "MySQL plugin interface: error reporting".
+ Same order as enum myisam_errors.
+*/
+const char *myisam_error_messages[] =
+{
+ "online backup impossible with --external-locking",
+ "backup archive format has too recent version (%u) (current: %u)"
+};
+
+static inline void myisam_error_messages_dummy_validator()
+{
+ compile_time_assert((sizeof(myisam_error_messages) /
+ sizeof(myisam_error_messages[0])) ==
+ (-MYISAM_ERR_LAST-1));
+}
diff -Nrup a/storage/myisam/mi_test2.c b/storage/myisam/mi_test2.c
--- a/storage/myisam/mi_test2.c 2008-04-01 15:44:57 +02:00
+++ b/storage/myisam/mi_test2.c 2008-05-07 17:30:58 +02:00
@@ -215,7 +215,7 @@ int main(int argc, char *argv[])
&create_info,create_flag))
goto err;
if (use_log)
- mi_log(1);
+ mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
if (!(file=mi_open(filename,2,HA_OPEN_ABORT_IF_LOCKED)))
goto err;
if (!silent)
@@ -657,10 +657,10 @@ int main(int argc, char *argv[])
sprintf((char*) key2,"%6d",k);
min_key.key= key;
- min_key.length= USE_WHOLE_KEY;
+ min_key.keypart_map= HA_WHOLE_KEY;
min_key.flag= HA_READ_AFTER_KEY;
max_key.key= key2;
- max_key.length= USE_WHOLE_KEY;
+ max_key.keypart_map= HA_WHOLE_KEY;
max_key.flag= HA_READ_BEFORE_KEY;
range_records= mi_records_in_range(file, 0, &min_key, &max_key);
records=0;
diff -Nrup a/storage/myisam/mi_test3.c b/storage/myisam/mi_test3.c
--- a/storage/myisam/mi_test3.c 2008-04-01 15:44:57 +02:00
+++ b/storage/myisam/mi_test3.c 2008-05-07 17:30:58 +02:00
@@ -169,7 +169,7 @@ void start_test(int id)
MI_INFO *file,*file1,*file2=0,*lock;
if (use_log)
- mi_log(1);
+ mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
if (!(file1=mi_open(filename,O_RDWR,HA_OPEN_WAIT_IF_LOCKED)) ||
!(file2=mi_open(filename,O_RDWR,HA_OPEN_WAIT_IF_LOCKED)))
{
@@ -214,7 +214,7 @@ void start_test(int id)
mi_close(file1);
mi_close(file2);
if (use_log)
- mi_log(0);
+ mi_log(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_LOGICAL, NULL, NULL);
if (error)
{
printf("%2d: Aborted\n",id); fflush(stdout);
diff -Nrup a/storage/myisam/mi_test_all.sh b/storage/myisam/mi_test_all.sh
--- a/storage/myisam/mi_test_all.sh 2007-07-28 13:36:15 +02:00
+++ b/storage/myisam/mi_test_all.sh 2008-05-07 17:30:58 +02:00
@@ -135,8 +135,10 @@ echo "mi_test2$suffix $silent -L -K -R1
./mi_test2$suffix $silent -m10000 -e16384 -E16384 -K -L
./myisamchk$suffix -sm test2
+/bin/rm myisam.log
./mi_test2$suffix $silent -L -K -W -P -m50 -l
./myisamlog$suffix
+/bin/rm myisam.log
./mi_test2$suffix $silent -L -K -W -P -m50 -l -b100
./myisamlog$suffix
time ./mi_test2$suffix $silent
diff -Nrup a/storage/myisam/mi_update.c b/storage/myisam/mi_update.c
--- a/storage/myisam/mi_update.c 2008-02-18 23:29:35 +01:00
+++ b/storage/myisam/mi_update.c 2008-05-07 17:30:58 +02:00
@@ -170,7 +170,7 @@ int mi_update(register MI_INFO *info, co
info->update= (HA_STATE_CHANGED | HA_STATE_ROW_CHANGED | HA_STATE_AKTIV |
key_changed);
- myisam_log_record(MI_LOG_UPDATE,info,newrec,info->lastpos,0);
+ myisam_log_record_logical(MI_LOG_UPDATE, info, newrec, info->lastpos, 0);
/*
Every myisam function that updates myisam table must end with
call to _mi_writeinfo(). If operation (second param of
@@ -185,8 +185,9 @@ int mi_update(register MI_INFO *info, co
allow_break(); /* Allow SIGHUP & SIGINT */
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (update)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (update)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
DBUG_RETURN(0);
@@ -231,7 +232,8 @@ err:
key_changed);
err_end:
- myisam_log_record(MI_LOG_UPDATE,info,newrec,info->lastpos,my_errno);
+ myisam_log_record_logical(MI_LOG_UPDATE, info, newrec,
+ info->lastpos, my_errno);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
if (save_errno == HA_ERR_KEY_NOT_FOUND)
diff -Nrup a/storage/myisam/mi_write.c b/storage/myisam/mi_write.c
--- a/storage/myisam/mi_write.c 2008-04-09 07:41:37 +02:00
+++ b/storage/myisam/mi_write.c 2008-05-07 17:30:58 +02:00
@@ -154,12 +154,13 @@ int mi_write(MI_INFO *info, uchar *recor
HA_STATE_ROW_CHANGED);
info->state->records++;
info->lastpos=filepos;
- myisam_log_record(MI_LOG_WRITE,info,record,filepos,0);
+ myisam_log_record_logical(MI_LOG_WRITE, info, record, filepos, 0);
VOID(_mi_writeinfo(info, WRITEINFO_UPDATE_KEYFILE));
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (update)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (update)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
@@ -231,7 +232,7 @@ err:
my_errno=save_errno;
err2:
save_errno=my_errno;
- myisam_log_record(MI_LOG_WRITE,info,record,filepos,my_errno);
+ myisam_log_record_logical(MI_LOG_WRITE, info, record, filepos, my_errno);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
DBUG_RETURN(my_errno=save_errno);
diff -Nrup a/storage/myisam/myisam_backup_engine.cc b/storage/myisam/myisam_backup_engine.cc
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/storage/myisam/myisam_backup_engine.cc 2008-05-07 17:30:59 +02:00
@@ -0,0 +1,1934 @@
+/* Copyright (C) 2007 MySQL AB
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; version 2 of the License.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program; if not, write to the Free Software
+ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+*/
+
+/**
+ @file
+ Online backup engine for the MyISAM storage engine.
+
+ @see myisam_backup
+*/
+
+#define MYSQL_SERVER 1 // need it to have mysql_tmpdir defined
+#include "mysql_priv.h"
+#include "ha_myisam.h"
+#include "myisamdef.h" // to access dfile and kfile
+#include "backup/backup_engine.h"
+#include "backup/backup_aux.h" // for build_table_list()
+#include <hash.h>
+
+/**
+ Online backup engine for the MyISAM storage engine.
+
+ Reference of the Online Backup API:
+ http://forge.mysql.com/source/OnlineBackup.
+
+ Here is how the MyISAM online backup works.
+ It is online because we dirtily copy the data and index files,
+ and the tables maintain a physical idempotent log of changes done to them
+ during the copy process, applying this log to the dirty copy yields a clean
+ table corresponding to how the original table was when logging ended.
+ Idempotent means that if you apply such log to a table, then applying it a
+ second time has no effect.
+
+ A condition for this to work is that any update done to a table after the
+ copy process started must be present in the log. See the comment of
+ mi_log_start_physical() for how this is ensured.
+
+ HOW THE BACKUP WORKS
+
+ In Backup::begin(), we instruct all needed tables to do backup
+ logging; this does not have to wait for existing updates to complete,
+ neither does it stall new updates.
+
+ Then we dirtily copy them in Backup::get_data(). That copy is intensive on
+ the hard drive, so can be optionally throttled (via a configurable sleep).
+
+ When the copy process is done with tables, it signals the backup kernel
+ that it is ready to lock tables (to create a validity point).
+ To not waste its time until the Backup::prelock() request is sent by the
+ backup kernel, the copy process starts copying the log.
+
+ Now the Backup::prelock() request comes.
+ To finish the backup, we need to synchronize (=read-lock) all tables of the
+ backup (thus creating a consistent state accross them), stop logging for
+ all of them, and unlock tables. This lock can wait for a long time if there
+ is a long running update. If it waited a long time, other drivers which have
+ already executed their lock(), would stay locked for a long time. To avoid
+ that, we do all the locking work before Backup::lock(), in
+ Backup::prelock() (called before lock() on any driver). Backup::prelock()
+ itself is not allowed to block, because it is called from the backup
+ kernel's thread: so it launches a separate thread (which will issue a LOCK
+ TABLES READ on our tables) and does not wait for completion of LOCK TABLES
+ READ: it immediately returns backup::OK which means "I have not completed my
+ preparations for locking".
+
+ In Backup::get_data(), the driver monitors the status of the locking
+ thread, and when finally that thread has managed to get its locks, we stop
+ logging and reply backup::READY.
+
+ So note the difference: this time, we have to wait for all updates to
+ finish, and stall new ones.
+
+ Next Backup::get_data() calls, if there are, send the final tail of the
+ log.
+
+ Backup::lock() comes, it's an empty operation for the driver.
+
+ Later we get a Backup::unlock() request. That kills the locking thread,
+ which thus unlocks tables. And Backup::end() cleans up memory.
+
+ HOW THE RESTORE WORKS
+
+ In Restore::send_data() we receive data which we write to tables (those
+ tables have just been created with their correct structure, but no data, by
+ the backup kernel). We similarly restore the log.
+
+ In Restore::end(), we apply the log to tables, making them clean.
+ If of the index file we backed up only the header (an option), we here do
+ an index rebuild.
+ Voila, the table is ready to work.
+
+ @todo if an index rebuild is needed, possibly do it at backup time.
+*/
+namespace myisam_backup {
+
+using backup::byte;
+using backup::result_t;
+using backup::version_t;
+using backup::Table_list;
+using backup::Table_ref;
+using backup::Buffer;
+
+/**
+ The current version of the format stored in MyISAM backup images by this
+ code. Increase it when making a backward-incompatible change.
+*/
+#define MYISAM_BACKUP_VERSION 1
+
+/** Like Table_ref but with file name added */
+class Myisam_table_ref
+{
+public:
+ Myisam_table_ref(const Table_ref &);
+protected:
+ String db, name;
+ String file_name; ///< concatenation of db and table name
+};
+
+
+Myisam_table_ref::Myisam_table_ref(const Table_ref &tbl)
+{
+ const char *db_arg= tbl.db().name().ptr();
+ const char *name_arg= tbl.name().ptr();
+ /**
+ We keep local copies of the db and name. This is because during restore,
+ the Table_ref is apparently modified before the Table_restore is done
+ (symptom is that starting from second Table_restore::send_data() we see
+ Table_ref being garbage, and this is a problem in
+ Table_restore::post_restore()). Rafal suspects a bug.
+ @todo Once fixed, we can replace "String db,name" by "Table_ref &ref",
+ and this will save memory.
+ As Rafal is changing relevant code now, it may go away.
+ */
+ db.append(db_arg);
+ name.append(name_arg);
+ /*
+ Note: this way below will break with non-ASCII characters;
+ what driver should be passed is what engine used to create the table, ie
+ output of build_table_filename().
+ Just replace "building" with "bûilding" in backup.test to see issues.
+ This is remembered in WL#4060.
+
+ But note, when we repair the table, we use open_temporary_table() which
+ requires db and table name separated.
+ */
+ file_name.append("./");
+ file_name.append(db_arg);
+ file_name.append("/");
+ file_name.append(name_arg);
+}
+
+
+/**
+ Backup engine class. It is the master class: a Backup_engine creates a
+ Backup_driver and a corresponding Restore_driver. @see backup::Engine.
+*/
+class Engine: public Backup_engine
+{
+ public:
+ Engine() {}
+ virtual const version_t version() const { return MYISAM_BACKUP_VERSION; };
+ virtual result_t get_backup(const uint32, const Table_list &,
+ Backup_driver* &);
+ virtual result_t get_restore(const version_t, const uint32,
+ const Table_list &,Restore_driver* &);
+ virtual void free() { delete this; }
+};
+
+/*************************
+ *
+ * BACKUP FUNCTIONALITY
+ *
+ *************************/
+
+class Object_backup;
+
+
+/**
+ Handles backup orders received from the backup kernel (implements the API).
+*/
+class Backup: public Backup_driver
+{
+public:
+ Backup(const Table_list &);
+ virtual ~Backup();
+ /** Estimates total size of backup. @todo improve it */
+ virtual size_t size() { return UNKNOWN_SIZE; };
+ /** Estimates size of backup before lock. @todo improve it */
+ virtual size_t init_size() { return UNKNOWN_SIZE; };
+ virtual result_t begin(const size_t);
+ virtual result_t end();
+ virtual result_t get_data(Buffer &);
+ virtual result_t prelock();
+ virtual result_t lock();
+ virtual result_t unlock();
+ virtual result_t cancel()
+ {
+ return backup::OK ; // free() will be called and suffice
+ };
+ virtual void free() { delete this; };
+ void lock_tables_TL_READ_NO_INSERT();
+
+private:
+ enum { DUMPING_DATA_INDEX_FILES,
+ DUMPING_LOG_FILE_BEFORE_TABLES_ARE_LOCKED,
+ DUMPING_LOG_FILE_AFTER_TABLES_ARE_LOCKED,
+ DONE, ERROR } state;
+ Object_backup *image; ///< object in backup currently
+ uint stream; ///< which stream we are currently writing to
+ char backup_log_name[FN_REFLEN];
+ /**
+ All db||table names in a HASH structure. Passed to MyISAM functions for
+ them to detect if a table is part of the backup (=> should do logging) or
+ not.
+ */
+ HASH *hash_of_tables;
+ /**
+ Locking of tables goes through these states. It is a delicate variable
+ which must be set correctly after inspecting thread-safety and race
+ conditions.
+ */
+ enum { LOCK_NOT_STARTED, LOCK_STARTED, LOCK_ACQUIRED, LOCK_ERROR }
+ lock_state;
+ /**
+ The locking thread (so that we can kill it). Creating a validity point is
+ only possible by locking all tables (it is the only way to have tables
+ consistent with each other, as we have no UNDO log). But locking via
+ thr_lock() is blocking. So, to have a non-blocking prelock() call, this
+ locking is done in a separate thread (named "the locking thread").
+ */
+ THD *lock_thd;
+ bool cannot_delete_lock_thd;
+ pthread_cond_t COND_lock_state; ///< for communication with locking thread
+ void kill_locking_thread();
+ static const size_t bytes_between_sleeps= 10*1024*1024;
+ /** After copying bytes_between_sleeps we sleep sleep_time */
+ ulong sleep_time;
+ size_t bytes_since_last_sleep; ///< how many bytes sent since we last slept
+};
+
+/**
+ When we send a backup packet to the backup kernel, we prefix it with a code
+ which tells which type of file this packet belongs to. Starts at 1 because
+ garbage is often zeros and we want to spot it.
+*/
+enum enum_file_code { DATA_FILE_CODE= 1,
+ WHOLE_INDEX_FILE_CODE, HEADER_INDEX_FILE_CODE,
+ LOG_FILE_CODE };
+
+/** An object to backup; in practice, a table or the log */
+class Object_backup
+{
+public:
+ virtual result_t get_data(Buffer &)= 0;
+ virtual ~Object_backup() {};
+ bool internal_error() { return state == ERROR; }
+ /**
+ The only reason to have an end() and call it from the destructor, instead
+ of putting the code into the destructor, is that when the caller does a
+ "delete image", it cannot be told about errors, while if the caller does
+ "image->end()" (and then "delete image") it can see an error.
+ */
+ virtual result_t end()= 0; ///< cleanups
+protected:
+ enum { OK, ERROR } state; ///< serves to detect an error during construction
+};
+
+
+/**
+ An object to back up is made of one or more such files. This class does not
+ open the file, user has to open it. This class provides a helper method if
+ its user wants to close the file.
+*/
+class File_backup
+{
+public:
+ File_backup() : fd(-1), backup_file_size(0) {}
+
+ /**
+ Initializes the object.
+
+ @param fd_arg file descriptor to attach to
+ @param file_size_arg copy should stop after copying that many bytes
+ @param file_code_arg code to store at start of each sent data packet
+ */
+
+ void init(int fd_arg, my_off_t file_size_arg, enum_file_code file_code_arg)
+ { fd= fd_arg; file_size= file_size_arg; file_code= file_code_arg; }
+
+ result_t get_data(Buffer &);
+ result_t close_file();
+private:
+ int fd; ///< file descriptor
+ /**
+ After backing up that many bytes of the file, we can stop. In case of
+ ftruncate() happening to the file, we may even copy less than this size.
+ */
+ my_off_t file_size;
+ enum_file_code file_code; ///< code stored at start of each backup block
+ my_off_t backup_file_size; ///< how much of the file we already backed up
+};
+
+
+/** Handles backing up a single table */
+class Table_backup: public Myisam_table_ref, public Object_backup
+{
+public:
+ Table_backup(const backup::Table_ref &);
+ virtual ~Table_backup();
+ virtual result_t get_data(Buffer &);
+ virtual result_t end(); ///< cleanups
+private:
+ File_backup dfile_backup, kfile_backup;
+ enum { DATA_FILE, INDEX_FILE } in_file; ///< which file we are dumping now
+};
+
+
+/** Handles backing up the log */
+class Log_backup: public Object_backup
+{
+public:
+ Log_backup(const char *);
+ virtual ~Log_backup();
+ virtual result_t get_data(Buffer &);
+ virtual result_t end();
+private:
+ const char *log_name;
+ File_backup log_file_backup;
+ bool log_deleted; ///< if we have already deleted the log or not
+};
+
+
+/**
+ Creates a backup driver, per the backup API. @see backup::Engine.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Engine::get_backup(const uint32, const Table_list &tables,
+ Backup_driver* &drv)
+{
+ Backup *ptr= new Backup(tables);
+ if (unlikely(!ptr))
+ return backup::ERROR;
+ drv= ptr;
+ return backup::OK;
+}
+
+
+Backup::Backup(const Table_list &tables):
+ Backup_driver(tables), state(ERROR), image(NULL), stream(1),
+ hash_of_tables(NULL), lock_state(LOCK_NOT_STARTED), lock_thd(NULL),
+ cannot_delete_lock_thd(FALSE), bytes_since_last_sleep(0)
+{
+ /*
+ Driver is not ready at this point, so state is ERROR.
+ This constructor cannot fail, otherwise begin() would have to detect it.
+ */
+ pthread_cond_init(&COND_lock_state, NULL);
+}
+
+
+/** Kills the locking thread when it is time to unlock tables */
+
+void Backup::kill_locking_thread()
+{
+ DBUG_ENTER("myisam_backup::Backup::kill_locking_thread");
+ /*
+ If everything worked well, when unlock() calls us we kill the thread and
+ so when free() calls us the locking thread is already dead here
+ (LOCK_ERROR).
+ */
+retry:
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ /* If thread started and not already dead, kill it */
+ if ((lock_state != LOCK_NOT_STARTED) & (lock_state != LOCK_ERROR))
+ {
+ /*
+ If the locking thread has not yet created THD (very unlikely), wait
+ for it.
+ */
+ if (unlikely(lock_thd == NULL))
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_PRINT("info",("lock_thd not yet set"));
+ sleep(1);
+ goto retry;
+ }
+ /*
+ Locking thread had time to create its THD, may be inside table locking
+ (waiting for others to release locks etc), wake it up and kill it. Or it
+ may have locked tables successfully, and be waiting for us to kill it.
+ To do that we will use lock_thd, but how to be sure that lock_thd is not
+ being deleted now? One way would be to hold THR_LOCK_myisam but
+ THD::awake() can't bear it (same mutex locked twice).
+ Another way is to take lock_thd->LOCK_delete (THD::awake() requires it
+ anyway), but again that requires that lock_thd is not deleted while we
+ access the mutex. We cannot hold THR_LOCK_myisam to get LOCK_delete,
+ because that could deadlock if a some other thread is doing a KILL on
+ the locking thread (it would indeed take LOCK_delete and then
+ THR_LOCK_myisam to wake up the locking thread).
+ So So we set a flag:
+ */
+ cannot_delete_lock_thd= TRUE;
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ /*
+ So now lock_thd cannot be destroyed.
+ We kill the thread (which will in particular work if it is waiting for
+ some table locks).
+ */
+ pthread_mutex_lock(&lock_thd->LOCK_delete);
+ lock_thd->awake(THD::KILL_CONNECTION);
+ pthread_mutex_unlock(&lock_thd->LOCK_delete);
+ /* won't look at lock_thd anymore, allow its deletion */
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ cannot_delete_lock_thd= FALSE;
+ /* we wake up thread if it was blocked on the bool above */
+ pthread_cond_broadcast(&COND_lock_state);
+ /* And we wait for the thread to inform of its death */
+ while (lock_state != LOCK_ERROR)
+ pthread_cond_wait(&COND_lock_state, &THR_LOCK_myisam);
+ }
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ This destructor is only called by the class' free().
+ It cleans up any leftover the driver could have. It is safe to call it at
+ any point. In a normal (no error) situation, the hash freeing is the only
+ operation done here, all the rest should already have been done by earlier
+ stages.
+*/
+
+Backup::~Backup()
+{
+ DBUG_ENTER("myisam_backup::Backup::~Backup");
+ /* If we had already started backup logging, we must dirtily stop it */
+ mi_log(MI_LOG_ACTION_CLOSE_INCONSISTENT, MI_LOG_PHYSICAL, NULL, NULL);
+ delete image;
+ if (hash_of_tables)
+ {
+ hash_free(hash_of_tables);
+ delete hash_of_tables;
+ hash_of_tables= NULL;
+ }
+ kill_locking_thread();
+ pthread_cond_destroy(&COND_lock_state);
+ DBUG_VOID_RETURN;
+}
+
+
+/** Usual parameter to hash_init() */
+
+static uchar
+*backup_get_table_from_hash_key(const uchar *lsc, size_t *length,
+ my_bool not_used __attribute__ ((unused)))
+{
+ const ::LEX_STRING *ls= reinterpret_cast<const ::LEX_STRING *>(lsc);
+ *length= ls->length;
+ return reinterpret_cast< uchar *>(ls->str);
+}
+
+
+/** Usual parameter to hash_init() */
+
+static void backup_free_hash_key(void *lsv)
+{
+ my_free(lsv, MYF(MY_WME));
+}
+
+
+#define SET_STATE_TO_ERROR_AND_DBUG_RETURN { \
+ state= ERROR; \
+ DBUG_PRINT("error",("driver got an error at %s:%d",__FILE__,__LINE__)); \
+ DBUG_RETURN(backup::ERROR); }
+
+/* use this one only in constructors */
+#define SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN { \
+ state= ERROR; \
+ DBUG_PRINT("error",("driver got an error at %s:%d",__FILE__,__LINE__)); \
+ DBUG_VOID_RETURN; }
+
+
+/**
+ Sets MyISAM in a state ready for the copy to start. I.e. builds
+ a hash of tables and starts MyISAM physical logging for those tables.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Backup::begin(const size_t)
+{
+ DBUG_ENTER("myisam_backup::Backup::begin");
+ DBUG_PRINT("info",("%lu tables", m_tables.count()));
+
+ /*
+ per the API, all significant allocations (large mem, opening files) must
+ not be in the constructor but in begin() or later.
+ */
+ DBUG_ASSERT(!hash_of_tables); // no double begin() call or reuse of driver
+ DBUG_ASSERT(m_tables.count() > 0); // or bug in the backup kernel
+ /*
+ If external locking is on, some other processes may modify our tables
+ while we are copying them, those modifications will not reach the log,
+ backup will be corrupted.
+ */
+ if (!my_disable_locking || !myisam_single_user)
+ {
+ my_error(ER_GET_ERRMSG, MYF(0),
+ MYISAM_ERR_NO_BACKUP_WITH_EXTERNAL_LOCKING,
+ MYISAM_ERR(MYISAM_ERR_NO_BACKUP_WITH_EXTERNAL_LOCKING), "MyISAM");
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ hash_of_tables= new HASH;
+ if (!hash_of_tables ||
+ hash_init(hash_of_tables, &my_charset_bin, m_tables.count(), 0, 0,
+ (hash_get_key)backup_get_table_from_hash_key,
+ (hash_free_key)backup_free_hash_key, 0))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ /* Build the hash of tables for the MyISAM layer (mi_backup_log.c etc) */
+ for (uint n=0 ; n < m_tables.count() ; n++ )
+ {
+ char unique_file_name[FN_REFLEN], *str;
+ size_t str_len;
+ ::LEX_STRING *hash_key;
+
+ my_realpath(unique_file_name,
+ fn_format(unique_file_name,m_tables[n].name().ptr(),
+ m_tables[n].db().name().ptr(),
+ MI_NAME_IEXT,
+ MY_UNPACK_FILENAME), MYF(MY_WME));
+ str_len= strlen(unique_file_name);
+ my_multi_malloc(MYF(MY_WME),
+ &hash_key, sizeof(*hash_key),
+ &str, static_cast<uint>(str_len), NullS);
+ if (!hash_key)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ memcpy(str, unique_file_name, str_len);
+ hash_key->length= str_len;
+ hash_key->str= str;
+ if (my_hash_insert(hash_of_tables,
+ reinterpret_cast< uchar *>(hash_key)))
+ {
+ my_free(hash_key, MYF(MY_WME));
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ DBUG_PRINT("info",("table '%.*s' inserted in hash",
+ static_cast<int>(hash_key->length), hash_key->str));
+ }
+
+ {
+ THD *thd= current_thd;
+ /*
+ If tmpdir is in RAM (/dev/shm etc), we may exhaust it if our log is big
+ */
+ my_snprintf(backup_log_name, sizeof(backup_log_name),
+ "%s/%s%lx_%lx_%x-backuplog", mysql_tmpdir,
+ tmp_file_prefix, current_pid, thd->thread_id,
+ thd->tmp_table++); // it's not a tmp table but what...
+ unpack_filename(backup_log_name, backup_log_name);
+ }
+
+ {
+ /**
+ Until there exists a framework by which the user tells, via SQL,
+ indications on how it wants the backup, and by which the backup kernel
+ tells it to the driver (API), we resort to this.
+ */
+ char *env_arg= getenv("MYISAM_BACKUP_NO_INDEX");
+ /* By default we log index pages */
+ mi_log_index_pages_physical= !(env_arg && atoi(env_arg));
+ env_arg= getenv("MYISAM_BACKUP_SLEEP");
+ /*
+ By default we don't sleep at all; however, 500 ms every 10MB gives a
+ low penalty on clients, so it can be a good choice.
+ */
+ sleep_time= env_arg ? atoi(env_arg) : 0;
+ }
+
+ if (mi_log(MI_LOG_ACTION_OPEN, MI_LOG_PHYSICAL,
+ backup_log_name, hash_of_tables))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ state= DUMPING_DATA_INDEX_FILES;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ If some error happened, end() is not called but free() is. So we do all
+ cleanup in free() i.e. in the destructor, and nothing here.
+
+ @return Operation status
+ @retval backup::OK
+*/
+
+result_t Backup::end()
+{
+ DBUG_ENTER("myisam_backup::Backup::end");
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Sends backup data for tables and log to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status (see the API for when they are returned)
+ @retval backup::OK
+ @retval backup::DONE
+ @retval backup::READY
+ @retval backup::ERROR
+*/
+
+result_t Backup::get_data(Buffer &buf)
+{
+ result_t ret;
+ DBUG_ENTER("myisam::backup::Backup::get_data");
+ DBUG_PRINT("enter",("stream %d",stream));
+
+ /* we are currently on stream 'stream' */
+ buf.table_num= stream;
+
+ /*
+ Rafal and I agreed that one single ERROR from the driver will cause the
+ upper layer to not call the driver anymore except for free().
+ */
+ DBUG_ASSERT(state != ERROR);
+ DBUG_ASSERT(buf.data != NULL); // to check that caller gave room
+
+ if (state == DONE)
+ {
+ /*
+ We never come here, because after returning from the call where we sent
+ the last piece of the last stream (when we set our internal state to
+ DONE), all streams were closed, so the upper layer wouldn't call us
+ again. At least it was so during testing. But if it calls us, we do all
+ that the API expects us to do:
+ */
+ buf.size= buf.table_num= 0;
+ buf.last= TRUE;
+ DBUG_RETURN(backup::DONE);
+ }
+
+ if (unlikely(image == NULL))
+ {
+ /*
+ Let's create it.
+ Table 0 will be image 1 on stream 1. Table N will be image N+1 on stream
+ N+1. Log will be image 0 on stream 0.
+ */
+ if (stream >= 1)
+ image= new Table_backup(m_tables[stream-1]);
+ else
+ image= new Log_backup(backup_log_name);
+ if (image == NULL || image->internal_error())
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+
+ if ((ret= image->get_data(buf)) != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ if (sleep_time)
+ {
+ bytes_since_last_sleep+= buf.size;
+ /* sched_yield() is not as flexible (higher penalty) as sleep() */
+ if (bytes_since_last_sleep > bytes_between_sleeps)
+ {
+ my_sleep(sleep_time * 1000UL);
+ bytes_since_last_sleep= 0;
+ }
+ }
+
+ if (state == DUMPING_LOG_FILE_BEFORE_TABLES_ARE_LOCKED)
+ {
+ DBUG_ASSERT(stream == 0);
+ /*
+ We are sending the log; even if reached its EOF, some more may be
+ appended to it before prelock() ends, so this is not the stream's end.
+ */
+ buf.last= FALSE;
+ /*
+ API docs say we should return READY, but Rafal says OK is better (one
+ READY to signal end of initial phase; then OKs; one READY to signal end
+ of prelock(); then OKs).
+ */
+ if (lock_state == LOCK_NOT_STARTED)
+ DBUG_RETURN(backup::OK);
+ /* Let's see if the locking thread has finished locking all tables */
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ if (lock_state == LOCK_STARTED) // not yet
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ DBUG_RETURN(backup::OK);
+ }
+ if (lock_state != LOCK_ACQUIRED) // it failed, so do we
+ {
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ DBUG_PRINT("info",("locking thread acquired locks on tables"));
+
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ if (mi_log(MI_LOG_ACTION_CLOSE_CONSISTENT, MI_LOG_PHYSICAL, NULL, NULL))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ state= DUMPING_LOG_FILE_AFTER_TABLES_ARE_LOCKED;
+ /* signal "end of prepare-for-lock, ready for lock()" */
+ DBUG_RETURN(backup::READY);
+ }
+ else if (buf.last)
+ {
+ /*
+ we are sending the last chunk of the image, next call will be about the
+ next image:
+ */
+ if (image->end() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ delete image;
+ image= NULL; /* next call of this function should open the next object */
+ stream++; /* and send it on the next stream */
+ if (state == DUMPING_DATA_INDEX_FILES && stream > m_tables.count())
+ {
+ /* all tables done */
+ stream= 0; // send the log on stream 0
+ state= DUMPING_LOG_FILE_BEFORE_TABLES_ARE_LOCKED;
+ ret= backup::READY; // end of initial phase
+ }
+ else if (state == DUMPING_LOG_FILE_AFTER_TABLES_ARE_LOCKED) // log done
+ state= DONE;
+ }
+
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Creates a validity point by locking all tables. This is the only job of the
+ locking thread: call this function which locks tables, then wait for being
+ killed (which will unlock tables).
+
+ @todo GUILHEM_TODO use sql/backup/be_thread.cc instead.
+
+ @todo use a method which does not open closed tables. This will be needed
+ when backing up lots of tables (more than the limit of open file
+ descriptors).
+*/
+
+void Backup::lock_tables_TL_READ_NO_INSERT()
+{
+ THD *thd;
+ TABLE_LIST *tables_in_TABLE_LIST_form=NULL ; ///< for open_and_lock_tables()
+ DBUG_ENTER("myisam::backup::Backup::lock_tables_TL_READ_NO_INSERT");
+
+ thd= new THD;
+ if (unlikely(!thd))
+ goto end2;
+ thd->thread_stack = reinterpret_cast< char *>(&thd);
+ pthread_mutex_lock(&LOCK_thread_count);
+ thd->thread_id= thread_id++;
+ pthread_mutex_unlock(&LOCK_thread_count);
+ if (unlikely(thd->store_globals())) // for a proper MEM_ROOT
+ goto end2;
+ thd->init_for_queries(); // opening tables needs a proper LEX
+ thd->command= COM_DAEMON;
+ thd->system_thread= SYSTEM_THREAD_BACKUP;
+ thd->version= refresh_version;
+ thd->set_time();
+ thd->main_security_ctx.host_or_ip= "";
+ thd->client_capabilities= 0;
+ my_net_init(&thd->net, 0);
+ thd->main_security_ctx.master_access= ~0;
+ thd->main_security_ctx.priv_user= 0;
+ thd->real_id= pthread_self();
+ /*
+ Making this thread visible to SHOW PROCESSLIST is useful for
+ troubleshooting a backup job (why does it stall etc).
+ */
+ pthread_mutex_lock(&LOCK_thread_count);
+ threads.append(thd);
+ pthread_mutex_unlock(&LOCK_thread_count);
+ lex_start(thd);
+ mysql_reset_thd_for_next_command(thd);
+ /*
+ As locking tables can be a long operation, we need to support
+ cancellability during that time. So we publish our THD now to the thread
+ which created us (the "master" thread), so that it can kill us early if
+ needed.
+ */
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ lock_thd= thd;
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ /*
+ We need TL_READ_NO_INSERT (and not TL_READ) because we want to prevent
+ concurrent inserts (we indeed need to freeze the tables to correspond to
+ a position in the binlog).
+ */
+ tables_in_TABLE_LIST_form=
+ backup::build_table_list(m_tables, TL_READ_NO_INSERT);
+ if (!tables_in_TABLE_LIST_form)
+ goto end2;
+ if (open_and_lock_tables(thd, tables_in_TABLE_LIST_form))
+ goto end;
+
+ DBUG_PRINT("info",("MyISAM backup locking thread got locks"));
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ thd->enter_cond(&COND_lock_state, &THR_LOCK_myisam,
+ "MyISAM backup: holding table locks");
+ /* show master thread that we got locks */
+ lock_state= LOCK_ACQUIRED;
+ /* and wait for it to kill us */
+ while (!thd->killed)
+ pthread_cond_wait(&COND_lock_state, &THR_LOCK_myisam);
+ thd->exit_cond("MyISAM backup: terminating");
+
+end:
+ DBUG_PRINT("info",("MyISAM backup locking thread dying"));
+ close_thread_tables(thd);
+end2:
+ pthread_mutex_lock(&THR_LOCK_myisam);
+ while (cannot_delete_lock_thd)
+ {
+ /* master thread is looking at our THD; wait for authorization */
+ pthread_cond_wait(&COND_lock_state, &THR_LOCK_myisam);
+ }
+ lock_state= LOCK_ERROR;
+ pthread_cond_broadcast(&COND_lock_state);
+ pthread_mutex_unlock(&THR_LOCK_myisam);
+ backup::free_table_list(tables_in_TABLE_LIST_form);
+ net_end(&thd->net);
+ delete thd;
+ DBUG_VOID_RETURN;
+}
+
+
+/** Entry point for the locking thread */
+
+pthread_handler_t separate_thread_for_locking(void *arg)
+{
+ my_thread_init();
+ DBUG_PRINT("info", ("myisam_backup::separate_thread_for_locking"));
+ pthread_detach_this_thread();
+ (static_cast<Backup *>(arg))->lock_tables_TL_READ_NO_INSERT();
+ my_thread_end();
+ pthread_exit(0);
+ return 0;
+}
+
+
+/**
+ Launches a separate thread ("locking thread") which will lock
+ tables. Locking in a separate thread is needed to have a non-blocking
+ prelock() (given that thr_lock() is blocking). prelock() is indeed not
+ allowed to block, or it would block the entire backup kernel (see "HOW THE
+ BACKUP WORKS" at the start of this file).
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Backup::prelock()
+{
+ DBUG_ENTER("myisam_backup::Backup::prelock");
+ /* we are going to launch a thread, we need to remember to kill it */
+ lock_state= LOCK_STARTED;
+ {
+ pthread_t th;
+ if (pthread_create(&th, &connection_attrib,
+ separate_thread_for_locking, this))
+ {
+ lock_state= LOCK_ERROR;
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+ }
+ DBUG_RETURN(backup::OK);
+}
+
+
+result_t Backup::lock()
+{
+ DBUG_ENTER("myisam_backup::Backup::lock");
+ /* locking was done in prelock() already, nothing to do */
+ DBUG_RETURN(backup::OK);
+}
+
+
+result_t Backup::unlock()
+{
+ DBUG_ENTER("myisam_backup::Backup::unlock");
+ /* kill the locking thread which owns table locks, it will unlock them */
+ kill_locking_thread();
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Backs up the log.
+
+ @todo For now we read the log file from disk. We could instead try to
+ "steal" it from its IO_CACHE; that might reduce the log portion which goes
+ to disk, if the backup thread is fast enough to catch up on client threads
+ filling the log.
+*/
+
+Log_backup::Log_backup(const char *log_name_arg) : log_name(log_name_arg),
+ log_deleted(FALSE)
+{
+ DBUG_ENTER("myisam_backup::Log_backup::Log_backup");
+ int fd= my_open(log_name, O_RDONLY, MYF(MY_WME));
+ if (fd < 0)
+ SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN;
+ /*
+ Log is alone on the shared stream for now, so LOG_FILE_CODE is useless,
+ except that it allows us to verify that what restore sends us is really a
+ log.
+ */
+ log_file_backup.init(fd, ~(ULL(0)), LOG_FILE_CODE);
+ state= OK;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Closes and deletes the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Log_backup::end()
+{
+ DBUG_ENTER("myisam_backup::Log_backup::end");
+ /*
+ Log is safe in the stream, or backup is cancelled, so we don't need it
+ anymore.
+ */
+ if (log_file_backup.close_file() != backup::OK ||
+ (!log_deleted && my_delete(log_name, MYF(MY_WME))))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ log_deleted= TRUE;
+ DBUG_RETURN(backup::OK);
+}
+
+
+Log_backup::~Log_backup()
+{
+ /*
+ If all went well, we don't do anything here.
+ All possible failures in end() below use MYF_WME so my_error() will be
+ called.
+ */
+ end();
+}
+
+
+/** The header of a MYI index file always fits in this size */
+#define MAX_INDEX_HEADER_SIZE (64*1024)
+
+
+/**
+ Opens a MyISAM table for backing it up.
+
+ @param tbl The table to open
+*/
+
+Table_backup::Table_backup(const backup::Table_ref &tbl) :
+ Myisam_table_ref(tbl)
+{
+ MI_INFO *mi_info;
+ File dfiledes= -1, kfiledes= -1;
+ my_off_t file_size;
+ DBUG_ENTER("myisam_backup::Table_backup::Table_backup");
+ DBUG_PRINT("info",("Initializing backup image for table %s",
+ file_name.ptr()));
+ /*
+ Here we use low-level mi_* functions as all we want is a pair of file
+ descriptors.
+ O_RDONLY is not ok, as it forces all instances of the table to be
+ read-only (sets HA_OPTION_READ_ONLY_DATA of share->options).
+ We don't use HA_OPEN_FOR_REPAIR so will fail to back up a known corrupted
+ table (would be a corrupted backup).
+ */
+ mi_info= mi_open(file_name.ptr(), O_RDWR, 0);
+ if (!mi_info) // table does not exist or is corrupted? backup not ok
+ goto err;
+ /*
+ we create our own descriptors, to use my_read() (faster than my_pread()
+ which may use mutex).
+ */
+ dfiledes= my_open(mi_info->s->data_file_name, O_RDONLY, MYF(MY_WME));
+ kfiledes= my_open(mi_info->s->unique_file_name, O_RDONLY, MYF(MY_WME));
+ if ((dfiledes < 0) || (kfiledes < 0))
+ goto err;
+ mi_close(mi_info);
+ mi_info= NULL;
+ file_size= my_seek(dfiledes, 0, SEEK_END, MYF(MY_WME));
+ if (file_size == MY_FILEPOS_ERROR ||
+ my_seek(dfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR)
+ goto err;
+ dfile_backup.init(dfiledes, file_size, DATA_FILE_CODE);
+ if (mi_log_index_pages_physical)
+ {
+ file_size= my_seek(kfiledes, 0, SEEK_END, MYF(MY_WME));
+ if (file_size == MY_FILEPOS_ERROR ||
+ my_seek(kfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR)
+ goto err;
+ kfile_backup.init(kfiledes, file_size, WHOLE_INDEX_FILE_CODE);
+ }
+ else
+ kfile_backup.init(kfiledes,
+ MAX_INDEX_HEADER_SIZE /* upper limit */ ,
+ HEADER_INDEX_FILE_CODE);
+ in_file= DATA_FILE; // dump the data file first (no specific reason)
+ state= OK;
+ DBUG_VOID_RETURN;
+ /*
+ Note: we are copying an index file of a table, which may have instances in
+ the MySQL table cache, so after restore it will show up as
+ "warning: 1 client is using or hasn't closed the table properly".
+ Maybe do a quick index update on the table at the end of restore to
+ remove this warning. But how to know if the problem pre-dates backup ?
+ */
+err:
+ if (dfiledes > 0)
+ my_close(dfiledes, MYF(MY_WME));
+ if (kfiledes > 0)
+ my_close(kfiledes, MYF(MY_WME));
+ if (mi_info != NULL)
+ mi_close(mi_info);
+ SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN;
+}
+
+
+/**
+ Closes the MyISAM table.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Table_backup::end()
+{
+ DBUG_ENTER("myisam_backup::Table_backup::end");
+ /* even if one close fails we still want to try the other one */
+ if ((dfile_backup.close_file() != backup::OK) |
+ (kfile_backup.close_file() != backup::OK))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+Table_backup::~Table_backup()
+{
+ /* If all went well, we don't do anything here. */
+ end();
+}
+
+
+/**
+ Sends backup data for one table to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Table_backup::get_data(Buffer &buf)
+{
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Table_backup::get_data");
+ switch (in_file)
+ {
+ case DATA_FILE:
+ ret= dfile_backup.get_data(buf);
+ if (buf.last) // move to dumping the index file...
+ {
+ in_file= INDEX_FILE;
+ buf.last= FALSE; // ... so this is not the last buffer on this stream
+ }
+ break;
+ case INDEX_FILE:
+ ret= kfile_backup.get_data(buf);
+ break;
+ default:
+ DBUG_ASSERT(0);
+ ret= backup::ERROR;
+ };
+ if (ret != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Sends backup data for the log to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Log_backup::get_data(Buffer &buf)
+{
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Log_backup::get_data");
+ /*
+ See, we detect a log write error encountered by the MyISAM myisam_log*
+ and mi_log* functions, every time we read a packet from the log file.
+ */
+ if (((ret= log_file_backup.get_data(buf)) != backup::OK) ||
+ (myisam_physical_log.hard_write_error_in_the_past == -1))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Closes a file in backup.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t File_backup::close_file()
+{
+ int ret;
+ if (fd < 0)
+ return backup::OK;
+ ret= my_close(fd, MYF(MY_WME));
+ fd= -1;
+ return ret ? backup::ERROR : backup::OK;
+}
+
+
+/**
+ Sends backup data for a single file to the backup kernel.
+
+ @param buf reference to Buffer where data should be put
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t File_backup::get_data(Buffer &buf)
+{
+ size_t res, howmuch= buf.size;
+ result_t ret= backup::OK;
+
+ DBUG_ENTER("myisam_backup::File_backup::get_data");
+
+ buf.size= 1;
+ DBUG_ASSERT(howmuch >= 2); // need at least 2 bytes
+ *buf.data= static_cast<uchar>(file_code);
+ howmuch--;
+
+ if (backup_file_size >= file_size)
+ res= 0; // we don't have to read/send the rest of file
+ else
+ {
+ res= my_read(fd, buf.data + 1, howmuch, MYF(MY_WME));
+ // DBUG_DUMP("sending",buf_ptr-1, 16);
+ }
+ if (res == (size_t)(-1))
+ {
+ ret= backup::ERROR;
+ goto end;
+ }
+ backup_file_size+= res;
+ if (res == 0) // end of file
+ {
+ buf.size= 0; // don't even send a packet
+ buf.last= TRUE;
+ goto end;
+ }
+ buf.size+= res;
+ buf.last= FALSE;
+end:
+ DBUG_PRINT("info",("ret %d buf.last %d buf.size %u",
+ ret, buf.last, static_cast<uint>(buf.size)));
+ DBUG_RETURN(ret);
+}
+
+
+/**************************************
+ *
+ * RESTORE FUNCTIONALITY
+ *
+ **************************************/
+
+class Object_restore;
+
+/**
+ Handles restore orders received from the backup kernel (implements the
+ API).
+*/
+class Restore: public Restore_driver
+{
+public:
+ Restore(const Table_list &tables);
+ virtual ~Restore();
+ virtual result_t begin(const size_t);
+ virtual result_t end();
+ virtual result_t send_data(Buffer &buf);
+ virtual result_t cancel()
+ {
+ /* Nothing to do in cancel(); free() will suffice */
+ return backup::OK;
+ };
+ virtual void free() { delete this; };
+
+private:
+ enum { PUMPING, DONE, ERROR } state;
+ uint images_left; ///< how many images left to restore
+ Object_restore **images; ///< one for the log and one per table
+ char restore_log_name[FN_REFLEN];
+};
+
+
+/** An object to restore; in practice, a table or the log */
+class Object_restore
+{
+public:
+ virtual result_t send_data(const Buffer &buf)= 0;
+ virtual ~Object_restore() {};
+ /**
+ Closes the object, post_restore() can later be called. Whereas in
+ Object_backup, closing is done in end() (there is no close()), here we
+ have a dedicated close() method. This is because we must close tables and
+ the log then apply the log then repair indices: we need to close way
+ before end()).
+ */
+ virtual result_t close()= 0;
+ /** Does additional restore operations between close() and end() */
+ virtual result_t post_restore()= 0;
+ bool internal_error() { return state == ERROR; }
+ virtual result_t end()= 0; ///< cleanups
+protected:
+ enum { OK, ERROR } state;
+};
+
+
+/**
+ An object to restore is made of one or more such files. This class does not
+ open the file, user has to open it. This class provides a helper method if
+ its user wants to close the file.
+*/
+class File_restore
+{
+public:
+ File_restore() : fd(-1) {}
+ void init(int fd_arg) { fd= fd_arg; }
+ result_t send_data(const Buffer &);
+ result_t close_file();
+private:
+ int fd; ///< file descriptor
+};
+
+
+/** Handles restoring a single table */
+class Table_restore: public Object_restore, public Myisam_table_ref
+{
+public:
+ Table_restore(const Table_ref &tbl);
+ virtual result_t send_data(const Buffer &buf);
+ virtual ~Table_restore();
+ virtual result_t close();
+ virtual result_t post_restore();
+ virtual result_t end(); ///< cleanups
+ private:
+ File_restore dfile_restore, kfile_restore;
+ bool rebuild_index; ///< if we have to rebuild index or not
+ THD *thd; ///< rebuilding index requires a THD
+};
+
+
+/** Handles restoring the log */
+class Log_restore: public Object_restore
+{
+public:
+ Log_restore(const char *log_name_arg);
+ virtual result_t send_data(const Buffer &buf);
+ virtual ~Log_restore();
+ virtual result_t close();
+ virtual result_t post_restore();
+ virtual result_t end();
+private:
+ const char *log_name;
+ File_restore log_file_restore;
+ bool log_deleted; ///< if we have already deleted the log or not
+};
+
+
+/**
+ Creates a restore driver, per the backup API. @see backup::Engine.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Engine::get_restore(const version_t ver, const uint32,
+ const Table_list &tables, Restore_driver* &drv)
+{
+ if (ver > MYISAM_BACKUP_VERSION)
+ {
+ char errbuff[200];
+ my_snprintf(errbuff, sizeof(errbuff),
+ MYISAM_ERR(MYISAM_ERR_BACKUP_TOO_RECENT),
+ ver, MYISAM_BACKUP_VERSION);
+ my_error(ER_GET_ERRMSG, MYF(0),
+ MYISAM_ERR_BACKUP_TOO_RECENT, errbuff, "MyISAM");
+ return backup::ERROR;
+ }
+
+ Restore *ptr= new Restore(tables);
+ if (unlikely(!ptr))
+ return backup::ERROR;
+ drv= ptr;
+ return backup::OK;
+}
+
+
+Restore::Restore(const Table_list &tables):
+ Restore_driver(tables), state(ERROR), images_left(0), images(NULL)
+{
+ /* This constructor cannot fail otherwise begin() would have to detect it */
+}
+
+
+/**
+ This destructor is only called by the class' free(). It cleans up any
+ leftover the driver could have. It is safe to call it at any point. In a
+ normal (no error) situation, it does nothing, all should already have been
+ done by earlier stages.
+*/
+
+Restore::~Restore()
+{
+ DBUG_ENTER("myisam_backup::Restore::~Restore");
+ if (images)
+ {
+ for (uint n= 0; n <= m_tables.count(); ++n)
+ delete images[n];
+ delete[] images;
+ }
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Sets MyISAM in a state ready for us to restore. I.e. creates a temporary
+ file to host the log's restored copy.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Restore::begin(const size_t)
+{
+ THD *thd= current_thd;
+ DBUG_ENTER("myisam_backup::Restore::begin");
+ my_snprintf(restore_log_name, sizeof(restore_log_name),
+ "%s/%s%lx_%lx_%x-restorelog", mysql_tmpdir,
+ tmp_file_prefix, current_pid, thd->thread_id,
+ thd->tmp_table++);
+ unpack_filename(restore_log_name, restore_log_name);
+
+ DBUG_ASSERT(m_tables.count() > 0); // or bug in the backup kernel
+ images_left= 1 + m_tables.count();
+ images= new Object_restore*[images_left];
+ if (unlikely(!images))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ bzero(images, images_left * sizeof(*images));
+ state= PUMPING;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ If no error happened, we have to apply the log and possibly repair
+ indexes; this has to be done here and not in the destructor (as it has to
+ be done only in case of success, while a destructor runs in all cases).
+ Because we have no "end of stream" notifications yet, when we come here all
+ our tables/logs are opened. and log is not applied (both things which could
+ be done in send_data() if we knew end-of-stream). Repairing indexes, on the
+ other hand, really has to be done here.
+
+ @todo selective restore (this is just passing a proper function which
+ checks if the table is in a hash of tables).
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Restore::end()
+{
+ DBUG_ENTER("myisam_backup::Restore::end");
+ /*
+ Rafal said currently end() is called in case of error but said he'll fix
+ that (only free() will be called)
+ */
+ DBUG_ASSERT(state != ERROR);
+ if (images)
+ {
+ for (uint n=0; n <= m_tables.count(); ++n)
+ if (images[n] && images[n]->close() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ /*
+ Tables are closed. Apply backup log if it exists (it does not exist if
+ it was empty at backup time), this is post_restore() of images[0]. Then
+ repair indices if needed (post_restore() of other images).
+ */
+ for (uint n=0; n <= m_tables.count(); ++n)
+ if (images[n] && images[n]->post_restore() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ /*
+ By doing here the work of the destructor we can test the return code of
+ end(). We don't do it for tables as they will do nothing in end()
+ (except freeing their memory) so that can be left to the destructor.
+ */
+ if (images[0] && images[0]->end() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Receives and restores data for tables and log from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Restore::send_data(Buffer &buf)
+{
+ result_t ret;
+ uint stream= buf.table_num;
+ DBUG_ENTER("myisam_backup::Restore::send_data");
+ DBUG_PRINT("enter",("Got packet with %u bytes from stream %d",
+ static_cast<uint>(buf.size), buf.table_num));
+
+ if (state == DONE)
+ {
+ /* we never come here */
+ DBUG_PRINT("info",("Ignoring the packet (all objects already restored)"));
+ DBUG_RETURN(backup::DONE);
+ }
+
+ Object_restore *image= images[stream];
+
+ /*
+ We create an image when we see a new stream.
+ Still we have N open tables during the last table's restore.
+ But when Rafal implements that the last buffer of a stream has
+ buf.last==TRUE (soon), we can close tables earlier.
+ */
+ if (!image)
+ {
+ if (stream >= 1)
+ image= new Table_restore(m_tables[stream-1]);
+ else
+ image= new Log_restore(restore_log_name);
+ images[stream]= image;
+ if (unlikely(!image || image->internal_error()))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ }
+
+ if ((ret= image->send_data(buf)) != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+
+ /* for when we have "end of stream" notifications: */
+#ifdef TODO_HAVE_END_OF_STREAM
+ if (buf.last)
+ {
+ if (image->close() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ images_left--;
+ if (images_left == 0)
+ {
+ state= DONE;
+ /* DONE means done with all send_data() calls, but we have more work */
+ DBUG_RETURN(backup::DONE);
+ }
+ }
+#endif
+
+ DBUG_RETURN(backup::OK);
+}
+
+
+
+/**
+ Restores the log.
+
+ @param log_name_arg Name under which log should be created
+*/
+
+Log_restore::Log_restore(const char *log_name_arg) : log_name(log_name_arg)
+{
+ DBUG_ENTER("myisam_backup::Log_restore::Log_restore");
+ int fd= my_create(log_name, 0, O_WRONLY, MYF(MY_WME));
+ if (fd < 0)
+ {
+ log_deleted= TRUE;
+ SET_STATE_TO_ERROR_AND_DBUG_VOID_RETURN;
+ }
+ log_deleted= FALSE;
+ log_file_restore.init(fd);
+ state= OK;
+ DBUG_VOID_RETURN;
+}
+
+
+/**
+ Closes the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Log_restore::close()
+{
+ DBUG_ENTER("myisam_backup::Log_restore::close");
+ if (log_file_restore.close_file() != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Applies the log to restored tables, to make them consistent.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Log_restore::post_restore()
+{
+ MI_EXAMINE_LOG_PARAM mi_exl;
+ DBUG_ENTER("myisam_backup::Log_restore::post_restore");
+ mi_examine_log_param_init(&mi_exl);
+ mi_exl.log_filename= log_name;
+ mi_exl.update= 1;
+ /*
+ For max_files, the assumption is that at backup time the server had
+ enough file descriptors and so should have that many now.
+ */
+ mi_exl.max_files= open_files_limit;
+ if (mi_examine_log(&mi_exl))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Closes and deletes the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Log_restore::end()
+{
+ DBUG_ENTER("myisam_backup::Log_restore::end");
+ /* log is applied so we don't need it anymore */
+ if (close() != backup::OK ||
+ (!log_deleted && my_delete(log_name, MYF(MY_WME))))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ log_deleted= TRUE;
+ DBUG_RETURN(backup::OK);
+}
+
+
+Log_restore::~Log_restore()
+{
+ /* If all went well, we don't do anything here. */
+ end();
+}
+
+
+/** Opens a MyISAM table for restoring it */
+
+Table_restore::Table_restore(const Table_ref &tbl):
+ Myisam_table_ref(tbl), rebuild_index(FALSE)
+{
+ MI_INFO *mi_info;
+ File dfiledes= -1, kfiledes= -1;
+ DBUG_ENTER("myisam_backup::Table_restore::Table_restore");
+ DBUG_PRINT("enter",("Initializing backup image for table %s",
+ file_name.ptr()));
+ /*
+ Here we use low-level mi_* functions as all we want is a pair of file
+ descriptors.
+ Though we only want to write (O_WRONLY), the SQL layer uses only O_RDONLY
+ and O_RDWR, so here we don't try to be original.
+ */
+ mi_info= mi_open(file_name.ptr(), O_RDWR, 0);
+ if (!mi_info)
+ {
+ /* table does not exist or is corrupted? not normal, it's just created */
+ goto err;
+ }
+ /*
+ It's ok to copy the kfile descriptor and write() to it as the upper layers
+ guarantee that we are the only user of the brand new table (nobody will
+ lseek() under our feet).
+ */
+ if (((dfiledes= my_dup(mi_info->dfile, MYF(MY_WME))) < 0) ||
+ ((kfiledes= my_dup(mi_info->s->kfile, MYF(MY_WME))) < 0))
+ goto err;
+ /*
+ We are going to my_write() to the files without updating the table's
+ state (mi_info->state). If we called mi_close() only at end of restore,
+ that function may write its out-of-date state on the table.
+ */
+ mi_close(mi_info);
+ mi_info= NULL;
+ /* seek them at start, because we use my_write() */
+ if ((my_seek(dfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR) ||
+ (my_seek(kfiledes, 0, SEEK_SET, MYF(MY_WME)) == MY_FILEPOS_ERROR))
+ goto err;
+ dfile_restore.init(dfiledes);
+ kfile_restore.init(kfiledes);
+ thd= current_thd;
+ state= OK;
+ DBUG_VOID_RETURN;
+err:
+ if (dfiledes > 0)
+ my_close(dfiledes, MYF(MY_WME));
+ if (kfiledes > 0)
+ my_close(kfiledes, MYF(MY_WME));
+ if (mi_info != NULL)
+ mi_close(mi_info);
+}
+
+
+/**
+ Closes a table.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Table_restore::close()
+{
+ DBUG_ENTER("myisam_backup::Table_restore::close");
+ DBUG_PRINT("info",("table: %s", file_name.ptr()));
+ if ((dfile_restore.close_file() != backup::OK) |
+ (kfile_restore.close_file() != backup::OK))
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Closes a table.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Table_restore::end()
+{
+ return close();
+}
+
+
+Table_restore::~Table_restore()
+{
+ end();
+}
+
+
+/**
+ Repairs table's index if needed. Has to be done after applying the log.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Table_restore::post_restore()
+{
+ HA_CHECK_OPT check_opt;
+ TABLE *table= NULL;
+ int error;
+ Vio* save_vio;
+ DBUG_ENTER("myisam_backup::Table_restore::post_restore");
+
+ if (!rebuild_index)
+ {
+ MI_INFO *mi_info;
+ MYISAM_SHARE *share;
+ /*
+ Table was copied while it was possibly open by other clients; we need to
+ correct open_count to not trigger superfluous warning messages or repair
+ by --myisam-recover. If we rebuild the index, that will automatically
+ fix open_count.
+ */
+ mi_info= mi_open(file_name.ptr(), O_RDWR, HA_OPEN_FOR_REPAIR);
+ if ((error= (mi_info == NULL)))
+ goto err;
+ share= mi_info->s;
+ if (share->state.changed & STATE_BAD_OPEN_COUNT)
+ {
+ /* table already had a problem when backup started, leave open_count */
+ DBUG_PRINT("info", ("STATE_BAD_OPEN_COUNT is on"));
+ }
+ else
+ {
+ /* open_count>0 only because we copied while open, no problem */
+ share->state.open_count= 0;
+ /* force this correct open_count to disk */
+ error= mi_state_info_write(share, share->kfile, &share->state, 1);
+ }
+ error|= mi_close(mi_info);
+ goto err;
+ }
+
+ /*
+ myisamchk() as well as ha_myisam::repair() do a lot of operations before
+ and after mi_repair(); to not duplicate code we reuse one of them.
+ As we are in the server here, we use the one of the server.
+ A "new ha_myisam + ha_open()" is not sufficient as TABLE and TABLE_SHARE
+ are needed for ha_myisam::open(). So we use open_temporary_table() which
+ sets up all fine without touching thread's structure (and so, without
+ causing problems to locks, without interfering with close_thread_tables()
+ which would be done by another driver in the same thread etc).
+ Note that as the table has just been created, and in theory is protected
+ from any usage, by the upper backup layer, opening it with
+ open_temporary_table() is correct.
+ */
+ char path[FN_REFLEN];
+ build_table_filename(path, sizeof(path), db.ptr(), name.ptr(), "", 0);
+ table= open_temporary_table(thd, path, db.ptr(), name.ptr(),
+ false, OTM_OPEN);
+
+ if ((error= (!table || !table->file)))
+ goto err;
+
+ check_opt.init();
+ check_opt.flags|= T_VERY_SILENT | T_QUICK;
+ /*
+ We do not want repair() to spam us with messages (protocol->store() etc).
+ Just send them to the error log, and report the failure in case of
+ problems.
+ Note that ha_myisam::restore() does not do that (merely uses the same
+ check_opt.flags as us), as it is allowed to return an array of errors.
+ */
+ save_vio= thd->net.vio;
+ thd->net.vio= NULL;
+ error= table->file->ha_repair(thd,&check_opt) != 0;
+ thd->net.vio= save_vio;
+
+err:
+ if (table)
+ {
+ intern_close_table(table);
+ my_free(table, MYF(MY_WME));
+ }
+ if (error)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(backup::OK);
+}
+
+
+/**
+ Receives and restores data for one table from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Table_restore::send_data(const Buffer &buf)
+{
+ enum enum_file_code file_code= static_cast<enum enum_file_code>(*buf.data);
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Table_restore::send_data");
+
+ switch (file_code)
+ {
+ case DATA_FILE_CODE:
+ ret= dfile_restore.send_data(buf);
+ break;
+ case HEADER_INDEX_FILE_CODE:
+ rebuild_index= TRUE; // because we are given only the index's header
+ // fall through
+ case WHOLE_INDEX_FILE_CODE:
+ ret= kfile_restore.send_data(buf);
+ break;
+ default:
+ DBUG_PRINT("info",("packet with code %d I didn't expect", file_code));
+ DBUG_ASSERT(0);
+ ret= backup::ERROR;
+ }
+ if (ret != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Receives and restores data for the log from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t Log_restore::send_data(const Buffer &buf)
+{
+ enum enum_file_code file_code= static_cast<enum enum_file_code>(*buf.data);
+ result_t ret;
+ DBUG_ENTER("myisam_backup::Log_restore::send_data");
+
+ ret= (file_code == LOG_FILE_CODE) ? log_file_restore.send_data(buf) :
+ backup::ERROR;
+ if (ret != backup::OK)
+ SET_STATE_TO_ERROR_AND_DBUG_RETURN;
+ DBUG_RETURN(ret);
+}
+
+
+/**
+ Receives and restores data for one single file from the backup kernel.
+
+ @param buf reference to Buffer where data is
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t File_restore::send_data(const Buffer &buf)
+{
+ size_t howmuch= buf.size;
+
+ DBUG_ENTER("myisam_backup::File_restore::send_data");
+ //DBUG_DUMP("receiving",buf.data + 1, 16);
+
+ // We should receive same buffers as those made at backup time
+ DBUG_ASSERT(howmuch >= 2);
+ howmuch--; // skip the first byte which contains the code
+ size_t res= my_write(fd, buf.data +1, howmuch, MYF(MY_WME));
+
+ DBUG_RETURN((res != howmuch) ? backup::ERROR : backup::OK);
+}
+
+
+/**
+ Closes a file in restore.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+result_t File_restore::close_file()
+{
+ int ret;
+ if (fd < 0)
+ return backup::OK;
+ ret= my_close(fd, MYF(MY_WME));
+ fd= -1;
+ return ret ? backup::ERROR : backup::OK;
+}
+
+
+} // myisam_backup namespace
+
+
+/**
+ Returns the backup Engine used by this storage engine, per the API.
+
+ @return Operation status
+ @retval backup::OK
+ @retval backup::ERROR
+*/
+
+Backup_result_t myisam_backup_engine(handlerton *self, Backup_engine* &be)
+{
+ be= new myisam_backup::Engine();
+
+ if (unlikely(!be))
+ return backup::ERROR;
+
+ return backup::OK;
+}
diff -Nrup a/storage/myisam/myisam_ftdump.c b/storage/myisam/myisam_ftdump.c
--- a/storage/myisam/myisam_ftdump.c 2007-10-12 18:03:01 +02:00
+++ b/storage/myisam/myisam_ftdump.c 2008-05-07 17:30:58 +02:00
@@ -98,7 +98,8 @@ int main(int argc,char *argv[])
if ((inx >= info->s->base.keys) ||
!(info->s->keyinfo[inx].flag & HA_FULLTEXT))
{
- printf("Key %d in table %s is not a FULLTEXT key\n", inx, info->filename);
+ printf("Key %d in table %s is not a FULLTEXT key\n", inx,
+ info->s->unresolv_file_name);
goto err;
}
diff -Nrup a/storage/myisam/myisamchk.c b/storage/myisam/myisamchk.c
--- a/storage/myisam/myisamchk.c 2008-04-01 15:44:57 +02:00
+++ b/storage/myisam/myisamchk.c 2008-05-07 17:30:58 +02:00
@@ -1068,7 +1068,8 @@ static int myisamchk(MI_CHECK *param, ch
error=mi_sort_index(param,info,filename);
if (!error)
share->state.changed&= ~(STATE_CHANGED | STATE_CRASHED |
- STATE_CRASHED_ON_REPAIR);
+ STATE_CRASHED_ON_REPAIR |
+ STATE_BAD_OPEN_COUNT);
else
mi_mark_crashed(info);
}
@@ -1124,7 +1125,8 @@ static int myisamchk(MI_CHECK *param, ch
(param->testflag & T_UPDATE_STATE))
info->update|=HA_STATE_CHANGED | HA_STATE_ROW_CHANGED;
share->state.changed&= ~(STATE_CHANGED | STATE_CRASHED |
- STATE_CRASHED_ON_REPAIR);
+ STATE_CRASHED_ON_REPAIR |
+ STATE_BAD_OPEN_COUNT);
}
else if (!mi_is_crashed(info) &&
(param->testflag & T_UPDATE_STATE))
diff -Nrup a/storage/myisam/myisamdef.h b/storage/myisam/myisamdef.h
--- a/storage/myisam/myisamdef.h 2008-04-01 15:44:57 +02:00
+++ b/storage/myisam/myisamdef.h 2008-05-07 17:30:58 +02:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* This file is included by all internal myisam files */
+/**
+ @file
+ This file is included by all internal myisam files
+*/
#include "myisam.h" /* Structs & some defines */
#include "myisampack.h" /* packing of keys */
@@ -21,6 +24,7 @@
#ifdef THREAD
#include <my_pthread.h>
#include <thr_lock.h>
+#include <my_atomic.h>
#else
#include <my_no_pthread.h>
#endif
@@ -154,6 +158,7 @@ typedef struct st_mi_isam_pack {
#define MAX_NONMAPPED_INSERTS 1000
+/** Information shared by all open instances of the same table */
typedef struct st_mi_isam_share { /* Shared between opens */
MI_STATE_INFO state;
MI_BASE_INFO base;
@@ -220,6 +225,27 @@ typedef struct st_mi_isam_share { /* Sha
uint nonmmaped_inserts; /* counter of writing in non-mmaped
area */
rw_lock_t mmap_lock;
+ /**
+ If this table is doing physical logging (1) or not (0).
+ Set under MI_INFO::physical_logging_rwlock and THR_LOCK_myisam.
+ Read under either one of the two locks above.
+ */
+ volatile int32 physical_logging;
+ /** For protecting MYISAM_SHARE::physical_logging */
+ my_atomic_rwlock_t physical_logging_rwlock;
+ /**
+ File name before resolving any symlink or expanding directory.
+ Used by physical logging. In order to restore to a data directory with a
+ different path from the original one, we need unresolved relative names.
+ */
+ char *unresolv_file_name;
+ /**
+ If we already stored MI_LOG_OPEN in physical log for this share.
+ Set to TRUE only by writer thread under THR_LOCK_myisam_log atomically
+ with logging the MI_LOG_OPEN; set to FALSE only by mi_log_stop_physical()
+ after closing the log.
+ */
+ my_bool MI_LOG_OPEN_stored_in_physical_log;
} MYISAM_SHARE;
@@ -235,6 +261,7 @@ typedef struct st_mi_bit_buff { /* Used
typedef my_bool (*index_cond_func_t)(void *param);
+/** Information local to the table's instance */
struct st_myisam_info {
MYISAM_SHARE *s; /* Shared between open:s */
MI_STATUS_INFO *state,save_state;
@@ -246,7 +273,6 @@ struct st_myisam_info {
MEM_ROOT ft_memroot; /* used by the parser */
MYSQL_FTPARSER_PARAM *ftparser_param; /* share info between init/deinit */
LIST in_use; /* Thread using this table */
- char *filename; /* parameter to open filename */
uchar *buff, /* Temp area for key */
*lastkey,*lastkey2; /* Last used search key */
uchar *first_mbr_key; /* Searhed spatial key */
@@ -382,6 +408,11 @@ typedef struct st_mi_sort_param
#define STATE_NOT_ANALYZED 8
#define STATE_NOT_OPTIMIZED_KEYS 16
#define STATE_NOT_SORTED_PAGES 32
+/**
+ If open_count>0 the first time we opened this table; cleared after
+ successful check or repair
+*/
+#define STATE_BAD_OPEN_COUNT 64
/* options to mi_read_cache */
@@ -469,7 +500,7 @@ typedef struct st_mi_sort_param
#define mi_unique_store(A,B) mi_int4store((A),(B))
#ifdef THREAD
-extern pthread_mutex_t THR_LOCK_myisam;
+extern pthread_mutex_t THR_LOCK_myisam_log;
#endif
#if !defined(THREAD) || defined(DONT_USE_RW_LOCKS)
#define rw_wrlock(A) {}
@@ -483,8 +514,8 @@ extern LIST *myisam_open_list;
extern uchar NEAR myisam_file_magic[],NEAR myisam_pack_file_magic[];
extern uint NEAR myisam_read_vec[],NEAR myisam_readnext_vec[];
extern uint myisam_quick_table_bits;
-extern File myisam_log_file;
extern ulong myisam_pid;
+extern const HASH *mi_log_tables_physical;
/* This is used by _mi_calc_xxx_key_length och _mi_store_key */
@@ -685,14 +716,48 @@ typedef struct st_mi_block_info { /* Par
#define SORT_BUFFER_INIT (2048L*1024L-MALLOC_OVERHEAD)
#define MIN_SORT_BUFFER (4096-MALLOC_OVERHEAD)
+/** Commands storable in MyISAM logs (L/P= in logical/physical log) */
enum myisam_log_commands {
- MI_LOG_OPEN,MI_LOG_WRITE,MI_LOG_UPDATE,MI_LOG_DELETE,MI_LOG_CLOSE,MI_LOG_EXTRA,MI_LOG_LOCK,MI_LOG_DELETE_ALL
+ MI_LOG_OPEN, /**< when mi_open() LP */
+ MI_LOG_WRITE, /**< when mi_write() L */
+ MI_LOG_UPDATE, /**< when mi_update() L */
+ MI_LOG_DELETE, /**< when mi_delete() L */
+ MI_LOG_CLOSE, /**< when mi_close() LP */
+ MI_LOG_EXTRA, /**< when mi_extra() L */
+ MI_LOG_LOCK, /**< when mi_lock_database() L */
+ MI_LOG_DELETE_ALL, /**< when mi_delete_all LP */
+ MI_LOG_WRITE_BYTES_MYD, /**< when MyISAM writes to the data file P */
+ MI_LOG_WRITE_BYTES_MYI, /**< when MyISAM writes to the index file P */
+ MI_LOG_CHSIZE_MYI, /**< when MyISAM changes size of index file P */
+ MI_LOG_END_SENTINEL /**< keep this one unused and last */
};
-
-#define myisam_log(a,b,c,d) if (myisam_log_file >= 0) _myisam_log(a,b,c,d)
-#define myisam_log_command(a,b,c,d,e) if (myisam_log_file >= 0) _myisam_log_command(a,b,c,d,e)
-#define myisam_log_record(a,b,c,d,e) if (myisam_log_file >= 0) _myisam_log_record(a,b,c,d,e)
-
+extern const char *mi_log_command_name[];
+/** Logs a command if this log is open */
+#define myisam_log_command_logical(command, info_or_share, buffert, length, result) \
+ if (my_b_inited(&myisam_logical_log)) \
+ _myisam_log_command(&myisam_logical_log, command, info_or_share, \
+ buffert, length, result)
+/** Logs a command involving a record if this log is open */
+#define myisam_log_record_logical(command, info, record, filepos, result) \
+ if (my_b_inited(&myisam_logical_log)) \
+ _myisam_log_record_logical(command, info, record, filepos, result)
+/** If log record stores numerical info in long format */
+#define MI_LOG_BIG_NUMBERS 128
+
+/**
+ MyISAM-specific errors (not generic enough to be HA_ERR), sent to the
+ caller wrapped inside ER_GET_ERRMSG. Not yet stabilized, so not yet
+ exported to myisam.h.
+*/
+enum myisam_errors
+{
+ /* decrease, starting with -1 */
+ MYISAM_ERR_NO_BACKUP_WITH_EXTERNAL_LOCKING= -1,
+ MYISAM_ERR_BACKUP_TOO_RECENT= -2,
+ MYISAM_ERR_LAST=-3 /**< keep it last and unused */
+ /* use only numbers<0, to not collide with OS errors, ER_, HA_ERR etc */
+};
+#define MYISAM_ERR(errnumber) myisam_error_messages[-errnumber-1]
#define fast_mi_writeinfo(INFO) if (!(INFO)->s->tot_locks) (void) _mi_writeinfo((INFO),0)
#define fast_mi_readinfo(INFO) ((INFO)->lock_type == F_UNLCK) && _mi_readinfo((INFO),F_RDLCK,1)
@@ -706,14 +771,57 @@ extern uint _mi_pack_get_block_info(MI_I
MI_BLOCK_INFO *info, uchar **rec_buff_p,
File file, my_off_t filepos);
extern void _my_store_blob_length(uchar *pos,uint pack_length,uint length);
-extern void _myisam_log(enum myisam_log_commands command,MI_INFO *info,
- const uchar *buffert,uint length);
-extern void _myisam_log_command(enum myisam_log_commands command,
- MI_INFO *info, const uchar *buffert,
- uint length, int result);
-extern void _myisam_log_record(enum myisam_log_commands command,MI_INFO *info,
- const uchar *record,my_off_t filepos,
- int result);
+extern void _myisam_log_command(IO_CACHE *log,
+ enum myisam_log_commands command,
+ void *info_or_share, const uchar *buffert,
+ uint length, int result);
+extern void _myisam_log_record_logical(enum myisam_log_commands command,
+ MI_INFO *info, const uchar *record,
+ my_off_t filepos, int result);
+extern void myisam_log_pwrite_physical(enum myisam_log_commands command,
+ MYISAM_SHARE *share,
+ const uchar *buffert, uint length,
+ my_off_t filepos);
+extern void myisam_log_chsize_kfile_physical(MYISAM_SHARE *share,
+ my_off_t new_length);
+#ifdef HAVE_MYISAM_PHYSICAL_LOGGING
+static inline int32 mi_get_physical_logging_state(MYISAM_SHARE *share)
+{
+ int32 ret;
+ my_atomic_rwlock_rdlock(&share->physical_logging_rwlock);
+ ret= my_atomic_load32(&share->physical_logging);
+ my_atomic_rwlock_rdunlock(&share->physical_logging_rwlock);
+ return ret;
+}
+static inline void
+mi_set_physical_logging_state(MYISAM_SHARE *share, int32 new_state)
+{
+ my_atomic_rwlock_wrlock(&share->physical_logging_rwlock);
+ my_atomic_store32(&share->physical_logging, new_state);
+ my_atomic_rwlock_wrunlock(&share->physical_logging_rwlock);
+}
+#else
+#define mi_get_physical_logging_state(share) 0
+#define mi_set_physical_logging_state(share, new_state)
+#endif
+/**
+ IN and OUT structure for instructing how to apply a MyISAM log and later
+ getting statistics about this log.
+*/
+typedef struct mi_examine_log_param
+{
+ uint verbose, update, max_files, re_open_count, recover, prefix_remove,
+ opt_processes;
+ ulong number_of_commands;
+ my_off_t start_offset,record_pos;
+ const char *log_filename, *filepath, *write_filename, *record_pos_file;
+ /** Count of commands found in log and their errors */
+ ulong com_count[MI_LOG_END_SENTINEL][3];
+ my_bool (*table_selection_hook)(const char *); /**< to filter tables */
+} MI_EXAMINE_LOG_PARAM;
+extern void mi_examine_log_param_init(MI_EXAMINE_LOG_PARAM *param);
+extern int mi_examine_log(MI_EXAMINE_LOG_PARAM *param);
+
extern void mi_report_error(int errcode, const char *file_name);
extern my_bool _mi_memmap_file(MI_INFO *info);
extern void _mi_unmap_file(MI_INFO *info);
@@ -729,8 +837,10 @@ extern size_t mi_nommap_pread(MI_INFO *i
extern size_t mi_nommap_pwrite(MI_INFO *info, const uchar *Buffer,
size_t Count, my_off_t offset, myf MyFlags);
-uint mi_state_info_write(File file, MI_STATE_INFO *state, uint pWrite);
+uint mi_state_info_write(MYISAM_SHARE *share, File file,
+ MI_STATE_INFO *state, uint pWrite);
uchar *mi_state_info_read(uchar *ptr, MI_STATE_INFO *state);
+int mi_remap_file_and_write_state_for_unlock(MI_INFO *info);
uint mi_state_info_read_dsk(File file, MI_STATE_INFO *state, my_bool pRead);
uint mi_base_info_write(File file, MI_BASE_INFO *base);
uchar *my_n_base_info_read(uchar *ptr, MI_BASE_INFO *base);
@@ -794,6 +904,12 @@ int _create_index_by_sort(MI_SORT_PARAM
extern void mi_set_index_cond_func(MI_INFO *info, index_cond_func_t func,
void *func_arg);
+
+extern const char *myisam_error_messages[];
+extern my_bool mi_log_index_pages_physical;
+extern IO_CACHE myisam_logical_log, myisam_physical_log;
+extern pthread_mutex_t THR_LOCK_myisam;
+
#ifdef __cplusplus
}
#endif
diff -Nrup a/storage/myisam/myisamlog.c b/storage/myisam/myisamlog.c
--- a/storage/myisam/myisamlog.c 2008-02-19 22:53:31 +01:00
+++ b/storage/myisam/myisamlog.c 2008-05-07 17:30:59 +02:00
@@ -13,7 +13,15 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* write whats in isam.log */
+/**
+ @file
+ Utility to display and apply a MyISAM logical or physical log to tables.
+
+ Prints what is in a MyISAM (logical or physical/backup) log, optionally
+ applies the changes to tables (all tables or only a set specified on the
+ command line). Works standalone (tables must not be modified by the
+ server during this).
+*/
#ifndef USE_MY_FUNC
#define USE_MY_FUNC
@@ -26,60 +34,15 @@
#include <sys/resource.h>
#endif
-#define FILENAME(A) (A ? A->show_name : "Unknown")
-
-struct file_info {
- long process;
- int filenr,id;
- uint rnd;
- char *name, *show_name;
- uchar *record;
- MI_INFO *isam;
- my_bool closed, used;
- ulong accessed;
-};
-
-struct test_if_open_param {
- char * name;
- int max_id;
-};
-
-struct st_access_param
-{
- ulong min_accessed;
- struct file_info *found;
-};
-
#define NO_FILEPOS (ulong) ~0L
-extern int main(int argc,char * *argv);
static void get_options(int *argc,char ***argv);
-static int examine_log(char * file_name,char **table_names);
-static int read_string(IO_CACHE *file,uchar* *to,uint length);
-static int file_info_compare(void *cmp_arg, void *a,void *b);
-static int test_if_open(struct file_info *key,element_count count,
- struct test_if_open_param *param);
-static void fix_blob_pointers(MI_INFO *isam,uchar *record);
-static int test_when_accessed(struct file_info *key,element_count count,
- struct st_access_param *access_param);
-static void file_info_free(struct file_info *info);
-static int close_some_file(TREE *tree);
-static int reopen_closed_file(TREE *tree,struct file_info *file_info);
-static int find_record_with_key(struct file_info *file_info,uchar *record);
-static void printf_log(const char *str,...);
-static my_bool cmp_filename(struct file_info *file_info,char * name);
-
-static uint verbose=0,update=0,test_info=0,max_files=0,re_open_count=0,
- recover=0,prefix_remove=0,opt_processes=0;
-static char *log_filename=0, *filepath=0, *write_filename=0;
-static char *record_pos_file= 0;
-static ulong com_count[10][3],number_of_commands=(ulong) ~0L,
- isamlog_process;
-static my_off_t isamlog_filepos,start_offset=0,record_pos= HA_OFFSET_ERROR;
-static const char *command_name[]=
-{"open","write","update","delete","close","extra","lock","re-open",
- "delete-all", NullS};
+static my_bool matches_list_of_tables(const char *isam_file_name);
+
+static MI_EXAMINE_LOG_PARAM mi_exl;
+static char **table_names;
+static uint test_info=0;
int main(int argc, char **argv)
{
@@ -87,40 +50,60 @@ int main(int argc, char **argv)
ulong total_count,total_error,total_recover;
MY_INIT(argv[0]);
- log_filename=myisam_log_filename;
+ mi_examine_log_param_init(&mi_exl);
+ mi_exl.log_filename= myisam_logical_log_filename; /* the default */
get_options(&argc,&argv);
+ if (argv[0]) /* some table names passed on command line */
+ {
+ table_names= argv;
+ mi_exl.table_selection_hook= matches_list_of_tables;
+ }
+
/* Number of MyISAM files we can have open at one time */
- max_files= (my_set_max_open_files(min(max_files,8))-6)/2;
- if (update)
+ mi_exl.max_files= (my_set_max_open_files(max(mi_exl.max_files,8))-6)/2;
+
+ /*
+ Program must work in all conditions: support symbolic links.
+ It should not be a security risk.
+ */
+#ifdef USE_SYMDIR
+ my_use_symdir= 1;
+#endif
+
+ if (mi_exl.update)
printf("Trying to %s MyISAM files according to log '%s'\n",
- (recover ? "recover" : "update"),log_filename);
- error= examine_log(log_filename,argv);
- if (update && ! error)
+ (mi_exl.recover ? "recover" : "update"),mi_exl.log_filename);
+
+ error= mi_examine_log(&mi_exl);
+
+ if (mi_exl.update && ! error)
puts("Tables updated successfully");
total_count=total_error=total_recover=0;
- for (i=first=0 ; command_name[i] ; i++)
+ for (i=first=0 ; mi_log_command_name[i] ; i++)
{
- if (com_count[i][0])
+ if (mi_exl.com_count[i][0])
{
if (!first++)
{
- if (verbose || update)
+ if (mi_exl.verbose || mi_exl.update)
puts("");
- puts("Commands Used count Errors Recover errors");
+ puts("Commands Used count Errors"
+ " Recover errors");
}
- printf("%-12s%9ld%10ld%17ld\n",command_name[i],com_count[i][0],
- com_count[i][1],com_count[i][2]);
- total_count+=com_count[i][0];
- total_error+=com_count[i][1];
- total_recover+=com_count[i][2];
+ printf("%-20s%9ld%10ld%15ld\n", mi_log_command_name[i],
+ mi_exl.com_count[i][0],
+ mi_exl.com_count[i][1],mi_exl.com_count[i][2]);
+ total_count+=mi_exl.com_count[i][0];
+ total_error+=mi_exl.com_count[i][1];
+ total_recover+=mi_exl.com_count[i][2];
}
}
if (total_count)
printf("%-12s%9ld%10ld%17ld\n","Total",total_count,total_error,
total_recover);
- if (re_open_count)
+ if (mi_exl.re_open_count)
printf("Had to do %d re-open because of too few possibly open files\n",
- re_open_count);
+ mi_exl.re_open_count);
VOID(mi_panic(HA_PANIC_CLOSE));
my_free_open_file_info();
my_end(test_info ? MY_CHECK_ERROR | MY_GIVE_INFO : MY_CHECK_ERROR);
@@ -156,11 +139,11 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- number_of_commands=(ulong) atol(pos);
+ mi_exl.number_of_commands= (ulong) atol(pos);
pos=" ";
break;
case 'u':
- update=1;
+ mi_exl.update=1;
break;
case 'f':
if (! *++pos)
@@ -170,7 +153,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- max_files=(uint) atoi(pos);
+ mi_exl.max_files=(uint) atoi(pos);
pos=" ";
break;
case 'i':
@@ -184,7 +167,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- start_offset=(my_off_t) strtoll(pos,NULL,10);
+ mi_exl.start_offset=(my_off_t) strtoll(pos,NULL,10);
pos=" ";
break;
case 'p':
@@ -195,14 +178,14 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- prefix_remove=atoi(pos);
+ mi_exl.prefix_remove=atoi(pos);
break;
case 'r':
- update=1;
- recover++;
+ mi_exl.update=1;
+ mi_exl.recover++;
break;
case 'P':
- opt_processes=1;
+ mi_exl.opt_processes=1;
break;
case 'R':
if (! *++pos)
@@ -212,14 +195,14 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- record_pos_file=(char*) pos;
+ mi_exl.record_pos_file=(char*) pos;
if (!--*argc)
goto err;
- record_pos=(my_off_t) strtoll(*(++*argv),NULL,10);
+ mi_exl.record_pos=(my_off_t) strtoll(*(++*argv),NULL,10);
pos=" ";
break;
case 'v':
- verbose++;
+ mi_exl.verbose++;
break;
case 'w':
if (! *++pos)
@@ -229,7 +212,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- write_filename=(char*) pos;
+ mi_exl.write_filename=(char*) pos;
pos=" ";
break;
case 'F':
@@ -240,7 +223,7 @@ static void get_options(register int *ar
else
pos= *(++*argv);
}
- filepath= (char*) pos;
+ mi_exl.filepath= (char*) pos;
pos=" ";
break;
case 'V':
@@ -249,13 +232,13 @@ static void get_options(register int *ar
case 'I':
case '?':
#include <help_start.h>
- printf("%s Ver 1.4 for %s at %s\n",my_progname,SYSTEM_TYPE,
+ printf("%s Ver 2.0 for %s at %s\n",my_progname,SYSTEM_TYPE,
MACHINE_TYPE);
puts("By Monty, for your professional use\n");
if (version)
break;
puts("Write info about whats in a MyISAM log file.");
- printf("If no file name is given %s is used\n",log_filename);
+ printf("If no file name is given %s is used\n",mi_exl.log_filename);
puts("");
printf(usage,my_progname);
puts("");
@@ -286,7 +269,7 @@ static void get_options(register int *ar
}
if (*argc >= 1)
{
- log_filename=(char*) pos;
+ mi_exl.log_filename=(char*) pos;
(*argc)--;
(*argv)++;
}
@@ -298,552 +281,19 @@ static void get_options(register int *ar
}
-static int examine_log(char * file_name, char **table_names)
-{
- uint command,result,files_open;
- ulong access_time,length;
- my_off_t filepos;
- int lock_command,mi_result;
- char isam_file_name[FN_REFLEN],llbuff[21],llbuff2[21];
- uchar head[20];
- uchar* buff;
- struct test_if_open_param open_param;
- IO_CACHE cache;
- File file;
- FILE *write_file;
- enum ha_extra_function extra_command;
- TREE tree;
- struct file_info file_info,*curr_file_info;
- DBUG_ENTER("examine_log");
-
- if ((file=my_open(file_name,O_RDONLY,MYF(MY_WME))) < 0)
- DBUG_RETURN(1);
- write_file=0;
- if (write_filename)
- {
- if (!(write_file=my_fopen(write_filename,O_WRONLY,MYF(MY_WME))))
- {
- my_close(file,MYF(0));
- DBUG_RETURN(1);
- }
- }
-
- init_io_cache(&cache,file,0,READ_CACHE,start_offset,0,MYF(0));
- bzero((uchar*) com_count,sizeof(com_count));
- init_tree(&tree,0,0,sizeof(file_info),(qsort_cmp2) file_info_compare,1,
- (tree_element_free) file_info_free, NULL);
- VOID(init_key_cache(dflt_key_cache,KEY_CACHE_BLOCK_SIZE,KEY_CACHE_SIZE,
- 0, 0));
-
- files_open=0; access_time=0;
- while (access_time++ != number_of_commands &&
- !my_b_read(&cache,(uchar*) head,9))
- {
- isamlog_filepos=my_b_tell(&cache)-9L;
- file_info.filenr= mi_uint2korr(head+1);
- isamlog_process=file_info.process=(long) mi_uint4korr(head+3);
- if (!opt_processes)
- file_info.process=0;
- result= mi_uint2korr(head+7);
- if ((curr_file_info=(struct file_info*) tree_search(&tree, &file_info,
- tree.custom_arg)))
- {
- curr_file_info->accessed=access_time;
- if (update && curr_file_info->used && curr_file_info->closed)
- {
- if (reopen_closed_file(&tree,curr_file_info))
- {
- command=sizeof(com_count)/sizeof(com_count[0][0])/3;
- result=0;
- goto com_err;
- }
- }
- }
- command=(uint) head[0];
- if (command < sizeof(com_count)/sizeof(com_count[0][0])/3 &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- {
- com_count[command][0]++;
- if (result)
- com_count[command][1]++;
- }
- switch ((enum myisam_log_commands) command) {
- case MI_LOG_OPEN:
- if (!table_names[0])
- {
- com_count[command][0]--; /* Must be counted explicite */
- if (result)
- com_count[command][1]--;
- }
-
- if (curr_file_info)
- printf("\nWarning: %s is opened with same process and filenumber\n"
- "Maybe you should use the -P option ?\n",
- curr_file_info->show_name);
- if (my_b_read(&cache,(uchar*) head,2))
- goto err;
- file_info.name=0;
- file_info.show_name=0;
- file_info.record=0;
- if (read_string(&cache,(uchar**) &file_info.name,
- (uint) mi_uint2korr(head)))
- goto err;
- {
- uint i;
- char *pos,*to;
-
- /* Fix if old DOS files to new format */
- for (pos=file_info.name; (pos=strchr(pos,'\\')) ; pos++)
- *pos= '/';
-
- pos=file_info.name;
- for (i=0 ; i < prefix_remove ; i++)
- {
- char *next;
- if (!(next=strchr(pos,'/')))
- break;
- pos=next+1;
- }
- to=isam_file_name;
- if (filepath)
- to=convert_dirname(isam_file_name,filepath,NullS);
- strmov(to,pos);
- fn_ext(isam_file_name)[0]=0; /* Remove extension */
- }
- open_param.name=file_info.name;
- open_param.max_id=0;
- VOID(tree_walk(&tree,(tree_walk_action) test_if_open,(void*) &open_param,
- left_root_right));
- file_info.id=open_param.max_id+1;
- /*
- * In the line below +10 is added to accomodate '<' and '>' chars
- * plus '\0' at the end, so that there is place for 7 digits.
- * It is improbable that same table can have that many entries in
- * the table cache.
- * The additional space is needed for the sprintf commands two lines
- * below.
- */
- file_info.show_name=my_memdup(isam_file_name,
- (uint) strlen(isam_file_name)+10,
- MYF(MY_WME));
- if (file_info.id > 1)
- sprintf(strend(file_info.show_name),"<%d>",file_info.id);
- file_info.closed=1;
- file_info.accessed=access_time;
- file_info.used=1;
- if (table_names[0])
- {
- char **name;
- file_info.used=0;
- for (name=table_names ; *name ; name++)
- {
- if (!strcmp(*name,isam_file_name))
- file_info.used=1; /* Update/log only this */
- }
- }
- if (update && file_info.used)
- {
- if (files_open >= max_files)
- {
- if (close_some_file(&tree))
- goto com_err;
- files_open--;
- }
- if (!(file_info.isam= mi_open(isam_file_name,O_RDWR,
- HA_OPEN_WAIT_IF_LOCKED)))
- goto com_err;
- if (!(file_info.record=my_malloc(file_info.isam->s->base.reclength,
- MYF(MY_WME))))
- goto end;
- files_open++;
- file_info.closed=0;
- }
- VOID(tree_insert(&tree, (uchar*) &file_info, 0, tree.custom_arg));
- if (file_info.used)
- {
- if (verbose && !record_pos_file)
- printf_log("%s: open -> %d",file_info.show_name, file_info.filenr);
- com_count[command][0]++;
- if (result)
- com_count[command][1]++;
- }
- break;
- case MI_LOG_CLOSE:
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s -> %d",FILENAME(curr_file_info),
- command_name[command],result);
- if (curr_file_info)
- {
- if (!curr_file_info->closed)
- files_open--;
- VOID(tree_delete(&tree, (uchar*) curr_file_info, 0, tree.custom_arg));
- }
- break;
- case MI_LOG_EXTRA:
- if (my_b_read(&cache,(uchar*) head,1))
- goto err;
- extra_command=(enum ha_extra_function) head[0];
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s(%d) -> %d",FILENAME(curr_file_info),
- command_name[command], (int) extra_command,result);
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (mi_extra(curr_file_info->isam, extra_command, 0) != (int) result)
- {
- fflush(stdout);
- VOID(fprintf(stderr,
- "Warning: error %d, expected %d on command %s at %s\n",
- my_errno,result,command_name[command],
- llstr(isamlog_filepos,llbuff)));
- fflush(stderr);
- }
- }
- break;
- case MI_LOG_DELETE:
- if (my_b_read(&cache,(uchar*) head,8))
- goto err;
- filepos=mi_sizekorr(head);
- if (verbose && (!record_pos_file ||
- ((record_pos == filepos || record_pos == NO_FILEPOS) &&
- !cmp_filename(curr_file_info,record_pos_file))) &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s at %ld -> %d",FILENAME(curr_file_info),
- command_name[command],(long) filepos,result);
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
- {
- if (!recover)
- goto com_err;
- if (verbose)
- printf_log("error: Didn't find row to delete with mi_rrnd");
- com_count[command][2]++; /* Mark error */
- }
- mi_result=mi_delete(curr_file_info->isam,curr_file_info->record);
- if ((mi_result == 0 && result) ||
- (mi_result && (uint) my_errno != result))
- {
- if (!recover)
- goto com_err;
- if (mi_result)
- com_count[command][2]++; /* Mark error */
- if (verbose)
- printf_log("error: Got result %d from mi_delete instead of %d",
- mi_result, result);
- }
- }
- break;
- case MI_LOG_WRITE:
- case MI_LOG_UPDATE:
- if (my_b_read(&cache,(uchar*) head,12))
- goto err;
- filepos=mi_sizekorr(head);
- length=mi_uint4korr(head+8);
- buff=0;
- if (read_string(&cache,&buff,(uint) length))
- goto err;
- if ((!record_pos_file ||
- ((record_pos == filepos || record_pos == NO_FILEPOS) &&
- !cmp_filename(curr_file_info,record_pos_file))) &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- {
- if (write_file &&
- (my_fwrite(write_file,buff,length,MYF(MY_WAIT_IF_FULL | MY_NABP))))
- goto end;
- if (verbose)
- printf_log("%s: %s at %ld, length=%ld -> %d",
- FILENAME(curr_file_info),
- command_name[command], filepos,length,result);
- }
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (curr_file_info->isam->s->base.blobs)
- fix_blob_pointers(curr_file_info->isam,buff);
- if ((enum myisam_log_commands) command == MI_LOG_UPDATE)
- {
- if (mi_rrnd(curr_file_info->isam,curr_file_info->record,filepos))
- {
- if (!recover)
- {
- result=0;
- goto com_err;
- }
- if (verbose)
- printf_log("error: Didn't find row to update with mi_rrnd");
- if (recover == 1 || result ||
- find_record_with_key(curr_file_info,buff))
- {
- com_count[command][2]++; /* Mark error */
- break;
- }
- }
- mi_result=mi_update(curr_file_info->isam,curr_file_info->record,
- buff);
- if ((mi_result == 0 && result) ||
- (mi_result && (uint) my_errno != result))
- {
- if (!recover)
- goto com_err;
- if (verbose)
- printf_log("error: Got result %d from mi_update instead of %d",
- mi_result, result);
- if (mi_result)
- com_count[command][2]++; /* Mark error */
- }
- }
- else
- {
- mi_result=mi_write(curr_file_info->isam,buff);
- if ((mi_result == 0 && result) ||
- (mi_result && (uint) my_errno != result))
- {
- if (!recover)
- goto com_err;
- if (verbose)
- printf_log("error: Got result %d from mi_write instead of %d",
- mi_result, result);
- if (mi_result)
- com_count[command][2]++; /* Mark error */
- }
- if (!recover && filepos != curr_file_info->isam->lastpos)
- {
- printf("error: Wrote at position: %s, should have been %s",
- llstr(curr_file_info->isam->lastpos,llbuff),
- llstr(filepos,llbuff2));
- goto end;
- }
- }
- }
- my_free(buff,MYF(0));
- break;
- case MI_LOG_LOCK:
- if (my_b_read(&cache,(uchar*) head,sizeof(lock_command)))
- goto err;
- memcpy_fixed(&lock_command,head,sizeof(lock_command));
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s(%d) -> %d\n",FILENAME(curr_file_info),
- command_name[command],lock_command,result);
- if (update && curr_file_info && !curr_file_info->closed)
- {
- if (mi_lock_database(curr_file_info->isam,lock_command) !=
- (int) result)
- goto com_err;
- }
- break;
- case MI_LOG_DELETE_ALL:
- if (verbose && !record_pos_file &&
- (!table_names[0] || (curr_file_info && curr_file_info->used)))
- printf_log("%s: %s -> %d\n",FILENAME(curr_file_info),
- command_name[command],result);
- break;
- default:
- fflush(stdout);
- VOID(fprintf(stderr,
- "Error: found unknown command %d in logfile, aborted\n",
- command));
- fflush(stderr);
- goto end;
- }
- }
- end_key_cache(dflt_key_cache,1);
- delete_tree(&tree);
- VOID(end_io_cache(&cache));
- VOID(my_close(file,MYF(0)));
- if (write_file && my_fclose(write_file,MYF(MY_WME)))
- DBUG_RETURN(1);
- DBUG_RETURN(0);
-
- err:
- fflush(stdout);
- VOID(fprintf(stderr,"Got error %d when reading from logfile\n",my_errno));
- fflush(stderr);
- goto end;
- com_err:
- fflush(stdout);
- VOID(fprintf(stderr,"Got error %d, expected %d on command %s at %s\n",
- my_errno,result,command_name[command],
- llstr(isamlog_filepos,llbuff)));
- fflush(stderr);
- end:
- end_key_cache(dflt_key_cache, 1);
- delete_tree(&tree);
- VOID(end_io_cache(&cache));
- VOID(my_close(file,MYF(0)));
- if (write_file)
- VOID(my_fclose(write_file,MYF(MY_WME)));
- DBUG_RETURN(1);
-}
-
-
-static int read_string(IO_CACHE *file, register uchar* *to, register uint length)
-{
- DBUG_ENTER("read_string");
-
- if (*to)
- my_free((uchar*) *to,MYF(0));
- if (!(*to= (uchar*) my_malloc(length+1,MYF(MY_WME))) ||
- my_b_read(file,(uchar*) *to,length))
- {
- if (*to)
- my_free(*to,MYF(0));
- *to= 0;
- DBUG_RETURN(1);
- }
- *((char*) *to+length)= '\0';
- DBUG_RETURN (0);
-} /* read_string */
-
-
-static int file_info_compare(void* cmp_arg __attribute__((unused)),
- void *a, void *b)
-{
- long lint;
-
- if ((lint=((struct file_info*) a)->process -
- ((struct file_info*) b)->process))
- return lint < 0L ? -1 : 1;
- return ((struct file_info*) a)->filenr - ((struct file_info*) b)->filenr;
-}
-
- /* ARGSUSED */
-
-static int test_if_open (struct file_info *key,
- element_count count __attribute__((unused)),
- struct test_if_open_param *param)
-{
- if (!strcmp(key->name,param->name) && key->id > param->max_id)
- param->max_id=key->id;
- return 0;
-}
-
-
-static void fix_blob_pointers(MI_INFO *info, uchar *record)
-{
- uchar *pos;
- MI_BLOB *blob,*end;
-
- pos=record+info->s->base.reclength;
- for (end=info->blobs+info->s->base.blobs, blob= info->blobs;
- blob != end ;
- blob++)
- {
- memcpy_fixed(record+blob->offset+blob->pack_length,&pos,sizeof(char*));
- pos+=_mi_calc_blob_length(blob->pack_length,record+blob->offset);
- }
-}
-
- /* close the file with hasn't been accessed for the longest time */
- /* ARGSUSED */
-
-static int test_when_accessed (struct file_info *key,
- element_count count __attribute__((unused)),
- struct st_access_param *access_param)
+static my_bool matches_list_of_tables(const char *isam_file_name)
{
- if (key->accessed < access_param->min_accessed && ! key->closed)
+ if (table_names && table_names[0])
{
- access_param->min_accessed=key->accessed;
- access_param->found=key;
- }
- return 0;
-}
-
-
-static void file_info_free(struct file_info *fileinfo)
-{
- DBUG_ENTER("file_info_free");
- if (update)
- {
- if (!fileinfo->closed)
- VOID(mi_close(fileinfo->isam));
- if (fileinfo->record)
- my_free(fileinfo->record,MYF(0));
- }
- my_free(fileinfo->name,MYF(0));
- my_free(fileinfo->show_name,MYF(0));
- DBUG_VOID_RETURN;
-}
-
-
-
-static int close_some_file(TREE *tree)
-{
- struct st_access_param access_param;
-
- access_param.min_accessed=LONG_MAX;
- access_param.found=0;
-
- VOID(tree_walk(tree,(tree_walk_action) test_when_accessed,
- (void*) &access_param,left_root_right));
- if (!access_param.found)
- return 1; /* No open file that is possibly to close */
- if (mi_close(access_param.found->isam))
- return 1;
- access_param.found->closed=1;
- return 0;
-}
-
-
-static int reopen_closed_file(TREE *tree, struct file_info *fileinfo)
-{
- char name[FN_REFLEN];
- if (close_some_file(tree))
- return 1; /* No file to close */
- strmov(name,fileinfo->show_name);
- if (fileinfo->id > 1)
- *strrchr(name,'<')='\0'; /* Remove "<id>" */
-
- if (!(fileinfo->isam= mi_open(name,O_RDWR,HA_OPEN_WAIT_IF_LOCKED)))
- return 1;
- fileinfo->closed=0;
- re_open_count++;
- return 0;
-}
-
- /* Try to find record with uniq key */
-
-static int find_record_with_key(struct file_info *file_info, uchar *record)
-{
- uint key;
- MI_INFO *info=file_info->isam;
- uchar tmp_key[MI_MAX_KEY_BUFF];
-
- for (key=0 ; key < info->s->base.keys ; key++)
- {
- if (mi_is_key_active(info->s->state.key_map, key) &&
- info->s->keyinfo[key].flag & HA_NOSAME)
+ char **name;
+ for (name= table_names ; *name ; name++)
{
- VOID(_mi_make_key(info,key,tmp_key,record,0L));
- return mi_rkey(info,file_info->record,(int) key,tmp_key,0,
- HA_READ_KEY_EXACT);
+ if (!strcmp(*name, isam_file_name))
+ return 1;
}
+ return 0;
}
return 1;
-}
-
-
-static void printf_log(const char *format,...)
-{
- char llbuff[21];
- va_list args;
- va_start(args,format);
- if (verbose > 2)
- printf("%9s:",llstr(isamlog_filepos,llbuff));
- if (verbose > 1)
- printf("%5ld ",isamlog_process); /* Write process number */
- (void) vprintf((char*) format,args);
- putchar('\n');
- va_end(args);
-}
-
-
-static my_bool cmp_filename(struct file_info *file_info, char * name)
-{
- if (!file_info)
- return 1;
- return strcmp(file_info->name,name) ? 1 : 0;
}
#include "mi_extrafunc.h"
diff -Nrup a/storage/myisam/myisampack.c b/storage/myisam/myisampack.c
--- a/storage/myisam/myisampack.c 2008-04-01 15:44:57 +02:00
+++ b/storage/myisam/myisampack.c 2008-05-07 17:30:59 +02:00
@@ -506,9 +506,12 @@ static int compress(PACK_MRG_INFO *mrg,c
/* Create temporary or join file */
if (backup)
- VOID(fn_format(org_name,isam_file->filename,"",MI_NAME_DEXT,2));
+ VOID(fn_format(org_name,isam_file->s->unresolv_file_name,
+ "", MI_NAME_DEXT, MY_REPLACE_EXT));
else
- VOID(fn_format(org_name,isam_file->filename,"",MI_NAME_DEXT,2+4+16));
+ VOID(fn_format(org_name,isam_file->s->unresolv_file_name,
+ "", MI_NAME_DEXT,
+ MY_REPLACE_EXT|MY_UNPACK_FILENAME|MY_RESOLVE_SYMLINKS));
if (!test_only && result_table)
{
/* Make a new indexfile based on first file in list */
@@ -693,7 +696,9 @@ static int compress(PACK_MRG_INFO *mrg,c
{
if (backup)
{
- if (my_rename(org_name,make_old_name(temp_name,isam_file->filename),
+ if (my_rename(org_name,
+ make_old_name(temp_name,
+ isam_file->s->unresolv_file_name),
MYF(MY_WME)))
error=1;
else
@@ -2977,7 +2982,7 @@ static int save_state(MI_INFO *isam_file
VOID(my_chsize(share->kfile, share->base.keystart, 0, MYF(0)));
if (share->base.keys)
isamchk_neaded=1;
- DBUG_RETURN(mi_state_info_write(share->kfile,&share->state,1+2));
+ DBUG_RETURN(mi_state_info_write(share, share->kfile, &share->state, 1+2));
}
@@ -3010,7 +3015,7 @@ static int save_state_mrg(File file,PACK
if (isam_file->s->base.keys)
isamchk_neaded=1;
state.changed=STATE_CHANGED | STATE_NOT_ANALYZED; /* Force check of table */
- DBUG_RETURN (mi_state_info_write(file,&state,1+2));
+ DBUG_RETURN (mi_state_info_write(isam_file->s, file, &state, 1+2));
}
diff -Nrup a/storage/myisammrg/ha_myisammrg.cc b/storage/myisammrg/ha_myisammrg.cc
--- a/storage/myisammrg/ha_myisammrg.cc 2008-04-29 11:27:24 +02:00
+++ b/storage/myisammrg/ha_myisammrg.cc 2008-05-07 17:30:59 +02:00
@@ -506,7 +506,7 @@ int ha_myisammrg::attach_children(void)
DBUG_PRINT("error",("reclength: %lu mean_rec_length: %lu",
table->s->reclength, stats.mean_rec_length));
if (test_if_locked & HA_OPEN_FOR_REPAIR)
- myrg_print_wrong_table(file->open_tables->table->filename);
+ myrg_print_wrong_table(file->open_tables->table->s->unresolv_file_name);
error= HA_ERR_WRONG_MRG_TABLE_DEF;
goto err;
}
@@ -530,14 +530,14 @@ int ha_myisammrg::attach_children(void)
u_table->table->s->base.fields, false))
{
DBUG_PRINT("error", ("table definition mismatch: '%s'",
- u_table->table->filename));
+ u_table->table->s->unresolv_file_name));
error= HA_ERR_WRONG_MRG_TABLE_DEF;
if (!(this->test_if_locked & HA_OPEN_FOR_REPAIR))
{
my_free((uchar*) recinfo, MYF(0));
goto err;
}
- myrg_print_wrong_table(u_table->table->filename);
+ myrg_print_wrong_table(u_table->table->s->unresolv_file_name);
}
}
my_free((uchar*) recinfo, MYF(0));
@@ -1009,7 +1009,7 @@ void ha_myisammrg::update_create_info(HA
if (!(ptr = (TABLE_LIST *) thd->calloc(sizeof(TABLE_LIST))))
goto err;
- split_file_name(open_table->table->filename, &db, &name);
+ split_file_name(open_table->table->s->unresolv_file_name, &db, &name);
if (!(ptr->table_name= thd->strmake(name.str, name.length)))
goto err;
if (db.length && !(ptr->db= thd->strmake(db.str, db.length)))
@@ -1127,7 +1127,7 @@ void ha_myisammrg::append_create_info(St
LEX_STRING db, name;
LINT_INIT(db.str);
- split_file_name(open_table->table->filename, &db, &name);
+ split_file_name(open_table->table->s->unresolv_file_name, &db, &name);
if (open_table != first)
packet->append(',');
/* Report database for mapped table if it isn't in current database */
diff -Nrup a/storage/myisammrg/myrg_info.c b/storage/myisammrg/myrg_info.c
--- a/storage/myisammrg/myrg_info.c 2006-12-31 01:06:41 +01:00
+++ b/storage/myisammrg/myrg_info.c 2008-05-07 17:30:59 +02:00
@@ -50,7 +50,8 @@ int myrg_status(MYRG_INFO *info,register
info->records+=file->table->s->state.state.records;
info->del+=file->table->s->state.state.del;
DBUG_PRINT("info2",("table: %s, offset: %lu",
- file->table->filename,(ulong) file->file_offset));
+ file->table->s->unresolv_file_name,
+ (ulong) file->file_offset));
}
x->records= info->records;
x->deleted= info->del;
diff -Nrup a/storage/myisammrg/myrg_open.c b/storage/myisammrg/myrg_open.c
--- a/storage/myisammrg/myrg_open.c 2007-12-12 09:14:05 +01:00
+++ b/storage/myisammrg/myrg_open.c 2008-05-07 17:30:59 +02:00
@@ -414,7 +414,7 @@ int myrg_attach_children(MYRG_INFO *m_in
while ((myisam= (*callback)(callback_param)))
{
DBUG_PRINT("myrg", ("child_nr: %u table: '%s'",
- child_nr, myisam->filename));
+ child_nr, myisam->s->unresolv_file_name));
DBUG_ASSERT(child_nr < m_info->tables);
/* Special handling when the first child is attached. */
@@ -441,12 +441,12 @@ int myrg_attach_children(MYRG_INFO *m_in
if (m_info->reclength != myisam->s->base.reclength)
{
DBUG_PRINT("error", ("definition mismatch table: '%s' repair: %d",
- myisam->filename,
+ myisam->s->unresolv_file_name,
(handle_locking & HA_OPEN_FOR_REPAIR)));
my_errno= HA_ERR_WRONG_MRG_TABLE_DEF;
if (handle_locking & HA_OPEN_FOR_REPAIR)
{
- myrg_print_wrong_table(myisam->filename);
+ myrg_print_wrong_table(myisam->s->unresolv_file_name);
continue;
}
goto err;
diff -Nrup a/storage/myisammrg/myrg_rrnd.c b/storage/myisammrg/myrg_rrnd.c
--- a/storage/myisammrg/myrg_rrnd.c 2007-05-10 11:59:33 +02:00
+++ b/storage/myisammrg/myrg_rrnd.c 2008-05-07 17:30:59 +02:00
@@ -111,6 +111,6 @@ static MYRG_TABLE *find_table(MYRG_TABLE
start=mid;
}
DBUG_PRINT("info",("offset: %lu, table: %s",
- (ulong) pos, start->table->filename));
+ (ulong) pos, start->table->s->unresolv_file_name));
DBUG_RETURN(start);
}
| Thread |
|---|
| • bk commit into 6.0 tree (istruewing:1.2616) WL#866 | Ingo Struewing | 7 May |
