Below is the list of changes that have just been committed into a local
6.0 repository of istruewing. When istruewing does a push these changes
will be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html
ChangeSet@stripped, 2008-05-09 12:27:23+02:00, istruewing@stripped +73 -0
WL#866 - Online backup driver for MyISAM
Patch from Guilhem Bichot ported to a current backup kernel.
When this driver (file myisam_backup_engine.cc) is to do a backup,
it starts physical logging of changes for target tables (file
mi_log.c), dirtily copies tables, then the physical log.
When this driver does a restore, it copies back tables and the log,
applies the log (file mi_examine_log.c).
The data file is always backed up; for the index file two methods are
available: either only its 64KB header is copied and then restore will
repair indexes, or the whole index file is copied.
Backup starts and runs without disturbing any running or new update
except at the very end: when creating the validity point it locks all
backed-up MyISAM tables with a read lock (so, stalls new update
statements and waits for running update statements to finish).
I recommend to readers to read in order myisam_backup_engine.cc,
mi_log.c then other files.
GUILHEM_TODO tag identifies the most urgent todo.
Fixes for misc bugs found along the way of development:
missing initialization in Bitmap<64>, issues with the dbug library.
dbug/dbug.c@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +1 -0
bugfix provided by Serg, for when doing "SET DEBUG=-d,etc"
(process hung).
include/atomic/generic-msvc.h@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped
+11 -6
Interlocked* functions use LONG as 32-bit integer type; my_atomic.h
rather provides int32 (=int on Windows), int* does not naturally
cast to LONG* and build fails; adding explicit casts.
include/atomic/nolock.h@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +4 -1
Changed comment.
include/keycache.h@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +11 -2
A block now has an optional post_write_arg. A key cache has a global
post_write function which is called after any block is written to its
file. The block's post_write_arg is set through key_cache_write().
The key cache's post_write is set when opening MyISAM tables.
This is used by MyISAM's physical logging: when a key page
is flushed to the file, the write is also recorded in the physical
log, thanks to such callback.
include/my_global.h@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +5 -3
When testing a C++ file against "gcc -Wold-style-cast", these casts
surfaced. STATIC_CAST means "a static cast which works in
C and in C++, without -Wold-style-cast warnings".
include/my_sys.h@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +27 -11
New members of IO_CACHE:
* post_write (called by my_b_flush_io_cache()),
* hard_write_error_in_the_past: like the existing 'error' except that
once set it is not reset until cache is reinitialized; 'error' is
reset at next _my_b_write() call for example. Is useful when we want
to lazily (once in a while) monitor if an IO_CACHE got a write error
(and so, file is not usable) instead of monitoring each IO_CACHE
write. These are used by MyISAM's online backup
(myisam_backup_engine.cc) to test if the physical log got a write
error.
include/myisam.h@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +17 -3
declarations' update.
mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test@stripped, 2008-05-09
12:27:18+02:00, istruewing@stripped +1 -0
don't pollute rest of the calling test by keeping a lock.
mysql-test/lib/mtr_report.pl@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped +2
-0
Suppress an intentionally provoked error.
mysql-test/mysql-test-run.pl@stripped, 2008-05-09 12:27:18+02:00, istruewing@stripped
+22 -0
ability to call myisamlog from tests (for new test t/myisamlog.test)
look for myisamlog in storage/myisam/(debug|release)
mysql-test/r/backup_myisam1.result@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +7 -0
result
mysql-test/r/backup_myisam1.result@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +0 -0
mysql-test/r/backup_myisam2.result@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +53 -0
result is ok: checksums match.
mysql-test/r/backup_myisam2.result@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +0 -0
mysql-test/r/backup_no_be.result@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+1 -1
We are using the native MyISAM backup driver now.
mysql-test/r/debug_sync.result@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+1 -1
Post-merge fix.
mysql-test/r/myisamlog.result@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+48 -0
result
mysql-test/r/myisamlog.result@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +0
-0
mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result@stripped, 2008-05-09
12:27:19+02:00, istruewing@stripped +1 -0
result update
mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result@stripped, 2008-05-09
12:27:19+02:00, istruewing@stripped +1 -0
result update
mysql-test/t/backup_charsets.test@stripped, 2008-05-09 12:27:19+02:00,
istruewing@stripped +2 -0
Suppress backup test on embedded server.
mysql-test/t/backup_myisam1-master.opt@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +1 -0
test with external locking
mysql-test/t/backup_myisam1-master.opt@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +0 -0
mysql-test/t/backup_myisam1.test@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+28 -0
test that backup fails with external locking
Suppress backup test on embedded server.
Added cleanup.
Fixed result from new report format.
mysql-test/t/backup_myisam1.test@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+0 -0
mysql-test/t/backup_myisam2.test@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+84 -0
test of records >64k (see MI_LOG_BIG_NUMBERS in mi_log.c)
Suppress backup test on embedded server.
Added cleanup.
Fixed result from new report format.
mysql-test/t/backup_myisam2.test@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+0 -0
mysql-test/t/backup_no_be.test@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+2 -1
Suppress backup test on embedded server.
Fixed a typo.
mysql-test/t/key_cache.test@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +3
-3
As the size of the BLOCK_LINK structure has grown (to accomodate
the new post_write_arg member), the number
of available blocks for a fixed key cache size has decreased
fix for 64-bit pointers (one pointer was added to the key cache
block structure, so less blocks)
mysql-test/t/myisamlog-master.opt@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +1 -0
To test the MyISAM logical log, which cannot be enabled on the fly.
mysql-test/t/myisamlog-master.opt@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +0 -0
mysql-test/t/myisamlog.test@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +39
-0
Test of MyISAM logical logging and the myisamlog utility:
see if they can repopulate a table.
Suppress test on embedded server.
mysql-test/t/myisamlog.test@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +0
-0
mysys/CMakeLists.txt@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +1 -1
include my_dup.c in mysys under Windows
mysys/mf_iocache.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +53 -19
Calling the new post_write hook. Note that it does not work with
SEQ_READ_APPEND IO_CACHEs (we don't need a post_write for them now).
Setting IO_CACHE::hard_write_error_in_the_past at the same time we
set IO_CACHE::error to -1 in functions which do writes.
IO_CACHE::hard_write_error_in_the_past is however not reset for
each write, that's its difference from IO_CACHE::error.
mysys/mf_keycache.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +60 -17
A block now has an optional post_write_arg which is used by
keycache->post_write after flushing the block.
We make sure that all pwrite() call keycache->post_write, by replacing
them by a new key_cache_pwrite().
This is used by MyISAM's physical logging.
mysys/my_thr_init.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +15 -4
A dedicated mutex for the MyISAM log (better not use THR_LOCK_myisam
which is already used at every mi_open()/mi_close(), and also it made
coding easier, see comments in _myisam_log_command()).
In my_thread_end(), do a DBUG_POP() so that if the thread has used
SET DEBUG=, the memory allocated by that SET is freed.
sql/backup/stream_v1_transport.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+3 -2
Fix an inifinite loop. EOS went undetected before.
Preliminarily reduced default block size.
sql/item_func.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +22 -7
debug_sync_point() implicitely releases any user-level lock held
by the current thread (like GET_LOCK()) ones; this is incorrect for
testing: assert that it does not hold a lock, and a comment to suggest
proper usage.
sql/mysqld.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +7 -7
emphasizing that the MyISAM log set from the command line is the
logical one.
sql/sql_bitmap.h@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +2 -2
Fix for valgrind error (unrelated to MyISAM online backup driver):
Bitmap<64> needs to initialize in its constructor, like the any-other-size
Bitmap does. Bug specific of 6.0.
sql/sql_cache.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +3 -2
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
sql/sql_class.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +3 -2
For the "locking threads" created by some online backup drivers
(MyISAM one and default lock-based one), THD::awake() must abort
locks. That if() dates from a time where the only possible system
threads were the delayed insert system thread and the slave threads.
sql/sql_load.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +1 -2
cast not needed
sql/sql_repl.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +4 -1
new prototype
sql/sql_repl.h@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +1 -1
new prototype
sql/sql_table.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +3 -0
As start_bulk_insert() has been called, end_bulk_insert() must be
called (bug found thanks to new assertion in mi_locking.c)
storage/myisam/CMakeLists.txt@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+1 -1
include new files in build
storage/myisam/Makefile.am@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +1
-1
include new files in build
storage/myisam/ha_myisam.cc@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+12 -7
MyISAM now has an online backup driver, see myisam_backup_engine.cc.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name.
Unset STATE_BAD_OPEN_COUNT after a successful check or repair.
Fix for doxygen.err warnings.
storage/myisam/ha_myisam.h@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +4
-0
MyISAM now has an online backup driver
storage/myisam/mi_check.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +83
-11
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging.
Note: REPAIR/OPTIMIZE TABLE are not handled by
physical logging; they should be blocked by the MySQL online backup
kernel: when they delete/create/rename files, they are as much
a problem as ALTER TABLE. So we assert for these cases.
At end of INSERT SELECT, if the table was originally empty,
an index rebuild is done. This uses my_pwrite() and my_chsize(),
which we thus have to write to the physical log.
storage/myisam/mi_close.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +12
-2
If we failed to flush the key cache, we need to mark the table as
corrupted (fix for an unlikely bug).
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging.
If the table has logged a MI_LOG_OPEN to the physical log (because it
has logged some writes there), it needs to log a MI_LOG_CLOSE too.
Regarding the added assertion, see revision comment in mi_locking.c.
storage/myisam/mi_create.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +9
-1
Ensure that no table is truncated if it is doing physical
logging. We *could* support TRUNCATE during an online backup of
MyISAM (switch off HTON_CAN_RECREATE during backup) but:
- it would require some synchro (no backup can start while recreate-based
truncate is running, etc)
- consistent-snapshot driver can't bear TRUNCATE anyway.
Another possibility is also to always switch off HTON_CAN_RECREATE.
storage/myisam/mi_delete.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +7
-4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_delete_all.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped
+4 -1
Log to the physical log the MI_DELETE_ALL operation
storage/myisam/mi_dynrec.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +23
-13
Log to the physical log the operation of writing bytes to the data file.
Always log _after_ the write (see comment at start of
mi_log_start_physical()).
storage/myisam/mi_examine_log.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+871 -0
This is the examine_log() function taken out of myisamlog.c, as we
now need it in the server too, to restore from an online backup.
It is renamed to mi_examine_log().
examine_log() used global variables of myisamlog.c, that is not possible
now so mi_examine_log() takes in parameter a new structure
MI_EXAMINE_LOG_PARAM whose content corresponds to the old global
variables.
The function is extended to understand new log commands found in
physical logs: MI_LOG_WRITE_BYTES_MYI, MI_LOG_WRITE_BYTES_MYD,
MI_LOG_CHSIZE_MYI, as well as MI_LOG_DELETE_ALL which wasn't replayed
properly (a bug).
Other bugfix: NO_FILEPOS was too small for 64-bit machines.
Other bugfix: MI_LOG_LOCK applying could not work because it didn't
remove the 'flag' added at the end of mi_lock_database().
To see the real code changes I made (because, most of lines are just
moved from myisamlog.c), you should diff mi_examine_log() with
examine_log() from the old myisamlog.c (I recommend it).
storage/myisam/mi_examine_log.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+0 -0
storage/myisam/mi_extra.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +59
-3
When we set up a write cache (HA_EXTRA_WRITE_CACHE) we give the cache
a post_write call, which will make sure that writes to the data file
will go to the physical log if needed.
It is more efficient that logging all my_b_write()s as it generates
less log records, and anyway we needed the post_write callback for
when logging is enabled while the table is in already the middle of
using a write cache, when non-logged my_b_write()s have passed already.
mi_state_info_write() needs the MYISAM_SHARE now, as it is used by
physical logging
storage/myisam/mi_locking.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +89
-46
Comments explaining why certain writes to the index file needn't go into
the physical log.
New function mi_remap_file_and_write_state_for_unlock(), used when
unlocking a table and in mi_backup_stop_logging_for_tables().
If logical log, we now always log MI_LOG_LOCK. Indeed, without it,
the MI_LOG_EXTRA failed to apply when command was HA_EXTRA_CACHE (see
the EACCES error in mi_extra():
missing MI_LOG_LOCK => info->lock_type==F_UNLCK => applying of
MI_LOG_EXTRA failed (so myisamlog.test got such warnings in
mysqltest.log: "Warning: error 13, expected 0 on command extra at 211").
We also make the storage of lock_type portable (independent of
endianess).
storage/myisam/mi_log.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +792
-79
mi_log() can now open a logical (==debug)or physical (==backup) log:
it thus receives new parameters (list of tables...); it can
close logs too. Opening and closing a physical log is more work
than a logical one, so additional functions are called.
Logging functions now use IO_CACHE instead of my_write(), to be
faster.
_myisam_log() merged into _myisam_log_command() (they were very
similar).
If _myisam_log_command() uses small numbers (file descriptor,
file offset buffer length) it stores them in few bytes; if they are big
it stores them in more bytes; how many bytes are used is denoted
by the highest bit of the 'command' (MI_LOG_BIG_NUMBERS).
That and the merge of _myisam_log() and _myisam_log_command()
imply that old myisamlog won't read new logical log, but it's ok: the
MyISAM log was so far only for debugging, advertised as such,
apparently used only by MySQL devs, and had a bug (MI_LOG_DELETE_ALL
was not replayed by myisamlog) which nobody reported.
_myisam_log_record() renamed to _myisam_log_record_logical()
(used only by logical logging).
Logging functions now need to check if the log is opened _inside the
log's mutex_, because the physical log can close at any time (while
the old logical log, was either open or closed for the
lifetime of the MySQL server process).
_myisam_log_command() may log MI_LOG_OPEN on top of its command
(needed for physical logging), if this is a physical log and
the first time this table's share writes to it.
_myisam_log_record() is used only by logical logging, renamed it.
storage/myisam/mi_open.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +114
-25
When opening a table, detect if it must do physical logging;
and remember with STATE_BAD_OPEN_COUNT if open_count>0 at first
open (this way, online backup can know if the open_count>0 which it
sees is a real problem or not); set key cache's post_write.
mi_state_info_write() needs the MYISAM_SHARE now, as it needs to
record the my_pwrite() done to the index file, into the physical log.
Note, for now all calls to mi_state_info_write() have the state
equal to the share's state but future projects may not have that,
that's why the two arguments stay.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name,
indeed physical logging needs this unresolved name, and
sometimes cannot have MI_INFO available, but always has MYISAM_SHARE
(see log_key_cache_flush_physical()). It is also more
efficient (why duplicate the same string in all MI_INFOs for a same
table?).
storage/myisam/mi_page.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +17
-15
passing "share" as post_write_arg to key_cache_write()
storage/myisam/mi_panic.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +11
-7
update for new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_rrnd.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +1 -3
cosmetic change
storage/myisam/mi_static.c@stripped, 2008-05-09 12:27:19+02:00, istruewing@stripped +38
-5
New physical log in MyISAM. MyISAM-specific error messages.
error message for when backup archive's format is too recent
storage/myisam/mi_test2.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +3 -3
update for new prototype. Complement to the fix of BUG#30094
(this complement has been approved by Serg and pushed in another
tree months ago); the bug caused random assertion failures.
storage/myisam/mi_test3.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +2 -2
update for new prototype
storage/myisam/mi_test_all.sh@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+2 -0
We need to remove logs from previous "mi_test_all" runs, or a bug
in writing myisam.log would continue to make the test fail even
though bug is fixed (mi_test2 would just append to the bad log).
storage/myisam/mi_update.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +6
-4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/mi_write.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +5 -4
update to new prototype.
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/myisam_backup_engine.cc@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +1934 -0
MyISAM online backup engine. See this file's first lines (namespace
myisam_backup) for a description of how it works.
This is the file which really drives the backup/restore inside MyISAM
(see it as the master which calls functions from other files to
perform some subtasks).
Note that Chuck has taken some code from this file and it is now
duplicated into sql/backup/ files, I should merge the two
to avoid duplication, when Chuck's work is pushed
Give error message if the backup archive which the kernel asks
us to restore is of a more recent version that ourselves (we cannot
parse it).
storage/myisam/myisam_backup_engine.cc@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +0 -0
storage/myisam/myisam_ftdump.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+2 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisam/myisamchk.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +4
-2
STATE_BAD_OPEN_COUNT reset on repair
storage/myisam/myisamdef.h@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+135 -19
Adding member "physical_logging" to MYISAM_SHARE, tells if the table
is currently doing physical logging or not.
It is read and set with atomic operations: this provides the needed
synchronization between the table-writer thread (if the table is being
written now) (which is the reader of "physical_logging") and the thread
turning physical logging on or off (which is the writer of
"physical_logging").
Adding member "MI_LOG_OPEN_stored_in_physical_log" to MYISAM_SHARE, to
remember if we already stored MI_LOG_OPEN in this physical log for this
table.
A new mutex THR_LOCK_myisam_log to protect MyISAM logs (instead of
THR_LOCK_myisam, for concurrency and coding reasons (explained
in _myisam_log_command()).
New commands which can be stored only in the physical log:
MI_LOG_WRITE_BYTES_MYD, MI_LOG_WRITE_BYTES_MYI, MI_LOG_CHSIZE_MYI.
Updates to myisam_log_* macros to adapt to the fact that logs are now
an IO_CACHE and not a plain OS file.
As most of the logic of myisamlog moved out of myisamlog.c into
mi_examine_log.c, mi_examine_log() cannot use globals of myisamlog.c
anymore; a new structure MI_EXAMINE_LOG_PARAM is introduced instead
(like MI_CHECK). mi_state_info_write() needs the MYISAM_SHARE now, as
it is used by physical logging.
Flag STATE_BAD_OPEN_COUNT.
Note: all extern vars touched in this file have their documentation
in the file where they are defined.
error for when backup archive's format is too recent
storage/myisam/myisamlog.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +74
-624
The main function of this file, examine_log(), is now needed in
the server to be able to do a restore from an online backup,
so moves to mi_examine_log.c.
I changed a meaningless min() to max().
New log commands (MI_LOG_etc) have long names so I widened the
space for displaying their name to 20 characters.
Due to backward-incompatibilities (see mi_log.c) we bump version
number to 2.0.
storage/myisam/myisampack.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped +10
-5
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name.
mi_state_info_write() needs MYISAM_SHARE as physical logging does.
storage/myisammrg/ha_myisammrg.cc@stripped, 2008-05-09 12:27:20+02:00,
istruewing@stripped +5 -5
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_info.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+2 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_open.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+3 -3
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
storage/myisammrg/myrg_rrnd.c@stripped, 2008-05-09 12:27:20+02:00, istruewing@stripped
+1 -1
MI_INFO::filename is now MYISAM_SHARE::unresolv_file_name
diff -Nrup a/dbug/dbug.c b/dbug/dbug.c
--- a/dbug/dbug.c 2008-03-28 19:03:59 +01:00
+++ b/dbug/dbug.c 2008-05-09 12:27:18 +02:00
@@ -1321,6 +1321,7 @@ static struct link *ListDel(struct link
free((void*) delme);
}
} while (*cur && *(cur=&((*cur)->next_link)));
+ ctlp++;
}
return head;
}
diff -Nrup a/include/atomic/generic-msvc.h b/include/atomic/generic-msvc.h
--- a/include/atomic/generic-msvc.h 2007-10-09 19:55:11 +02:00
+++ b/include/atomic/generic-msvc.h 2008-05-09 12:27:18 +02:00
@@ -21,25 +21,30 @@
intrinsics.
*/
#define MY_ATOMIC_MODE "msvc-intrinsics"
+/* the 32-bit type for Interlocked functions is LONG */
+#define IL_EXCHG_TYPE32 LONG
+#define IL_EXCHG_TYPEptr PVOID
#define IL_EXCHG_ADD32 InterlockedExchangeAdd
#define IL_COMP_EXCHG32 InterlockedCompareExchange
#define IL_COMP_EXCHGptr InterlockedCompareExchangePointer
#define IL_EXCHG32 InterlockedExchange
#define IL_EXCHGptr InterlockedExchangePointer
#define make_atomic_add_body(S) \
- v= IL_EXCHG_ADD ## S (a, v)
+ v= (int ## S)IL_EXCHG_ADD ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ##
S)(v))
#define make_atomic_cas_body(S) \
- int ## S initial_cmp= *cmp; \
- int ## S initial_a= IL_COMP_EXCHG ## S (a, set, initial_cmp); \
- if (!(ret= (initial_a == initial_cmp))) *cmp= initial_a;
+ IL_EXCHG_TYPE ## S initial_cmp= *cmp; \
+ IL_EXCHG_TYPE ## S initial_a= IL_COMP_EXCHG ## S ((volatile IL_EXCHG_TYPE ## S *)(a),
(IL_EXCHG_TYPE ## S)(set), initial_cmp); \
+ if (!(ret= (initial_a == initial_cmp))) *cmp= (int ## S)initial_a;
#define make_atomic_swap_body(S) \
- v= IL_EXCHG ## S (a, v)
+ v= IL_EXCHG ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ## S)(v))
#define make_atomic_load_body(S) \
ret= 0; /* avoid compiler warning */ \
- ret= IL_COMP_EXCHG ## S (a, ret, ret);
+ ret= IL_COMP_EXCHG ## S ((volatile IL_EXCHG_TYPE ## S *)(a), (IL_EXCHG_TYPE ## S)(ret),
(IL_EXCHG_TYPE ## S)(ret));
#else /* cleanup */
+#undef IL_EXCHG_TYPE32
+#undef IL_EXCHG_TYPEptr
#undef IL_EXCHG_ADD32
#undef IL_COMP_EXCHG32
#undef IL_COMP_EXCHGptr
diff -Nrup a/include/atomic/nolock.h b/include/atomic/nolock.h
--- a/include/atomic/nolock.h 2008-01-12 02:45:42 +01:00
+++ b/include/atomic/nolock.h 2008-05-09 12:27:18 +02:00
@@ -32,7 +32,10 @@
#endif
#ifdef make_atomic_cas_body
-
+/*
+ Type not used so minimal size (emptry struct has different size between C
+ and C++, zero-length array is gcc-specific).
+*/
typedef char my_atomic_rwlock_t __attribute__ ((unused));
#define my_atomic_rwlock_destroy(name)
#define my_atomic_rwlock_init(name)
diff -Nrup a/include/keycache.h b/include/keycache.h
--- a/include/keycache.h 2007-10-02 09:32:29 +02:00
+++ b/include/keycache.h 2008-05-09 12:27:18 +02:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* Key cache variable structures */
+/**
+ @file
+ Key cache API
+*/
#ifndef _keycache_h
#define _keycache_h
@@ -34,6 +37,10 @@ typedef struct st_keycache_wqueue
struct st_my_thread_var *last_thread; /* circular list of waiting threads */
} KEYCACHE_WQUEUE;
+/** Callback called when any block is flushed */
+typedef int (*KEYCACHE_POST_WRITE_CALLBACK)(void *arg, const uchar *buffert,
+ uint length, my_off_t filepos);
+
#define CHANGED_BLOCKS_HASH 128 /* must be power of 2 */
/*
@@ -81,6 +88,7 @@ typedef struct st_key_cache
KEYCACHE_WQUEUE waiting_for_block; /* requests waiting for a free block */
BLOCK_LINK *changed_blocks[CHANGED_BLOCKS_HASH]; /* hash for dirty file bl.*/
BLOCK_LINK *file_blocks[CHANGED_BLOCKS_HASH]; /* hash for other file bl.*/
+ KEYCACHE_POST_WRITE_CALLBACK post_write;/**< Called when flushing any block*/
/*
The following variables are and variables used to hold parameters for
@@ -124,7 +132,8 @@ extern int key_cache_insert(KEY_CACHE *k
extern int key_cache_write(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length,
- uint block_length,int force_write);
+ uint block_length, int force_write,
+ void *post_write_arg);
extern int flush_key_blocks(KEY_CACHE *keycache,
int file, enum flush_type type);
extern void end_key_cache(KEY_CACHE *keycache, my_bool cleanup);
diff -Nrup a/include/my_global.h b/include/my_global.h
--- a/include/my_global.h 2008-04-14 12:09:56 +02:00
+++ b/include/my_global.h 2008-05-09 12:27:18 +02:00
@@ -68,9 +68,11 @@
#ifdef __cplusplus
#define C_MODE_START extern "C" {
#define C_MODE_END }
+#define STATIC_CAST(TYPE) static_cast<TYPE>
#else
#define C_MODE_START
#define C_MODE_END
+#define STATIC_CAST(TYPE) (TYPE)
#endif
#if defined(_WIN32) || defined(_WIN64) || defined(__WIN32__) || defined(WIN32)
@@ -924,7 +926,7 @@ typedef long long my_ptrdiff_t;
#define my_offsetof(TYPE, MEMBER) \
((size_t)((char *)&(((TYPE *)0x10)->MEMBER) - (char*)0x10))
-#define NullS (char *) 0
+#define NullS STATIC_CAST(char *)(0)
/* Nowdays we do not support MessyDos */
#ifndef NEAR
#define NEAR /* Who needs segments ? */
@@ -1041,7 +1043,7 @@ typedef ulonglong my_off_t;
#else
typedef unsigned long my_off_t;
#endif
-#define MY_FILEPOS_ERROR (~(my_off_t) 0)
+#define MY_FILEPOS_ERROR (~STATIC_CAST(my_off_t)(0))
#if !defined(__WIN__)
typedef off_t os_off_t;
#endif
@@ -1078,7 +1080,7 @@ typedef char bool; /* Ordinary boolean
#define INT8(v) (int8) (v)
#define INT16(v) (int16) (v)
#define INT32(v) (int32) (v)
-#define MYF(v) (myf) (v)
+#define MYF(v) STATIC_CAST(myf)(v)
#ifndef LL
#ifdef HAVE_LONG_LONG
diff -Nrup a/include/my_sys.h b/include/my_sys.h
--- a/include/my_sys.h 2008-04-14 12:11:45 +02:00
+++ b/include/my_sys.h 2008-05-09 12:27:18 +02:00
@@ -13,6 +13,11 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
+/**
+ @file
+ mysys library API
+*/
+
#ifndef _my_sys_h
#define _my_sys_h
C_MODE_START
@@ -344,7 +349,10 @@ typedef struct st_dynamic_string
} DYNAMIC_STRING;
struct st_io_cache;
-typedef int (*IO_CACHE_CALLBACK)(struct st_io_cache*);
+/** Function called when certain events happen to an IO_CACHE */
+typedef int (*IO_CACHE_CALLBACK)(struct st_io_cache *cache,
+ const uchar *buffert, uint length,
+ my_off_t filepos);
#ifdef THREAD
typedef struct st_io_cache_share
@@ -443,21 +451,24 @@ typedef struct st_io_cache /* Used when
*/
enum cache_type type;
/*
- Callbacks when the actual read I/O happens. These were added and
- are currently used for binary logging of LOAD DATA INFILE - when a
- block is read from the file, we create a block create/append event, and
- when IO_CACHE is closed, we create an end event. These functions could,
- of course be used for other things
- */
- IO_CACHE_CALLBACK pre_read;
- IO_CACHE_CALLBACK post_read;
- IO_CACHE_CALLBACK pre_close;
+ Callbacks were added and are currently used for binary logging of LOAD
+ DATA INFILE - when a block is read from the file, we create a block
+ create/append event, and when IO_CACHE is closed, we create an end event;
+ also used to write the MyISAM WRITE_CACHE blocks to the MyISAM physical
+ log. These functions could, of course be used for other things. Note: some
+ callbacks share the same argument ("arg").
+ */
+ IO_CACHE_CALLBACK pre_read; /**< called before reading from disk */
+ IO_CACHE_CALLBACK post_read; /**< called after reading from disk */
+ IO_CACHE_CALLBACK pre_close; /**< called before ending the cache */
+ /** Called _after_ writing to disk; not honoured by SEQ_READ_APPEND */
+ IO_CACHE_CALLBACK post_write;
/*
Counts the number of times, when we were forced to use disk. We use it to
increase the binlog_cache_disk_use status variable.
*/
ulong disk_writes;
- void* arg; /* for use by pre/post_read */
+ void *arg; /**< used by pre/post_read,post_write */
char *file_name; /* if used with 'open_cached_file' */
char *dir,*prefix;
File file; /* file descriptor */
@@ -469,6 +480,11 @@ typedef struct st_io_cache /* Used when
partial.
*/
int seek_not_done,error;
+ /**
+ Cumulative 'error' since last [re]init_io_cache(). Useful if cache's user
+ polls for errors only once in a while.
+ */
+ int hard_write_error_in_the_past;
/* buffer_length is memory size allocated for buffer or write_buffer */
size_t buffer_length;
/* read_length is the same as buffer_length except when we use async io */
diff -Nrup a/include/myisam.h b/include/myisam.h
--- a/include/myisam.h 2008-04-01 17:13:53 +02:00
+++ b/include/myisam.h 2008-05-09 12:27:18 +02:00
@@ -13,7 +13,10 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/* This file should be included when using myisam_funktions */
+/**
+ @file
+ This file should be included when using MyISAM functions.
+*/
#ifndef _myisam_h
#define _myisam_h
@@ -32,6 +35,7 @@ extern "C" {
#endif
#include "my_handler.h"
#include <mysql/plugin.h>
+#include <hash.h>
/*
Limit max keys according to HA_MAX_POSSIBLE_KEY
@@ -249,8 +253,10 @@ typedef struct st_columndef /* column i
#endif
} MI_COLUMNDEF;
+/** Physical logging is always compiled in. Undefine if want to benchmark */
+#define HAVE_MYISAM_PHYSICAL_LOGGING 1
-extern char * myisam_log_filename; /* Name of logfile */
+extern char * myisam_logical_log_filename;
extern ulong myisam_block_size;
extern ulong myisam_concurrent_insert;
extern my_bool myisam_flush,myisam_delay_key_write,myisam_single_user;
@@ -295,7 +301,15 @@ extern int mi_extra(struct st_myisam_inf
extern int mi_reset(struct st_myisam_info *file);
extern ha_rows mi_records_in_range(MI_INFO *info, int inx,
key_range *min_key, key_range *max_key);
-extern int mi_log(int activate_log);
+/** Open/close actions allowed on a MyISAM log */
+enum enum_mi_log_action
+{
+ MI_LOG_ACTION_OPEN,
+ MI_LOG_ACTION_CLOSE_CONSISTENT, MI_LOG_ACTION_CLOSE_INCONSISTENT
+};
+enum enum_mi_log_type { MI_LOG_PHYSICAL, MI_LOG_LOGICAL };
+extern int mi_log(enum enum_mi_log_action action, enum enum_mi_log_type type,
+ const char *log_filename, const HASH *tables);
extern int mi_is_changed(struct st_myisam_info *info);
extern int mi_delete_all_rows(struct st_myisam_info *info);
extern ulong _mi_calc_blob_length(uint length , const uchar *pos);
diff -Nrup a/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test
b/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test
--- a/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test 2008-02-27 15:50:42
+01:00
+++ b/mysql-test/extra/binlog_tests/mix_innodb_myisam_binlog.test 2008-05-09 12:27:18
+02:00
@@ -315,6 +315,7 @@ disconnect con3;
connection con4;
select get_lock("a",10); # wait for rollback to finish
+do release_lock("a");
flush logs;
# we check that the error code of the "ROLLBACK" event is 0 and not
diff -Nrup a/mysql-test/lib/mtr_report.pl b/mysql-test/lib/mtr_report.pl
--- a/mysql-test/lib/mtr_report.pl 2008-05-05 21:33:22 +02:00
+++ b/mysql-test/lib/mtr_report.pl 2008-05-09 12:27:18 +02:00
@@ -342,6 +342,8 @@ sub mtr_report_stats ($) {
# ignore warning generated when backup engine selection algorithm is tested
($testname eq 'main.backup_no_be') and /Backup: Cannot create backup engine/ or
+ ($testname eq 'main.backup_myisam1') and
+ (/Backup: Can't initialize MyISAM backup driver/) or
/Sort aborted/ or
/Time-out in NDB/ or
/One can only use the --user.*root/ or
diff -Nrup a/mysql-test/mysql-test-run.pl b/mysql-test/mysql-test-run.pl
--- a/mysql-test/mysql-test-run.pl 2008-04-29 11:27:22 +02:00
+++ b/mysql-test/mysql-test-run.pl 2008-05-09 12:27:18 +02:00
@@ -142,6 +142,7 @@ our $exe_mysql;
our $exe_mysqladmin;
our $exe_mysql_upgrade;
our $exe_mysqlbinlog;
+our $exe_myisamlog;
our $exe_mysql_client_test;
our $exe_bug25714;
our $exe_mysqld;
@@ -1955,6 +1956,27 @@ sub environment_setup () {
" --debug=d:t:A,$path_vardir_trace/log/mysqlbinlog.trace";
}
$ENV{'MYSQL_BINLOG'}= $cmdline_mysqlbinlog;
+
+ # ----------------------------------------------------
+ # Setup env so childs can execute myisamlog
+ # ----------------------------------------------------
+
+ my $myisam_path= mtr_file_exists(vs_config_dirs("storage/myisam", ""),
+ "$glob_basedir/storage/myisam",
+ "$glob_basedir/bin");
+
+ $exe_myisamlog=
+ mtr_exe_exists("$myisam_path/myisamlog");
+
+ my $cmdline_myisamlog=
+ mtr_native_path($exe_myisamlog);
+
+ if ( $opt_debug )
+ {
+ $cmdline_myisamlog .=
+ " -#d:t:A,$path_vardir_trace/log/myisamlog.trace";
+ }
+ $ENV{'MYISAMLOG'}= $cmdline_myisamlog;
# ----------------------------------------------------
# Setup env so childs can execute mysql
diff -Nrup a/mysql-test/r/backup_myisam1.result b/mysql-test/r/backup_myisam1.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/backup_myisam1.result 2008-05-09 12:27:20 +02:00
@@ -0,0 +1,7 @@
+drop database if exists mysqltest;
+create database mysqltest;
+use mysqltest;
+CREATE TABLE t1 (a int) engine=myisam;
+BACKUP DATABASE mysqltest TO 'test.ba';
+ERROR HY000: Got error -1 'online backup impossible with --external-locking' from MyISAM
+DROP DATABASE mysqltest;
diff -Nrup a/mysql-test/r/backup_myisam2.result b/mysql-test/r/backup_myisam2.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/backup_myisam2.result 2008-05-09 12:27:20 +02:00
@@ -0,0 +1,53 @@
+drop database if exists mysqltest;
+create database mysqltest;
+use mysqltest;
+CREATE TABLE t1 (a longtext) engine=myisam;
+SELECT get_lock("data_prepare", 100);
+get_lock("data_prepare", 100)
+1
+SET SESSION debug="+d,query,enter,info,query,backup_debug";
+BACKUP DATABASE mysqltest TO 'test.ba';
+use mysqltest;
+insert into t1 values ("text");
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+update t1 set a=concat(a,a);
+select length(a) from t1;
+length(a)
+262144
+checksum table t1;
+Table Checksum
+mysqltest.t1 1728069308
+SELECT release_lock("data_prepare");
+release_lock("data_prepare")
+1
+backup_id
+#
+repair table t1 quick;
+Table Op Msg_type Msg_text
+mysqltest.t1 repair status OK
+DROP DATABASE mysqltest;
+RESTORE FROM 'test.ba';
+backup_id
+#
+select length(a) from t1;
+length(a)
+262144
+checksum table t1;
+Table Checksum
+mysqltest.t1 1728069308
+drop database mysqltest;
+SET SESSION debug="-d,query,enter,info,query,backup_debug";
diff -Nrup a/mysql-test/r/backup_no_be.result b/mysql-test/r/backup_no_be.result
--- a/mysql-test/r/backup_no_be.result 2008-04-16 20:23:01 +02:00
+++ b/mysql-test/r/backup_no_be.result 2008-05-09 12:27:19 +02:00
@@ -16,7 +16,7 @@ SELECT max(backup_id) INTO @id FROM mysq
WHERE command LIKE 'BACKUP DATABASE db1 %';
SELECT engines FROM mysql.online_backup WHERE backup_id=@id;
engines
-Default
+MyISAM
SET SESSION debug="d,backup_test_dummy_be_factory";
SELECT @@debug;
@@debug
diff -Nrup a/mysql-test/r/debug_sync.result b/mysql-test/r/debug_sync.result
--- a/mysql-test/r/debug_sync.result 2008-04-29 16:16:49 +02:00
+++ b/mysql-test/r/debug_sync.result 2008-05-09 12:27:19 +02:00
@@ -137,7 +137,7 @@ Variable_name Value
debug_sync ON - current signal: 'something'
SET DEBUG_SYNC= 'now WAIT_FOR nothing TIMEOUT 0';
Warnings:
-Warning 1673 debug sync point wait timed out
+Warning 1717 debug sync point wait timed out
SET DEBUG_SYNC= 'now SIGNAL nothing';
SHOW VARIABLES LIKE 'DEBUG_SYNC';
Variable_name Value
diff -Nrup a/mysql-test/r/myisamlog.result b/mysql-test/r/myisamlog.result
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/r/myisamlog.result 2008-05-09 12:27:20 +02:00
@@ -0,0 +1,48 @@
+create database if not exists mysqltest;
+use mysqltest;
+drop table if exists t1;
+create table t1 (a int, b varchar(100)) engine=myisam;
+insert into t1 values(1,'life'), (2,'file');
+insert into t1 select a*5, concat("A ",b) from t1;
+update t1 set b="A knife" where a=5;
+delete from t1 where a=10;
+select * from t1;
+a b
+1 life
+2 file
+5 A knife
+truncate table t1;
+flush table t1;
+Commands Used count Errors Recover errors
+open 16 0 0
+write 4 0 0
+update 1 0 0
+delete 1 0 0
+close 8 0 0
+extra 72 0 0
+lock 48 0 0
+Total 150 0 0
+select * from t1;
+a b
+flush table t1;
+Trying to update MyISAM files according to log 'VARDIR/master-data/myisam.log'
+Tables updated successfully
+
+Commands Used count Errors Recover errors
+open 17 0 0
+write 4 0 0
+update 1 0 0
+delete 1 0 0
+close 9 0 0
+extra 75 0 0
+lock 50 0 0
+Total 157 0 0
+select * from t1;
+a b
+1 life
+2 file
+5 A knife
+check table t1 extended;
+Table Op Msg_type Msg_text
+mysqltest.t1 check status OK
+drop database mysqltest;
diff -Nrup a/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
b/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result
--- a/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result 2008-04-01 15:44:50
+02:00
+++ b/mysql-test/suite/binlog/r/binlog_row_mix_innodb_myisam.result 2008-05-09 12:27:19
+02:00
@@ -433,6 +433,7 @@ insert into t2 select * from t1;
select get_lock("a",10);
get_lock("a",10)
1
+do release_lock("a");
flush logs;
select
(@a:=load_file("MYSQLTEST_VARDIR/tmp/mix_innodb_myisam_binlog.output"))
diff -Nrup a/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
b/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result
--- a/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2008-04-01 15:44:50
+02:00
+++ b/mysql-test/suite/binlog/r/binlog_stm_mix_innodb_myisam.result 2008-05-09 12:27:19
+02:00
@@ -376,6 +376,7 @@ insert into t2 select * from t1;
select get_lock("a",10);
get_lock("a",10)
1
+do release_lock("a");
flush logs;
select
(@a:=load_file("MYSQLTEST_VARDIR/tmp/mix_innodb_myisam_binlog.output"))
diff -Nrup a/mysql-test/t/backup_charsets.test b/mysql-test/t/backup_charsets.test
--- a/mysql-test/t/backup_charsets.test 2008-05-05 17:03:15 +02:00
+++ b/mysql-test/t/backup_charsets.test 2008-05-09 12:27:19 +02:00
@@ -17,6 +17,8 @@
# regardless of character set settings on backup and restore servers.
#
+--source include/not_embedded.inc
+
disable_warnings;
DROP DATABASE IF EXISTS backup_cs;
error 0,1;
diff -Nrup a/mysql-test/t/backup_myisam1-master.opt
b/mysql-test/t/backup_myisam1-master.opt
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/backup_myisam1-master.opt 2008-05-09 12:27:20 +02:00
@@ -0,0 +1 @@
+--external-locking=1
diff -Nrup a/mysql-test/t/backup_myisam1.test b/mysql-test/t/backup_myisam1.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/backup_myisam1.test 2008-05-09 12:27:20 +02:00
@@ -0,0 +1,28 @@
+# Test specific of MyISAM's online backup:
+# see if --external-locking=1 causes backup to fail as expected
+
+--source include/not_embedded.inc
+
+#
+# Cleanup from former test cases
+#
+--disable_warnings
+drop database if exists mysqltest;
+--enable_warnings
+--error 0,1
+--remove_file $MYSQLTEST_VARDIR/master-data/test.ba
+
+create database mysqltest;
+use mysqltest;
+CREATE TABLE t1 (a int) engine=myisam;
+
+--replace_column 1 #
+--error ER_GET_ERRMSG
+BACKUP DATABASE mysqltest TO 'test.ba';
+
+#
+# Cleanup from this test case
+#
+DROP DATABASE mysqltest;
+--remove_file $MYSQLTEST_VARDIR/master-data/test.ba
+
diff -Nrup a/mysql-test/t/backup_myisam2.test b/mysql-test/t/backup_myisam2.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/backup_myisam2.test 2008-05-09 12:27:20 +02:00
@@ -0,0 +1,84 @@
+# Tests specific of MyISAM's online backup
+
+--source include/not_embedded.inc
+--source include/have_debug.inc
+
+#
+# Cleanup from former test cases
+#
+--disable_warnings
+drop database if exists mysqltest;
+--enable_warnings
+--error 0,1
+--remove_file $MYSQLTEST_VARDIR/master-data/test.ba
+
+connect (backup,localhost,root,,);
+connect (restore,localhost,root,,);
+
+# Dedicated connection for GET_LOCK(). BACKUP_BREAKPOINT, present in
+# various places like mysql_delete(), causes current lock to be
+# released, that's why GET_LOCK() and DELETE must not be made by the
+# same connection.
+connect (syncer,localhost,root,,);
+
+connection backup;
+
+create database mysqltest;
+use mysqltest;
+
+# test of long records (causing records' length to be stored in a long
+# format in the backup log)
+
+CREATE TABLE t1 (a longtext) engine=myisam;
+
+connection syncer;
+# make sure we can update MyISAM tables before the backup finishes,
+# to test backup in online conditions
+SELECT get_lock("data_prepare", 100);
+
+connection backup;
+# prepare to block:
+SET SESSION debug="+d,query,enter,info,query,backup_debug";
+
+send BACKUP DATABASE mysqltest TO 'test.ba';
+
+connection restore;
+# Must wait to know when backup has entered lock.
+let $wait_condition = SELECT state = "debug_sync_point: data_prepare"
+ FROM INFORMATION_SCHEMA.PROCESSLIST
+ WHERE info LIKE "backup database %";
+--source include/wait_condition.inc
+
+use mysqltest;
+insert into t1 values ("text");
+let $1=16;
+while ($1)
+{
+ update t1 set a=concat(a,a);
+ dec $1;
+}
+select length(a) from t1;
+checksum table t1;
+
+connection syncer;
+SELECT release_lock("data_prepare");
+
+connection backup;
+--replace_column 1 #
+reap;
+repair table t1 quick;
+DROP DATABASE mysqltest;
+
+--replace_column 1 #
+RESTORE FROM 'test.ba';
+
+select length(a) from t1;
+checksum table t1;
+
+#
+# Cleanup from this test case
+#
+drop database mysqltest;
+--remove_file $MYSQLTEST_VARDIR/master-data/test.ba
+
+SET SESSION debug="-d,query,enter,info,query,backup_debug";
diff -Nrup a/mysql-test/t/backup_no_be.test b/mysql-test/t/backup_no_be.test
--- a/mysql-test/t/backup_no_be.test 2008-04-16 20:23:01 +02:00
+++ b/mysql-test/t/backup_no_be.test 2008-05-09 12:27:19 +02:00
@@ -1,4 +1,5 @@
--source include/have_debug.inc
+--source include/not_embedded.inc
#
# This test script tests the behaviour reported in BUG#34721
@@ -66,7 +67,7 @@ SELECT @@debug;
--remove_file $MYSQLTEST_VARDIR/master-data/db1.bak
--replace_column 1 #
BACKUP DATABASE db1 TO 'db1.bak';
---enable_warings
+--enable_warnings
# Don't show warning message code
--replace_column 2 #
diff -Nrup a/mysql-test/t/key_cache.test b/mysql-test/t/key_cache.test
--- a/mysql-test/t/key_cache.test 2008-03-27 17:43:16 +01:00
+++ b/mysql-test/t/key_cache.test 2008-05-09 12:27:19 +02:00
@@ -71,7 +71,7 @@ show status like 'key_blocks_used';
# Following results differs on 64 and 32 bit systems because of different
# pointer sizes, which takes up different amount of space in key cache
---replace_result 1812 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED
1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
+--replace_result 1805 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1663 KEY_BLOCKS_UNUSED
1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
insert into t1 values (1, 'qqqq'), (11, 'yyyy');
@@ -84,7 +84,7 @@ update t1 set p=2 where p=1;
update t2 set i=2 where i=1;
show status like 'key_blocks_used';
---replace_result 1808 KEY_BLOCKS_UNUSED 1789 KEY_BLOCKS_UNUSED 1670 KEY_BLOCKS_UNUSED
1814 KEY_BLOCKS_UNUSED 1820 KEY_BLOCKS_UNUSED
+--replace_result 1801 KEY_BLOCKS_UNUSED 1789 KEY_BLOCKS_UNUSED 1659 KEY_BLOCKS_UNUSED
1814 KEY_BLOCKS_UNUSED 1820 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
cache index t1 key (`primary`) in keycache1;
@@ -146,7 +146,7 @@ cache index t1,t2 in default;
drop table t1,t2,t3;
show status like 'key_blocks_used';
---replace_result 1812 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1674 KEY_BLOCKS_UNUSED
1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
+--replace_result 1805 KEY_BLOCKS_UNUSED 1793 KEY_BLOCKS_UNUSED 1663 KEY_BLOCKS_UNUSED
1818 KEY_BLOCKS_UNUSED 1824 KEY_BLOCKS_UNUSED
show status like 'key_blocks_unused';
diff -Nrup a/mysql-test/t/myisamlog-master.opt b/mysql-test/t/myisamlog-master.opt
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/myisamlog-master.opt 2008-05-09 12:27:20 +02:00
@@ -0,0 +1 @@
+--log-isam
diff -Nrup a/mysql-test/t/myisamlog.test b/mysql-test/t/myisamlog.test
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/myisamlog.test 2008-05-09 12:27:20 +02:00
@@ -0,0 +1,39 @@
+# Test of MyISAM logical logging and the myisamlog utility:
+# see if they can repopulate a table.
+
+--source include/not_embedded.inc
+
+disable_warnings;
+create database if not exists mysqltest;
+use mysqltest;
+drop table if exists t1;
+enable_warnings;
+# CREATE generates no entry in the logical log
+create table t1 (a int, b varchar(100)) engine=myisam;
+
+# we put some data into the table
+insert into t1 values(1,'life'), (2,'file');
+insert into t1 select a*5, concat("A ",b) from t1;
+update t1 set b="A knife" where a=5;
+delete from t1 where a=10;
+select * from t1;
+
+# wipe it out (TRUNCATE generates no entry in the logical log)
+truncate table t1;
+# close the table (as we are going to change it out of the server process)
+flush table t1;
+
+# look at log
+exec $MYISAMLOG $MYSQLTEST_VARDIR/master-data/myisam.log ;
+# should not have changed the table
+select * from t1;
+flush table t1;
+
+# apply log to empty table
+--replace_result $MYSQLTEST_VARDIR VARDIR
+exec $MYISAMLOG -F $MYSQLTEST_VARDIR/master-data -u
$MYSQLTEST_VARDIR/master-data/myisam.log ;
+
+# reopen the table and verify that content is back
+select * from t1;
+check table t1 extended;
+drop database mysqltest;
diff -Nrup a/mysys/CMakeLists.txt b/mysys/CMakeLists.txt
--- a/mysys/CMakeLists.txt 2007-11-08 00:18:26 +01:00
+++ b/mysys/CMakeLists.txt 2008-05-09 12:27:19 +02:00
@@ -35,7 +35,7 @@ SET(MYSYS_SOURCES array.c charset-def.c
my_clock.c my_compress.c my_conio.c my_copy.c my_crc32.c my_create.c my_delete.c
my_div.c my_error.c my_file.c my_fopen.c my_fstream.c my_gethostbyname.c
my_gethwaddr.c my_getopt.c my_getsystime.c my_getwd.c my_handler.c my_init.c
- my_lib.c my_lock.c my_lockmem.c my_malloc.c my_messnc.c
+ my_lib.c my_lock.c my_lockmem.c my_malloc.c my_messnc.c my_dup.c
my_mkdir.c my_mmap.c my_net.c my_once.c my_open.c my_pread.c my_pthread.c
my_quick.c my_read.c my_realloc.c my_redel.c my_rename.c my_seek.c my_sleep.c
my_static.c my_symlink.c my_symlink2.c my_sync.c my_thr_init.c my_wincond.c
diff -Nrup a/mysys/mf_iocache.c b/mysys/mf_iocache.c
--- a/mysys/mf_iocache.c 2007-10-11 17:07:33 +02:00
+++ b/mysys/mf_iocache.c 2008-05-09 12:27:19 +02:00
@@ -129,6 +129,9 @@ init_functions(IO_CACHE* info)
}
+/* FUNCTIONS TO SET UP OR RESET A CACHE */
+
+
/*
Initialize an IO_CACHE object
@@ -166,7 +169,7 @@ int init_io_cache(IO_CACHE *info, File f
info->file= file;
info->type= TYPE_NOT_SET; /* Don't set it until mutex are created */
info->pos_in_file= seek_offset;
- info->pre_close = info->pre_read = info->post_read = 0;
+ info->pre_close= info->pre_read= info->post_read= info->post_write= NULL;
info->arg = 0;
info->alloced_buffer = 0;
info->buffer=0;
@@ -278,7 +281,7 @@ int init_io_cache(IO_CACHE *info, File f
/* End_of_file may be changed by user later */
info->end_of_file= end_of_file;
- info->error=0;
+ info->error= info->hard_write_error_in_the_past= 0;
info->type= type;
init_functions(info);
#ifdef HAVE_AIOWAIT
@@ -405,7 +408,7 @@ my_bool reinit_io_cache(IO_CACHE *info,
}
}
info->type=type;
- info->error=0;
+ info->error= info->hard_write_error_in_the_past= 0;
init_functions(info);
#ifdef HAVE_AIOWAIT
@@ -422,6 +425,8 @@ my_bool reinit_io_cache(IO_CACHE *info,
} /* reinit_io_cache */
+/* FUNCTIONS TO DO READS FROM THE CACHE */
+
/*
Read buffered.
@@ -1471,14 +1476,19 @@ int _my_b_get(IO_CACHE *info)
uchar buff;
IO_CACHE_CALLBACK pre_read,post_read;
if ((pre_read = info->pre_read))
- (*pre_read)(info);
+ (*pre_read)(info, NULL, 0, 0);
if ((*(info)->read_function)(info,&buff,1))
return my_b_EOF;
if ((post_read = info->post_read))
- (*post_read)(info);
+ (*post_read)(info, NULL, 0, 0);
return (int) (uchar) buff;
}
+/* FUNCTIONS TO DO WRITES TO THE CACHE */
+
+#define set_hard_write_error(CACHE) \
+ ((CACHE)->error= (CACHE)->hard_write_error_in_the_past= -1)
+
/*
Write a byte buffer to IO_CACHE and flush to disk
if IO_CACHE is full.
@@ -1496,7 +1506,7 @@ int _my_b_write(register IO_CACHE *info,
if (info->pos_in_file+info->buffer_length > info->end_of_file)
{
my_errno=errno=EFBIG;
- return info->error = -1;
+ return set_hard_write_error(info);
}
rest_length= (size_t) (info->write_end - info->write_pos);
@@ -1520,13 +1530,15 @@ int _my_b_write(register IO_CACHE *info,
*/
if (my_seek(info->file,info->pos_in_file,MY_SEEK_SET,MYF(0)))
{
- info->error= -1;
+ set_hard_write_error(info);
return (1);
}
info->seek_not_done=0;
}
if (my_write(info->file, Buffer, length, info->myflags | MY_NABP))
- return info->error= -1;
+ return set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, length, info->pos_in_file);
#ifdef THREAD
/*
@@ -1571,7 +1583,7 @@ int my_b_append(register IO_CACHE *info,
*/
DBUG_ASSERT(!info->share);
#endif
-
+ DBUG_ASSERT(info->post_write == NULL); /* unsupported */
lock_append_buffer(info);
rest_length= (size_t) (info->write_end - info->write_pos);
if (Count <= rest_length)
@@ -1591,7 +1603,7 @@ int my_b_append(register IO_CACHE *info,
if (my_write(info->file,Buffer, length, info->myflags | MY_NABP))
{
unlock_append_buffer(info);
- return info->error= -1;
+ return set_hard_write_error(info);
}
Count-=length;
Buffer+=length;
@@ -1644,12 +1656,21 @@ int my_block_write(register IO_CACHE *in
{
/* Of no overlap, write everything without buffering */
if (pos + Count <= info->pos_in_file)
- return my_pwrite(info->file, Buffer, Count, pos,
- info->myflags | MY_NABP);
+ {
+ int ret= my_pwrite(info->file, Buffer, Count, pos,
+ info->myflags | MY_NABP);
+ if (unlikely(ret))
+ set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, Count, pos);
+ return ret;
+ }
/* Write the part of the block that is before buffer */
length= (uint) (info->pos_in_file - pos);
if (my_pwrite(info->file, Buffer, length, pos, info->myflags | MY_NABP))
- info->error= error= -1;
+ error= set_hard_write_error(info);
+ if (info->post_write)
+ (*(info->post_write))(info, Buffer, length, pos);
Buffer+=length;
pos+= length;
Count-= length;
@@ -1711,7 +1732,7 @@ int my_b_flush_io_cache(IO_CACHE *info,
if (info->file == -1)
{
if (real_open_cached_file(info))
- DBUG_RETURN((info->error= -1));
+ DBUG_RETURN(set_hard_write_error(info));
}
LOCK_APPEND_BUFFER;
@@ -1739,29 +1760,42 @@ int my_b_flush_io_cache(IO_CACHE *info,
MY_FILEPOS_ERROR)
{
UNLOCK_APPEND_BUFFER;
- DBUG_RETURN((info->error= -1));
+ DBUG_RETURN(set_hard_write_error(info));
}
if (!append_cache)
info->seek_not_done=0;
}
- if (!append_cache)
- info->pos_in_file+=length;
info->write_end= (info->write_buffer+info->buffer_length-
((pos_in_file+length) & (IO_SIZE-1)));
if (my_write(info->file,info->write_buffer,length,
info->myflags | MY_NABP))
- info->error= -1;
+ set_hard_write_error(info);
else
info->error= 0;
if (!append_cache)
{
+ /*
+ This post_write is really POST-write; callers depend on this! So
+ always call it after writing to the file, not before.
+ */
+ if (info->post_write)
+ (*(info->post_write))(info, info->write_buffer,
+ length, info->pos_in_file);
+ /*
+ The addition below will make the info->pos_in_file be the end of
+ written block; whereas the value we needed in post_write is the
+ value before the addition. That's why we called post_write before
+ this.
+ */
+ info->pos_in_file+=length;
set_if_bigger(info->end_of_file,(pos_in_file+length));
}
else
{
info->end_of_file+=(info->write_pos-info->append_read_pos);
DBUG_ASSERT(info->end_of_file == my_tell(info->file,MYF(0)));
+ DBUG_ASSERT(info->post_write == NULL); /* unsupported */
}
info->append_read_pos=info->write_pos=info->write_buffer;
@@ -1815,7 +1849,7 @@ int end_io_cache(IO_CACHE *info)
if ((pre_close=info->pre_close))
{
- (*pre_close)(info);
+ (*pre_close)(info, NULL, 0, 0);
info->pre_close= 0;
}
if (info->alloced_buffer)
diff -Nrup a/mysys/mf_keycache.c b/mysys/mf_keycache.c
--- a/mysys/mf_keycache.c 2008-04-01 15:44:51 +02:00
+++ b/mysys/mf_keycache.c 2008-05-09 12:27:19 +02:00
@@ -13,7 +13,8 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
+/**
+ @file
These functions handle keyblock cacheing for ISAM and MyISAM tables.
One cache can handle many files.
@@ -36,7 +37,9 @@
blocks_unused is the sum of never used blocks in the pool and of currently
free blocks. blocks_used is the number of blocks fetched from the pool and
as such gives the maximum number of in-use blocks at any time.
+*/
+/*
Key Cache Locking
=================
@@ -213,6 +216,7 @@ struct st_block_link
uint hits_left; /* number of hits left until promotion */
ulonglong last_hit_time; /* timestamp of the last hit */
KEYCACHE_CONDVAR *condvar; /* condition variable for 'no readers' event */
+ void *post_write_arg; /**< post_write's argument*/
};
KEY_CACHE dflt_key_cache_var;
@@ -401,6 +405,7 @@ int init_key_cache(KEY_CACHE *keycache,
keycache->in_init= 0;
pthread_mutex_init(&keycache->cache_lock, MY_MUTEX_INIT_FAST);
keycache->resize_queue.last_thread= NULL;
+ keycache->post_write= NULL;
}
keycache->key_cache_mem_size= use_mem;
@@ -764,12 +769,37 @@ void end_key_cache(KEY_CACHE *keycache,
{
pthread_mutex_destroy(&keycache->cache_lock);
keycache->key_cache_inited= keycache->can_be_used= 0;
+ keycache->post_write= NULL;
KEYCACHE_DEBUG_CLOSE;
}
DBUG_VOID_RETURN;
} /* end_key_cache */
+/**
+ Does a my_pwrite() to the file and then calls callback. Arguments are those
+ of my_pwrite() plus the callback and its argument.
+
+ @note The callback is really POST-write; callers depend on this! So always
+ call it after writing to the file, not before.
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 write or callback failed
+*/
+
+static inline int key_cache_pwrite(int Filedes, const uchar *Buffer,
+ uint Count, my_off_t offset, myf MyFlags,
+ KEYCACHE_POST_WRITE_CALLBACK callback,
+ void *callback_arg)
+{
+ int ret= my_pwrite(Filedes, Buffer, Count, offset, MyFlags);
+ if (callback)
+ ret|= (*callback)(callback_arg, Buffer, Count, offset);
+ return ret;
+}
+
+
#ifdef THREAD
/*
@@ -2207,11 +2237,14 @@ restart:
The call is thread safe because only the current
thread might change the block->hash_link value
*/
- error= my_pwrite(block->hash_link->file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
+ error= key_cache_pwrite(block->hash_link->file,
+ block->buffer + block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos +
+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write,
+ block->post_write_arg);
keycache_pthread_mutex_lock(&keycache->cache_lock);
/* Block status must not have changed. */
@@ -2954,6 +2987,8 @@ int key_cache_insert(KEY_CACHE *keycache
length length of the buffer
dont_write if is 0 then all dirty pages involved in writing
should have been flushed from key cache
+ post_write_arg argument which will be passed to key cache's
+ post_write callback
RETURN VALUE
0 if a success, 1 - otherwise.
@@ -2973,7 +3008,8 @@ int key_cache_write(KEY_CACHE *keycache,
File file, my_off_t filepos, int level,
uchar *buff, uint length,
uint block_length __attribute__((unused)),
- int dont_write)
+ int dont_write,
+ void *post_write_arg)
{
my_bool locked_and_incremented= FALSE;
int error=0;
@@ -2991,7 +3027,9 @@ int key_cache_write(KEY_CACHE *keycache,
/* Force writing from buff into disk. */
keycache->global_cache_w_requests++;
keycache->global_cache_write++;
- if (my_pwrite(file, buff, length, filepos, MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, buff, length, filepos,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, post_write_arg))
DBUG_RETURN(1);
/* purecov: end */
}
@@ -3066,8 +3104,10 @@ int key_cache_write(KEY_CACHE *keycache,
/* Used in the server. */
keycache->global_cache_write++;
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- if (my_pwrite(file, (uchar*) buff, read_length, filepos + offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, (uchar*) buff, read_length,
+ filepos + offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, post_write_arg))
error=1;
keycache_pthread_mutex_lock(&keycache->cache_lock);
}
@@ -3159,6 +3199,7 @@ int key_cache_write(KEY_CACHE *keycache,
*/
if (!(block->status & BLOCK_ERROR))
{
+ block->post_write_arg= post_write_arg;
#if !defined(SERIALIZED_READ_FROM_CACHE)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
#endif
@@ -3234,8 +3275,9 @@ no_key_cache:
keycache->global_cache_write++;
if (locked_and_incremented)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- if (my_pwrite(file, (uchar*) buff, length, filepos,
- MYF(MY_NABP | MY_WAIT_IF_FULL)))
+ if (key_cache_pwrite(file, (uchar*) buff, length, filepos,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, post_write_arg))
error=1;
if (locked_and_incremented)
keycache_pthread_mutex_lock(&keycache->cache_lock);
@@ -3461,11 +3503,12 @@ static int flush_cached_blocks(KEY_CACHE
(BLOCK_READ | BLOCK_IN_FLUSH | BLOCK_CHANGED | BLOCK_IN_USE));
block->status|= BLOCK_IN_FLUSHWRITE;
keycache_pthread_mutex_unlock(&keycache->cache_lock);
- error= my_pwrite(file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
+ error= key_cache_pwrite(file,
+ block->buffer+block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL),
+ keycache->post_write, block->post_write_arg);
keycache_pthread_mutex_lock(&keycache->cache_lock);
keycache->global_cache_write++;
if (error)
diff -Nrup a/mysys/my_thr_init.c b/mysys/my_thr_init.c
--- a/mysys/my_thr_init.c 2008-04-01 11:20:59 +02:00
+++ b/mysys/my_thr_init.c 2008-05-09 12:27:19 +02:00
@@ -13,8 +13,9 @@
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
-/*
- Functions to handle initializating and allocationg of all mysys & debug
+/**
+ @file
+ Functions for initialization and allocation of all mysys & debug
thread variables.
*/
@@ -29,8 +30,12 @@ pthread_key(struct st_my_thread_var*, TH
pthread_key(struct st_my_thread_var, THR_KEY_mysys);
#endif /* USE_TLS */
pthread_mutex_t THR_LOCK_malloc,THR_LOCK_open,
- THR_LOCK_lock,THR_LOCK_isam,THR_LOCK_myisam,THR_LOCK_heap,
- THR_LOCK_net, THR_LOCK_charset, THR_LOCK_threads, THR_LOCK_time;
+ THR_LOCK_lock,THR_LOCK_isam,THR_LOCK_heap, THR_LOCK_net,
+ THR_LOCK_charset, THR_LOCK_threads, THR_LOCK_time;
+/** For insert/delete in the list of MyISAM open tables */
+pthread_mutex_t THR_LOCK_myisam;
+/** For writing to the MyISAM logs */
+pthread_mutex_t THR_LOCK_myisam_log;
pthread_cond_t THR_COND_threads;
uint THR_thread_count= 0;
uint my_thread_end_wait_time= 5;
@@ -143,6 +148,7 @@ my_bool my_thread_global_init(void)
pthread_mutex_init(&THR_LOCK_lock,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_isam,MY_MUTEX_INIT_SLOW);
pthread_mutex_init(&THR_LOCK_myisam,MY_MUTEX_INIT_SLOW);
+ pthread_mutex_init(&THR_LOCK_myisam_log,MY_MUTEX_INIT_SLOW);
pthread_mutex_init(&THR_LOCK_heap,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_net,MY_MUTEX_INIT_FAST);
pthread_mutex_init(&THR_LOCK_charset,MY_MUTEX_INIT_FAST);
@@ -209,6 +215,7 @@ void my_thread_global_end(void)
pthread_mutex_destroy(&THR_LOCK_lock);
pthread_mutex_destroy(&THR_LOCK_isam);
pthread_mutex_destroy(&THR_LOCK_myisam);
+ pthread_mutex_destroy(&THR_LOCK_myisam_log);
pthread_mutex_destroy(&THR_LOCK_heap);
pthread_mutex_destroy(&THR_LOCK_net);
pthread_mutex_destroy(&THR_LOCK_time);
@@ -332,6 +339,10 @@ void my_thread_end(void)
/* tmp->dbug is allocated inside DBUG library */
if (tmp->dbug)
{
+ /*
+ Frees memory allocated in SET DEBUG=...; does nothing if there were no
+ SET DEBUG in a thread.
+ */
DBUG_POP();
free(tmp->dbug);
tmp->dbug=0;
diff -Nrup a/sql/backup/stream_v1_transport.c b/sql/backup/stream_v1_transport.c
--- a/sql/backup/stream_v1_transport.c 2007-12-03 21:28:17 +01:00
+++ b/sql/backup/stream_v1_transport.c 2008-05-09 12:27:19 +02:00
@@ -30,7 +30,7 @@
@note Block size should be >= 8 bytes.
*/
-#define DEFAULT_BLOCK_SIZE (32*1024)
+#define DEFAULT_BLOCK_SIZE (16*1024)
/**
@brief Input buffer size.
@@ -1480,7 +1480,8 @@ int bstream_read_part(backup_stream *s,
saved= *data;
data->end= data->begin + howmuch;
- as_read(&s->stream,data,buf);
+ if (as_read(&s->stream,data,buf) == BSTREAM_EOS)
+ s->state= EOS;
s->buf.begin += data->begin - saved.begin;
s->buf.pos= s->buf.begin;
diff -Nrup a/sql/item_func.cc b/sql/item_func.cc
--- a/sql/item_func.cc 2008-04-29 11:27:22 +02:00
+++ b/sql/item_func.cc 2008-05-09 12:27:19 +02:00
@@ -3375,21 +3375,35 @@ longlong Item_master_pos_wait::val_int()
}
#ifdef EXTRA_DEBUG
+/**
+ When a connection con1 does a GET_LOCK() to synchronize with a another
+ connection con2 doing debug_sync_point(), con1 should not run any statements
+ after that until RELEASE_LOCK(). I.e.: con1 should dedicate itself to
+ synchronization only.
+ */
void debug_sync_point(const char* lock_name, uint lock_timeout)
{
THD* thd=current_thd;
User_level_lock* ull;
struct timespec abstime;
size_t lock_name_len;
+ DBUG_ENTER("debug_sync_point");
+ DBUG_PRINT("enter", ("lock_name: '%s'", lock_name));
lock_name_len= strlen(lock_name);
- pthread_mutex_lock(&LOCK_user_locks);
- if (thd->ull)
- {
- item_user_lock_release(thd->ull);
- thd->ull=0;
- }
+ /*
+ If thread already has a lock:
+ - we cannot automatically release it (it would likely cause
+ synchronization problems in tests, the test writer probably didn't want
+ this implicit release, as it cannot have inspected all code files to see
+ if one of them calls debug_sync_point())
+ - we cannot keep it either, because general user-level lock management
+ relies on one lock max per thread (THD::ull is not a list, GET_LOCK()
+ does an implicit release), and overwriting thd->ull is not ok.
+ */
+ DBUG_ASSERT(thd->ull == NULL);
+ pthread_mutex_lock(&LOCK_user_locks);
/*
If the lock has not been aquired by some client, we do not want to
create an entry for it, since we immediately release the lock. In
@@ -3401,7 +3415,7 @@ void debug_sync_point(const char* lock_n
lock_name_len))))
{
pthread_mutex_unlock(&LOCK_user_locks);
- return;
+ DBUG_VOID_RETURN;
}
ull->count++;
@@ -3453,6 +3467,7 @@ void debug_sync_point(const char* lock_n
thd->ull=0;
}
pthread_mutex_unlock(&LOCK_user_locks);
+ DBUG_VOID_RETURN;
}
#endif
diff -Nrup a/sql/mysqld.cc b/sql/mysqld.cc
--- a/sql/mysqld.cc 2008-05-05 21:37:26 +02:00
+++ b/sql/mysqld.cc 2008-05-09 12:27:19 +02:00
@@ -711,7 +711,7 @@ static bool kill_in_progress, segfaulted
#ifdef HAVE_STACK_TRACE_ON_SEGV
static my_bool opt_do_pstack;
#endif /* HAVE_STACK_TRACE_ON_SEGV */
-static my_bool opt_bootstrap, opt_myisam_log;
+static my_bool opt_bootstrap, opt_myisam_logical_log;
static int cleanup_done;
static ulong opt_specialflag, opt_myisam_block_size;
static char *opt_update_logname, *opt_binlog_index_name;
@@ -4155,8 +4155,8 @@ server.");
pthread_attr_setstacksize(&connection_attrib, NW_THD_STACKSIZE);
#endif
- if (opt_myisam_log)
- (void) mi_log(1);
+ if (opt_myisam_logical_log)
+ (void) mi_log(MI_LOG_ACTION_OPEN, MI_LOG_LOGICAL, NULL, NULL);
#if defined(HAVE_MLOCKALL) && defined(MCL_CURRENT) &&
!defined(EMBEDDED_LIBRARY)
if (locked_in_memory && !getuid())
@@ -5963,8 +5963,8 @@ Disable with --skip-large-pages.",
(uchar**) &log_error_file_ptr, (uchar**) &log_error_file_ptr, 0, GET_STR,
OPT_ARG, 0, 0, 0, 0, 0, 0},
{"log-isam", OPT_ISAM_LOG, "Log all MyISAM changes to file.",
- (uchar**) &myisam_log_filename, (uchar**) &myisam_log_filename, 0, GET_STR,
- OPT_ARG, 0, 0, 0, 0, 0, 0},
+ (uchar**) &myisam_logical_log_filename, (uchar**)
+ &myisam_logical_log_filename, 0, GET_STR, OPT_ARG, 0, 0, 0, 0, 0, 0},
{"log-long-format", '0',
"Log some extra information to update log. Please note that this option is deprecated;
see --log-short-format option.",
0, 0, 0, GET_NO_ARG, NO_ARG, 0, 0, 0, 0, 0, 0},
@@ -7598,7 +7598,7 @@ static void mysql_init_variables(void)
opt_tc_log_file= (char *)"tc.log"; // no hostname in tc_log file name !
opt_secure_auth= 0;
opt_secure_file_priv= 0;
- opt_bootstrap= opt_myisam_log= 0;
+ opt_bootstrap= opt_myisam_logical_log= 0;
mqh_used= 0;
segfaulted= kill_in_progress= 0;
cleanup_done= 0;
@@ -7885,7 +7885,7 @@ mysqld_get_one_option(int optid,
thd_startup_options|=OPTION_BIG_TABLES;
break;
case (int) OPT_ISAM_LOG:
- opt_myisam_log=1;
+ opt_myisam_logical_log=1;
break;
case (int) OPT_UPDATE_LOG:
opt_update_log=1;
diff -Nrup a/sql/sql_bitmap.h b/sql/sql_bitmap.h
--- a/sql/sql_bitmap.h 2008-03-27 18:44:47 +01:00
+++ b/sql/sql_bitmap.h 2008-05-09 12:27:19 +02:00
@@ -156,7 +156,7 @@ template <> class Bitmap<64>
{
ulonglong map;
public:
- Bitmap<64>() { map= 0; }
+ Bitmap<64>() { init(); }
#if defined(__NETWARE__) || defined(__MWERKS__)
/*
Metwork compiler gives error on Bitmap<64>
@@ -167,7 +167,7 @@ public:
#else
explicit Bitmap<64>(uint prefix_to_set) { set_prefix(prefix_to_set); }
#endif
- void init() { }
+ void init() { clear_all(); }
void init(uint prefix_to_set) { set_prefix(prefix_to_set); }
uint length() const { return 64; }
void set_bit(uint n) { map|= ((ulonglong)1) << n; }
diff -Nrup a/sql/sql_cache.cc b/sql/sql_cache.cc
--- a/sql/sql_cache.cc 2008-04-14 12:10:00 +02:00
+++ b/sql/sql_cache.cc 2008-05-09 12:27:19 +02:00
@@ -2705,8 +2705,9 @@ Query_cache::register_tables_from_list(T
{
char key[MAX_DBKEY_LENGTH];
uint32 db_length;
- uint key_length= filename_2_table_key(key, table->table->filename,
- &db_length);
+ uint key_length=
+ filename_2_table_key(key, table->table->s->unresolv_file_name,
+ &db_length);
(++block_table)->n= ++n;
/*
There are not callback function for for MyISAM, and engine data
diff -Nrup a/sql/sql_class.cc b/sql/sql_class.cc
--- a/sql/sql_class.cc 2008-04-29 11:30:03 +02:00
+++ b/sql/sql_class.cc 2008-05-09 12:27:19 +02:00
@@ -1034,8 +1034,9 @@ void THD::awake(THD::killed_state state_
if (mysys_var)
{
pthread_mutex_lock(&mysys_var->mutex);
- if (!system_thread) // Don't abort locks
- mysys_var->abort=1;
+ if (system_thread == NON_SYSTEM_THREAD ||
+ system_thread == SYSTEM_THREAD_BACKUP)
+ mysys_var->abort= 1; // abort locks
/*
This broadcast could be up in the air if the victim thread
exits the cond in the time between read and broadcast, but that is
diff -Nrup a/sql/sql_load.cc b/sql/sql_load.cc
--- a/sql/sql_load.cc 2008-04-14 12:10:02 +02:00
+++ b/sql/sql_load.cc 2008-05-09 12:27:19 +02:00
@@ -1170,8 +1170,7 @@ READ_INFO::READ_INFO(File file_par, uint
cache.read_function = _my_b_net_read;
if (mysql_bin_log.is_open())
- cache.pre_read = cache.pre_close =
- (IO_CACHE_CALLBACK) log_loaded_block;
+ cache.pre_read= cache.pre_close = log_loaded_block;
#endif
}
}
diff -Nrup a/sql/sql_repl.cc b/sql/sql_repl.cc
--- a/sql/sql_repl.cc 2008-03-08 11:14:45 +01:00
+++ b/sql/sql_repl.cc 2008-05-09 12:27:19 +02:00
@@ -1714,7 +1714,10 @@ err:
@param file pointer to io-cache
@return 0
*/
-int log_loaded_block(IO_CACHE* file)
+int log_loaded_block(IO_CACHE* file,
+ const uchar *buffert __attribute__((unused)),
+ uint count __attribute__((unused)),
+ my_off_t filepos __attribute__((unused)))
{
DBUG_ENTER("log_loaded_block");
LOAD_FILE_INFO *lf_info;
diff -Nrup a/sql/sql_repl.h b/sql/sql_repl.h
--- a/sql/sql_repl.h 2008-03-14 18:38:48 +01:00
+++ b/sql/sql_repl.h 2008-05-09 12:27:19 +02:00
@@ -60,7 +60,7 @@ typedef struct st_load_file_info
bool wrote_create_file, log_delayed;
} LOAD_FILE_INFO;
-int log_loaded_block(IO_CACHE* file);
+int log_loaded_block(IO_CACHE *, const uchar *, uint, my_off_t);
int init_replication_sys_vars();
#endif /* HAVE_REPLICATION */
diff -Nrup a/sql/sql_table.cc b/sql/sql_table.cc
--- a/sql/sql_table.cc 2008-04-29 11:27:24 +02:00
+++ b/sql/sql_table.cc 2008-05-09 12:27:19 +02:00
@@ -7245,7 +7245,10 @@ copy_data_between_tables(TABLE *from,TAB
(SQL_SELECT *) 0, HA_POS_ERROR,
1, &examined_rows)) ==
HA_POS_ERROR)
+ {
+ to->file->ha_end_bulk_insert();
goto err;
+ }
}
};
diff -Nrup a/storage/myisam/CMakeLists.txt b/storage/myisam/CMakeLists.txt
--- a/storage/myisam/CMakeLists.txt 2008-02-22 15:51:22 +01:00
+++ b/storage/myisam/CMakeLists.txt 2008-05-09 12:27:19 +02:00
@@ -33,7 +33,7 @@ SET(MYISAM_SOURCES ft_boolean_search.c
mi_rsame.c mi_rsamepos.c mi_scan.c mi_search.c mi_static.c mi_statrec.c
mi_unique.c mi_update.c mi_write.c rt_index.c rt_key.c rt_mbr.c
rt_split.c sort.c sp_key.c ft_eval.h mi_extrafunc.h myisamdef.h
- rt_index.h mi_rkey.c)
+ rt_index.h mi_rkey.c mi_examine_log.c myisam_backup_engine.cc)
IF(NOT SOURCE_SUBLIBS)
diff -Nrup a/storage/myisam/Makefile.am b/storage/myisam/Makefile.am
--- a/storage/myisam/Makefile.am 2007-10-12 18:03:01 +02:00
+++ b/storage/myisam/Makefile.am 2008-05-09 12:27:19 +02:00
@@ -98,7 +98,7 @@ libmyisam_a_SOURCES = mi_open.c mi_extra
mi_keycache.c mi_preload.c \
ft_parser.c ft_stopwords.c ft_static.c \
ft_update.c ft_boolean_search.c ft_nlq_search.c sort.c \
- ha_myisam.cc \
+ mi_examine_log.c ha_myisam.cc myisam_backup_engine.cc \
rt_index.c rt_key.c rt_mbr.c rt_split.c sp_key.c
CLEANFILES = test?.MY? FT?.MY? isam.log mi_test_all rt_test.MY? sp_test.MY?
diff -Nrup a/storage/myisam/ha_myisam.cc b/storage/myisam/ha_myisam.cc
--- a/storage/myisam/ha_myisam.cc 2008-04-14 12:10:05 +02:00
+++ b/storage/myisam/ha_myisam.cc 2008-05-09 12:27:19 +02:00
@@ -775,7 +775,8 @@ int ha_myisam::check(THD* thd, HA_CHECK_
file->update|=HA_STATE_CHANGED | HA_STATE_ROW_CHANGED;
pthread_mutex_lock(&share->intern_lock);
share->state.changed&= ~(STATE_CHANGED | STATE_CRASHED |
- STATE_CRASHED_ON_REPAIR);
+ STATE_CRASHED_ON_REPAIR |
+ STATE_BAD_OPEN_COUNT);
if (!(table->db_stat & HA_READ_ONLY))
error=update_state_info(¶m,file,UPDATE_TIME | UPDATE_OPEN_COUNT |
UPDATE_STAT);
@@ -939,7 +940,7 @@ int ha_myisam::repair(THD *thd, MI_CHECK
param.thd= thd;
param.tmpdir= &mysql_tmpdir_list;
param.out_flag= 0;
- strmov(fixed_name,file->filename);
+ strmov(fixed_name,file->s->unresolv_file_name);
// Don't lock tables if we have used LOCK TABLE
if (!thd->locked_tables &&
@@ -1019,7 +1020,8 @@ int ha_myisam::repair(THD *thd, MI_CHECK
if ((share->state.changed & STATE_CHANGED) || mi_is_crashed(file))
{
share->state.changed&= ~(STATE_CHANGED | STATE_CRASHED |
- STATE_CRASHED_ON_REPAIR);
+ STATE_CRASHED_ON_REPAIR |
+ STATE_BAD_OPEN_COUNT);
file->update|=HA_STATE_CHANGED | HA_STATE_ROW_CHANGED;
}
/*
@@ -1686,11 +1688,11 @@ int ha_myisam::info(uint flag)
if table is symlinked (Ie; Real name is not same as generated name)
*/
data_file_name= index_file_name= 0;
- fn_format(name_buff, file->filename, "", MI_NAME_DEXT,
+ fn_format(name_buff, file->s->unresolv_file_name, "", MI_NAME_DEXT,
MY_APPEND_EXT | MY_UNPACK_FILENAME);
if (strcmp(name_buff, misam_info.data_file_name))
data_file_name=misam_info.data_file_name;
- fn_format(name_buff, file->filename, "", MI_NAME_IEXT,
+ fn_format(name_buff, file->s->unresolv_file_name, "", MI_NAME_IEXT,
MY_APPEND_EXT | MY_UNPACK_FILENAME);
if (strcmp(name_buff, misam_info.index_file_name))
index_file_name=misam_info.index_file_name;
@@ -1977,6 +1979,9 @@ static int myisam_init(void *p)
myisam_hton->create= myisam_create_handler;
myisam_hton->panic= myisam_panic;
myisam_hton->flags= HTON_CAN_RECREATE | HTON_SUPPORT_LOG_TABLES;
+#if !defined(EMBEDDED_LIBRARY) && defined(HAVE_MYISAM_PHYSICAL_LOGGING)
+ myisam_hton->get_backup_engine= myisam_backup_engine;
+#endif
return 0;
}
@@ -2064,8 +2069,8 @@ mysql_declare_plugin_end;
@brief Register a named table with a call back function to the query cache.
@param thd The thread handle
- @param table_key A pointer to the table name in the table cache
- @param key_length The length of the table name
+ @param table_name A pointer to the table name in the table cache
+ @param table_name_len The length of the table name
@param[out] engine_callback The pointer to the storage engine call back
function, currently 0
@param[out] engine_data Engine data will be set to 0.
diff -Nrup a/storage/myisam/ha_myisam.h b/storage/myisam/ha_myisam.h
--- a/storage/myisam/ha_myisam.h 2008-01-10 23:04:54 +01:00
+++ b/storage/myisam/ha_myisam.h 2008-05-09 12:27:19 +02:00
@@ -170,3 +170,7 @@ private:
friend my_bool index_cond_func_myisam(void *arg);
};
+#if !defined(EMBEDDED_LIBRARY) && defined(HAVE_MYISAM_PHYSICAL_LOGGING)
+// If embedded, there is no online backup
+Backup_result_t myisam_backup_engine(handlerton *self, Backup_engine* &be);
+#endif
diff -Nrup a/storage/myisam/mi_check.c b/storage/myisam/mi_check.c
--- a/storage/myisam/mi_check.c 2008-04-01 15:44:56 +02:00
+++ b/storage/myisam/mi_check.c 2008-05-09 12:27:19 +02:00
@@ -85,6 +85,7 @@ static SORT_KEY_BLOCKS *alloc_key_blocks
uint buffer_length);
static ha_checksum mi_byte_checksum(const uchar *buf, uint length);
static void set_data_file_type(SORT_INFO *sort_info, MYISAM_SHARE *share);
+static int chsize_kfile(MI_INFO *info);
void myisamchk_init(MI_CHECK *param)
{
@@ -1667,7 +1668,7 @@ int mi_repair(MI_CHECK *param, register
{
VOID(fputs(" \r",stdout)); VOID(fflush(stdout));
}
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
{
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d",
@@ -1732,6 +1733,11 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ /*
+ File change like this is not handled in physical log. filecopy() above
+ is also not handled.
+ */
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -1991,6 +1997,7 @@ int mi_sort_index(MI_CHECK *param, regis
VOID(my_close(share->kfile,MYF(MY_WME)));
share->kfile = -1;
VOID(my_close(new_file,MYF(MY_WME)));
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->index_file_name,MI_NAME_IEXT,INDEX_TMP_EXT,0,
MYF(0)) ||
mi_open_keyfile(share))
@@ -2096,6 +2103,7 @@ static int sort_one_index(MI_CHECK *para
/* Fill block with zero and write it to the new index file */
length=mi_getint(buff);
bzero((uchar*) buff+length,keyinfo->block_length-length);
+ DBUG_ASSERT(!info->s->physical_logging);
if (my_pwrite(new_file,(uchar*) buff,(uint) keyinfo->block_length,
new_page_pos,MYF(MY_NABP | MY_WAIT_IF_FULL)))
{
@@ -2158,7 +2166,7 @@ int lock_file(MI_CHECK *param, File file
} /* lock_file */
- /* Copy a block between two files */
+ /* Copy a block between two files. Not handled by physical logging. */
int filecopy(MI_CHECK *param, File to,File from,my_off_t start,
my_off_t length, const char *type)
@@ -2508,15 +2516,18 @@ int mi_repair_by_sort(MI_CHECK *param, r
skr=share->base.reloc*share->base.min_pack_length;
#endif
if (skr != sort_info.filelength && !info->s->base.raid_type)
+ {
+ DBUG_ASSERT(!share->physical_logging);
if (my_chsize(info->dfile,skr,0,MYF(0)))
mi_check_print_warning(param,
"Can't change size of datafile, error: %d",
my_errno);
+ }
}
if (param->testflag & T_CALC_CHECKSUM)
info->state->checksum=param->glob_crc;
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d",
my_errno);
@@ -2545,6 +2556,7 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -3028,15 +3040,18 @@ int mi_repair_parallel(MI_CHECK *param,
skr=share->base.reloc*share->base.min_pack_length;
#endif
if (skr != sort_info.filelength && !info->s->base.raid_type)
+ {
+ DBUG_ASSERT(!share->physical_logging);
if (my_chsize(info->dfile,skr,0,MYF(0)))
mi_check_print_warning(param,
"Can't change size of datafile, error: %d",
my_errno);
+ }
}
if (param->testflag & T_CALC_CHECKSUM)
info->state->checksum=param->glob_crc;
- if (my_chsize(share->kfile,info->state->key_file_length,0,MYF(0)))
+ if (chsize_kfile(info))
mi_check_print_warning(param,
"Can't change size of indexfile, error: %d", my_errno);
@@ -3077,6 +3092,7 @@ err:
{
my_close(new_file,MYF(0));
info->dfile=new_file= -1;
+ DBUG_ASSERT(!share->physical_logging);
if (change_to_newfile(share->data_file_name,MI_NAME_DEXT,
DATA_TMP_EXT, share->base.raid_chunks,
(param->testflag & T_BACKUP_DATA ?
@@ -4057,9 +4073,16 @@ static int sort_insert_key(MI_SORT_PARAM
if (_mi_write_keypage(info, keyinfo, filepos, DFLT_INIT_HITS, anc_buff))
DBUG_RETURN(1);
}
- else if (my_pwrite(info->s->kfile,(uchar*) anc_buff,
- (uint) keyinfo->block_length,filepos, param->myf_rw))
- DBUG_RETURN(1);
+ else
+ {
+ if (my_pwrite(info->s->kfile, anc_buff,
+ keyinfo->block_length, filepos, param->myf_rw))
+ DBUG_RETURN(1);
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(info->s)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, info->s, anc_buff,
+ keyinfo->block_length, filepos);
+ }
DBUG_DUMP("buff",(uchar*) anc_buff,mi_getint(anc_buff));
/* Write separator-key to block in next level */
@@ -4162,9 +4185,17 @@ int flush_pending_blocks(MI_SORT_PARAM *
DFLT_INIT_HITS, key_block->buff))
DBUG_RETURN(1);
}
- else if (my_pwrite(info->s->kfile,(uchar*) key_block->buff,
- (uint) keyinfo->block_length,filepos, myf_rw))
- DBUG_RETURN(1);
+ else
+ {
+ if (my_pwrite(info->s->kfile, key_block->buff,
+ keyinfo->block_length,filepos, myf_rw))
+ DBUG_RETURN(1);
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(info->s)))
+ myisam_log_pwrite_physical(MI_LOG_WRITE_BYTES_MYI, info->s,
+ key_block->buff, keyinfo->block_length,
+ filepos);
+ }
DBUG_DUMP("buff",(uchar*) key_block->buff,length);
nod_flag=1;
}
@@ -4444,7 +4475,7 @@ int update_state_info(MI_CHECK *param, M
*/
if (info->lock_type == F_WRLCK)
share->state.state= *info->state;
- if (mi_state_info_write(share->kfile,&share->state,1+2))
+ if (mi_state_info_write(share, share->kfile, &share->state, 1+2))
goto err;
share->changed=0;
}
@@ -4727,4 +4758,45 @@ set_data_file_type(SORT_INFO *sort_info,
mi_setup_functions(&tmp);
share->delete_record=tmp.delete_record;
}
+}
+
+/**
+ Changes the size of an index file, and logs the operation to the physical
+ log if needed.
+
+ The only known case when my_chsize(kfile) can happen on a table doing
+ physical logging, is when the table was empty, bulk insert on it has been
+ done, it's the end of bulk insert: we re-enable indices (mi_repair*()): thus
+ my_chsize() is in fact a void operation (file already has grown, starting
+ from empty, info->state->key_file_length is up-to-date and so file already
+ has the requested size). We however log the operation, in case there are
+ unknown cases.
+
+ @param info table
+
+ @return Operation status
+ @retval 0 ok
+ @retval !=0 error
+*/
+
+static int chsize_kfile(MI_INFO *info)
+{
+ MYISAM_SHARE *share= info->s;
+ my_off_t new_length= info->state->key_file_length;
+ int ret;
+#ifndef DBUG_OFF
+ my_bool no_length_change=
+ (my_seek(share->kfile, 0L, MY_SEEK_END, MYF(0)) == new_length);
+#endif
+
+ ret= my_chsize(share->kfile, new_length, 0, MYF(0));
+
+ if (unlikely(mi_log_index_pages_physical &&
+ mi_get_physical_logging_state(share)))
+ {
+ DBUG_ASSERT(no_length_change);
+ myisam_log_chsize_kfile_physical(share, new_length);
+ }
+
+ return ret;
}
diff -Nrup a/storage/myisam/mi_close.c b/storage/myisam/mi_close.c
--- a/storage/myisam/mi_close.c 2007-05-10 11:59:32 +02:00
+++ b/storage/myisam/mi_close.c 2008-05-09 12:27:19 +02:00
@@ -52,6 +52,8 @@ int mi_close(register MI_INFO *info)
}
if (info->opt_flag & (READ_CACHE_USED | WRITE_CACHE_USED))
{
+ /* Logically there should not be a WRITE_CACHE at this stage */
+ DBUG_ASSERT(!(info->opt_flag & WRITE_CACHE_USED));
if (end_io_cache(&info->rec_cache))
error=my_errno;
info->opt_flag&= ~(READ_CACHE_USED | WRITE_CACHE_USED);
@@ -67,7 +69,11 @@ int mi_close(register MI_INFO *info)
flush_key_blocks(share->key_cache, share->kfile,
share->temporary ? FLUSH_IGNORE_CHANGED :
FLUSH_RELEASE))
+ {
error=my_errno;
+ mi_print_error(share, HA_ERR_CRASHED);
+ mi_mark_crashed(info); /* Mark that table must be checked */
+ }
if (share->kfile >= 0)
{
/*
@@ -77,7 +83,10 @@ int mi_close(register MI_INFO *info)
may be using the file at this point
*/
if (share->mode != O_RDONLY && mi_is_crashed(info))
- mi_state_info_write(share->kfile, &share->state, 1);
+ mi_state_info_write(share, share->kfile, &share->state, 1);
+ if (share->MI_LOG_OPEN_stored_in_physical_log)
+ _myisam_log_command(&myisam_physical_log, MI_LOG_CLOSE, share,
+ NULL, 0, error);
if (my_close(share->kfile,MYF(0)))
error = my_errno;
}
@@ -93,6 +102,7 @@ int mi_close(register MI_INFO *info)
#ifdef THREAD
thr_lock_delete(&share->lock);
VOID(pthread_mutex_destroy(&share->intern_lock));
+ my_atomic_rwlock_destroy(&share->physical_logging_rwlock);
{
int i,keys;
keys = share->state.header.keys;
@@ -113,7 +123,7 @@ int mi_close(register MI_INFO *info)
if (info->dfile >= 0 && my_close(info->dfile,MYF(0)))
error = my_errno;
- myisam_log_command(MI_LOG_CLOSE,info,NULL,0,error);
+ myisam_log_command_logical(MI_LOG_CLOSE, info, NULL, 0, error);
my_free((uchar*) info,MYF(0));
if (error)
diff -Nrup a/storage/myisam/mi_create.c b/storage/myisam/mi_create.c
--- a/storage/myisam/mi_create.c 2008-04-14 13:30:05 +02:00
+++ b/storage/myisam/mi_create.c 2008-05-09 12:27:19 +02:00
@@ -637,6 +637,14 @@ int mi_create(const char *name,uint keys
my_errno= HA_ERR_TABLE_EXIST;
goto err;
}
+ /*
+ TRUNCATE TABLE does not work with physical logging. If we changed TRUNCATE
+ to always use mi_delete_all_rows() (remove HTON_CAN_RECREATE from MyISAM)
+ this would solve the problem.
+ */
+ DBUG_ASSERT((options & HA_OPTION_TMP_TABLE) || !mi_log_tables_physical ||
+ !hash_search(mi_log_tables_physical, filename,
+ strlen(filename)));
if ((file= my_create_with_symlink(linkname_ptr, filename, 0, create_mode,
MYF(MY_WME | create_flag))) < 0)
@@ -702,7 +710,7 @@ int mi_create(const char *name,uint keys
}
DBUG_PRINT("info", ("write state info and base info"));
- if (mi_state_info_write(file, &share.state, 2) ||
+ if (mi_state_info_write(&share, file, &share.state, 2) ||
mi_base_info_write(file, &share.base))
goto err;
#ifndef DBUG_OFF
diff -Nrup a/storage/myisam/mi_delete.c b/storage/myisam/mi_delete.c
--- a/storage/myisam/mi_delete.c 2008-04-09 07:41:37 +02:00
+++ b/storage/myisam/mi_delete.c 2008-05-09 12:27:19 +02:00
@@ -100,13 +100,15 @@ int mi_delete(MI_INFO *info,const uchar
info->state->records--;
mi_sizestore(lastpos,info->lastpos);
- myisam_log_command(MI_LOG_DELETE,info,(uchar*) lastpos,sizeof(lastpos),0);
+ myisam_log_command_logical(MI_LOG_DELETE, info,
+ (uchar*) lastpos, sizeof(lastpos), 0);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
allow_break(); /* Allow SIGHUP & SIGINT */
if (info->invalidator != 0)
{
- DBUG_PRINT("info", ("invalidator... '%s' (delete)", info->filename));
- (*info->invalidator)(info->filename);
+ DBUG_PRINT("info", ("invalidator... '%s' (delete)",
+ info->s->unresolv_file_name));
+ (*info->invalidator)(info->s->unresolv_file_name);
info->invalidator=0;
}
DBUG_RETURN(0);
@@ -114,7 +116,8 @@ int mi_delete(MI_INFO *info,const uchar
err:
save_errno=my_errno;
mi_sizestore(lastpos,info->lastpos);
- myisam_log_command(MI_LOG_DELETE,info,(uchar*) lastpos, sizeof(lastpos),0);
+ myisam_log_command_logical(MI_LOG_DELETE, info,
+ (uchar*) lastpos, sizeof(lastpos), 0);
if (save_errno != HA_ERR_RECORD_CHANGED)
{
mi_print_error(info->s, HA_ERR_CRASHED);
diff -Nrup a/storage/myisam/mi_delete_all.c b/storage/myisam/mi_delete_all.c
--- a/storage/myisam/mi_delete_all.c 2008-03-28 16:16:49 +01:00
+++ b/storage/myisam/mi_delete_all.c 2008-05-09 12:27:19 +02:00
@@ -47,7 +47,7 @@ int mi_delete_all_rows(MI_INFO *info)
for (i=0 ; i < share->base.keys ; i++)
state->key_root[i]= HA_OFFSET_ERROR;
- myisam_log_command(MI_LOG_DELETE_ALL,info,(uchar*) 0,0,0);
+ myisam_log_command_logical(MI_LOG_DELETE_ALL, info, (uchar*) 0, 0, 0);
/*
If we are using delayed keys or if the user has done changes to the tables
since it was locked then there may be key blocks in the key cache
@@ -60,6 +60,9 @@ int mi_delete_all_rows(MI_INFO *info)
if (my_chsize(info->dfile, 0, 0, MYF(MY_WME)) ||
my_chsize(share->kfile, share->base.keystart, 0, MYF(MY_WME)) )
goto err;
+ if (unlikely(mi_get_physical_logging_state(info->s)))
+ _myisam_log_command(&myisam_physical_log, MI_LOG_DELETE_ALL, share,
+ NULL, 0, 0);
VOID(_mi_writeinfo(info,WRITEINFO_UPDATE_KEYFILE));
#ifdef HAVE_MMAP
/* Map again */
diff -Nrup a/storage/myisam/mi_dynrec.c b/storage/myisam/mi_dynrec.c
--- a/storage/myisam/mi_dynrec.c 2008-04-01 15:44:56 +02:00
+++ b/storage/myisam/mi_dynrec.c 2008-05-09 12:27:19 +02:00
@@ -193,9 +193,11 @@ size_t mi_nommap_pread(MI_INFO *info, uc
size_t mi_mmap_pwrite(MI_INFO *info, c 