Below is the list of changes that have just been committed into a local
5.1 repository of istruewing. When istruewing does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html
ChangeSet@stripped, 2007-01-31 18:49:07+01:00, istruewing@stripped +7 -0
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Resizing a key cache while it was in heavy use could crash the
server. There were several race conditions.
I reworked some of the algorithms to fix the race conditions.
No test case. Repeating the crashes requires heavy concurrent
load on the key cache. A test script is attached to the bug report.
More explanations to the changes are contained in a text file
attached to the bug report.
include/keycache.h@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +6 -0
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Added KEY_CACHE components in_resize and waiting_for_resize_cnt.
mysys/mf_keycache.c@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +2088 -466
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Changed resize_key_cache() to not disable the key cache
after the flush phase. Changed queue handling to use
standard functions. Wake all threads waiting on resize_queue.
We can now have read/write threads waiting there (see below).
Combined add_to_queue() and the wait loops that were always
following it to the new function wait_on_queue().
Combined release_queue() and the condition that was always
preceding it to the new function release_whole_queue().
Added code to flag and respect the exceptional situation
BLOCK_IN_EVICTION.
Rewrote the resize branch of find_key_block().
Added code to the eviction handling in find_key_block()
to catch more exceptional cases.
Changed key_cache_read(), key_cache_insert() and key_cache_write()
so that they lock keycache->cache_lock whenever the key cache is
initialized. Checking for a disabled cache and incrementing and
decrementing the "resize counter" is always done within the lock.
Locking and unlocking as well as counting the "resize counter" is
now done once outside the loop. All three functions can now handle
a NULL return from find_key_block. This happens in the flush phase
of a resize and demands direct file I/O. Care is taken for
secondary requests (PAGE_WAIT_TO_BE_READ) to wait in any case.
Moved block status changes behind the copying of buffer data.
key_cache_insert() does now read the block if the caller did
supply less data than a full cache block.
key_cache_write() does now take care of parallel running flushes
(BLOCK_FOR_UPDATE, BLOCK_IN_FLUSHWRITE).
Changed free_block() to un-initialize block variables in the
correct order and respect an exceptional BLOCK_IN_EVICTION state.
Changed flushing to take care for parallel running writes.
Changed flushing to avoid freeing blocks in eviction.
Changed flushing to consider that parallel writes can move blocks
from the file_blocks hash to the changed_blocks hash.
Changed flushing to take care for other parallel flushes.
Changed flushing to assure that it ends with everything flushed.
Added some comments and debugging statements. ;-)
mysys/my_static.c@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +0 -3
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Removed an unused global variable.
sql/handler.cc@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +2 -2
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Changed types of local variables to match their use
for init_key_cache().
sql/sql_table.cc@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +6 -1
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Changed TL_READ to TL_READ_NO_INSERT in mysql_preload_keys.
storage/myisam/ha_myisam.cc@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +1
-1
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Moved an automatic (stack) variable to the scope where it is used.
storage/myisam/mi_preload.c@stripped, 2007-01-31 18:49:03+01:00, istruewing@stripped +27
-0
Bug#17332 - changing key_buffer_size on a running server
can crash under load
Added some preliminary code to
- allow LOAD INDEX to load indexes of different block size,
- align load chunks to key cache blocks.
# This is a BitKeeper patch. What follows are the unified diffs for the
# set of deltas contained in the patch. The rest of the patch, the part
# that BitKeeper cares about, is below these diffs.
# User: istruewing
# Host: chilla.local
# Root: /home/mydev/mysql-5.1-bug17332
--- 1.62/mysys/mf_keycache.c 2007-01-31 18:49:15 +01:00
+++ 1.63/mysys/mf_keycache.c 2007-01-31 18:49:15 +01:00
@@ -36,6 +36,64 @@
blocks_unused is the sum of never used blocks in the pool and of currently
free blocks. blocks_used is the number of blocks fetched from the pool and
as such gives the maximum number of in-use blocks at any time.
+
+ Key Cache Locking
+ =================
+
+ All key cache locking is done with a single mutex per key cache:
+ keycache->cache_lock. This mutex is locked almost all the time
+ when executing code in this file (mf_keycache.c).
+ However it is released for I/O and some copy operations.
+
+ The cache_lock is also released when waiting for some event. Waiting
+ and signalling is done via condition variables. In most cases the
+ thread waits on its thread->suspend condition variable. Every thread
+ has a my_thread_var structure, which contains this variable and a
+ '*next' and '**prev' pointer. These pointers are used to insert the
+ thread into a wait queue.
+
+ NOTE: Since there is only one pair of queue pointers per thread, a
+ thread can be in one wait queue only.
+
+ Before starting to wait on its condition variable with
+ pthread_cond_wait(), the thread enters itself to a specific wait queue
+ with link_into_queue() (double linked with '*next' + '**prev') or
+ wait_on_queue() (single linked with '*next').
+
+ Another thread, when releasing a resource, looks up the waiting thread
+ in the related wait queue. It sends a signal with
+ pthread_cond_signal() to the waiting thread.
+
+ NOTE: Depending on the particular wait situation, either the sending
+ thread removes the waiting thread from the wait queue with
+ unlink_from_queue() or release_whole_queue() respectively, or the waiting
+ thread removes itself.
+
+ There is one exception from this locking scheme. Each block has a
+ reference to a condition variable (condvar). It holds a reference to
+ the thread->suspend condition variable, if that thread is waiting for
+ the block. When that thread is signalled, the reference is cleared.
+ This is similar to the above, but it clearly means that only one
+ thread can wait for a particular block. There is no queue in this
+ case. Strangely enough block->convar is used for waiting for the
+ assigned hash_link only. More precisely it is used to wait for all
+ requests to be unregistered from the assigned hash_link.
+
+ The resize_queue serves two purposes:
+ 1. Threads that want to do a resize wait there if in_resize is set.
+ This is not used in the server. The server refuses a second resize
+ request if one is already active. keycache->in_init is used for the
+ synchronization. See set_var.cc.
+ 2. Threads that want to access blocks during resize wait here during
+ the re-initialization phase.
+ When the resize is done, all threads on the queue are signalled.
+ Hypothetical resizers can compete for resizing, and read/write
+ requests will restart to request blocks from the freshly resized
+ cache. If the cache has been resized too small, it is disabled and
+ 'can_be_used' is false. In this case read/write requests bypass the
+ cache. Since they increment and decrement 'cnt_for_resize_op', the
+ next resizer can wait on the queue 'waiting_for_resize_cnt' until all
+ I/O finished.
*/
#include "mysys_priv.h"
@@ -111,12 +169,16 @@ struct st_hash_link
};
/* simple states of a block */
-#define BLOCK_ERROR 1 /* an error occured when performing disk i/o */
-#define BLOCK_READ 2 /* the is page in the block buffer */
-#define BLOCK_IN_SWITCH 4 /* block is preparing to read new page */
-#define BLOCK_REASSIGNED 8 /* block does not accept requests for old page */
-#define BLOCK_IN_FLUSH 16 /* block is in flush operation */
-#define BLOCK_CHANGED 32 /* block buffer contains a dirty page */
+#define BLOCK_ERROR 1 /* an error occured when performing file i/o */
+#define BLOCK_READ 2 /* file block is in the block buffer */
+#define BLOCK_IN_SWITCH 4 /* block is preparing to read new page */
+#define BLOCK_REASSIGNED 8 /* blk does not accept requests for old page */
+#define BLOCK_IN_FLUSH 16 /* block is selected for flush */
+#define BLOCK_CHANGED 32 /* block buffer contains a dirty page */
+#define BLOCK_IN_USE 64 /* block is not free */
+#define BLOCK_IN_EVICTION 128 /* block is selected for eviction */
+#define BLOCK_IN_FLUSHWRITE 256 /* block is in write to file */
+#define BLOCK_FOR_UPDATE 512 /* block is selected for buffer modification */
/* page status, returned by find_key_block */
#define PAGE_READ 0
@@ -153,14 +215,18 @@ KEY_CACHE *dflt_key_cache= &dflt_key_cac
static int flush_all_key_blocks(KEY_CACHE *keycache);
#ifdef THREAD
-static void link_into_queue(KEYCACHE_WQUEUE *wqueue,
- struct st_my_thread_var *thread);
-static void unlink_from_queue(KEYCACHE_WQUEUE *wqueue,
- struct st_my_thread_var *thread);
+static void wait_on_queue(KEYCACHE_WQUEUE *wqueue,
+ pthread_mutex_t *mutex);
+static void release_whole_queue(KEYCACHE_WQUEUE *wqueue);
+#else
+#define wait_on_queue(wqueue, mutex) KEYCACHE_DBUG_ASSERT(0);
+#define release_whole_queue(wqueue) /* release_whole_queue() */
#endif
static void free_block(KEY_CACHE *keycache, BLOCK_LINK *block);
+#if !defined(DBUG_OFF)
static void test_key_cache(KEY_CACHE *keycache,
const char *where, my_bool lock);
+#endif
#define KEYCACHE_HASH(f, pos) \
(((ulong) ((pos) >> keycache->key_cache_shift)+ \
@@ -253,6 +319,13 @@ static int keycache_pthread_cond_signal(
#define keycache_pthread_cond_signal pthread_cond_signal
#endif /* defined(KEYCACHE_DEBUG) */
+#if !defined(DBUG_OFF)
+#define inline /* disabled inline for easier debugging */
+static int fail_block(BLOCK_LINK *block);
+static int fail_hlink(HASH_LINK *hlink);
+static int cache_empty(KEY_CACHE *keycache);
+#endif
+
static inline uint next_power(uint value)
{
return (uint) my_round_up_to_next_power((uint32) value) << 1;
@@ -305,10 +378,19 @@ int init_key_cache(KEY_CACHE *keycache,
keycache->disk_blocks= -1;
if (! keycache->key_cache_inited)
{
- keycache->key_cache_inited= 1;
+ /*
+ Initialize these variables once only.
+ Their value must survive re-initialization during resizing.
+ */
+ keycache->in_resize= 0;
+ keycache->resize_in_flush= 0;
+ keycache->cnt_for_resize_op= 0;
+ keycache->waiting_for_resize_cnt.last_thread= NULL;
keycache->in_init= 0;
pthread_mutex_init(&keycache->cache_lock, MY_MUTEX_INIT_FAST);
keycache->resize_queue.last_thread= NULL;
+ /* Initialize this after the mutex. It is read asynchronously. */
+ keycache->key_cache_inited= 1;
}
keycache->key_cache_mem_size= use_mem;
@@ -320,7 +402,8 @@ int init_key_cache(KEY_CACHE *keycache,
blocks= (uint) (use_mem / (sizeof(BLOCK_LINK) + 2 * sizeof(HASH_LINK) +
sizeof(HASH_LINK*) * 5/4 + key_cache_block_size));
/* It doesn't make sense to have too few blocks (less than 8) */
- if (blocks >= 8 && keycache->disk_blocks < 0)
+ /* Comment to be deleted: disk_blocks is set to -1 above unconditionally. */
+ if (blocks >= 8)
{
for ( ; ; )
{
@@ -394,8 +477,6 @@ int init_key_cache(KEY_CACHE *keycache,
blocks * age_threshold / 100 :
blocks);
- keycache->cnt_for_resize_op= 0;
- keycache->resize_in_flush= 0;
keycache->can_be_used= 1;
keycache->waiting_for_hash_link.last_thread= NULL;
@@ -411,6 +492,11 @@ int init_key_cache(KEY_CACHE *keycache,
bzero((gptr) keycache->file_blocks,
sizeof(keycache->file_blocks[0]) * CHANGED_BLOCKS_HASH);
}
+ else
+ {
+ /* key_buffer_size is specified too small. Disable the cache. */
+ keycache->can_be_used= 0;
+ }
keycache->blocks= keycache->disk_blocks > 0 ? keycache->disk_blocks : 0;
DBUG_RETURN((int) keycache->disk_blocks);
@@ -469,10 +555,6 @@ int resize_key_cache(KEY_CACHE *keycache
uint age_threshold)
{
int blocks;
-#ifdef THREAD
- struct st_my_thread_var *thread;
- KEYCACHE_WQUEUE *wqueue;
-#endif
DBUG_ENTER("resize_key_cache");
if (!keycache->key_cache_inited)
@@ -488,54 +570,89 @@ int resize_key_cache(KEY_CACHE *keycache
keycache_pthread_mutex_lock(&keycache->cache_lock);
#ifdef THREAD
- wqueue= &keycache->resize_queue;
- thread= my_thread_var;
- link_into_queue(wqueue, thread);
-
- while (wqueue->last_thread->next != thread)
+ /*
+ We may need to wait for another thread which is doing a resize
+ already. This cannot happen in the MySQL server though. It allows
+ one resizer only. In set_var.cc keycache->in_init is used to block
+ multiple attempts.
+ */
+ while (keycache->in_resize)
{
- keycache_pthread_cond_wait(&thread->suspend, &keycache->cache_lock);
+ /* purecov: begin inspected */
+ wait_on_queue(&keycache->resize_queue, &keycache->cache_lock);
+ /* purecov: end */
}
#endif
- keycache->resize_in_flush= 1;
- if (flush_all_key_blocks(keycache))
+ /*
+ Mark the operation in progress. This blocks other threads from doing
+ a resize in parallel. It prohibits new blocks to enter the cache.
+ Read/write requests can bypass the cache during the flush phase.
+ */
+ keycache->in_resize= 1;
+
+ /* Need to flush only if keycache is enabled. */
+ if (keycache->can_be_used)
{
- /* TODO: if this happens, we should write a warning in the log file ! */
+ /* Start the flush phase. */
+ keycache->resize_in_flush= 1;
+
+ if (flush_all_key_blocks(keycache))
+ {
+ /* TODO: if this happens, we should write a warning in the log file ! */
+ keycache->resize_in_flush= 0;
+ blocks= 0;
+ keycache->can_be_used= 0;
+ goto finish;
+ }
+
+ /* End the flush phase. */
keycache->resize_in_flush= 0;
- blocks= 0;
- keycache->can_be_used= 0;
- goto finish;
}
- keycache->resize_in_flush= 0;
- keycache->can_be_used= 0;
+
#ifdef THREAD
+ /*
+ Some direct read/write operations (bypassing the cache) may still be
+ unfinished. Wait until they are done. If the key cache can be used,
+ direct I/O is done in increments of key_cache_block_size. That is,
+ every block is checked if it is in the cache. We need to wait for
+ pending I/O before re-initializing the cache, because we may change
+ the block size. Otherwise they could check for blocks at file
+ positions where the new block division has none. We do also want to
+ wait for I/O done when (if) the cache was disabled. It must not
+ run in parallel with normal cache operation.
+ */
while (keycache->cnt_for_resize_op)
{
- KEYCACHE_DBUG_PRINT("resize_key_cache: wait",
- ("suspend thread %ld", thread->id));
- keycache_pthread_cond_wait(&thread->suspend, &keycache->cache_lock);
+ wait_on_queue(&keycache->waiting_for_resize_cnt,
&keycache->cache_lock);
}
#else
KEYCACHE_DBUG_ASSERT(keycache->cnt_for_resize_op == 0);
#endif
+ /*
+ Free old cache structures, allocate new structures, and initialize
+ them. Note that the cache_lock mutex and the resize_queue are left
+ untouched. We do not lose the cache_lock and will release it only at
+ the end of this function.
+ */
end_key_cache(keycache, 0); /* Don't free mutex */
/* The following will work even if use_mem is 0 */
blocks= init_key_cache(keycache, key_cache_block_size, use_mem,
division_limit, age_threshold);
finish:
+ /*
+ Mark the resize finished. This allows other threads to start a
+ resize or to request new cache blocks.
+ */
+ keycache->in_resize= 0;
+
#ifdef THREAD
- unlink_from_queue(wqueue, thread);
- /* Signal for the next resize request to proceeed if any */
- if (wqueue->last_thread)
- {
- KEYCACHE_DBUG_PRINT("resize_key_cache: signal",
- ("thread %ld", wqueue->last_thread->next->id));
- keycache_pthread_cond_signal(&wqueue->last_thread->next->suspend);
- }
+ /* Signal waiting threads. */
+ release_whole_queue(&keycache->resize_queue);
#endif
+
keycache_pthread_mutex_unlock(&keycache->cache_lock);
DBUG_RETURN(blocks);
}
@@ -557,14 +674,8 @@ static inline void inc_counter_for_resiz
static inline void dec_counter_for_resize_op(KEY_CACHE *keycache)
{
#ifdef THREAD
- struct st_my_thread_var *last_thread;
- if (!--keycache->cnt_for_resize_op &&
- (last_thread= keycache->resize_queue.last_thread))
- {
- KEYCACHE_DBUG_PRINT("dec_counter_for_resize_op: signal",
- ("thread %ld", last_thread->next->id));
- keycache_pthread_cond_signal(&last_thread->next->suspend);
- }
+ if (!--keycache->cnt_for_resize_op)
+ release_whole_queue(&keycache->waiting_for_resize_cnt);
#else
keycache->cnt_for_resize_op--;
#endif
@@ -658,6 +769,7 @@ void end_key_cache(KEY_CACHE *keycache,
#ifdef THREAD
+
/*
Link a thread into double-linked queue of waiting threads.
@@ -673,12 +785,17 @@ void end_key_cache(KEY_CACHE *keycache,
Queue is represented by a circular list of the thread structures
The list is double-linked of the type (**prev,*next), accessed by
a pointer to the last element.
+
+ Since there is only one pair of queue pointers per thread, a
+ thread can be part of one wait queue only.
*/
static void link_into_queue(KEYCACHE_WQUEUE *wqueue,
struct st_my_thread_var *thread)
{
struct st_my_thread_var *last;
+
+ DBUG_ASSERT(!thread->next && !thread->prev);
if (! (last= wqueue->last_thread))
{
/* Queue is empty */
@@ -714,6 +831,7 @@ static void unlink_from_queue(KEYCACHE_W
struct st_my_thread_var *thread)
{
KEYCACHE_DBUG_PRINT("unlink_from_queue", ("thread %ld", thread->id));
+ DBUG_ASSERT(thread->next && thread->prev);
if (thread->next == thread)
/* The queue contains only one member */
wqueue->last_thread= NULL;
@@ -726,6 +844,13 @@ static void unlink_from_queue(KEYCACHE_W
thread->prev);
}
thread->next= NULL;
+#if !defined(DBUG_OFF)
+ /*
+ This makes it easier to see it's not in a chain during debugging.
+ And some DBUG_ASSERT() rely on it.
+ */
+ thread->prev= NULL;
+#endif
}
@@ -733,9 +858,9 @@ static void unlink_from_queue(KEYCACHE_W
Add a thread to single-linked queue of waiting threads
SYNOPSIS
- add_to_queue()
- wqueue pointer to the queue structure
- thread pointer to the thread to be added to the queue
+ wait_on_queue()
+ wqueue Pointer to the queue structure.
+ mutex Cache_lock to acquire after awake.
RETURN VALUE
none
@@ -744,12 +869,26 @@ static void unlink_from_queue(KEYCACHE_W
Queue is represented by a circular list of the thread structures
The list is single-linked of the type (*next), accessed by a pointer
to the last element.
+
+ Since there is only one pair of queue pointers per thread, a
+ thread can be part of one wait queue only.
+
+ The function protects against stray signals by verifying that the
+ current thread is unlinked from the queue when awaking. However,
+ since several threads can wait for the same event, it might be
+ necessary for the caller of the function to check again if the
+ condition for awake is indeed matched.
*/
-static inline void add_to_queue(KEYCACHE_WQUEUE *wqueue,
- struct st_my_thread_var *thread)
+static void wait_on_queue(KEYCACHE_WQUEUE *wqueue,
+ pthread_mutex_t *mutex)
{
struct st_my_thread_var *last;
+ struct st_my_thread_var *thread= my_thread_var;
+
+ /* Add to queue. */
+ DBUG_ASSERT(!thread->next);
+ DBUG_ASSERT(!thread->prev); /* Not required, but must be true anyway. */
if (! (last= wqueue->last_thread))
thread->next= thread;
else
@@ -758,6 +897,17 @@ static inline void add_to_queue(KEYCACHE
last->next= thread;
}
wqueue->last_thread= thread;
+
+ /*
+ Wait until thread is removed from queue by the signalling thread.
+ The loop protects against stray signals.
+ */
+ do
+ {
+ KEYCACHE_DBUG_PRINT("wait", ("suspend thread %ld", thread->id));
+ keycache_pthread_cond_wait(&thread->suspend, mutex);
+ }
+ while(thread->next);
}
@@ -765,36 +915,47 @@ static inline void add_to_queue(KEYCACHE
Remove all threads from queue signaling them to proceed
SYNOPSIS
- realease_queue()
- wqueue pointer to the queue structure
- thread pointer to the thread to be added to the queue
+ release_whole_queue()
+ wqueue pointer to the queue structure
RETURN VALUE
none
NOTES.
- See notes for add_to_queue
+ See notes for wait_on_queue().
When removed from the queue each thread is signaled via condition
variable thread->suspend.
*/
-static void release_queue(KEYCACHE_WQUEUE *wqueue)
+static void release_whole_queue(KEYCACHE_WQUEUE *wqueue)
{
- struct st_my_thread_var *last= wqueue->last_thread;
- struct st_my_thread_var *next= last->next;
+ struct st_my_thread_var *last;
+ struct st_my_thread_var *next;
struct st_my_thread_var *thread;
+
+ /* Queue may be empty. */
+ if (!(last= wqueue->last_thread))
+ return;
+
+ next= last->next;
do
{
thread=next;
- KEYCACHE_DBUG_PRINT("release_queue: signal", ("thread %ld", thread->id));
+ KEYCACHE_DBUG_PRINT("release_whole_queue: signal",
+ ("thread %ld", thread->id));
+ /* Signal the thread. */
keycache_pthread_cond_signal(&thread->suspend);
+ /* Take thread from queue. */
next=thread->next;
thread->next= NULL;
}
while (thread != last);
+
+ /* Now queue is definitely empty. */
wqueue->last_thread= NULL;
}
-#endif
+
+#endif /* THREAD */
/*
@@ -803,9 +964,19 @@ static void release_queue(KEYCACHE_WQUEU
static inline void unlink_changed(BLOCK_LINK *block)
{
+ DBUG_ASSERT(block->prev_changed && *block->prev_changed == block);
if (block->next_changed)
block->next_changed->prev_changed= block->prev_changed;
*block->prev_changed= block->next_changed;
+
+#if !defined(DBUG_OFF)
+ /*
+ This makes it easier to see it's not in a chain during debugging.
+ And some DBUG_ASSERT() rely on it.
+ */
+ block->next_changed= NULL;
+ block->prev_changed= NULL;
+#endif
}
@@ -815,6 +986,8 @@ static inline void unlink_changed(BLOCK_
static inline void link_changed(BLOCK_LINK *block, BLOCK_LINK **phead)
{
+ DBUG_ASSERT(!block->next_changed);
+ DBUG_ASSERT(!block->prev_changed);
block->prev_changed= phead;
if ((block->next_changed= *phead))
(*phead)->prev_changed= &block->next_changed;
@@ -823,13 +996,36 @@ static inline void link_changed(BLOCK_LI
/*
- Unlink a block from the chain of dirty/clean blocks, if it's asked for,
- and link it to the chain of clean blocks for the specified file
+ Link a block in a chain of clean blocks of a file.
+
+ SYNOPSIS
+ link_to_file_list()
+ keycache Key cache handle
+ block Block to relink
+ file File to be linked to
+ unlink If to unlink first
+
+ DESCRIPTION
+ Unlink a block from whichever chain it is linked in, if it's
+ asked for, and link it to the chain of clean blocks of the
+ specified file.
+
+ NOTE
+ Please do never set/clear BLOCK_CHANGED outside of
+ link_to_file_list() or link_to_changed_list().
+ You would risk to damage correct counting of changed blocks
+ and to find blocks in the wrong hash.
+
+ RETURN
+ void
*/
static void link_to_file_list(KEY_CACHE *keycache,
BLOCK_LINK *block, int file, my_bool unlink)
{
+ DBUG_ASSERT(block->status & BLOCK_IN_USE);
+ DBUG_ASSERT(block->hash_link && block->hash_link->block == block);
+ DBUG_ASSERT(block->hash_link->file == file);
if (unlink)
unlink_changed(block);
link_changed(block, &keycache->file_blocks[FILE_HASH(file)]);
@@ -843,13 +1039,34 @@ static void link_to_file_list(KEY_CACHE
/*
- Unlink a block from the chain of clean blocks for the specified
- file and link it to the chain of dirty blocks for this file
+ Re-link a block from the clean chain to the dirty chain of a file.
+
+ SYNOPSIS
+ link_to_changed_list()
+ keycache key cache handle
+ block block to relink
+
+ DESCRIPTION
+ Unlink a block from the chain of clean blocks of a file
+ and link it to the chain of dirty blocks of the same file.
+
+ NOTE
+ Please do never set/clear BLOCK_CHANGED outside of
+ link_to_file_list() or link_to_changed_list().
+ You would risk to damage correct counting of changed blocks
+ and to find blocks in the wrong hash.
+
+ RETURN
+ void
*/
-static inline void link_to_changed_list(KEY_CACHE *keycache,
- BLOCK_LINK *block)
+static void link_to_changed_list(KEY_CACHE *keycache,
+ BLOCK_LINK *block)
{
+ DBUG_ASSERT(block->status & BLOCK_IN_USE);
+ DBUG_ASSERT(!(block->status & BLOCK_CHANGED));
+ DBUG_ASSERT(block->hash_link && block->hash_link->block == block);
+
unlink_changed(block);
link_changed(block,
&keycache->changed_blocks[FILE_HASH(block->hash_link->file)]);
@@ -874,13 +1091,13 @@ static inline void link_to_changed_list(
none
NOTES.
- The LRU chain is represented by a curcular list of block structures.
+ The LRU ring is represented by a circular list of block structures.
The list is double-linked of the type (**prev,*next) type.
- The LRU chain is divided into two parts - hot and warm.
+ The LRU ring is divided into two parts - hot and warm.
There are two pointers to access the last blocks of these two
parts. The beginning of the warm part follows right after the
end of the hot part.
- Only blocks of the warm part can be used for replacement.
+ Only blocks of the warm part can be used for eviction.
The first block from the beginning of this subchain is always
taken for eviction (keycache->last_used->next)
@@ -893,6 +1110,9 @@ static inline void link_to_changed_list(
+----| beg |---->...----| end |----+
+------+ +------+ins
first for eviction
+
+ It is also possible that the block is selected for eviction and thus
+ not linked in the LRU ring.
*/
static void link_block(KEY_CACHE *keycache, BLOCK_LINK *block, my_bool hot,
@@ -901,7 +1121,12 @@ static void link_block(KEY_CACHE *keycac
BLOCK_LINK *ins;
BLOCK_LINK **pins;
- KEYCACHE_DBUG_ASSERT(! (block->hash_link &&
block->hash_link->requests));
+ DBUG_ASSERT((block->status & ~BLOCK_CHANGED) == (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(block->hash_link); /*backptr to block NULL from free_block()*/
+ DBUG_ASSERT(!block->requests);
+ DBUG_ASSERT(block->prev_changed && *block->prev_changed == block);
+ DBUG_ASSERT(!block->next_used);
+ DBUG_ASSERT(!block->prev_used);
#ifdef THREAD
if (!hot && keycache->waiting_for_block.last_thread)
{
@@ -930,6 +1155,29 @@ static void link_block(KEY_CACHE *keycac
}
while (thread != last_thread);
hash_link->block= block;
+ /*
+ NOTE: We assigned the block to the hash_link and signalled the
+ requesting thread(s). But it is possible that other threads runs
+ first. These threads see the hash_link assigned to a block which
+ is assigned to another hash_link and not marked BLOCK_IN_SWITCH.
+ This can be a problem for functions that do not select the block
+ via its hash_link: flush and free. They do only see a block which
+ is in a "normal" state and don't know that it will be evicted soon.
+
+ We cannot set BLOCK_IN_SWITCH here because only one of the
+ requesting threads must handle the eviction. All others must wait
+ for it to complete. If we set the flag here, the threads would not
+ know who is in charge of the eviction. Without the flag, the first
+ thread takes the stick and sets the flag.
+
+ But we need to note in the block that is has been selected for
+ eviction. It must not be freed. The evicting thread will not
+ expect the block in the free list. Before freeing we could also
+ check if block->requests > 1. But I think including another flag
+ in the check of block->status is slightly more efficient and
+ probably easier to read.
+ */
+ block->status|= BLOCK_IN_EVICTION;
KEYCACHE_THREAD_TRACE("link_block: after signaling");
#if defined(KEYCACHE_DEBUG)
KEYCACHE_DBUG_PRINT("link_block",
@@ -956,7 +1204,7 @@ static void link_block(KEY_CACHE *keycac
}
else
{
- /* The LRU chain is empty */
+ /* The LRU ring is empty. Let the block point to itself. */
keycache->used_last= keycache->used_ins= block->next_used= block;
block->prev_used= &block->next_used;
}
@@ -990,6 +1238,13 @@ static void link_block(KEY_CACHE *keycac
static void unlink_block(KEY_CACHE *keycache, BLOCK_LINK *block)
{
+ DBUG_ASSERT((block->status & ~BLOCK_CHANGED) == (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(block->hash_link); /*backptr to block NULL from free_block()*/
+ DBUG_ASSERT(!block->requests);
+ DBUG_ASSERT(block->prev_changed && *block->prev_changed == block);
+ DBUG_ASSERT(block->next_used && block->prev_used &&
+ (block->next_used->prev_used == &block->next_used) &&
+ (*block->prev_used == block));
if (block->next_used == block)
/* The list contains only one member */
keycache->used_last= keycache->used_ins= NULL;
@@ -1003,6 +1258,13 @@ static void unlink_block(KEY_CACHE *keyc
keycache->used_ins=STRUCT_PTR(BLOCK_LINK, next_used, block->prev_used);
}
block->next_used= NULL;
+#if !defined(DBUG_OFF)
+ /*
+ This makes it easier to see it's not in a chain during debugging.
+ And some DBUG_ASSERT() rely on it.
+ */
+ block->prev_used= NULL;
+#endif
KEYCACHE_THREAD_TRACE("unlink_block");
#if defined(KEYCACHE_DEBUG)
@@ -1017,12 +1279,27 @@ static void unlink_block(KEY_CACHE *keyc
/*
- Register requests for a block
+ Register requests for a block.
+
+ SYNOPSIS
+ reg_requests()
+ keycache Pointer to a key cache data structure.
+ block Pointer to the block to register a request on.
+ count Number of requests. Always 1.
+
+ NOTE
+ The first request unlinks the block from the LRU ring. This means
+ that it is protected against eveiction.
+
+ RETURN
+ void
*/
static void reg_requests(KEY_CACHE *keycache, BLOCK_LINK *block, int count)
{
- if (! block->requests)
- /* First request for the block unlinks it */
+ DBUG_ASSERT(block->status & BLOCK_IN_USE);
+ DBUG_ASSERT(block->hash_link);
+
+ if (!block->requests)
unlink_block(keycache, block);
block->requests+=count;
}
@@ -1042,7 +1319,7 @@ static void reg_requests(KEY_CACHE *keyc
none
NOTES.
- Every linking to the LRU chain decrements by one a special block
+ Every linking to the LRU ring decrements by one a special block
counter (if it's positive). If the at_end parameter is TRUE the block is
added either at the end of warm sub-chain or at the end of hot sub-chain.
It is added to the hot subchain if its counter is zero and number of
@@ -1055,11 +1332,20 @@ static void reg_requests(KEY_CACHE *keyc
At the same time the block at the very beginning of the hot subchain
might be moved to the beginning of the warm subchain if it stays untouched
for a too long time (this time is determined by parameter age_threshold).
+
+ It is also possible that the block is selected for eviction and thus
+ not linked in the LRU ring.
*/
static void unreg_request(KEY_CACHE *keycache,
BLOCK_LINK *block, int at_end)
{
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(block->hash_link); /*backptr to block NULL from free_block()*/
+ DBUG_ASSERT(block->requests);
+ DBUG_ASSERT(block->prev_changed && *block->prev_changed == block);
+ DBUG_ASSERT(!block->next_used);
+ DBUG_ASSERT(!block->prev_used);
if (! --block->requests)
{
my_bool hot;
@@ -1078,9 +1364,22 @@ static void unreg_request(KEY_CACHE *key
link_block(keycache, block, hot, (my_bool)at_end);
block->last_hit_time= keycache->keycache_time;
keycache->keycache_time++;
+ /*
+ At this place, the block might be in the LRU ring or not. If an
+ evicter was waiting for a block, it was selected for eviction and
+ not linked in the LRU ring.
+ */
+ /*
+ Check if we should link a hot block to the warm block sub-chain.
+ It is possible that we select the same block as above. But it can
+ also be another block. In any case a block from the LRU ring is
+ selected. In other words it works even if the above block was
+ selected for eviction and not linked in the LRU ring. Since this
+ happens only if the LRU ring is empty, the block selected below
+ would be NULL and the rest of the function skipped.
+ */
block= keycache->used_ins;
- /* Check if we should link a hot block to the warm block */
if (block && keycache->keycache_time - block->last_hit_time >
keycache->age_threshold)
{
@@ -1101,8 +1400,14 @@ static void unreg_request(KEY_CACHE *key
Remove a reader of the page in block
*/
-static inline void remove_reader(BLOCK_LINK *block)
+static void remove_reader(BLOCK_LINK *block)
{
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(block->hash_link && block->hash_link->block == block);
+ DBUG_ASSERT(block->prev_changed && *block->prev_changed == block);
+ DBUG_ASSERT(!block->next_used);
+ DBUG_ASSERT(!block->prev_used);
+ DBUG_ASSERT(block->hash_link->requests);
#ifdef THREAD
if (! --block->hash_link->requests && block->condvar)
keycache_pthread_cond_signal(block->condvar);
@@ -1117,19 +1422,34 @@ static inline void remove_reader(BLOCK_L
signals on its termination
*/
-static inline void wait_for_readers(KEY_CACHE *keycache __attribute__((unused)),
- BLOCK_LINK *block)
+static void wait_for_readers(KEY_CACHE *keycache,
+ BLOCK_LINK *block)
{
#ifdef THREAD
struct st_my_thread_var *thread= my_thread_var;
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(!(block->status & (BLOCK_ERROR | BLOCK_IN_FLUSH |
+ BLOCK_CHANGED)));
+ DBUG_ASSERT(block->hash_link);
+ DBUG_ASSERT(block->hash_link->block == block);
+ /* Linked in file_blocks or changed_blocks hash. */
+ DBUG_ASSERT(block->prev_changed && *block->prev_changed == block);
+ /* Not linked in LRU ring. */
+ DBUG_ASSERT(!block->next_used);
+ DBUG_ASSERT(!block->prev_used);
while (block->hash_link->requests)
{
KEYCACHE_DBUG_PRINT("wait_for_readers: wait",
("suspend thread %ld block %u",
thread->id, BLOCK_NUMBER(block)));
+ /* There must be no other waiter. We have no queue here. */
+ DBUG_ASSERT(!block->condvar);
block->condvar= &thread->suspend;
keycache_pthread_cond_wait(&thread->suspend, &keycache->cache_lock);
block->condvar= NULL;
+ /* The other thread might have freed the block in between. */
+ if (!block->hash_link)
+ break;
}
#else
KEYCACHE_DBUG_ASSERT(block->hash_link->requests == 0);
@@ -1355,77 +1675,312 @@ static BLOCK_LINK *find_key_block(KEY_CA
#endif
restart:
- /* Find the hash link for the requested page (file, filepos) */
+ /*
+ If the flush phase of a resize operation fails, the cache is left
+ unusable. This will be detected only after "goto restart".
+ */
+ if (!keycache->can_be_used)
+ DBUG_RETURN(0);
+
+ /*
+ Find the hash_link for the requested file block (file, filepos). We
+ do always get a hash_link here. It has registered our request so
+ that no other thread can use it for another file block until we
+ release the request (which is done by remove_reader() usually). The
+ hash_link can have a block assigned to it or not. If there is a
+ block, it may be assigned to this hash_link or not. In cases where a
+ block is evicted from the cache, it is taken from the LRU ring and
+ referenced by the new hash_link. But the block can still be assigned
+ to its old hash_link for some time if it needs to be flushed first,
+ or if there are other threads still reading it.
+
+ Summary:
+ hash_link is always returned.
+ hash_link->block can be:
+ - NULL or
+ - not assigned to this hash_link or
+ - assigned to this hash_link. If assigned, the block can have
+ - invalid data (when freshly assigned) or
+ - valid data. Valid data can be
+ - changed over the file contents (dirty) or
+ - not changed (clean).
+ */
hash_link= get_hash_link(keycache, file, filepos);
+ DBUG_ASSERT((hash_link->file == file) && (hash_link->diskpos ==
filepos));
page_status= -1;
if ((block= hash_link->block) &&
block->hash_link == hash_link && (block->status & BLOCK_READ))
+ {
+ /* Assigned block with valid (changed or unchanged) contents. */
page_status= PAGE_READ;
+ }
+ /*
+ else (page_status == -1)
+ - block == NULL or
+ - block not assigned to this hash_link or
+ - block assigned but not yet read from file (invalid data).
+ */
- if (wrmode && keycache->resize_in_flush)
+ if (keycache->in_resize)
{
- /* This is a write request during the flush phase of a resize operation */
+ /* This is a request during a resize operation */
- if (page_status != PAGE_READ)
+ if (!block)
{
- /* We don't need the page in the cache: we are going to write on disk */
+ struct st_my_thread_var *thread;
+
+ /*
+ The file block is not in the cache. We don't need it in the
+ cache: we are going to read or write directly to file. Cancel
+ the request. We can simply decrement hash_link->requests because
+ we did not release cache_lock since increasing it. So no other
+ thread can wait for our request to become released.
+ */
+ if (!--hash_link->requests)
+ {
+ /*
+ We are the only one to request this hash_link (this file/pos).
+ Free the hash_link.
+ */
+ unlink_hash(keycache, hash_link);
+ DBUG_RETURN(0);
+ }
+
+ /*
+ More requests on the hash_link. Someone tries to evict a block
+ for this hash_link (could have started before resizing started).
+ This means that the LRU ring is empty. Otherwise a block could
+ be assigned immediately. Behave like a thread that wants to
+ evict a block for this file/pos. Add to the queue of threads
+ waiting for a block. Wait until there is one assigned.
+
+ Refresh the request on the hash-link so that it cannot be reused
+ for another file/pos.
+ */
+ hash_link->requests++;
+ thread= my_thread_var;
+ thread->opt_info= (void *) hash_link;
+ link_into_queue(&keycache->waiting_for_block, thread);
+ do
+ {
+ KEYCACHE_DBUG_PRINT("find_key_block: wait",
+ ("suspend thread %ld", thread->id));
+ keycache_pthread_cond_wait(&thread->suspend,
+ &keycache->cache_lock);
+ } while (thread->next);
+ thread->opt_info= NULL;
+ /*
+ A block should now be assigned to the hash_link. But it may
+ still need to be evicted. Anyway, we should re-check the
+ situation. page_status must be set correctly.
+ */
hash_link->requests--;
- unlink_hash(keycache, hash_link);
- return 0;
+ goto restart;
+ } /* end of if (!block) */
+
+ /*
+ There is a block for this file/pos in the cache. Register a
+ request on it. This unlinks it from the LRU ring (if it is there)
+ and hence protects it against eviction (if not already in
+ eviction). We need this for returning the block to the caller, for
+ calling remove_reader() (for debugging purposes), and for calling
+ free_block(). The only case where we don't need the request is if
+ the block is in eviction. In that case we have to unregister the
+ request later.
+ */
+ reg_requests(keycache, block, 1);
+
+ if (page_status != PAGE_READ)
+ {
+ /*
+ - block not assigned to this hash_link or
+ - block assigned but not yet read from file (invalid data).
+
+ This must be a block in eviction. It will be read soon. We need
+ to wait here until this happened. Otherwise the caller could
+ access a wrong block or a block which is in read. While waiting
+ we cannot lose hash_link nor block. We have registered a request
+ on the hash_link. Everything can happen to the block but changes
+ in the hash_link -> block relationship. In other words:
+ everything can happen to the block but free or another completed
+ eviction.
+
+ Note that we bahave like a secondary requestor here. We just
+ cannot return with PAGE_WAIT_TO_BE_READ. This would work for
+ read requests and writes on dirty blocks that are not in flush
+ only. Waiting here on COND_FOR_REQUESTED works in all
+ situations.
+ */
+ DBUG_ASSERT(((block->hash_link != hash_link) &&
+ (block->status & (BLOCK_IN_EVICTION | BLOCK_IN_SWITCH))) ||
+ ((block->hash_link == hash_link) &&
+ !(block->status & BLOCK_READ)));
+ wait_on_queue(&block->wqueue[COND_FOR_REQUESTED],
&keycache->cache_lock);
+ /*
+ Here we can trust that the block has been assigned to this
+ hash_link (block->hash_link == hash_link) and read into the
+ buffer (BLOCK_READ). The worst things possible here are that the
+ block is in free (BLOCK_REASSIGNED). But the block is still
+ assigned to the hash_link. The freeing thread waits until we
+ release our request on the hash_link. The block must not be
+ again in eviction because we registered an request on it before
+ starting to wait.
+ */
+ DBUG_ASSERT(block->hash_link == hash_link);
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(!(block->status & (BLOCK_IN_EVICTION | BLOCK_IN_SWITCH)));
}
- if (!(block->status & BLOCK_IN_FLUSH))
+ /*
+ The block is in the cache. Assigned to the hash_link. Valid data.
+ Note that in case of page_st == PAGE_READ, the block can be marked
+ for eviction. In any case it can be marked for freeing.
+ */
+
+ if (!wrmode)
{
- hash_link->requests--;
+ /* A reader can just read the block. */
+ *page_st= PAGE_READ;
+ DBUG_ASSERT((hash_link->file == file) &&
+ (hash_link->diskpos == filepos) &&
+ (block->hash_link == hash_link));
+ DBUG_RETURN(block);
+ }
+
+ /*
+ This is a writer. No two writers for the same block can exist.
+ This must be assured by locks outside of the key cache.
+ */
+ DBUG_ASSERT(!(block->status & BLOCK_FOR_UPDATE) || fail_block(block));
+
+ while (block->status & BLOCK_IN_FLUSH)
+ {
+ /*
+ Wait until the block is flushed to file. Do not release the
+ request on the hash_link yet to prevent that the block is freed
+ or reassigned while we wait. While we wait, several things can
+ happen to the block, including another flush. But the block
+ cannot be reassigned to another hash_link until we release our
+ request on it. But it can be marked BLOCK_REASSIGNED from free
+ or eviction, while they wait for us to release the hash_link.
+ */
+ wait_on_queue(&block->wqueue[COND_FOR_SAVED], &keycache->cache_lock);
/*
- Remove block to invalidate the page in the block buffer
- as we are going to write directly on disk.
- Although we have an exlusive lock for the updated key part
- the control can be yieded by the current thread as we might
+ If the flush phase failed, the resize could have finished while
+ we waited here.
+ */
+ if (!keycache->in_resize)
+ {
+ remove_reader(block);
+ unreg_request(keycache, block, 1);
+ goto restart;
+ }
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(!(block->status & BLOCK_FOR_UPDATE) || fail_block(block));
+ DBUG_ASSERT(block->hash_link == hash_link);
+ }
+
+ if (block->status & BLOCK_CHANGED)
+ {
+ /*
+ We want to write a block with changed contents. If the cache
+ block size is bigger than the callers block size (e.g. MyISAM),
+ the caller may replace part of the block only. Changes of the
+ other part of the block must be preserved. Since the block has
+ not yet been selected for flush, we can still add our changes.
+ */
+ *page_st= PAGE_READ;
+ DBUG_ASSERT((hash_link->file == file) &&
+ (hash_link->diskpos == filepos) &&
+ (block->hash_link == hash_link));
+ DBUG_RETURN(block);
+ }
+
+ /*
+ This is a write request for a clean block. We do not want to have
+ new dirty blocks in the cache while resizing. We will free the
+ block and write directly to file. If the block is in eviction or
+ in free, we just let it go.
+
+ Unregister from the hash_link. This must be done before freeing
+ the block. And it must be done if not freeing the block. Because
+ we could have waited above, we need to call remove_reader(). Other
+ threads could wait for us to release our request on the hash_link.
+ */
+ remove_reader(block);
+
+ /* If the block is not in eviction and not in free, we can free it. */
+ if (!(block->status & (BLOCK_IN_EVICTION | BLOCK_IN_SWITCH |
+ BLOCK_REASSIGNED)))
+ {
+ /*
+ Free block as we are going to write directly to file.
+ Although we have an exlusive lock for the updated key part,
+ the control can be yielded by the current thread as we might
have unfinished readers of other key parts in the block
buffer. Still we are guaranteed not to have any readers
of the key part we are writing into until the block is
- removed from the cache as we set the BLOCL_REASSIGNED
+ removed from the cache as we set the BLOCK_REASSIGNED
flag (see the code below that handles reading requests).
*/
free_block(keycache, block);
- return 0;
}
- /* Wait intil the page is flushed on disk */
- hash_link->requests--;
+ else
{
-#ifdef THREAD
- struct st_my_thread_var *thread= my_thread_var;
- add_to_queue(&block->wqueue[COND_FOR_SAVED], thread);
- do
- {
- KEYCACHE_DBUG_PRINT("find_key_block: wait",
- ("suspend thread %ld", thread->id));
- keycache_pthread_cond_wait(&thread->suspend,
- &keycache->cache_lock);
- }
- while(thread->next);
-#else
- KEYCACHE_DBUG_ASSERT(0);
/*
- Given the use of "resize_in_flush", it seems impossible
- that this whole branch is ever entered in single-threaded case
- because "(wrmode && keycache->resize_in_flush)" cannot be true.
- TODO: Check this, and then put the whole branch into the
- "#ifdef THREAD" guard.
+ The block will be evicted/freed soon. Don't touch it in any way.
+ Unregister the request that we registered above.
*/
-#endif
+ unreg_request(keycache, block, 1);
+
+ /*
+ The block is still assigned to the hash_link (the file/pos that
+ we are goig to write to). Wait until the eviction/free is
+ complete. Otherwise the direct write could complete before all
+ readers are done with the block. So they could read outdated
+ data.
+
+ Comment to be deleted: This was the reason why I experienced
+ index corruptions during resize. Since I introduced the wait
+ loop here, they are gone.
+
+ Since we released our request on the hash_link, it can be reused
+ for another file/pos. Hence we cannot just check for
+ block->hash_link == hash_link. As long as the resize is
+ proceeding the block cannot be reassigned to the same file/pos
+ again. So we can terminate the loop when the block is no longer
+ assigned to this file/pos.
+ */
+ do
+ {
+ wait_on_queue(&block->wqueue[COND_FOR_SAVED],
+ &keycache->cache_lock);
+ /*
+ If the flush phase failed, the resize could have finished
+ while we waited here.
+ */
+ if (!keycache->in_resize)
+ goto restart;
+ } while (block->hash_link &&
+ (block->hash_link->file == file) &&
+ (block->hash_link->diskpos == filepos));
}
- /* Invalidate page in the block if it has not been done yet */
- if (block->status)
- free_block(keycache, block);
- return 0;
+ DBUG_RETURN(0);
}
if (page_status == PAGE_READ &&
- (block->status & (BLOCK_IN_SWITCH | BLOCK_REASSIGNED)))
+ (block->status & (BLOCK_IN_EVICTION | BLOCK_IN_SWITCH |
+ BLOCK_REASSIGNED)))
{
- /* This is a request for a page to be removed from cache */
+ /*
+ This is a request for a block to be removed from cache. The block
+ is assigned to this hash_link and contains valid data, but is
+ marked for eviction or to be freed. Possible reasons why it has
+ not yet been evicted/freed can be a flush before reassignment
+ (BLOCK_IN_SWITCH), readers of the block have not finished yet
+ (BLOCK_REASSIGNED), or the evicting thread did not yet awake after
+ the block has been selected for it (BLOCK_IN_EVICTION).
+ */
KEYCACHE_DBUG_PRINT("find_key_block",
("request for old page in block %u "
@@ -1436,43 +1991,58 @@ restart:
all others are to be suspended, then resubmitted
*/
if (!wrmode && !(block->status & BLOCK_REASSIGNED))
+ {
+ /*
+ This is a read request and the block not yet reassigned. We can
+ register our request and proceed. This unlinks the block from
+ the LRU ring and protects it against eviction.
+ */
reg_requests(keycache, block, 1);
+ }
else
{
+ /*
+ Either this is a write request for a block that is in eviction
+ or in free. We must not use it any more. Instead we must evict
+ another block. But we cannot do this before the eviction/free is
+ done. Otherwise we would find the same hash_link + block again
+ and again.
+
+ Or this is a read request for a block in eviction/free that does
+ not require a flush, but waits for readers to finish with the
+ block. We do not read this block to let the eviction/free happen
+ as soon as possible. Again we must wait so that we don't find
+ the same hash_link + block again and again.
+ */
+ DBUG_ASSERT(hash_link->requests);
hash_link->requests--;
KEYCACHE_DBUG_PRINT("find_key_block",
("request waiting for old page to be saved"));
- {
-#ifdef THREAD
- struct st_my_thread_var *thread= my_thread_var;
- /* Put the request into the queue of those waiting for the old page */
- add_to_queue(&block->wqueue[COND_FOR_SAVED], thread);
- /* Wait until the request can be resubmitted */
- do
- {
- KEYCACHE_DBUG_PRINT("find_key_block: wait",
- ("suspend thread %ld", thread->id));
- keycache_pthread_cond_wait(&thread->suspend,
- &keycache->cache_lock);
- }
- while(thread->next);
-#else
- KEYCACHE_DBUG_ASSERT(0);
- /* No parallel requests in single-threaded case */
-#endif
- }
+ wait_on_queue(&block->wqueue[COND_FOR_SAVED], &keycache->cache_lock);
KEYCACHE_DBUG_PRINT("find_key_block",
("request for old page resubmitted"));
- /* Resubmit the request */
+ /*
+ The block is no longer assigned to this hash_link.
+ Get another one.
+ */
goto restart;
}
}
else
{
- /* This is a request for a new page or for a page not to be removed */
+ /*
+ This is a request for a new block or for a block not to be removed.
+ Either
+ - block == NULL or
+ - block not assigned to this hash_link or
+ - block assigned but not yet read from file,
+ or
+ - block assigned with valid (changed or unchanged) data and
+ - it will not be reassigned/freed.
+ */
if (! block)
{
- /* No block is assigned for the page yet */
+ /* No block is assigned to the hash_link yet. */
if (keycache->blocks_unused)
{
if (keycache->free_block_list)
@@ -1481,28 +2051,42 @@ restart:
block= keycache->free_block_list;
keycache->free_block_list= block->next_used;
block->next_used= NULL;
+ DBUG_ASSERT(!block->prev_used);
+ DBUG_ASSERT(!block->next_changed);
+ DBUG_ASSERT(!block->prev_changed);
+ DBUG_ASSERT(!block->hash_link);
+ DBUG_ASSERT(!block->status);
+ DBUG_ASSERT(!block->requests);
}
else
{
/* There are some never used blocks, take first of them */
+ DBUG_ASSERT(keycache->blocks_used < (ulong) keycache->disk_blocks);
block= &keycache->block_root[keycache->blocks_used];
block->buffer= ADD_TO_PTR(keycache->block_mem,
((ulong) keycache->blocks_used*
keycache->key_cache_block_size),
byte*);
keycache->blocks_used++;
+ DBUG_ASSERT(!block->next_used);
+ DBUG_ASSERT(!block->prev_used);
+ DBUG_ASSERT(!block->next_changed);
+ DBUG_ASSERT(!block->prev_changed);
+ DBUG_ASSERT(!block->hash_link);
+ DBUG_ASSERT(!block->status);
+ DBUG_ASSERT(!block->requests);
}
keycache->blocks_unused--;
- block->status= 0;
+ block->status= BLOCK_IN_USE;
block->length= 0;
block->offset= keycache->key_cache_block_size;
block->requests= 1;
block->temperature= BLOCK_COLD;
block->hits_left= init_hits_left;
block->last_hit_time= 0;
- link_to_file_list(keycache, block, file, 0);
block->hash_link= hash_link;
hash_link->block= block;
+ link_to_file_list(keycache, block, file, 0);
page_status= PAGE_TO_BE_READ;
KEYCACHE_DBUG_PRINT("find_key_block",
("got free or never used block %u",
@@ -1510,17 +2094,26 @@ restart:
}
else
{
- /* There are no never used blocks, use a block from the LRU chain */
-
- /*
- Wait until a new block is added to the LRU chain;
- several threads might wait here for the same page,
- all of them must get the same block
+ /*
+ There are no free blocks and no never used blocks, use a block
+ from the LRU ring.
*/
#ifdef THREAD
if (! keycache->used_last)
{
+ /*
+ The LRU ring is empty. Wait until a new block is added to
+ it. Several threads might wait here for the same hash_link,
+ all of them must get the same block. While waiting for a
+ block, after a block is selected for this hash_link, other
+ threads can run first before this one awakes. During this
+ time interval other threads find this hash_link pointing to
+ the block, which is still assigned to another hash_link. In
+ this case the block is not marked BLOCK_IN_SWITCH yet, but
+ it is marked BLOCK_IN_EVICTION.
+ */
+
struct st_my_thread_var *thread= my_thread_var;
thread->opt_info= (void *) hash_link;
link_into_queue(&keycache->waiting_for_block, thread);
@@ -1533,24 +2126,50 @@ restart:
}
while (thread->next);
thread->opt_info= NULL;
+ /* Assert that block has a request registered. */
+ DBUG_ASSERT(hash_link->block->requests);
+ /* Assert that block is not in LRU ring. */
+ DBUG_ASSERT(!hash_link->block->next_used);
+ DBUG_ASSERT(!hash_link->block->prev_used);
}
#else
KEYCACHE_DBUG_ASSERT(keycache->used_last);
#endif
+ /*
+ If we waited above, hash_link->block has been assigned by
+ link_block(). Otherwise it is still NULL. In the latter case
+ we need to grab a block from the LRU ring ourselves.
+ */
block= hash_link->block;
if (! block)
{
- /*
- Take the first block from the LRU chain
- unlinking it from the chain
- */
+ /* Select the last block from the LRU ring. */
block= keycache->used_last->next_used;
block->hits_left= init_hits_left;
block->last_hit_time= 0;
- reg_requests(keycache, block,1);
hash_link->block= block;
+ /*
+ Register a request on the block. This unlinks it from the
+ LRU ring and protects it against eviction.
+ */
+ DBUG_ASSERT(!block->requests);
+ reg_requests(keycache, block,1);
+ /*
+ We do not need to set block->status|= BLOCK_IN_EVICTION here
+ because we will set block->status|= BLOCK_IN_SWITCH
+ immediately without releasing the lock in between. This does
+ also support debugging. When looking at the block, one can
+ see if the block has been selected by link_block() after the
+ LRU ring was empty, or if it was grabbed directly from the
+ LRU ring in this branch.
+ */
}
+ /*
+ If we had to wait above, there is a small chance that another
+ thread grabbed this block for the same file block already. But
+ in most cases the first condition is true.
+ */
if (block->hash_link != hash_link &&
! (block->status & BLOCK_IN_SWITCH) )
{
@@ -1565,46 +2184,117 @@ restart:
/* The block contains a dirty page - push it out of the cache */
KEYCACHE_DBUG_PRINT("find_key_block", ("block is dirty"));
+ if (block->status & BLOCK_IN_FLUSH)
+ {
+ /*
+ The block is marked for flush. If we do not wait here,
+ it could happen that we write the block, reassign it to
+ another file block, then, before the new owner can read
+ the new file block, the flusher writes the cache block
+ (wich still has the old contents) to the new file block!
+ */
+ wait_on_queue(&block->wqueue[COND_FOR_SAVED],
+ &keycache->cache_lock);
+ /*
+ The block is marked BLOCK_IN_SWITCH. It should be left
+ alone except for reading. No free, no write.
+ */
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ DBUG_ASSERT(!(block->status & (BLOCK_REASSIGNED |
+ BLOCK_CHANGED |
+ BLOCK_FOR_UPDATE)));
+ }
+ else
+ {
+ block->status|= BLOCK_IN_FLUSH | BLOCK_IN_FLUSHWRITE;
+ /*
+ BLOCK_IN_EVICTION may be true or not. Other flags must
+ have a fixed value.
+ */
+ DBUG_ASSERT((block->status & ~BLOCK_IN_EVICTION) ==
+ (BLOCK_READ | BLOCK_IN_SWITCH |
+ BLOCK_IN_FLUSH | BLOCK_IN_FLUSHWRITE |
+ BLOCK_CHANGED | BLOCK_IN_USE));
+ DBUG_ASSERT(block->hash_link);
- keycache_pthread_mutex_unlock(&keycache->cache_lock);
- /*
- The call is thread safe because only the current
- thread might change the block->hash_link value
- */
- error= my_pwrite(block->hash_link->file,
- block->buffer+block->offset,
- block->length - block->offset,
- block->hash_link->diskpos+ block->offset,
- MYF(MY_NABP | MY_WAIT_IF_FULL));
- keycache_pthread_mutex_lock(&keycache->cache_lock);
- keycache->global_cache_write++;
+ keycache_pthread_mutex_unlock(&keycache->cache_lock);
+ /*
+ The call is thread safe because only the current
+ thread might change the block->hash_link value
+ */
+ error= my_pwrite(block->hash_link->file,
+ block->buffer+block->offset,
+ block->length - block->offset,
+ block->hash_link->diskpos+ block->offset,
+ MYF(MY_NABP | MY_WAIT_IF_FULL));
+ keycache_pthread_mutex_lock(&keycache->cache_lock);
+
+ /* Block status must not have changed. */
+ DBUG_ASSERT((block->status & ~BLOCK_IN_EVICTION) ==
+ (BLOCK_READ | BLOCK_IN_SWITCH |
+ BLOCK_IN_FLUSH | BLOCK_IN_FLUSHWRITE |
+ BLOCK_CHANGED | BLOCK_IN_USE) || fail_block(block));
+ keycache->global_cache_write++;
+ }
}
block->status|= BLOCK_REASSIGNED;
+ /*
+ The block comes from the LRU ring. It must have a hash_link
+ assigned.
+ */
+ DBUG_ASSERT(block->hash_link);
if (block->hash_link)
{
/*
+ All pending requests for this page must be resubmitted.
+ This must be done before waiting for readers. They could
+ wait for the flush to complete. And we must also do it
+ after the wait. Flushers might try to free the block while
+ we wait. They would wait until the reassignment is
+ complete. Also the block status must reflect the correct
+ situation: The block is not changed nor in flush any more.
+ Note that we must not change the BLOCK_CHANGED flag
+ outside of link_to_file_list() so that it is always in the
+ correct queue and the *blocks_changed counters are
+ correct.
+ */
+ block->status&= ~(BLOCK_IN_FLUSH | BLOCK_IN_FLUSHWRITE);
+ link_to_file_list(keycache, block, block->hash_link->file, 1);
+ release_whole_queue(&block->wqueue[COND_FOR_SAVED]);
+ /*
+ The block is still assigned to its old hash_link.
Wait until all pending read requests
for this page are executed
(we could have avoided this waiting, if we had read
a page in the cache in a sweep, without yielding control)
*/
wait_for_readers(keycache, block);
+ DBUG_ASSERT(block->hash_link && block->hash_link->block ==
block &&
+ block->prev_changed);
+ /* The reader must not have been a writer. */
+ DBUG_ASSERT(!(block->status & BLOCK_CHANGED));
- /* Remove the hash link for this page from the hash table */
+ /* Wake flushers that might have found the block in between. */
+ release_whole_queue(&block->wqueue[COND_FOR_SAVED]);
+
+ /* Remove the hash link for the old file block from the hash. */
unlink_hash(keycache, block->hash_link);
- /* All pending requests for this page must be resubmitted */
-#ifdef THREAD
- if (block->wqueue[COND_FOR_SAVED].last_thread)
- release_queue(&block->wqueue[COND_FOR_SAVED]);
-#endif
+
+ /*
+ For sanity checks link_to_file_list() asserts that block
+ and hash_link refer to each other. Hence we need to assign
+ the hash_link first, but then we would not know if it was
+ linked before. Hence we would not know if to unlink it. So
+ unlink it here and call link_to_file_list(..., FALSE).
+ */
+ unlink_changed(block);
}
- link_to_file_list(keycache, block, file,
- (my_bool)(block->hash_link ? 1 : 0));
- block->status= error? BLOCK_ERROR : 0;
+ block->status= error ? BLOCK_ERROR : BLOCK_IN_USE ;
block->length= 0;
block->offset= keycache->key_cache_block_size;
block->hash_link= hash_link;
+ link_to_file_list(keycache, block, file, 0);
page_status= PAGE_TO_BE_READ;
KEYCACHE_DBUG_ASSERT(block->hash_link->block == block);
@@ -1612,7 +2302,20 @@ restart:
}
else
{
- /* This is for secondary requests for a new page only */
+ /*
+ Either (block->hash_link == hash_link),
+ or (block->status & BLOCK_IN_SWITCH).
+
+ This is for secondary requests for a new file block only.
+ Either it is already assigned to the new hash_link meanwhile
+ (if we had to wait due to empty LRU), or it is already in
+ eviction by another thread. Since this block has been
+ grabbed from the LRU ring and attached to this hash_link,
+ another thread cannot grab the same block from the LRU ring
+ anymore. If the block is in eviction already, it must become
+ attached to the same hash_link and as such destined for the
+ same file block.
+ */
KEYCACHE_DBUG_PRINT("find_key_block",
("block->hash_link: %p hash_link: %p "
"block->status: %u", block->hash_link,
@@ -1622,10 +2325,40 @@ restart:
PAGE_READ : PAGE_WAIT_TO_BE_READ);
}
}
- keycache->global_cache_read++;
+ /*
+ Comment to be deleted: keycache->global_cache_read++; moved to
+ read_block(). At this place it was counted for primary and
+ secondary requests. Better count it where the actual read is done.
+ */
}
else
{
+ /*
+ Block is not NULL. This hash_link points to a block.
+ Either
+ - block not assigned to this hash_link (yet) or
+ - block assigned but not yet read from file,
+ or
+ - block assigned with valid (changed or unchanged) data and
+ - it will not be reassigned/freed.
+
+ The first condition means hash_link points to a block in
+ eviction. This is not necessarily marked by BLOCK_IN_SWITCH yet.
+ But then it is marked BLOCK_IN_EVICTION. See the NOTE in
+ link_block(). In both cases it is destined for this hash_link
+ and its file block address. When this hash_link got its block
+ address, the block was removed from the LRU ring and cannot be
+ selected for eviction (for another hash_link) again.
+
+ Register a request on the block. This is another protection
+ against eviction.
+ */
+ DBUG_ASSERT(((block->hash_link != hash_link) &&
+ (block->status & (BLOCK_IN_EVICTION | BLOCK_IN_SWITCH))) ||
+ ((block->hash_link == hash_link) &&
+ !(block->status & BLOCK_READ)) ||
+ ((block->status & BLOCK_READ) &&
+ !(block->status & (BLOCK_IN_EVICTION | BLOCK_IN_SWITCH))));
reg_requests(keycache, block, 1);
KEYCACHE_DBUG_PRINT("find_key_block",
("block->hash_link: %p hash_link: %p "
@@ -1638,6 +2371,16 @@ restart:
}
KEYCACHE_DBUG_ASSERT(page_status != -1);
+ /* Same assert basically, but be very sure. */
+ KEYCACHE_DBUG_ASSERT(block);
+ /* Assert that block has a request and is not in LRU ring. */
+ DBUG_ASSERT(block->requests);
+ DBUG_ASSERT(!block->next_used);
+ DBUG_ASSERT(!block->prev_used);
+ /* Assert that we return the correct block. */
+ DBUG_ASSERT((page_status == PAGE_WAIT_TO_BE_READ) ||
+ ((block->hash_link->file == file) &&
+ (block->hash_link->diskpos == filepos)));
*page_st=page_status;
KEYCACHE_DBUG_PRINT("find_key_block",
("fd: %d pos: %lu block->status: %u page_status: %u",
@@ -1677,7 +2420,7 @@ restart:
portion is less than read_length, but not less than min_length.
*/
-static void read_block(KEY_CACHE *keycache __attribute__((unused)),
+static void read_block(KEY_CACHE *keycache,
BLOCK_LINK *block, uint read_length,
uint min_length, my_bool primary)
{
@@ -1689,13 +2432,27 @@ static void read_block(KEY_CACHE *keycac
if (primary)
{
/*
- This code is executed only by threads
- that submitted primary requests
+ This code is executed only by threads that submitted primary
+ requests. Until block->status contains BLOCK_READ, all other
+ request for the block become secondary requests. For a primary
+ request the block must be properly initialized.
*/
+ DBUG_ASSERT(((block->status & ~BLOCK_FOR_UPDATE) == BLOCK_IN_USE) ||
+ fail_block(block));
+ DBUG_ASSERT((block->length == 0) || fail_block(block));
+ DBUG_ASSERT((block->offset == keycache->key_cache_block_size) ||
+ fail_block(block));
+ DBUG_ASSERT((block->requests > 0) || fail_block(block));
KEYCACHE_DBUG_PRINT("read_block",
("page to be read by primary request"));
+ /*
+ Comment to be deleted: keycache->global_cache_read++; moved here
+ from find_key_block(). At this place it counts primary requests
+ only.
+ */
+ keycache->global_cache_read++;
/* Page is not in buffer yet, is to be read from disk */
keycache_pthread_mutex_unlock(&keycache->cache_lock);
/*
@@ -1705,47 +2462,51 @@ static void read_block(KEY_CACHE *keycac
got_length= my_pread(block->hash_link->file, block->buffer,
read_length, block->hash_link->diskpos, MYF(0));
keycache_pthread_mutex_lock(&keycache->cache_lock);
+ /*
+ The block can now have been marked for free (in case of
+ FLUSH_RELEASE). Otherwise the state must be unchanged.
+ */
+ DBUG_ASSERT(((block->status & ~(BLOCK_REASSIGNED |
+ BLOCK_FOR_UPDATE)) == BLOCK_IN_USE) ||
+ fail_block(block));
+ DBUG_ASSERT((block->length == 0) || fail_block(block));
+ DBUG_ASSERT((block->offset == keycache->key_cache_block_size) ||
+ fail_block(block));
+ DBUG_ASSERT((block->requests > 0) || fail_block(block));
+
if (got_length < min_length)
block->status|= BLOCK_ERROR;
else
{
- block->status= BLOCK_READ;
+ /* Comment to be deleted: Do not kill other block status flags. */
+ block->status|= BLOCK_READ;
block->length= got_length;
+ /*
+ Do not set block->offset here. If this block is marked
+ BLOCK_CHANGED later, we want to flush only the modified part. So
+ only a writer may set block->offset down from
+ keycache->key_cache_block_size.
+ */
}
KEYCACHE_DBUG_PRINT("read_block",
("primary request: new page in cache"));
/* Signal that all pending requests for this page now can be processed */
-#ifdef THREAD
- if (block->wqueue[COND_FOR_REQUESTED].last_thread)
- release_queue(&block->wqueue[COND_FOR_REQUESTED]);
-#endif
+ release_whole_queue(&block->wqueue[COND_FOR_REQUESTED]);
}
else
{
/*
- This code is executed only by threads
- that submitted secondary requests
+ This code is executed only by threads that submitted secondary
+ requests. At this point it could happen that the cache block is
+ not yet assigned to the hash_link for the requested file block.
+ But at awake from the wait this should be the case. Unfortunately
+ we cannot assert this here because we do not know the hash_link
+ for the requested file block nor the file and position. So we have
+ to assert this in the caller.
*/
KEYCACHE_DBUG_PRINT("read_block",
("secondary request waiting for new page to be read"));
- {
-#ifdef THREAD
- struct st_my_thread_var *thread= my_thread_var;
- /* Put the request into a queue and wait until it can be processed */
- add_to_queue(&block->wqueue[COND_FOR_REQUESTED], thread);
- do
- {
- KEYCACHE_DBUG_PRINT("read_block: wait",
- ("suspend thread %ld", thread->id));
- keycache_pthread_cond_wait(&thread->suspend,
- &keycache->cache_lock);
- }
- while (thread->next);
-#else
- KEYCACHE_DBUG_ASSERT(0);
- /* No parallel requests in single-threaded case */
-#endif
- }
+ wait_on_queue(&block->wqueue[COND_FOR_REQUESTED],
&keycache->cache_lock);
KEYCACHE_DBUG_PRINT("read_block",
("secondary request: new page in cache"));
}
@@ -1786,32 +2547,59 @@ byte *key_cache_read(KEY_CACHE *keycache
uint block_length __attribute__((unused)),
int return_buffer __attribute__((unused)))
{
+ my_bool incremented= FALSE;
int error=0;
- uint offset= 0;
byte *start= buff;
DBUG_ENTER("key_cache_read");
DBUG_PRINT("enter", ("fd: %u pos: %lu length: %u",
(uint) file, (ulong) filepos, length));
- if (keycache->can_be_used)
+ if (keycache->key_cache_inited)
{
/* Key cache is used */
reg1 BLOCK_LINK *block;
uint read_length;
+ uint offset;
uint status;
int page_st;
+ /*
+ When the key cache is once initialized, we use the cache_lock to
+ reliably distinguish the cases of normal operation, resizing, and
+ disabled cache. We always increment and decrement
+ 'cnt_for_resize_op' so that a resizer can wait for pending I/O.
+ */
+ keycache_pthread_mutex_lock(&keycache->cache_lock);
+ /*
+ Cache resizing has two phases: Flushing and re-initializing. In
+ the flush phase read requests are allowed to bypass the cache for
+ blocks not in the cache. find_key_block() returns NULL in this
+ case.
+
+ After the flush phase new I/O requests must wait until the
+ re-initialization is done. The re-initialization can be done only
+ if no I/O request is in progress. The reason is that
+ key_cache_block_size can change. With enabled cache, I/O is done
+ in chunks of key_cache_block_size. Every chunk tries to use a
+ cache block first. If the block size changes in the middle, a
+ block could be missed and old data could be read.
+ */
+ while (keycache->in_resize && !keycache->resize_in_flush)
+ wait_on_queue(&keycache->resize_queue, &keycache->cache_lock);
+ /* Register the I/O for the next resize. */
+ inc_counter_for_resize_op(keycache);
+ incremented= TRUE;
+ /* Requested data may not always be aligned to cache blocks. */
offset= (uint) (filepos & (keycache->key_cache_block_size-1));
/* Read data in key_cache_block_size increments */
do
{
- keycache_pthread_mutex_lock(&keycache->cache_lock);
+ /* Cache could be disabled in a later iteration. */
if (!keycache->can_be_used)
- {
- keycache_pthread_mutex_unlock(&keycache->cache_lock);
goto no_key_cache;
- }
+ /* Start reading at the beginning of the cache block. */
filepos-= offset;
+ /* Do not read beyond the end of the cache block. */
read_length= length;
set_if_smaller(read_length, keycache->key_cache_block_size-offset);
KEYCACHE_DBUG_ASSERT(read_length > 0);
@@ -1821,34 +2609,64 @@ byte *key_cache_read(KEY_CACHE *keycache
return_buffer=0;
#endif
- inc_counter_for_resize_op(keycache);
+ /* Request the cache block that matches file/pos. */
keycache->global_cache_r_requests++;
block=find_key_block(keycache, file, filepos, level, 0, &page_st);
- if (block->status != BLOCK_ERROR && page_st != PAGE_READ)
- {
- /* The requested page is to be read into the block buffer */
- read_block(keycache, block,
- keycache->key_cache_block_size, read_length+offset,
- (my_bool)(page_st == PAGE_TO_BE_READ));
- }
- else if (! (block->status & BLOCK_ERROR) &&
- block->length < read_length + offset)
+ if (!block)
{
/*
- Impossible if nothing goes wrong:
- this could only happen if we are using a file with
- small key blocks and are trying to read outside the file
+ This happens only for requests submitted during key cache
+ resize. The block is not in the cache and shall not go in.
+ Read directly from file.
*/
- my_errno= -1;
- block->status|= BLOCK_ERROR;
+ keycache->global_cache_read++;
+ keycache_pthread_mutex_unlock(&keycache->cache_lock);
+ if (my_pread(file, (byte*) buff, read_length,
+ filepos + offset, MYF(MY_NABP)))
+ {
+ error= 1;
+ }
+ keycache_pthread_mutex_lock(&keycache->cache_lock);
+ goto next_block;
+ }
+ if (block->status != BLOCK_ERROR)
+ {
+ if (page_st != PAGE_READ)
+ {
+ /* The requested page is to be read into the block buffer */
+ read_block(keycache, block,
+ keycache->key_cache_block_size, read_length+offset,
+ (my_bool)(page_st == PAGE_TO_BE_READ));
+ /*
+ A secondary request must now have the block assigned to the
+ requested file block. It does not hurt to check it for
+ primary requests too.
+ */
+ DBUG_ASSERT(keycache->can_be_used);
+ DBUG_ASSERT(block->hash_link->file == file);
+ DBUG_ASSERT(block->hash_link->diskpos == filepos);
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
+ }
+ else if (block->length < read_length + offset)
+ {
+ /*
+ Impossible if nothing goes wrong:
+ this could only happen if we are using a file with
+ small key blocks and are trying to read outside the file
+ */
+ my_errno= -1;
+ block->status|= BLOCK_ERROR;
+ }
}
+ /* block status may have added BLOCK_ERROR in the above 'if'. */
if (! ((status= block->status) & BLOCK_ERROR))
{
#ifndef THREAD
if (! return_buffer)
#endif
{
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
#if !defined(SERIALIZED_READ_FROM_CACHE)
keycache_pthread_mutex_unlock(&keycache->cache_lock);
#endif
@@ -1861,44 +2679,63 @@ byte *key_cache_read(KEY_CACHE *keycache
#if !defined(SERIALIZED_READ_FROM_CACHE)
keycache_pthread_mutex_lock(&keycache->cache_lock);
+ DBUG_ASSERT(block->status & (BLOCK_READ | BLOCK_IN_USE));
#endif
}
}
remove_reader(block);
+
/*
- Link the block into the LRU chain
- if it's the last submitted request for the block
+ Link the block into the LRU ring if it's the last submitted
+ request for the block. This enables eviction for the block.
*/
unreg_request(keycache, block, 1);
- dec_counter_for_resize_op(keycache);
-
- keycache_pthread_mutex_unlock(&keycache->cache_lock);
-
if (status & BLOCK_ERROR)
- DBUG_RETURN((byte *) 0);
+ {
+ error= 1;
+ break;
+ }
#ifndef THREAD
/* This is only true if we where able to read everything in one block */
if (return_buffer)
DBUG_RETURN(block->buffer);
#endif
+ next_block:
buff+= read_length;
filepos+= read_length+offset;
offset= 0;
} while ((length-= read_length));
- DBUG_RETURN(start);
+ goto end;
}
-no_key_cache: /* Key cache is not used */
+no_key_cache:
+ /* Key cache is not used */
- /* We can't use mutex here as the key cache may not be initialized */
+ if (keycache->key_cache_inited && !incremented)
+ {
+ keycache_pthread_mutex_lock(&keycache->cache_lock);
+ inc_counter_for_resize_op(keycache);
+ incremented= TRUE;
+ }
keycache->global_cache_r_requests++;
keycache->global_cache_read++;
- if (my_pread(file, (byte*) buff, length, filepos+offset, MYF(MY_NABP)))
+ if (incremented)
+ keycache_pthread_mutex_unlock(&keycache->cache_lock);
+ if (my_pread(file, (byte*) buff, length, filepos, MYF(MY_NABP)))
error= 1;
+ if (incremented)
+ keycache_pthread_mutex_lock(&keycache->cache_lock);
+
+end:
+ if (incremented)
+ {
+ dec_counter_for_resize_op(keycache);
+ keycache_pthread_mutex_unlock(&keycache->cache_lock);
+ }
DBUG_RETURN(error ? (byte*) 0 : start);
}
@@ -1927,92 +2764,219 @@ int key_cache_insert(KEY_CACHE *keycache
File file, my_off_t filepos, int level,
byte *buff, uint length)
{
+ int error= 0;
DBUG_ENTER("key_cache_insert");
DBUG_PRINT("enter", ("fd: %u pos: %lu length: %u",
(uint) file,(ulong) filepos, length));
- if (keycache->can_be_used)
+ if (keycache->key_cache_inited)
{
/* Key cache is used */
reg1 BLOCK_LINK *block;
uint read_length;
- int page_st;
- int error;
uint offset;
+ int page_st;
+ my_bool incremented= FALSE;
+ /*
+ When the keycache is once initialized, we use the cache_lock to
+ reliably distinguish the cases of normal operation, resizing, and
+ disabled cache. We always increment and decrement
+ 'cnt_for_resize_op' so that a resizer can wait for pending I/O.
+ */
+ keycache_pthread_mutex_lock(&keycache->cache_lock);
+ /*
+ We do not load index data into a disabled cache nor into an
+ ongoing resize.
+ */
+ if (!keycache->can_be_used || keycache->in_resize)
+ goto no_key_cache;
+ /* Register the pseudo I/O for the next resize. */
+ inc_counter_for_resize_op(keycache);
+ incremented= TRUE;
+ /* Loaded data may not always be aligned to cache blocks. */
offset= (uint) (filepos & (keycache->key_cache_block_size-1));
+ /* Load data in key_cache_block_size increments. */
do
{
- keycache_pthread_mutex_lock(&keycache->cache_lock);
- if (!keycache->can_be_used)
- {
- keycache_pthread_mutex_unlock(&keycache->cache_lock);
- DBUG_RETURN(0);
- }
- /* Read data into key cache from buff in key_cache_block_size incr. */
+ /* Cache could be disabled or resizing in a later iteration. */
+ if (!keycache->can_be_used || keycache->in_resize)
+ goto no_key_cache;
+ /* Start loading at the beginning of the cache block. */
filepos-= offset;
+ /* Do not load beyond the end of the cache block. */
read_length= length;
set_if_smaller(read_length, keycache->key_cache_block_size-offset);
KEYCACHE_DBUG_ASSERT(read_length > 0);
- inc_counter_for_resize_op(keycache);
+ /* The block has been read by the caller already. */
+ keycache->global_cache_read++;
+ /* Request the cache block that matches file/pos. */
keycache->global_cache_r_requests++;
block= find_key_block(keycache, file, filepos, level, 0, &page_st);
- if (block->status != BLOCK_ERROR && page_st != PAGE_READ)
+ if (!block)
{
- /* The requested page is to be read into the block buffer */
-#if !defined(SERIALIZED_READ_FROM_CACHE)
- keycache_pthread_mutex_unlock(&keycache->cache_lock);
/*
- Here other threads may step in and register as secondary readers.
- They will register in block->wqueue[COND_FOR_REQUESTED].
+ This happens only for requests submitted during key cache
+ resize. The block is not in the cache and shall not go in.
+ Stop loading index data.
*/
-#endif
+ goto no_key_cache;
+ }
+ if (block->status != BLOCK_ERROR)
+ {
+ if ((page_st == PAGE_WAIT_TO_BE_READ) ||
+ ((page_st == PAGE_TO_BE_READ) &&
+ (offset || (read_length < keycache->key_cache_block_size))))
+ {
+ /*
