List:Maria Storage Engine« Previous MessageNext Message »
From:Michael Widenius Date:December 2 2008 10:03pm
Subject:bzr commit into MySQL/Maria:mysql-maria branch (monty:2701) WL#3262
View as plain text  
#At bzr+ssh://bk-internal.mysql.com/bzrroot/server/mysql-maria/

 2701 Michael Widenius	2008-12-03
      WL#3262 add mutex lock order checking to safemutex (also called safe_mutex_deadlock_detector)
      This writes a warning on stderr if one uses mutex in different order,
      like if one in one case would lock mutex in the order A,B and in another case
      would lock mutex in the order B,A
      This is inspired by and loosely based on the LOCKDEP patch by Jonas
      Wrong mutex order is either fixed or mutex are marked with MYF_NO_DEADLOCK_DETECTION
      if used inconsistently (need to be fixed by server team)
modified:
  KNOWN_BUGS.txt
  include/hash.h
  include/my_pthread.h
  mysys/Makefile.am
  mysys/hash.c
  mysys/my_init.c
  mysys/my_pthread.c
  mysys/my_sleep.c
  mysys/my_thr_init.c
  mysys/mysys_priv.h
  mysys/thr_mutex.c
  sql/event_queue.cc
  sql/event_scheduler.cc
  sql/events.cc
  sql/ha_ndbcluster_binlog.cc
  sql/log.cc
  sql/mysqld.cc
  sql/protocol.cc
  sql/rpl_mi.cc
  sql/set_var.cc
  sql/sp_cache.cc
  sql/sp_cache.h
  sql/sql_class.cc
  sql/sql_insert.cc
  sql/sql_show.cc
  storage/innobase/handler/ha_innodb.cc
  storage/maria/ha_maria.cc
  storage/maria/ha_maria.h
  storage/maria/ma_close.c
  storage/maria/ma_key.c
  storage/maria/ma_loghandler.c
  storage/maria/ma_state.c
  storage/maria/ma_test1.c
  storage/maria/ma_test2.c

per-file messages:
  KNOWN_BUGS.txt
    Added information that one need to dump and restore Maria tables
  include/hash.h
    Added prototype function for walking over all elements in a hash
  include/my_pthread.h
    Added my_pthread_mutex_init() and my_pthread_mutex_lock(); These should be used if one wants to disable mutex order checking.
    Changed names of the nonposix mutex_init functions to not conflict with my_phread_mutex_init()
    Added and extended structures for mutex deadlock detection.
    New arguments to sage_mutex_init() and safe_mutex_lock() to allow one to disable mutex order checking.
    Added variable 'safe_mutex_deadlock_detector' to enable/disable deadlock detection for all pthread_mutex_init()
  mysys/Makefile.am
    Added cleaning of test files
    Added test_thr_mutex
  mysys/hash.c
    Added hash_iterate() to iterate over all elements in a hash
    More comments
  mysys/my_init.c
    Added calls to destory all mutex uses by mysys()
    Added waiting for threads to end before calling TERMINATE() to list not freed memory
  mysys/my_pthread.c
    Changed names to free my_pthread_mutex_init() for mutex-lock-order-checking
  mysys/my_sleep.c
    Fixed too long wait if using 1000000L as argument
  mysys/my_thr_init.c
    Mark THR_LOCK_threads and THR_LOCK_malloc to not have mutex deadlock detection.
    (We can't have it enabled for this as these are internal mutex used by the detector  
    Call my_thread_init() early as we need thread specific variables enabled for the following pthread_mutex_init()
    Move code to wait for threads to end to my_wait_for_other_threads_to_die()
    Don't destroy mutex and conditions unless all threads have died
    Added my_thread_destroy_mutex() to destroy all mutex used by the mysys thread system
    Name the thread specific mutex as "mysys_var->mutex"
    Added my_thread_var_mutex_in_use() to return pointer to mutex in use or 0 if thread variables are not initialized
  mysys/mysys_priv.h
    Added prototypes for functions used internally with mutex-wrong-usage detection
  mysys/thr_mutex.c
    Added runtime detection of mutex used in conflicting order
    See WL#3262 or test_thr_mutex.c for examples
    The base idea is for each mutex have two hashes:
    - mutex->locked_mutex points to all mutex used after this one
    - mutex->used_mutex points to all mutex which has this mutex in it's mutex->locked_mutex
    There is a wrong mutex order if any mutex currently locked before this mutex is in the mutex->locked_mutex hash
  sql/event_queue.cc
    Mark mutex used inconsistently (need to be fixed by server team)
  sql/event_scheduler.cc
    Declare the right order to take the mutex
  sql/events.cc
    Mark mutex used inconsistently (need to be fixed by server team)
  sql/ha_ndbcluster_binlog.cc
    Mark mutex used inconsistently (need to be fixed by server team)
  sql/log.cc
    Mark mutex used inconsistently (need to be fixed by server team)
  sql/mysqld.cc
    Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
    This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
    Added call to free sp cache.
    Add destruction of LOCK_server_started and COND_server_started.
    Added register_mutex_order() function to register in which order mutex should be taken
    (to initiailize mutex_deadlock_detector).
    Added option to turn off safe_mutex_deadlock_detector
  sql/protocol.cc
    Fixed wrong argument to DBUG_PRINT (found by valgrind)
  sql/rpl_mi.cc
    Mark mutex used inconsistently (need to be fixed by server team)
  sql/set_var.cc
    Remove wrong locking of LOCK_global_system_variables when reading and setting log variables
    (would cause inconsistent mutex order).
    Update global variables outside of logger.unlock() as LOCK_global_system_variables has to be taken before logger locks
    Reviewed by gluh
  sql/sp_cache.cc
    Added function to destroy mutex used by sp cache
  sql/sp_cache.h
    Added function to destroy mutex used by sp cache
  sql/sql_class.cc
    Use pthread_mutex_trylock instead of pthread_mutex_unlock() when sending kill signal to thread
    This is needed to avoid wrong mutex order as normally one takes 'current_mutex' before mysys_var->mutex.
    Register order in which LOCK_delete and mysys_var->mutex is taken
  sql/sql_insert.cc
    Give a name for Delayed_insert::mutex
    Mark mutex used inconsistently (need to be fixed by server team)
    Move closing of tables outside of di->mutex (to avoid wrong mutex order)
  sql/sql_show.cc
    Don't keep LOCK_global_system_variables locked over value->show_type() as this leads to wrong mutex order
  storage/innobase/handler/ha_innodb.cc
    Disable safe_muted_deadlock_detector for innobase intern mutex (to speed up page cache initialization)
  storage/maria/ha_maria.cc
    Added flag to ha_maria::info() to signal if we need to lock table share or not.
    This is needed to avoid locking mutex in wrong order
  storage/maria/ha_maria.h
    Added flag to ha_maria::info() to signal if we need to lock table share or not.
  storage/maria/ma_close.c
    Destroy key_del_lock
    Simplify freeing ftparser_param
  storage/maria/ma_key.c
    Better comment
  storage/maria/ma_loghandler.c
    Mark mutex used inconsistently (need to be fixed by sanja)
  storage/maria/ma_state.c
    More comments
  storage/maria/ma_test1.c
    Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
  storage/maria/ma_test2.c
    Ensure that safe_mutex_deadlock_detector is always on (should be, this is just for safety)
=== modified file 'KNOWN_BUGS.txt'
--- a/KNOWN_BUGS.txt	2008-07-09 12:07:38 +0000
+++ b/KNOWN_BUGS.txt	2008-12-02 22:02:52 +0000
@@ -20,14 +20,29 @@ If you have found a bug that is not list
 http://bugs.mysql.com/ so that we can either fix it for next release
 or in the worst case add it here for others to know!
 
+IMPORTANT:
+
+If you have been using a MySQL-5.1-Maria-alpha build and upgrading to
+MySQL-5.1-Maria-beta you MUST run maria_chk --recover on all your
+Maria tables.  This is because we made an incompatible change of how
+transaction id is stored and old transaction id's must be reset!
+
+cd mysql-data-directory
+maria_chk --recover */*.MAI
+
+As the Maria-1.5 engine is now in beta we will do our best to not
+introduce any incompatible changes in the data format for the Maria
+tables; If this would be ever be needed, we will, if possible, support
+both the old and the new version to make upgrades as easy as possible.
 
 Known bugs that we are working on and will be fixed shortly
 ===========================================================
 
-- We have some instabilities in log writing that is under investigatation
+- We have some time ago some instabilities in log writing that is was 
+  under investigation but we haven't been able to repeat in a while.
   This causes mainly assert to triggers in the code and sometimes
   the log handler doesn't start up after restart.
-  Most of this should now be fixed...
+  Most of this should now be fixed.
 
 - INSERT on a duplicate key against a key inserted by another connection
   that has not yet ended will give a duplicate key error instead of

=== modified file 'include/hash.h'
--- a/include/hash.h	2008-03-18 09:45:36 +0000
+++ b/include/hash.h	2008-12-02 22:02:52 +0000
@@ -32,6 +32,7 @@ extern "C" {
 
 typedef uchar *(*hash_get_key)(const uchar *,size_t*,my_bool);
 typedef void (*hash_free_key)(void *);
+typedef my_bool (*hash_walk_action)(void *,void *);
 
 typedef struct st_hash {
   size_t key_offset,key_length;		/* Length of key if const length */
@@ -66,6 +67,7 @@ my_bool hash_delete(HASH *hash,uchar *re
 my_bool hash_update(HASH *hash,uchar *record,uchar *old_key,size_t old_key_length);
 void hash_replace(HASH *hash, HASH_SEARCH_STATE *state, uchar *new_row);
 my_bool hash_check(HASH *hash);			/* Only in debug library */
+my_bool hash_iterate(HASH *hash, hash_walk_action action, void *argument);
 
 #define hash_clear(H) bzero((char*) (H),sizeof(*(H)))
 #define hash_inited(H) ((H)->array.buffer != 0)

=== modified file 'include/my_pthread.h'
--- a/include/my_pthread.h	2008-08-31 15:20:20 +0000
+++ b/include/my_pthread.h	2008-12-02 22:02:52 +0000
@@ -239,13 +239,13 @@ int my_sigwait(const sigset_t *set,int *
 
 #ifdef HAVE_NONPOSIX_PTHREAD_MUTEX_INIT
 #ifndef SAFE_MUTEX
-#define pthread_mutex_init(a,b) my_pthread_mutex_init((a),(b))
-extern int my_pthread_mutex_init(pthread_mutex_t *mp,
-				 const pthread_mutexattr_t *attr);
+#define pthread_mutex_init(a,b) my_pthread_mutex_noposix_init((a),(b))
+extern int my_pthread_mutex_noposix_init(pthread_mutex_t *mp,
+                                         const pthread_mutexattr_t *attr);
 #endif /* SAFE_MUTEX */
-#define pthread_cond_init(a,b) my_pthread_cond_init((a),(b))
-extern int my_pthread_cond_init(pthread_cond_t *mp,
-				const pthread_condattr_t *attr);
+#define pthread_cond_init(a,b) my_pthread_cond_noposix_init((a),(b))
+extern int my_pthread_cond_noposix_init(pthread_cond_t *mp,
+                                        const pthread_condattr_t *attr);
 #endif /* HAVE_NONPOSIX_PTHREAD_MUTEX_INIT */
 
 #if defined(HAVE_SIGTHREADMASK) && !defined(HAVE_PTHREAD_SIGMASK)
@@ -449,18 +449,33 @@ int my_pthread_mutex_trylock(pthread_mut
 #if defined(__NETWARE__) && !defined(SAFE_MUTEX_DETECT_DESTROY)
 #define SAFE_MUTEX_DETECT_DESTROY
 #endif
+struct st_hash;
 
 typedef struct st_safe_mutex_t
 {
   pthread_mutex_t global,mutex;
   const char *file, *name;
   uint line,count;
+  myf create_flags, active_flags;
+  ulong id;
   pthread_t thread;
+  struct st_hash *locked_mutex, *used_mutex;
+  struct st_safe_mutex_t *prev, *next;
 #ifdef SAFE_MUTEX_DETECT_DESTROY
   struct st_safe_mutex_info_t *info;	/* to track destroying of mutexes */
 #endif
 } safe_mutex_t;
 
+typedef struct st_safe_mutex_deadlock_t
+{
+  const char *file, *name;
+  safe_mutex_t *mutex;
+  uint line;
+  ulong count;
+  ulong id;
+  my_bool warning_only;
+} safe_mutex_deadlock_t;
+
 #ifdef SAFE_MUTEX_DETECT_DESTROY
 /*
   Used to track the destroying of mutexes. This needs to be a seperate
@@ -478,8 +493,10 @@ typedef struct st_safe_mutex_info_t
 #endif /* SAFE_MUTEX_DETECT_DESTROY */
 
 int safe_mutex_init(safe_mutex_t *mp, const pthread_mutexattr_t *attr,
-                    const char *file, uint line, const char *name);
-int safe_mutex_lock(safe_mutex_t *mp, my_bool try_lock, const char *file, uint line);
+                    const char *name, myf my_flags,
+                    const char *file, uint line);
+int safe_mutex_lock(safe_mutex_t *mp, myf my_flags, const char *file,
+                    uint line);
 int safe_mutex_unlock(safe_mutex_t *mp,const char *file, uint line);
 int safe_mutex_destroy(safe_mutex_t *mp,const char *file, uint line);
 int safe_cond_wait(pthread_cond_t *cond, safe_mutex_t *mp,const char *file,
@@ -488,8 +505,12 @@ int safe_cond_timedwait(pthread_cond_t *
 			struct timespec *abstime, const char *file, uint line);
 void safe_mutex_global_init(void);
 void safe_mutex_end(FILE *file);
+void safe_mutex_free_deadlock_data(safe_mutex_t *mp);
 
 	/* Wrappers if safe mutex is actually used */
+#define MYF_TRY_LOCK              1
+#define MYF_NO_DEADLOCK_DETECTION 2
+
 #ifdef SAFE_MUTEX
 #undef pthread_mutex_init
 #undef pthread_mutex_lock
@@ -501,13 +522,15 @@ void safe_mutex_end(FILE *file);
 #undef pthread_cond_wait
 #undef pthread_cond_timedwait
 #undef pthread_mutex_trylock
-#define pthread_mutex_init(A,B) safe_mutex_init((A),(B),__FILE__,__LINE__,#A)
-#define pthread_mutex_lock(A) safe_mutex_lock((A), FALSE, __FILE__, __LINE__)
+#define my_pthread_mutex_init(A,B,C,D) safe_mutex_init((A),(B),(C),(D),__FILE__,__LINE__)
+#define pthread_mutex_init(A,B) safe_mutex_init((A),(B),#A,0,__FILE__,__LINE__)
+#define pthread_mutex_lock(A) safe_mutex_lock((A), 0, __FILE__, __LINE__)
+#define my_pthread_mutex_lock(A,B) safe_mutex_lock((A), (B), __FILE__, __LINE__)
 #define pthread_mutex_unlock(A) safe_mutex_unlock((A),__FILE__,__LINE__)
 #define pthread_mutex_destroy(A) safe_mutex_destroy((A),__FILE__,__LINE__)
 #define pthread_cond_wait(A,B) safe_cond_wait((A),(B),__FILE__,__LINE__)
 #define pthread_cond_timedwait(A,B,C) safe_cond_timedwait((A),(B),(C),__FILE__,__LINE__)
-#define pthread_mutex_trylock(A) safe_mutex_lock((A), TRUE, __FILE__, __LINE__)
+#define pthread_mutex_trylock(A) safe_mutex_lock((A), MYF_TRY_LOCK, __FILE__, __LINE__)
 #define pthread_mutex_t safe_mutex_t
 #define safe_mutex_assert_owner(mp) \
           DBUG_ASSERT((mp)->count > 0 && \
@@ -516,8 +539,11 @@ void safe_mutex_end(FILE *file);
           DBUG_ASSERT(! (mp)->count || \
                       ! pthread_equal(pthread_self(), (mp)->thread))
 #else
+#define my_pthread_mutex_init(A,B,C,D) pthread_mutex_init((A),(B))
+#define my_pthread_mutex_lock(A,B) pthread_mutex_lock(A)
 #define safe_mutex_assert_owner(mp)
 #define safe_mutex_assert_not_owner(mp)
+#define safe_mutex_free_deadlock_data(mp)
 #endif /* SAFE_MUTEX */
 
 #if defined(MY_PTHREAD_FASTMUTEX) && !defined(SAFE_MUTEX)
@@ -685,6 +711,7 @@ struct st_my_thread_var
   void *opt_info;
   uint  lock_type; /* used by conditional release the queue */
   void  *stack_ends_here;
+  safe_mutex_t *mutex_in_use;
 #ifndef DBUG_OFF
   void *dbug;
   char name[THREAD_NAME_SIZE+1];
@@ -693,7 +720,9 @@ struct st_my_thread_var
 
 extern struct st_my_thread_var *_my_thread_var(void) __attribute__ ((const));
 extern void **my_thread_var_dbug();
+extern safe_mutex_t **my_thread_var_mutex_in_use();
 extern uint my_thread_end_wait_time;
+extern my_bool safe_mutex_deadlock_detector;
 #define my_thread_var (_my_thread_var())
 #define my_errno my_thread_var->thr_errno
 /*

=== modified file 'mysys/Makefile.am'
--- a/mysys/Makefile.am	2008-10-11 08:27:03 +0000
+++ b/mysys/Makefile.am	2008-12-02 22:02:52 +0000
@@ -79,6 +79,13 @@ DEFS =			-DDEFAULT_BASEDIR=\"$(prefix)\"
 # I hope this always does the right thing. Otherwise this is only test programs
 FLAGS=$(DEFS) $(INCLUDES) $(CPPFLAGS) $(CFLAGS) @NOINST_LDFLAGS@
 
+CLEANFILES =		test_bitmap$(EXEEXT) test_priority_queue$(EXEEXT) \
+			test_thr_alarm$(EXEEXT) test_thr_lock$(EXEEXT) \
+			test_vsnprintf$(EXEEXT) test_io_cache$(EXEEXT) \
+			test_dir$(EXEEXT) test_charset$(EXEEXT) \
+			testhash$(EXEEXT) test_gethwaddr$(EXEEXT) \
+			test_base64$(EXEEXT) test_thr_mutex$(EXEEXT)
+
 #
 # The CP .. RM stuff is to avoid problems with some compilers (like alpha ccc)
 # which automaticly removes the object files you use to compile a final program
@@ -129,6 +136,9 @@ test_base64$(EXEEXT): base64.c $(LIBRARI
 	$(LINK) $(FLAGS) -DMAIN  ./test_base64.c $(LDADD) $(LIBS)
 	$(RM) -f ./test_base64.c
 
+test_thr_mutex$(EXEEXT): test_thr_mutex.c $(LIBRARIES)
+	$(LINK) $(FLAGS) $(srcdir)/test_thr_mutex.c $(LDADD) $(LIBS)
+
 
 # Don't update the files from bitkeeper
 %::SCCS/s.%

=== modified file 'mysys/hash.c'
--- a/mysys/hash.c	2008-05-29 18:39:25 +0000
+++ b/mysys/hash.c	2008-12-02 22:02:52 +0000
@@ -304,7 +304,13 @@ static int hashcmp(const HASH *hash, HAS
 }
 
 
-	/* Write a hash-key to the hash-index */
+/**
+   Write a hash-key to the hash-index
+
+   @return
+   @retval  0  ok
+   @retval  1  Duplicate key or out of memory
+*/
 
 my_bool my_hash_insert(HASH *info,const uchar *record)
 {
@@ -443,11 +449,21 @@ my_bool my_hash_insert(HASH *info,const 
 }
 
 
-/******************************************************************************
-** Remove one record from hash-table. The record with the same record
-** ptr is removed.
-** if there is a free-function it's called for record if found
-******************************************************************************/
+/**
+   Remove one record from hash-table.
+
+   @fn    hash_delete()
+   @param hash		Hash tree
+   @param record	Row to be deleted
+
+   @notes
+   The record with the same record ptr is removed.
+   If there is a free-function it's called if record was found.
+
+   @return
+   @retval  0  ok
+   @retval  1 Record not found
+*/
 
 my_bool hash_delete(HASH *hash,uchar *record)
 {
@@ -656,6 +672,37 @@ void hash_replace(HASH *hash, HASH_SEARC
 }
 
 
+/**
+   Iterate over all elements in hash and call function with the element
+
+   @param hash     hash array
+   @param action   function to call for each argument
+   @param argument second argument for call to action
+
+   @notes
+   If one of functions calls returns 1 then the iteration aborts
+
+   @retval 0  ok
+   @retval 1  iteration aborted becasue action returned 1
+*/
+
+my_bool hash_iterate(HASH *hash, hash_walk_action action, void *argument)
+{
+  uint records, i;
+  HASH_LINK *data;
+
+  records= hash->records;
+  data= dynamic_element(&hash->array,0,HASH_LINK*);
+
+  for (i= 0 ; i < records ; i++)
+  {
+    if ((*action)(data[i].data, argument))
+      return 1;
+  }
+  return 0;
+}
+
+
 #ifndef DBUG_OFF
 
 my_bool hash_check(HASH *hash)

=== modified file 'mysys/my_init.c'
--- a/mysys/my_init.c	2008-04-28 16:24:05 +0000
+++ b/mysys/my_init.c	2008-12-02 22:02:52 +0000
@@ -165,6 +165,9 @@ void my_end(int infoflag)
   free_charsets();
   my_error_unregister_all();
   my_once_free();
+#ifdef THREAD
+  my_thread_destroy_mutex();
+#endif
 
   if ((infoflag & MY_GIVE_INFO) || print_info)
   {
@@ -195,6 +198,8 @@ Voluntary context switches %ld, Involunt
     fprintf(info_file,"\nRun time: %.1f\n",(double) clock()/CLOCKS_PER_SEC);
 #endif
 #if defined(SAFEMALLOC)
+    /* Wait for other threads to free mysys_var */
+    (void) my_wait_for_other_threads_to_die(1);
     TERMINATE(stderr, (infoflag & MY_GIVE_INFO) != 0);
 #elif defined(__WIN__) && defined(_MSC_VER)
    _CrtSetReportMode( _CRT_WARN, _CRTDBG_MODE_FILE );

=== modified file 'mysys/my_pthread.c'
--- a/mysys/my_pthread.c	2007-07-30 19:45:06 +0000
+++ b/mysys/my_pthread.c	2008-12-02 22:02:52 +0000
@@ -429,7 +429,8 @@ int sigwait(sigset_t *setp, int *sigp)
 
 #include <netdb.h>
 
-int my_pthread_mutex_init(pthread_mutex_t *mp, const pthread_mutexattr_t *attr)
+int my_pthread_mutex_noposix_init(pthread_mutex_t *mp,
+                                  const pthread_mutexattr_t *attr)
 {
   int error;
   if (!attr)
@@ -439,7 +440,8 @@ int my_pthread_mutex_init(pthread_mutex_
   return error;
 }
 
-int my_pthread_cond_init(pthread_cond_t *mp, const pthread_condattr_t *attr)
+int my_pthread_cond_noposix_init(pthread_cond_t *mp,
+                                 const pthread_condattr_t *attr)
 {
   int error;
   if (!attr)

=== modified file 'mysys/my_sleep.c'
--- a/mysys/my_sleep.c	2006-12-23 19:20:40 +0000
+++ b/mysys/my_sleep.c	2008-12-02 22:02:52 +0000
@@ -30,7 +30,7 @@ void my_sleep(ulong m_seconds)
   t.tv_usec= m_seconds % 1000000L;
   select(0,0,0,0,&t); /* sleep */
 #else
-  uint sec=    (uint) (m_seconds / 1000000L);
+  uint sec=    (uint) ((m_seconds + 999999L) / 1000000L);
   ulong start= (ulong) time((time_t*) 0);
   while ((ulong) time((time_t*) 0) < start+sec);
 #endif

=== modified file 'mysys/my_thr_init.c'
--- a/mysys/my_thr_init.c	2008-10-10 15:28:41 +0000
+++ b/mysys/my_thr_init.c	2008-12-02 22:02:52 +0000
@@ -115,6 +115,15 @@ my_bool my_thread_global_init(void)
   }
 #endif /* TARGET_OS_LINUX */
 
+  /* Mutex used by my_thread_init() and after my_thread_destroy_mutex() */
+  my_pthread_mutex_init(&THR_LOCK_threads, MY_MUTEX_INIT_FAST, 
+                        "THR_LOCK_threads", MYF_NO_DEADLOCK_DETECTION);
+  my_pthread_mutex_init(&THR_LOCK_malloc, MY_MUTEX_INIT_FAST,
+                        "THR_LOCK_malloc", MYF_NO_DEADLOCK_DETECTION);
+
+  if (my_thread_init())
+    return 1;
+
 #ifdef PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP
   /*
     Set mutex type to "fast" a.k.a "adaptive"
@@ -138,7 +147,7 @@ my_bool my_thread_global_init(void)
                             PTHREAD_MUTEX_ERRORCHECK);
 #endif
 
-  pthread_mutex_init(&THR_LOCK_malloc,MY_MUTEX_INIT_FAST);
+  /* Mutex uses by mysys */
   pthread_mutex_init(&THR_LOCK_open,MY_MUTEX_INIT_FAST);
   pthread_mutex_init(&THR_LOCK_lock,MY_MUTEX_INIT_FAST);
   pthread_mutex_init(&THR_LOCK_isam,MY_MUTEX_INIT_SLOW);
@@ -146,7 +155,6 @@ my_bool my_thread_global_init(void)
   pthread_mutex_init(&THR_LOCK_heap,MY_MUTEX_INIT_FAST);
   pthread_mutex_init(&THR_LOCK_net,MY_MUTEX_INIT_FAST);
   pthread_mutex_init(&THR_LOCK_charset,MY_MUTEX_INIT_FAST);
-  pthread_mutex_init(&THR_LOCK_threads,MY_MUTEX_INIT_FAST);
   pthread_mutex_init(&THR_LOCK_time,MY_MUTEX_INIT_FAST);
   pthread_cond_init(&THR_COND_threads, NULL);
 #if defined( __WIN__) || defined(OS2)
@@ -158,44 +166,64 @@ my_bool my_thread_global_init(void)
 #ifndef HAVE_GETHOSTBYNAME_R
   pthread_mutex_init(&LOCK_gethostbyname_r,MY_MUTEX_INIT_SLOW);
 #endif
-  if (my_thread_init())
-  {
-    my_thread_global_end();			/* Clean up */
-    return 1;
-  }
   return 0;
 }
 
 
-void my_thread_global_end(void)
+/**
+   Wait for all threads in system to die
+   @fn    my_wait_for_other_threads_to_die()
+   @param number_of_threads  Wait until this number of threads
+
+   @retval  0  Less or equal to number_of_threads left
+   @retval  1  Wait failed
+*/
+
+my_bool my_wait_for_other_threads_to_die(uint number_of_threads)
 {
   struct timespec abstime;
   my_bool all_threads_killed= 1;
 
   set_timespec(abstime, my_thread_end_wait_time);
   pthread_mutex_lock(&THR_LOCK_threads);
-  while (THR_thread_count > 0)
+  while (THR_thread_count > number_of_threads)
   {
     int error= pthread_cond_timedwait(&THR_COND_threads, &THR_LOCK_threads,
                                       &abstime);
     if (error == ETIMEDOUT || error == ETIME)
     {
-#ifdef HAVE_PTHREAD_KILL
-      /*
-        We shouldn't give an error here, because if we don't have
-        pthread_kill(), programs like mysqld can't ensure that all threads
-        are killed when we enter here.
-      */
-      if (THR_thread_count)
-        fprintf(stderr,
-                "Error in my_thread_global_end(): %d threads didn't exit\n",
-                THR_thread_count);
-#endif
       all_threads_killed= 0;
       break;
     }
   }
   pthread_mutex_unlock(&THR_LOCK_threads);
+  return all_threads_killed;
+}
+
+
+/**
+   End the mysys thread system. Called when ending the last thread
+*/
+
+
+void my_thread_global_end(void)
+{
+  my_bool all_threads_killed;
+
+  if (!(all_threads_killed= my_wait_for_other_threads_to_die(0)))
+  {
+#ifdef HAVE_PTHREAD_KILL
+    /*
+      We shouldn't give an error here, because if we don't have
+      pthread_kill(), programs like mysqld can't ensure that all threads
+      are killed when we enter here.
+    */
+    if (THR_thread_count)
+      fprintf(stderr,
+              "Error in my_thread_global_end(): %d threads didn't exit\n",
+              THR_thread_count);
+#endif
+  }
 
   pthread_key_delete(THR_KEY_mysys);
 #ifdef PTHREAD_ADAPTIVE_MUTEX_INITIALIZER_NP
@@ -204,7 +232,25 @@ void my_thread_global_end(void)
 #ifdef PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP
   pthread_mutexattr_destroy(&my_errorcheck_mutexattr);
 #endif
-  pthread_mutex_destroy(&THR_LOCK_malloc);
+  if (all_threads_killed)
+  {
+    pthread_mutex_destroy(&THR_LOCK_threads);
+    pthread_cond_destroy(&THR_COND_threads);
+    pthread_mutex_destroy(&THR_LOCK_malloc);
+  }
+}
+
+/* Free all mutex used by mysys */
+
+void my_thread_destroy_mutex(void)
+{
+  struct st_my_thread_var *tmp;
+  tmp= my_pthread_getspecific(struct st_my_thread_var*,THR_KEY_mysys);
+  if (tmp)
+  {
+    safe_mutex_free_deadlock_data(&tmp->mutex);
+  }  
+
   pthread_mutex_destroy(&THR_LOCK_open);
   pthread_mutex_destroy(&THR_LOCK_lock);
   pthread_mutex_destroy(&THR_LOCK_isam);
@@ -213,11 +259,6 @@ void my_thread_global_end(void)
   pthread_mutex_destroy(&THR_LOCK_net);
   pthread_mutex_destroy(&THR_LOCK_time);
   pthread_mutex_destroy(&THR_LOCK_charset);
-  if (all_threads_killed)
-  {
-    pthread_mutex_destroy(&THR_LOCK_threads);
-    pthread_cond_destroy(&THR_COND_threads);
-  }
 #if !defined(HAVE_LOCALTIME_R) || !defined(HAVE_GMTIME_R)
   pthread_mutex_destroy(&LOCK_localtime_r);
 #endif
@@ -287,7 +328,8 @@ my_bool my_thread_init(void)
 #else
   tmp->pthread_self= pthread_self();
 #endif
-  pthread_mutex_init(&tmp->mutex,MY_MUTEX_INIT_FAST);
+  my_pthread_mutex_init(&tmp->mutex, MY_MUTEX_INIT_FAST, "mysys_var->mutex",
+                        0);
   pthread_cond_init(&tmp->suspend, NULL);
 
   tmp->stack_ends_here= &tmp + STACK_DIRECTION * my_thread_stack_size;
@@ -330,6 +372,13 @@ void my_thread_end(void)
 #endif
   if (tmp && tmp->init)
   {
+
+#if !defined(__bsdi__) && !defined(__OpenBSD__)
+ /* bsdi and openbsd 3.5 dumps core here */
+    pthread_cond_destroy(&tmp->suspend);
+#endif
+    pthread_mutex_destroy(&tmp->mutex);
+
 #if !defined(DBUG_OFF)
     /* tmp->dbug is allocated inside DBUG library */
     if (tmp->dbug)
@@ -339,12 +388,11 @@ void my_thread_end(void)
       tmp->dbug=0;
     }
 #endif
-#if !defined(__bsdi__) && !defined(__OpenBSD__)
- /* bsdi and openbsd 3.5 dumps core here */
-    pthread_cond_destroy(&tmp->suspend);
-#endif
-    pthread_mutex_destroy(&tmp->mutex);
 #if !defined(__WIN__) || defined(USE_TLS)
+#ifndef DBUG_OFF
+    /* To find bugs when accessing unallocated data */
+    bfill(tmp, sizeof(tmp), 0x8F);
+#endif
     free(tmp);
 #else
     tmp->init= 0;
@@ -399,6 +447,15 @@ extern void **my_thread_var_dbug()
 }
 #endif
 
+/* Return pointer to mutex_in_use */
+
+safe_mutex_t **my_thread_var_mutex_in_use()
+{
+  struct st_my_thread_var *tmp=
+    my_pthread_getspecific(struct st_my_thread_var*,THR_KEY_mysys);
+  return tmp ? &tmp->mutex_in_use : 0;
+}
+
 /****************************************************************************
   Get name of current thread.
 ****************************************************************************/

=== modified file 'mysys/mysys_priv.h'
--- a/mysys/mysys_priv.h	2007-07-30 08:33:50 +0000
+++ b/mysys/mysys_priv.h	2008-12-02 22:02:52 +0000
@@ -33,6 +33,7 @@ extern pthread_mutex_t THR_LOCK_charset,
 #include <my_no_pthread.h>
 #endif
 
+
 /*
   EDQUOT is used only in 3 C files only in mysys/. If it does not exist on
   system, we set it to some value which can never happen.
@@ -42,3 +43,5 @@ extern pthread_mutex_t THR_LOCK_charset,
 #endif
 
 void my_error_unregister_all(void);
+void my_thread_destroy_mutex(void);
+my_bool my_wait_for_other_threads_to_die(uint number_of_threads);

=== modified file 'mysys/thr_mutex.c'
--- a/mysys/thr_mutex.c	2007-12-16 15:03:44 +0000
+++ b/mysys/thr_mutex.c	2008-12-02 22:02:52 +0000
@@ -1,4 +1,4 @@
-/* Copyright (C) 2000-2003 MySQL AB
+/* Copyright (C) 2000-2008 MySQL AB, 2008 Sun Microsystems, Inc
 
    This program is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
@@ -24,6 +24,7 @@
 #include "mysys_priv.h"
 #include "my_static.h"
 #include <m_string.h>
+#include <hash.h>
 
 #ifndef DO_NOT_REMOVE_THREAD_WRAPPERS
 /* Remove wrappers */
@@ -34,28 +35,68 @@
 #undef pthread_mutex_destroy
 #undef pthread_cond_wait
 #undef pthread_cond_timedwait
+#undef safe_mutex_free_deadlock_data
 #ifdef HAVE_NONPOSIX_PTHREAD_MUTEX_INIT
-#define pthread_mutex_init(a,b) my_pthread_mutex_init((a),(b))
+#define pthread_mutex_init(a,b) my_pthread_noposix_mutex_init((a),(b))
 #endif
 #endif /* DO_NOT_REMOVE_THREAD_WRAPPERS */
 
 static pthread_mutex_t THR_LOCK_mutex;
 static ulong safe_mutex_count= 0;		/* Number of mutexes created */
+static ulong safe_mutex_id= 0;
+my_bool safe_mutex_deadlock_detector= 1;        /* On by default */
+
 #ifdef SAFE_MUTEX_DETECT_DESTROY
-static struct st_safe_mutex_info_t *safe_mutex_root= NULL;
+static struct st_safe_mutex_create_info_t *safe_mutex_create_root= NULL;
 #endif
 
+static my_bool add_used_to_locked_mutex(safe_mutex_t *used_mutex,
+                                        safe_mutex_deadlock_t *locked_mutex);
+static my_bool add_to_locked_mutex(safe_mutex_deadlock_t *locked_mutex,
+                                   safe_mutex_t *current_mutex);
+static my_bool remove_from_locked_mutex(safe_mutex_t *mp,
+                                        safe_mutex_t *delete_mutex);
+static my_bool remove_from_used_mutex(safe_mutex_deadlock_t *locked_mutex,
+                                      safe_mutex_t *mutex);
+static void print_deadlock_warning(safe_mutex_t *new_mutex,
+                                   safe_mutex_t *conflicting_mutex);
+
 void safe_mutex_global_init(void)
 {
   pthread_mutex_init(&THR_LOCK_mutex,MY_MUTEX_INIT_FAST);
+  safe_mutex_id= safe_mutex_count= 0;
+  safe_mutex_deadlock_detector= 1;
+
+#ifdef SAFE_MUTEX_DETECT_DESTROY
+  safe_mutex_create_root= 0;
+#endif
+}
+
+static inline void remove_from_active_list(safe_mutex_t *mp)
+{
+  if (!(mp->active_flags & (MYF_NO_DEADLOCK_DETECTION | MYF_TRY_LOCK)))
+  {
+    /* Remove mutex from active mutex linked list */
+    if (mp->next)
+      mp->next->prev= mp->prev;
+    if (mp->prev)
+      mp->prev->next= mp->next;
+    else
+      *my_thread_var_mutex_in_use()= mp->next;
+  }
+  mp->prev= mp->next= 0;
 }
 
 
 int safe_mutex_init(safe_mutex_t *mp,
 		    const pthread_mutexattr_t *attr __attribute__((unused)),
+                    const char *name,
+                    myf my_flags,
 		    const char *file,
-		    uint line, const char *name)
+		    uint line)
 {
+  DBUG_ENTER("safe_mutex_init");
+  DBUG_PRINT("enter",("mutex: 0x%lx  name: %s", (ulong) mp, name));
   bzero((char*) mp,sizeof(*mp));
   pthread_mutex_init(&mp->global,MY_MUTEX_INIT_ERRCHK);
   pthread_mutex_init(&mp->mutex,attr);
@@ -65,6 +106,36 @@ int safe_mutex_init(safe_mutex_t *mp,
   /* Skip the very common '&' prefix from the autogenerated name */
   mp->name= name[0] == '&' ? name + 1 : name;
 
+  if (safe_mutex_deadlock_detector && !( my_flags & MYF_NO_DEADLOCK_DETECTION))
+  {
+    if (!my_multi_malloc(MY_FAE | MY_WME,
+                         &mp->locked_mutex, sizeof(*mp->locked_mutex),
+                         &mp->used_mutex, sizeof(*mp->used_mutex), NullS))
+    {
+      /* Disable deadlock handling for this mutex */
+      my_flags|= MYF_NO_DEADLOCK_DETECTION;
+    }
+    else
+    {
+      pthread_mutex_lock(&THR_LOCK_mutex);
+      mp->id= ++safe_mutex_id;
+      pthread_mutex_unlock(&THR_LOCK_mutex);
+      hash_init(mp->locked_mutex, &my_charset_bin,
+                1000,
+                offsetof(safe_mutex_deadlock_t, id),
+                sizeof(mp->id),
+                0, 0, HASH_UNIQUE);
+      hash_init(mp->used_mutex, &my_charset_bin,
+                1000,
+                offsetof(safe_mutex_t, id),
+                sizeof(mp->id),
+                0, 0, HASH_UNIQUE);
+    }
+  }
+  else
+    my_flags|= MYF_NO_DEADLOCK_DETECTION;
+  mp->create_flags= my_flags;
+
 #ifdef SAFE_MUTEX_DETECT_DESTROY
   /*
     Monitor the freeing of mutexes.  This code depends on single thread init
@@ -72,7 +143,7 @@ int safe_mutex_init(safe_mutex_t *mp,
   */
   if ((mp->info= (safe_mutex_info_t *) malloc(sizeof(safe_mutex_info_t))))
   {
-    struct st_safe_mutex_info_t *info =mp->info;
+    struct st_safe_mutex_info_t *info= mp->info;
 
     info->init_file= file;
     info->init_line= line;
@@ -80,20 +151,21 @@ int safe_mutex_init(safe_mutex_t *mp,
     info->next= NULL;
 
     pthread_mutex_lock(&THR_LOCK_mutex);
-    if ((info->next= safe_mutex_root))
-      safe_mutex_root->prev= info;
-    safe_mutex_root= info;
+    if ((info->next= safe_mutex_create_root))
+      safe_mutex_create_root->prev= info;
+    safe_mutex_create_root= info;
     safe_mutex_count++;
     pthread_mutex_unlock(&THR_LOCK_mutex);
   }
 #else
   thread_safe_increment(safe_mutex_count, &THR_LOCK_mutex);
 #endif /* SAFE_MUTEX_DETECT_DESTROY */
-  return 0;
+  DBUG_RETURN(0);
 }
 
 
-int safe_mutex_lock(safe_mutex_t *mp, my_bool try_lock, const char *file, uint line)
+int safe_mutex_lock(safe_mutex_t *mp, myf my_flags, const char *file,
+                    uint line)
 {
   int error;
   DBUG_PRINT("mutex", ("%s (0x%lx) locking", mp->name ? mp->name : "Null",
@@ -110,12 +182,13 @@ int safe_mutex_lock(safe_mutex_t *mp, my
   pthread_mutex_lock(&mp->global);
   if (mp->count > 0)
   {
-    if (try_lock)
-    {
-      pthread_mutex_unlock(&mp->global);
-      return EBUSY;
-    }
-    else if (pthread_equal(pthread_self(),mp->thread))
+    /*
+      Check that we are not trying to lock mutex twice. This is an error
+      even if we are using 'try_lock' as it's not portably what happens
+      if you lock the mutex many times and this is in any case bad
+      behaviour that should not be encouraged
+    */
+    if (pthread_equal(pthread_self(),mp->thread))
     {
       fprintf(stderr,
               "safe_mutex: Trying to lock mutex at %s, line %d, when the"
@@ -143,7 +216,7 @@ int safe_mutex_lock(safe_mutex_t *mp, my
       instead just return EBUSY, since this is the expected behaviour
       of trylock().
    */
-  if (try_lock)
+  if (my_flags & MYF_TRY_LOCK)
   {
     error= pthread_mutex_trylock(&mp->mutex);
     if (error == EBUSY)
@@ -169,7 +242,93 @@ int safe_mutex_lock(safe_mutex_t *mp, my
   }
   mp->file= file;
   mp->line= line;
+  mp->active_flags= mp->create_flags | my_flags;
   pthread_mutex_unlock(&mp->global);
+
+  /* Deadlock detection */
+
+  mp->prev= mp->next= 0;
+  if (!(mp->active_flags & (MYF_TRY_LOCK | MYF_NO_DEADLOCK_DETECTION)))
+  {
+    safe_mutex_t **mutex_in_use= my_thread_var_mutex_in_use();
+
+    if (!mutex_in_use)
+    {
+      /* thread has not called my_thread_init() */
+      mp->active_flags|= MYF_NO_DEADLOCK_DETECTION;
+    }
+    else
+    {
+      safe_mutex_t *mutex_root;
+      if ((mutex_root= *mutex_in_use))   /* If not first locked */
+      {
+        /*
+          Protect locked_mutex against changes if a mutex is deleted
+        */
+        pthread_mutex_lock(&THR_LOCK_mutex);
+
+        if (!hash_search(mutex_root->locked_mutex, (uchar*) &mp->id, 0))
+        {
+          safe_mutex_deadlock_t *deadlock;
+          safe_mutex_t *mutex;
+
+          /* Create object to store mutex info */
+          if (!(deadlock= my_malloc(sizeof(*deadlock),
+                                    MYF(MY_ZEROFILL | MY_WME | MY_FAE))))
+            goto abort_loop;
+          deadlock->name= mp->name;
+          deadlock->id= mp->id;
+          deadlock->mutex= mp;
+          /* The following is useful for debugging wrong mutex usage */
+          deadlock->file= file;
+          deadlock->line= line;
+
+          /* Check if potential deadlock */
+          mutex= mutex_root;
+          do
+          {
+            if (hash_search(mp->locked_mutex, (uchar*) &mutex->id, 0))
+            {
+              print_deadlock_warning(mp, mutex);
+              /* Mark wrong usage to avoid future warnings for same error */
+              deadlock->warning_only= 1;
+              add_to_locked_mutex(deadlock, mutex_root);
+              DBUG_ASSERT(deadlock->count > 0);
+              goto abort_loop;
+            }
+          }
+          while ((mutex= mutex->next));
+
+          /*
+            Copy current mutex and all mutex that has been locked
+            after current mutex (mp->locked_mutex) to all mutex that
+            was locked before previous mutex (mutex_root->used_mutex)
+
+            For example if A->B would have been done before and we
+            are now locking (C) in B->C, then we would add C into
+            B->locked_mutex and A->locked_mutex
+          */
+          hash_iterate(mutex_root->used_mutex,
+                       (hash_walk_action) add_used_to_locked_mutex,
+                       deadlock);
+
+          /*
+            Copy all current mutex and all mutex locked after current one
+            into the prev mutex
+          */
+          add_used_to_locked_mutex(mutex_root, deadlock);
+          DBUG_ASSERT(deadlock->count > 0);
+        }
+  abort_loop:
+        pthread_mutex_unlock(&THR_LOCK_mutex);
+      }
+      /* Link mutex into mutex_in_use list */
+      if ((mp->next= *mutex_in_use))
+        (*mutex_in_use)->prev= mp;
+      *mutex_in_use= mp;
+    }
+  }
+
   DBUG_PRINT("mutex", ("%s (0x%lx) locked", mp->name, (ulong) mp));
   return error;
 }
@@ -182,7 +341,9 @@ int safe_mutex_unlock(safe_mutex_t *mp,c
   pthread_mutex_lock(&mp->global);
   if (mp->count == 0)
   {
-    fprintf(stderr,"safe_mutex: Trying to unlock mutex %s that wasn't locked at %s, line %d\n"
+    fprintf(stderr,
+            "safe_mutex: Trying to unlock mutex %s that wasn't locked at "
+            "%s, line %d\n"
             "Last used at %s, line: %d\n",
 	    mp->name ? mp->name : "Null", file, line,
             mp->file ? mp->file : "Null", mp->line);
@@ -191,7 +352,9 @@ int safe_mutex_unlock(safe_mutex_t *mp,c
   }
   if (!pthread_equal(pthread_self(),mp->thread))
   {
-    fprintf(stderr,"safe_mutex: Trying to unlock mutex %s at %s, line %d that was locked by "
+    fprintf(stderr,
+            "safe_mutex: Trying to unlock mutex %s at %s, line %d that was "
+            "locked by "
             "another thread at: %s, line: %d\n",
 	    mp->name, file, line, mp->file, mp->line);
     fflush(stderr);
@@ -199,6 +362,9 @@ int safe_mutex_unlock(safe_mutex_t *mp,c
   }
   mp->thread= 0;
   mp->count--;
+
+  remove_from_active_list(mp);
+
 #ifdef __WIN__
   pthread_mutex_unlock(&mp->mutex);
   error=0;
@@ -206,8 +372,9 @@ int safe_mutex_unlock(safe_mutex_t *mp,c
   error=pthread_mutex_unlock(&mp->mutex);
   if (error)
   {
-    fprintf(stderr,"safe_mutex: Got error: %d (%d) when trying to unlock mutex %s at %s, "
-            "line %d\n", error, errno, mp->name, file, line);
+    fprintf(stderr,
+            "safe_mutex: Got error: %d (%d) when trying to unlock mutex "
+            "%s at %s, line %d\n", error, errno, mp->name, file, line);
     fflush(stderr);
     abort();
   }
@@ -221,18 +388,23 @@ int safe_cond_wait(pthread_cond_t *cond,
 		   uint line)
 {
   int error;
+  safe_mutex_t save_state;
+
   pthread_mutex_lock(&mp->global);
   if (mp->count == 0)
   {
-    fprintf(stderr,"safe_mutex: Trying to cond_wait on a unlocked mutex %s at %s, line %d\n",
+    fprintf(stderr,
+            "safe_mutex: Trying to cond_wait on a unlocked mutex %s at %s, "
+            "line %d\n",
             mp->name ? mp->name : "Null", file, line);
     fflush(stderr);
     abort();
   }
   if (!pthread_equal(pthread_self(),mp->thread))
   {
-    fprintf(stderr,"safe_mutex: Trying to cond_wait on a mutex %s at %s, line %d that was "
-            "locked by another thread at: %s, line: %d\n",
+    fprintf(stderr,
+            "safe_mutex: Trying to cond_wait on a mutex %s at %s, line %d "
+            "that was locked by another thread at: %s, line: %d\n",
 	    mp->name, file, line, mp->file, mp->line);
     fflush(stderr);
     abort();
@@ -240,26 +412,37 @@ int safe_cond_wait(pthread_cond_t *cond,
 
   if (mp->count-- != 1)
   {
-    fprintf(stderr,"safe_mutex:  Count was %d on locked mutex %s at %s, line %d\n",
+    fprintf(stderr,
+            "safe_mutex:  Count was %d on locked mutex %s at %s, line %d\n",
 	    mp->count+1, mp->name, file, line);
     fflush(stderr);
     abort();
   }
+  save_state= *mp;
+  remove_from_active_list(mp);
   pthread_mutex_unlock(&mp->global);
   error=pthread_cond_wait(cond,&mp->mutex);
   pthread_mutex_lock(&mp->global);
+
   if (error)
   {
-    fprintf(stderr,"safe_mutex: Got error: %d (%d) when doing a safe_mutex_wait on %s at %s, "
-            "line %d\n", error, errno, mp->name, file, line);
+    fprintf(stderr,
+            "safe_mutex: Got error: %d (%d) when doing a safe_mutex_wait on "
+            "%s at %s, line %d\n", error, errno, mp->name, file, line);
     fflush(stderr);
     abort();
   }
-  mp->thread=pthread_self();
+  /* Restore state as it was before */
+  mp->thread=       save_state.thread;
+  mp->active_flags= save_state.active_flags;
+  mp->next=         save_state.next;
+  mp->prev=         save_state.prev;
+
   if (mp->count++)
   {
     fprintf(stderr,
-	    "safe_mutex:  Count was %d in thread 0x%lx when locking mutex %s at %s, line %d\n",
+	    "safe_mutex:  Count was %d in thread 0x%lx when locking mutex %s "
+            "at %s, line %d\n",
 	    mp->count-1, my_thread_dbug_id(), mp->name, file, line);
     fflush(stderr);
     abort();
@@ -276,33 +459,44 @@ int safe_cond_timedwait(pthread_cond_t *
 			const char *file, uint line)
 {
   int error;
+  safe_mutex_t save_state;
+
   pthread_mutex_lock(&mp->global);
   if (mp->count != 1 || !pthread_equal(pthread_self(),mp->thread))
   {
-    fprintf(stderr,"safe_mutex: Trying to cond_wait at %s, line %d on a not hold mutex %s\n",
+    fprintf(stderr,
+            "safe_mutex: Trying to cond_wait at %s, line %d on a not hold "
+            "mutex %s\n",
             file, line, mp->name ? mp->name : "Null");
     fflush(stderr);
     abort();
   }
   mp->count--;					/* Mutex will be released */
+  save_state= *mp;
+  remove_from_active_list(mp);
   pthread_mutex_unlock(&mp->global);
   error=pthread_cond_timedwait(cond,&mp->mutex,abstime);
 #ifdef EXTRA_DEBUG
   if (error && (error != EINTR && error != ETIMEDOUT && error != ETIME))
   {
     fprintf(stderr,
-            "safe_mutex: Got error: %d (%d) when doing a safe_mutex_timedwait on %s at %s, "
-            "line %d\n",
+            "safe_mutex: Got error: %d (%d) when doing a safe_mutex_timedwait "
+            "on %s at %s, line %d\n",
             error, errno, mp->name, file, line);
   }
 #endif
   pthread_mutex_lock(&mp->global);
-  mp->thread=pthread_self();
+  /* Restore state as it was before */
+  mp->thread=       save_state.thread;
+  mp->active_flags= save_state.active_flags;
+  mp->next=         save_state.next;
+  mp->prev=         save_state.prev;
+
   if (mp->count++)
   {
     fprintf(stderr,
-	    "safe_mutex:  Count was %d in thread 0x%lx when locking mutex %s at %s, line %d "
-            "(error: %d (%d))\n",
+	    "safe_mutex:  Count was %d in thread 0x%lx when locking mutex "
+            "%s at %s, line %d (error: %d (%d))\n",
 	    mp->count-1, my_thread_dbug_id(), mp->name, file, line,
             error, error);
     fflush(stderr);
@@ -318,6 +512,8 @@ int safe_cond_timedwait(pthread_cond_t *
 int safe_mutex_destroy(safe_mutex_t *mp, const char *file, uint line)
 {
   int error=0;
+  DBUG_ENTER("safe_mutex_destroy");
+  DBUG_PRINT("enter", ("mutex: 0x%lx  name: %s", (ulong) mp, mp->name));
   if (!mp->file)
   {
     fprintf(stderr,
@@ -328,12 +524,17 @@ int safe_mutex_destroy(safe_mutex_t *mp,
   }
   if (mp->count != 0)
   {
-    fprintf(stderr,"safe_mutex: Trying to destroy a mutex %s that was locked at %s, "
+    fprintf(stderr,
+            "safe_mutex: Trying to destroy a mutex %s that was locked at %s, "
             "line %d at %s, line %d\n",
 	    mp->name, mp->file, mp->line, file, line);
     fflush(stderr);
     abort();
   }
+
+  /* Free all entries that points to this one */
+  safe_mutex_free_deadlock_data(mp);
+
 #ifdef __WIN__ 
   pthread_mutex_destroy(&mp->global);
   pthread_mutex_destroy(&mp->mutex);
@@ -354,7 +555,7 @@ int safe_mutex_destroy(safe_mutex_t *mp,
     if (info->prev)
       info->prev->next = info->next;
     else
-      safe_mutex_root = info->next;
+      safe_mutex_create_root = info->next;
     if (info->next)
       info->next->prev = info->prev;
     safe_mutex_count--;
@@ -366,10 +567,36 @@ int safe_mutex_destroy(safe_mutex_t *mp,
 #else
   thread_safe_sub(safe_mutex_count, 1, &THR_LOCK_mutex);
 #endif /* SAFE_MUTEX_DETECT_DESTROY */
-  return error;
+  DBUG_RETURN(error);
 }
 
 
+/**
+  Free all data related to deadlock detection
+
+  This is also useful together with safemalloc when you don't want to
+  have reports of not freed memory for mysys mutexes.
+*/
+
+void safe_mutex_free_deadlock_data(safe_mutex_t *mp)
+{
+  /* Free all entries that points to this one */
+  if (!(mp->create_flags & MYF_NO_DEADLOCK_DETECTION))
+  {
+    pthread_mutex_lock(&THR_LOCK_mutex);
+    hash_iterate(mp->used_mutex, (hash_walk_action) remove_from_locked_mutex,
+                 mp);
+    hash_iterate(mp->locked_mutex, (hash_walk_action) remove_from_used_mutex,
+                 mp);
+    pthread_mutex_unlock(&THR_LOCK_mutex);
+
+    hash_free(mp->used_mutex);
+    hash_free(mp->locked_mutex);
+    my_free(mp->locked_mutex, 0);
+    mp->create_flags|= MYF_NO_DEADLOCK_DETECTION;
+  }
+}
+
 /*
   Free global resources and check that all mutex has been destroyed
 
@@ -400,7 +627,7 @@ void safe_mutex_end(FILE *file __attribu
   }
   {
     struct st_safe_mutex_info_t *ptr;
-    for (ptr= safe_mutex_root ; ptr ; ptr= ptr->next)
+    for (ptr= safe_mutex_create_root ; ptr ; ptr= ptr->next)
     {
       fprintf(file, "\tMutex %s initiated at line %4u in '%s'\n",
 	      ptr->name, ptr->init_line, ptr->init_file);
@@ -410,6 +637,127 @@ void safe_mutex_end(FILE *file __attribu
 #endif /* SAFE_MUTEX_DETECT_DESTROY */
 }
 
+
+static my_bool add_used_to_locked_mutex(safe_mutex_t *used_mutex,
+                                        safe_mutex_deadlock_t *locked_mutex)
+{
+  /* Add mutex to all parent of the current mutex */
+  if (!locked_mutex->warning_only)
+  {
+    (void) hash_iterate(locked_mutex->mutex->locked_mutex,
+                        (hash_walk_action) add_to_locked_mutex,
+                        used_mutex);
+    /* mark that locked_mutex is locked after used_mutex */
+    (void) add_to_locked_mutex(locked_mutex, used_mutex);
+  }
+  return 0;
+}
+
+
+/**
+   register that locked_mutex was locked after current_mutex
+*/
+
+static my_bool add_to_locked_mutex(safe_mutex_deadlock_t *locked_mutex,
+                                   safe_mutex_t *current_mutex)
+{
+  DBUG_ENTER("add_to_locked_mutex");
+  DBUG_PRINT("info", ("inserting 0x%lx  into  0x%lx  (id: %lu -> %lu)",
+                      (ulong) locked_mutex, (long) current_mutex,
+                      locked_mutex->id, current_mutex->id));
+  if (my_hash_insert(current_mutex->locked_mutex, (uchar*) locked_mutex))
+  {
+    /* Got mutex through two paths; ignore */
+    DBUG_RETURN(0);
+  }
+  locked_mutex->count++;
+  if (my_hash_insert(locked_mutex->mutex->used_mutex,
+                     (uchar*) current_mutex))
+  {
+    DBUG_ASSERT(0);
+  }
+  DBUG_RETURN(0);
+}
+
+
+/**
+  Remove mutex from the locked mutex hash
+  @fn    remove_from_used_mutex()
+  @param mp            Mutex that has delete_mutex in it's locked_mutex hash
+  @param delete_mutex  Mutex should be removed from the hash
+
+  @notes
+    safe_mutex_deadlock_t entries in the locked hash are shared.
+    When counter goes to 0, we delete the safe_mutex_deadlock_t entry.
+*/
+
+static my_bool remove_from_locked_mutex(safe_mutex_t *mp,
+                                        safe_mutex_t *delete_mutex)
+{
+  safe_mutex_deadlock_t *found;
+  DBUG_ENTER("remove_from_locked_mutex");
+  DBUG_PRINT("enter", ("delete_mutex: 0x%lx  mutex: 0x%lx  (id: %lu <- %lu)",
+                       (ulong) delete_mutex, (ulong) mp, 
+                       delete_mutex->id, mp->id));
+
+  found= (safe_mutex_deadlock_t *) hash_search(mp->locked_mutex,
+                                               (uchar*) &delete_mutex->id, 0);
+  DBUG_ASSERT(found);
+  if (found)
+  {
+    if (hash_delete(mp->locked_mutex, (uchar*) found))
+    {
+      DBUG_ASSERT(0);
+    }
+    if (!--found->count)
+      my_free(found, MYF(0));
+  }
+  DBUG_RETURN(0);
+}
+
+static my_bool remove_from_used_mutex(safe_mutex_deadlock_t *locked_mutex,
+                                      safe_mutex_t *mutex)
+{
+  DBUG_ENTER("remove_from_used_mutex");
+  DBUG_PRINT("enter", ("delete_mutex: 0x%lx  mutex: 0x%lx  (id: %lu <- %lu)",
+                       (ulong) mutex, (ulong) locked_mutex, 
+                       mutex->id, locked_mutex->id));
+  if (hash_delete(locked_mutex->mutex->used_mutex, (uchar*) mutex))
+  {
+    DBUG_ASSERT(0);
+  }
+  if (!--locked_mutex->count)
+    my_free(locked_mutex, MYF(0));
+  DBUG_RETURN(0);
+}
+
+
+static void print_deadlock_warning(safe_mutex_t *new_mutex,
+                                   safe_mutex_t *parent_mutex)
+{
+  safe_mutex_t *mutex_root;
+  DBUG_ENTER("print_deadlock_warning");
+  DBUG_PRINT("enter", ("mutex: %s  parent: %s",
+                       new_mutex->name, parent_mutex->name));
+
+  fprintf(stderr, "safe_mutex: Found wrong usage of mutex "
+          "'%s' and '%s'\n",
+          parent_mutex->name, new_mutex->name);
+  fprintf(stderr, "Mutex currently locked (in reverse order):\n");
+  fprintf(stderr, "%-32.32s  %s  line %u\n", new_mutex->name, new_mutex->file,
+          new_mutex->line);
+  for (mutex_root= *my_thread_var_mutex_in_use() ;
+       mutex_root;
+       mutex_root= mutex_root->next)
+  {
+    fprintf(stderr, "%-32.32s  %s  line %u\n", mutex_root->name,
+            mutex_root->file, mutex_root->line);
+  }
+  fflush(stderr);
+  DBUG_VOID_RETURN;
+}
+
+
 #endif /* THREAD && SAFE_MUTEX */
 
 #if defined(THREAD) && defined(MY_PTHREAD_FASTMUTEX) && !defined(SAFE_MUTEX)

=== modified file 'sql/event_queue.cc'
--- a/sql/event_queue.cc	2008-05-09 07:43:02 +0000
+++ b/sql/event_queue.cc	2008-12-02 22:02:52 +0000
@@ -94,7 +94,12 @@ Event_queue::Event_queue()
    mutex_queue_data_attempting_lock(FALSE),
    waiting_on_cond(FALSE)
 {
-  pthread_mutex_init(&LOCK_event_queue, MY_MUTEX_INIT_FAST);
+  /*
+    Inconsisent usage between LOCK_event_queue and LOCK_scheduler_state and
+    LOCK_open
+  */
+  my_pthread_mutex_init(&LOCK_event_queue, MY_MUTEX_INIT_FAST,
+                        "LOCK_event_queue", MYF_NO_DEADLOCK_DETECTION);
   pthread_cond_init(&COND_queue_state, NULL);
 }
 

=== modified file 'sql/event_scheduler.cc'
--- a/sql/event_scheduler.cc	2007-11-14 11:25:43 +0000
+++ b/sql/event_scheduler.cc	2008-12-02 22:02:52 +0000
@@ -350,6 +350,14 @@ Event_scheduler::Event_scheduler(Event_q
 {
   pthread_mutex_init(&LOCK_scheduler_state, MY_MUTEX_INIT_FAST);
   pthread_cond_init(&COND_state, NULL);
+
+#ifdef SAFE_MUTEX
+  /* Ensure right mutex order */
+  pthread_mutex_lock(&LOCK_scheduler_state);
+  pthread_mutex_lock(&LOCK_global_system_variables);
+  pthread_mutex_unlock(&LOCK_global_system_variables);
+  pthread_mutex_unlock(&LOCK_scheduler_state);
+#endif
 }
 
 

=== modified file 'sql/events.cc'
--- a/sql/events.cc	2008-10-10 15:28:41 +0000
+++ b/sql/events.cc	2008-12-02 22:02:52 +0000
@@ -995,7 +995,12 @@ Events::deinit()
 void
 Events::init_mutexes()
 {
-  pthread_mutex_init(&LOCK_event_metadata, MY_MUTEX_INIT_FAST);
+  /*
+    Inconsisent usage between LOCK_event_metadata and LOCK_scheduler_state
+    and LOCK_open
+  */
+  my_pthread_mutex_init(&LOCK_event_metadata, MY_MUTEX_INIT_FAST,
+                        "LOCK_event_metadata", MYF_NO_DEADLOCK_DETECTION);
 }
 
 

=== modified file 'sql/ha_ndbcluster_binlog.cc'
--- a/sql/ha_ndbcluster_binlog.cc	2008-02-26 16:34:02 +0000
+++ b/sql/ha_ndbcluster_binlog.cc	2008-12-02 22:02:52 +0000
@@ -1550,7 +1550,9 @@ end:
       dict->forceGCPWait();
 
     int max_timeout= opt_ndb_sync_timeout;
-    (void) pthread_mutex_lock(&ndb_schema_object->mutex);
+    /* Inconsistent usage of ndb_schema_object->mutex and LOCK_open */
+    (void) my_pthread_mutex_lock(&ndb_schema_object->mutex,
+                                 MYF_NO_DEADLOCK_DETECTION);
     if (have_lock_open)
     {
       safe_mutex_assert_owner(&LOCK_open);

=== modified file 'sql/log.cc'
--- a/sql/log.cc	2008-10-10 15:28:41 +0000
+++ b/sql/log.cc	2008-12-02 22:02:52 +0000
@@ -2403,7 +2403,12 @@ void MYSQL_BIN_LOG::init_pthread_objects
   DBUG_ASSERT(inited == 0);
   inited= 1;
   (void) pthread_mutex_init(&LOCK_log, MY_MUTEX_INIT_SLOW);
-  (void) pthread_mutex_init(&LOCK_index, MY_MUTEX_INIT_SLOW);
+  /*
+    LOCK_index and LOCK_log are taken in wrong order
+    Can be seen with 'mysql-test-run ndb.ndb_binlog_basic'
+  */ 
+  (void) my_pthread_mutex_init(&LOCK_index, MY_MUTEX_INIT_SLOW, "LOCK_index",
+                               MYF_NO_DEADLOCK_DETECTION);
   (void) pthread_cond_init(&update_cond, 0);
 }
 

=== modified file 'sql/mysqld.cc'
--- a/sql/mysqld.cc	2008-12-02 20:00:29 +0000
+++ b/sql/mysqld.cc	2008-12-02 22:02:52 +0000
@@ -757,6 +757,7 @@ static ulong find_bit_type_or_exit(const
                                    const char *option);
 static void clean_up(bool print_message);
 static int test_if_case_insensitive(const char *dir_name);
+static void register_mutex_order();
 
 #ifndef EMBEDDED_LIBRARY
 static void usage(void);
@@ -902,9 +903,19 @@ static void close_connections(void)
       pthread_mutex_lock(&tmp->mysys_var->mutex);
       if (tmp->mysys_var->current_cond)
       {
-	pthread_mutex_lock(tmp->mysys_var->current_mutex);
-	pthread_cond_broadcast(tmp->mysys_var->current_cond);
-	pthread_mutex_unlock(tmp->mysys_var->current_mutex);
+        uint i;
+        for (i=0; i < 2; i++)
+        {
+          int ret= pthread_mutex_trylock(tmp->mysys_var->current_mutex);
+          pthread_cond_broadcast(tmp->mysys_var->current_cond);
+          if (!ret)
+          {
+            /* Thread has surely got the signal, unlock and abort */
+            pthread_mutex_unlock(tmp->mysys_var->current_mutex);
+            break;
+          }
+          sleep(1);
+        }
       }
       pthread_mutex_unlock(&tmp->mysys_var->mutex);
     }
@@ -1244,6 +1255,7 @@ void clean_up(bool print_message)
   wt_end();
   delete_elements(&key_caches, (void (*)(const char*, uchar*)) free_key_cache);
   multi_keycache_free();
+  sp_cache_end();
   free_status_vars();
   end_thr_alarm(1);			/* Free allocated memory */
   my_free_open_file_info();
@@ -1336,6 +1348,7 @@ static void wait_for_signal_thread_to_en
 
 static void clean_up_mutexes()
 {
+  DBUG_ENTER("clean_up_mutexes");
   (void) pthread_mutex_destroy(&LOCK_mysql_create_db);
   (void) pthread_mutex_destroy(&LOCK_lock_db);
   (void) pthread_mutex_destroy(&LOCK_Acl);
@@ -1367,6 +1380,8 @@ static void clean_up_mutexes()
   (void) pthread_mutex_destroy(&LOCK_rpl_status);
   (void) pthread_cond_destroy(&COND_rpl_status);
 #endif
+  (void) pthread_mutex_destroy(&LOCK_server_started);
+  (void) pthread_cond_destroy(&COND_server_started);
   (void) pthread_mutex_destroy(&LOCK_active_mi);
   (void) rwlock_destroy(&LOCK_sys_init_connect);
   (void) rwlock_destroy(&LOCK_sys_init_slave);
@@ -1381,11 +1396,35 @@ static void clean_up_mutexes()
   (void) pthread_cond_destroy(&COND_thread_cache);
   (void) pthread_cond_destroy(&COND_flush_thread_cache);
   (void) pthread_cond_destroy(&COND_manager);
+  DBUG_VOID_RETURN;
 }
 
 #endif /*EMBEDDED_LIBRARY*/
 
 
+/**
+   Register order of mutex for wrong mutex deadlock detector
+
+   By aquiring all mutex in order here, the mutex order detector in
+   mysys/thr_mutex.c, will give a warning on first wrong mutex usage!
+*/
+
+static void register_mutex_order()
+{
+#ifdef SAFE_MUTEX
+  /*
+    We must have LOCK_open before LOCK_global_system_variables because
+    LOCK_open is hold while sql_plugin.c::intern_sys_var_ptr() is called.
+  */
+  pthread_mutex_lock(&LOCK_open);
+  pthread_mutex_lock(&LOCK_global_system_variables);
+
+  pthread_mutex_unlock(&LOCK_global_system_variables);
+  pthread_mutex_unlock(&LOCK_open);
+#endif
+}
+
+
 /****************************************************************************
 ** Init IP and UNIX socket
 ****************************************************************************/
@@ -3549,6 +3588,7 @@ static int init_thread_environment()
     sql_print_error("Can't create thread-keys");
     return 1;
   }
+  register_mutex_order();
   return 0;
 }
 
@@ -5464,7 +5504,7 @@ enum options_mysqld
   OPT_NDB_REPORT_THRESH_BINLOG_EPOCH_SLIP,
   OPT_NDB_REPORT_THRESH_BINLOG_MEM_USAGE,
   OPT_NDB_USE_COPYING_ALTER_TABLE,
-  OPT_SKIP_SAFEMALLOC,
+  OPT_SKIP_SAFEMALLOC, OPT_MUTEX_DEADLOCK_DETECTOR,
   OPT_TEMP_POOL, OPT_TX_ISOLATION, OPT_COMPLETION_TYPE,
   OPT_SKIP_STACK_TRACE, OPT_SKIP_SYMLINKS,
   OPT_MAX_BINLOG_DUMP_EVENTS, OPT_SPORADIC_BINLOG_DUMP_FAIL,
@@ -6002,6 +6042,13 @@ master-ssl",
 #endif /* HAVE_REPLICATION */
   {"memlock", OPT_MEMLOCK, "Lock mysqld in memory.", (uchar**) &locked_in_memory,
    (uchar**) &locked_in_memory, 0, GET_BOOL, NO_ARG, 0, 0, 0, 0, 0, 0},
+#ifdef SAFE_MUTEX
+  {"mutex-deadlock-detector", OPT_MUTEX_DEADLOCK_DETECTOR,
+   "Enable checking of wrong mutex usage.",
+   (uchar**) &safe_mutex_deadlock_detector,
+   (uchar**) &safe_mutex_deadlock_detector,
+   0, GET_BOOL, NO_ARG, 1, 0, 0, 0, 0, 0},
+#endif
   {"myisam-recover", OPT_MYISAM_RECOVER,
    "Syntax: myisam-recover[=option[,option...]], where option can be DEFAULT, BACKUP, FORCE or QUICK.",
    (uchar**) &myisam_recover_options_str, (uchar**) &myisam_recover_options_str, 0,

=== modified file 'sql/protocol.cc'
--- a/sql/protocol.cc	2008-10-10 15:28:41 +0000
+++ b/sql/protocol.cc	2008-12-02 22:02:52 +0000
@@ -790,8 +790,8 @@ bool Protocol_text::store(const char *fr
 {
   CHARSET_INFO *tocs= this->thd->variables.character_set_results;
 #ifndef DBUG_OFF
-  DBUG_PRINT("info", ("Protocol_text::store field %u (%u): %s", field_pos,
-                      field_count, from));
+  DBUG_PRINT("info", ("Protocol_text::store field %u (%u): %*s", field_pos,
+                      field_count, (int) length, from));
   DBUG_ASSERT(field_pos < field_count);
   DBUG_ASSERT(field_types == 0 ||
 	      field_types[field_pos] == MYSQL_TYPE_DECIMAL ||

=== modified file 'sql/rpl_mi.cc'
--- a/sql/rpl_mi.cc	2007-12-14 13:21:37 +0000
+++ b/sql/rpl_mi.cc	2008-12-02 22:02:52 +0000
@@ -38,11 +38,26 @@ Master_info::Master_info()
   ssl_cipher[0]= 0; ssl_key[0]= 0;
 
   bzero((char*) &file, sizeof(file));
-  pthread_mutex_init(&run_lock, MY_MUTEX_INIT_FAST);
-  pthread_mutex_init(&data_lock, MY_MUTEX_INIT_FAST);
+  /*
+    We have to use MYF_NO_DEADLOCK_DETECTION because mysqld doesn't
+    lock run_lock and data_lock consistently.
+    Should be fixed as this can easily lead to deadlocks
+  */
+  my_pthread_mutex_init(&run_lock, MY_MUTEX_INIT_FAST, 
+                        "Master_info::run_lock", MYF_NO_DEADLOCK_DETECTION);
+  my_pthread_mutex_init(&data_lock, MY_MUTEX_INIT_FAST,
+                        "Master_info::data_lock", MYF_NO_DEADLOCK_DETECTION);
   pthread_cond_init(&data_cond, NULL);
   pthread_cond_init(&start_cond, NULL);
   pthread_cond_init(&stop_cond, NULL);
+
+#ifdef SAFE_MUTEX
+  /* Define mutex order for locks to find wrong lock usage */
+  pthread_mutex_lock(&data_lock);
+  pthread_mutex_lock(&run_lock);
+  pthread_mutex_unlock(&run_lock);
+  pthread_mutex_unlock(&data_lock);
+#endif
 }
 
 Master_info::~Master_info()

=== modified file 'sql/set_var.cc'
--- a/sql/set_var.cc	2008-10-10 15:28:41 +0000
+++ b/sql/set_var.cc	2008-12-02 22:02:52 +0000
@@ -2403,7 +2403,6 @@ end:
 bool sys_var_log_state::update(THD *thd, set_var *var)
 {
   bool res;
-  pthread_mutex_lock(&LOCK_global_system_variables);
   if (!var->save_result.ulong_value)
   {
     logger.deactivate_log_handler(thd, log_type);
@@ -2411,15 +2410,12 @@ bool sys_var_log_state::update(THD *thd,
   }
   else
     res= logger.activate_log_handler(thd, log_type);
-  pthread_mutex_unlock(&LOCK_global_system_variables);
   return res;
 }
 
 void sys_var_log_state::set_default(THD *thd, enum_var_type type)
 {
-  pthread_mutex_lock(&LOCK_global_system_variables);
   logger.deactivate_log_handler(thd, log_type);
-  pthread_mutex_unlock(&LOCK_global_system_variables);
 }
 
 
@@ -2515,23 +2511,18 @@ bool update_sys_var_str_path(THD *thd, s
     goto err;
   }
 
-  pthread_mutex_lock(&LOCK_global_system_variables);
   logger.lock_exclusive();
 
   if (file_log && log_state)
     file_log->close(0);
-  old_value= var_str->value;
-  var_str->value= res;
-  var_str->value_length= str_length;
-  my_free(old_value, MYF(MY_ALLOW_ZERO_PTR));
   if (file_log && log_state)
   {
     switch (log_type) {
     case QUERY_LOG_SLOW:
-      file_log->open_slow_log(sys_var_slow_log_path.value);
+      file_log->open_slow_log(res);
       break;
     case QUERY_LOG_GENERAL:
-      file_log->open_query_log(sys_var_general_log_path.value);
+      file_log->open_query_log(res);
       break;
     default:
       DBUG_ASSERT(0);
@@ -2539,6 +2530,13 @@ bool update_sys_var_str_path(THD *thd, s
   }
 
   logger.unlock();
+
+  /* update global variable */
+  pthread_mutex_lock(&LOCK_global_system_variables);
+  old_value= var_str->value;
+  var_str->value= res;
+  var_str->value_length= str_length;
+  my_free(old_value, MYF(MY_ALLOW_ZERO_PTR));
   pthread_mutex_unlock(&LOCK_global_system_variables);
 
 err:
@@ -2578,26 +2576,22 @@ static void sys_default_slow_log_path(TH
 
 bool sys_var_log_output::update(THD *thd, set_var *var)
 {
-  pthread_mutex_lock(&LOCK_global_system_variables);
   logger.lock_exclusive();
   logger.init_slow_log(var->save_result.ulong_value);
   logger.init_general_log(var->save_result.ulong_value);
   *value= var->save_result.ulong_value;
   logger.unlock();
-  pthread_mutex_unlock(&LOCK_global_system_variables);
   return 0;
 }
 
 
 void sys_var_log_output::set_default(THD *thd, enum_var_type type)
 {
-  pthread_mutex_lock(&LOCK_global_system_variables);
   logger.lock_exclusive();
   logger.init_slow_log(LOG_FILE);
   logger.init_general_log(LOG_FILE);
   *value= LOG_FILE;
   logger.unlock();
-  pthread_mutex_unlock(&LOCK_global_system_variables);
 }
 
 

=== modified file 'sql/sp_cache.cc'
--- a/sql/sp_cache.cc	2008-07-03 19:41:22 +0000
+++ b/sql/sp_cache.cc	2008-12-02 22:02:52 +0000
@@ -106,6 +106,12 @@ void sp_cache_clear(sp_cache **cp)
 }
 
 
+void sp_cache_end()
+{
+  pthread_mutex_destroy(&Cversion_lock);
+}
+
+
 /*
   Insert a routine into the cache.
 

=== modified file 'sql/sp_cache.h'
--- a/sql/sp_cache.h	2008-07-03 19:41:22 +0000
+++ b/sql/sp_cache.h	2008-12-02 22:02:52 +0000
@@ -53,6 +53,7 @@ class sp_cache;
 */
 
 void sp_cache_init();
+void sp_cache_end();
 void sp_cache_clear(sp_cache **cp);
 void sp_cache_insert(sp_cache **cp, sp_head *sp);
 sp_head *sp_cache_lookup(sp_cache **cp, sp_name *name);

=== modified file 'sql/sql_class.cc'
--- a/sql/sql_class.cc	2008-10-24 21:39:19 +0000
+++ b/sql/sql_class.cc	2008-12-02 22:02:52 +0000
@@ -978,6 +978,14 @@ void add_diff_to_status(STATUS_VAR *to_v
     *(to++)+= *(from++) - *(dec++);
 }
 
+#define SECONDS_TO_WAIT_FOR_KILL 2
+#if !defined(__WIN__) && defined(HAVE_SELECT)
+/* my_sleep() can wait for sub second times */
+#define WAIT_FOR_KILL_TRY_TIMES 20
+#else
+#define WAIT_FOR_KILL_TRY_TIMES 2
+#endif
+
 
 void THD::awake(THD::killed_state state_to_set)
 {
@@ -1032,12 +1040,35 @@ void THD::awake(THD::killed_state state_
       we issue a second KILL or the status it's waiting for happens).
       It's true that we have set its thd->killed but it may not
       see it immediately and so may have time to reach the cond_wait().
+
+      We have to do the loop with trylock, because if we would use
+      pthread_mutex_lock(), we can cause a deadlock as we are here locking
+      the mysys_var->mutex and mysys_var->current_mutex in a different order
+      than in the thread we are trying to kill.
+      We only sleep for 2 seconds as we don't want to have LOCK_delete
+      locked too long time.
+
+      There is a small change we may not succeed in aborting a thread that
+      is not yet waiting for a mutex, but as this happens only for a
+      thread that was doing something else when the kill was issued and
+      which should detect the kill flag before it starts to wait, this
+      should be good enough.
     */
     if (mysys_var->current_cond && mysys_var->current_mutex)
     {
-      pthread_mutex_lock(mysys_var->current_mutex);
-      pthread_cond_broadcast(mysys_var->current_cond);
-      pthread_mutex_unlock(mysys_var->current_mutex);
+      uint i;
+      for (i= 0; i < WAIT_FOR_KILL_TRY_TIMES * SECONDS_TO_WAIT_FOR_KILL; i++)
+      {
+        int ret= pthread_mutex_trylock(mysys_var->current_mutex);
+        pthread_cond_broadcast(mysys_var->current_cond);
+        if (!ret)
+        {
+          /* Signal is sure to get through */
+          pthread_mutex_unlock(mysys_var->current_mutex);
+          break;
+        }
+      }
+      my_sleep(1000000L / WAIT_FOR_KILL_TRY_TIMES);
     }
     pthread_mutex_unlock(&mysys_var->mutex);
   }
@@ -1073,6 +1104,15 @@ bool THD::store_globals()
     created in another thread
   */
   thr_lock_info_init(&lock_info);
+
+#ifdef SAFE_MUTEX
+  /* Register order of mutex for wrong mutex deadlock detector */
+  pthread_mutex_lock(&LOCK_delete);
+  pthread_mutex_lock(&mysys_var->mutex);
+
+  pthread_mutex_unlock(&mysys_var->mutex);
+  pthread_mutex_unlock(&LOCK_delete);
+#endif
   return 0;
 }
 

=== modified file 'sql/sql_insert.cc'
--- a/sql/sql_insert.cc	2008-10-10 15:28:41 +0000
+++ b/sql/sql_insert.cc	2008-12-02 22:02:52 +0000
@@ -1719,7 +1719,8 @@ public:
     thd.system_thread= SYSTEM_THREAD_DELAYED_INSERT;
     thd.security_ctx->host_or_ip= "";
     bzero((char*) &info,sizeof(info));
-    pthread_mutex_init(&mutex,MY_MUTEX_INIT_FAST);
+    my_pthread_mutex_init(&mutex, MY_MUTEX_INIT_FAST, "Delayed_insert::mutex",
+                          0);
     pthread_cond_init(&cond,NULL);
     pthread_cond_init(&cond_client,NULL);
     VOID(pthread_mutex_lock(&LOCK_thread_count));
@@ -2224,7 +2225,8 @@ void kill_delayed_threads(void)
 	  in handle_delayed_insert()
 	*/
 	if (&di->mutex != di->thd.mysys_var->current_mutex)
-	  pthread_mutex_lock(di->thd.mysys_var->current_mutex);
+	  my_pthread_mutex_lock(di->thd.mysys_var->current_mutex, 
+                                MYF_NO_DEADLOCK_DETECTION);
 	pthread_cond_broadcast(di->thd.mysys_var->current_cond);
 	if (&di->mutex != di->thd.mysys_var->current_mutex)
 	  pthread_mutex_unlock(di->thd.mysys_var->current_mutex);
@@ -2470,13 +2472,14 @@ end:
     clients
   */
 
-  close_thread_tables(thd);			// Free the table
   di->table=0;
   di->dead= 1;                                  // If error
   thd->killed= THD::KILL_CONNECTION;	        // If error
-  pthread_cond_broadcast(&di->cond_client);	// Safety
   pthread_mutex_unlock(&di->mutex);
 
+  close_thread_tables(thd);			// Free the table
+  pthread_cond_broadcast(&di->cond_client);	// Safety
+
   pthread_mutex_lock(&LOCK_delayed_create);	// Because of delayed_get_table
   pthread_mutex_lock(&LOCK_delayed_insert);	
   delete di;

=== modified file 'sql/sql_show.cc'
--- a/sql/sql_show.cc	2008-10-10 15:28:41 +0000
+++ b/sql/sql_show.cc	2008-12-02 22:02:52 +0000
@@ -2102,7 +2102,6 @@ static bool show_status_array(THD *thd, 
         const char *pos, *end;                  // We assign a lot of const's
 
         pthread_mutex_lock(&LOCK_global_system_variables);
-
         if (show_type == SHOW_SYS)
         {
           show_type= ((sys_var*) value)->show_type();
@@ -2183,14 +2182,14 @@ static bool show_status_array(THD *thd, 
           DBUG_ASSERT(0);
           break;
         }
+        pthread_mutex_unlock(&LOCK_global_system_variables);
+
         restore_record(table, s->default_values);
         table->field[0]->store(name_buffer, strlen(name_buffer),
                                system_charset_info);
         table->field[1]->store(pos, (uint32) (end - pos), system_charset_info);
         table->field[1]->set_notnull();
 
-        pthread_mutex_unlock(&LOCK_global_system_variables);
-
         if (schema_table_store_record(thd, table))
           DBUG_RETURN(TRUE);
       }

=== modified file 'storage/innobase/handler/ha_innodb.cc'
--- a/storage/innobase/handler/ha_innodb.cc	2008-08-20 22:18:33 +0000
+++ b/storage/innobase/handler/ha_innodb.cc	2008-12-02 22:02:52 +0000
@@ -1659,8 +1659,15 @@ innobase_init(
 
 	srv_sizeof_trx_t_in_ha_innodb_cc = sizeof(trx_t);
 
+#ifdef SAFE_MUTEX
+        /* Disable deadlock detection as it's very slow for the buffer pool */
+        my_bool old_safe_mutex_deadlock_detector;
+        safe_mutex_deadlock_detector= 0;
+#endif
 	err = innobase_start_or_create_for_mysql();
-
+#ifdef SAFE_MUTEX
+        safe_mutex_deadlock_detector= old_safe_mutex_deadlock_detector;
+#endif
 	if (err != DB_SUCCESS) {
 		my_free(internal_innobase_data_file_path,
 						MYF(MY_ALLOW_ZERO_PTR));

=== modified file 'storage/maria/ha_maria.cc'
--- a/storage/maria/ha_maria.cc	2008-11-03 13:53:22 +0000
+++ b/storage/maria/ha_maria.cc	2008-12-02 22:02:52 +0000
@@ -1460,7 +1460,7 @@ int ha_maria::repair(THD *thd, HA_CHECK 
                                      (local_testflag &
                                       T_STATISTICS ? UPDATE_STAT : 0));
     info(HA_STATUS_NO_LOCK | HA_STATUS_TIME | HA_STATUS_VARIABLE |
-         HA_STATUS_CONST);
+         HA_STATUS_CONST, 0);
     if (rows != file->state->records && !(param->testflag & T_VERY_SILENT))
     {
       char llbuff[22], llbuff2[22];
@@ -2148,6 +2148,11 @@ void ha_maria::position(const uchar *rec
 
 int ha_maria::info(uint flag)
 {
+  return info(flag, table->s->tmp_table == NO_TMP_TABLE);
+}
+
+int ha_maria::info(uint flag, my_bool lock_table_share)
+{
   MARIA_INFO maria_info;
   char name_buff[FN_REFLEN];
 
@@ -2173,7 +2178,7 @@ int ha_maria::info(uint flag)
     stats.block_size= maria_block_size;
 
     /* Update share */
-    if (share->tmp_table == NO_TMP_TABLE)
+    if (lock_table_share)
       pthread_mutex_lock(&share->mutex);
     share->keys_in_use.set_prefix(share->keys);
     share->keys_in_use.intersect_extended(maria_info.key_map);
@@ -2186,7 +2191,7 @@ int ha_maria::info(uint flag)
       for (end= to+ share->key_parts ; to < end ; to++, from++)
         *to= (ulong) (*from + 0.5);
     }
-    if (share->tmp_table == NO_TMP_TABLE)
+    if (lock_table_share)
       pthread_mutex_unlock(&share->mutex);
 
     /*

=== modified file 'storage/maria/ha_maria.h'
--- a/storage/maria/ha_maria.h	2008-10-31 23:14:58 +0000
+++ b/storage/maria/ha_maria.h	2008-12-02 22:02:52 +0000
@@ -112,6 +112,7 @@ public:
   int restart_rnd_next(uchar * buf);
   void position(const uchar * record);
   int info(uint);
+  int info(uint, my_bool);
   int extra(enum ha_extra_function operation);
   int extra_opt(enum ha_extra_function operation, ulong cache_size);
   int reset(void);

=== modified file 'storage/maria/ma_close.c'
--- a/storage/maria/ma_close.c	2008-10-15 12:44:31 +0000
+++ b/storage/maria/ma_close.c	2008-12-02 22:02:52 +0000
@@ -114,6 +114,7 @@ int maria_close(register MARIA_HA *info)
     }
 #ifdef THREAD
     thr_lock_delete(&share->lock);
+    (void) pthread_mutex_destroy(&share->key_del_lock);
     {
       int i,keys;
       keys = share->state.header.keys;
@@ -162,14 +163,10 @@ int maria_close(register MARIA_HA *info)
   pthread_mutex_unlock(&share->intern_lock);
   if (share_can_be_freed)
   {
-    VOID(pthread_mutex_destroy(&share->intern_lock));
+    (void) pthread_mutex_destroy(&share->intern_lock);
     my_free((uchar *)share, MYF(0));
   }
-  if (info->ftparser_param)
-  {
-    my_free((uchar*)info->ftparser_param, MYF(0));
-    info->ftparser_param= 0;
-  }
+  my_free(info->ftparser_param, MYF(MY_ALLOW_ZERO_PTR));
   if (info->dfile.file >= 0)
   {
     /*

=== modified file 'storage/maria/ma_key.c'
--- a/storage/maria/ma_key.c	2008-11-04 10:54:04 +0000
+++ b/storage/maria/ma_key.c	2008-12-02 22:02:52 +0000
@@ -67,14 +67,16 @@ static int _ma_put_key_in_record(MARIA_H
   Prefix bytes 244 to 249 are reserved for negative transid, that can be used
   when we pack transid relative to each other on a key block.
 
-  We have to store transid in high-byte-first order to be able to do a
-  fast byte-per-byte comparision of them without packing them up.
+  We have to store transid in high-byte-first order so that we can compare
+  them unpacked byte per byte and as soon we find a difference we know
+  which is smaller.
 
   For example, assuming we the following data:
 
   key_data:               1                (4 byte integer)
   pointer_to_row:         2 << 8 + 3 = 515 (page 2, row 3)
-  table_create_transid    1000             Defined at create table time
+  table_create_transid    1000             Defined at create table time and
+                                           stored in table definition
   transid                 1010	           Transaction that created row
   delete_transid          2011             Transaction that deleted row
 
@@ -86,8 +88,9 @@ static int _ma_put_key_in_record(MARIA_H
 
   00 00 00 01     Key data (1 stored high byte first)
   00 00 00 47	  (515 << 1) + 1         ;  The last 1 is marker that key cont.
-  15              ((1000-1010) << 1) + 1 ;  The last 1 is marker that key cont.
-  FB 07 E6        length byte and  ((2011 - 1000) << 1) = 07 E6
+  15              ((1010-1000) << 1) + 1 ;  The last 1 is marker that key cont.
+  FB 07 E6        Length byte (FE = 249 + 2 means 2 bytes) and 
+                  ((2011 - 1000) << 1) = 07 E6
 */
 
 uint transid_store_packed(MARIA_HA *info, uchar *to, ulonglong trid)

=== modified file 'storage/maria/ma_loghandler.c'
--- a/storage/maria/ma_loghandler.c	2008-10-31 13:06:56 +0000
+++ b/storage/maria/ma_loghandler.c	2008-12-02 22:02:52 +0000
@@ -1431,7 +1431,9 @@ static my_bool translog_buffer_init(stru
   /* list of waiting buffer ready threads */
   buffer->waiting_flush= 0;
   /* lock for the buffer. Current buffer also lock the handler */
-  if (pthread_mutex_init(&buffer->mutex, MY_MUTEX_INIT_FAST) ||
+  if (my_pthread_mutex_init(&buffer->mutex, MY_MUTEX_INIT_FAST,
+                            "translog_buffer->mutex",
+                            MYF_NO_DEADLOCK_DETECTION) ||
       pthread_cond_init(&buffer->prev_sent_to_disk_cond, 0))
     DBUG_RETURN(1);
   buffer->is_closing_buffer= 0;

=== modified file 'storage/maria/ma_state.c'
--- a/storage/maria/ma_state.c	2008-11-03 13:53:22 +0000
+++ b/storage/maria/ma_state.c	2008-12-02 22:02:52 +0000
@@ -366,6 +366,15 @@ my_bool _ma_check_status(void *param)
 
 /**
    @brief write hook at end of trans to store status for all used table
+
+   @Notes
+   This function must be called under trnman_lock in trnman_end_trn()
+   because of the following reasons:
+   - After trnman_end_trn() is called, the current transaction will be
+   regarded as committed and all used tables state_history will be
+   visible to other transactions.  To do this, we loop over all used
+   tables and create/update a history entries that contains the correct
+   state_history for them.
 */
 
 my_bool _ma_trnman_end_trans_hook(TRN *trn, my_bool commit,
@@ -390,6 +399,13 @@ my_bool _ma_trnman_end_trans_hook(TRN *t
           trnman_exists_active_transactions(share->state_history->trid,
                                             trn->commit_trid, 1))
       {
+        /*
+          There exist transactions that are still using the current
+          share->state_history.  Create a new history item for this
+          commit and add it first in the state_history list. This
+          ensures that all history items are stored in the list in
+          decresing trid order.
+        */
         if (!(history= my_malloc(sizeof(*history), MYF(MY_WME))))
         {
           /* purecov: begin inspected */

=== modified file 'storage/maria/ma_test1.c'
--- a/storage/maria/ma_test1.c	2008-06-30 09:13:08 +0000
+++ b/storage/maria/ma_test1.c	2008-12-02 22:02:52 +0000
@@ -70,6 +70,9 @@ extern int _ma_flush_table_files(MARIA_H
 
 int main(int argc,char *argv[])
 {
+#if defined(SAFE_MUTEX) && defined(THREAD)
+  safe_mutex_deadlock_detector= 1;
+#endif
   MY_INIT(argv[0]);
   get_options(argc,argv);
   maria_data_root= (char *)".";

=== modified file 'storage/maria/ma_test2.c'
--- a/storage/maria/ma_test2.c	2008-06-26 05:18:28 +0000
+++ b/storage/maria/ma_test2.c	2008-12-02 22:02:52 +0000
@@ -72,6 +72,10 @@ int main(int argc, char *argv[])
   const char *filename;
   char *blob_buffer;
   MARIA_CREATE_INFO create_info;
+
+#if defined(SAFE_MUTEX) && defined(THREAD)
+  safe_mutex_deadlock_detector= 1;
+#endif
   MY_INIT(argv[0]);
 
   filename= "test2";

Thread
bzr commit into MySQL/Maria:mysql-maria branch (monty:2701) WL#3262Michael Widenius2 Dec
  • Re: bzr commit into MySQL/Maria:mysql-maria branch (monty:2701) WL#3262Guilhem Bichot12 Dec
    • Re: bzr commit into MySQL/Maria:mysql-maria branch (monty:2701) WL#3262Michael Widenius15 Dec