List:Internals« Previous MessageNext Message »
From:Stewart Smith Date:May 2 2014 10:29pm
Subject:InnoDB mutex atomics implementation incorrect on POWER and ARM
View as plain text  
Hi all,

seeing as the page to register an account for filing a MySQL bug doesn't
work (even from a text based browser) and just times out or gives a 500
Internal Server Error.... I'll "file" this one here.

So, at least for MySQL 5.6 (I haven't looked at previous versions, but I
strongly suspect the last few have this as well), the code in
sync0sync.ic for manipulating and loocking at mutex->lock_word is
missing a barrier.

This would affect multi CPU systems of both POWER and ARM (although I
haven't tried on ARM). I could incredibly reliably reproduce on a POWER8
system but likely also hit on POWER7.

I've reproduced *very* reliably with a frest MySQL 5.6.17 build (from
source) on a big endian host (e.g. Fedora) with a simple mysqlslap run:
mysqlslap --concurrency=128 --iterations=2 --number-int-cols=2
--number-char-cols=3 --auto-generate-sql --number-of-queries=10000

(this could probably be paired down)

What's interesting to note is that the test suite passes but a simple
very small stress test like the above makes the server hit an assert in
the InnoDB list code (in create read view) that's protected by.. yep,
one of these mutexes. Obviously nobody's hammering this too hard on
multicore ARM yet.

If you disable the use of GCC atomics for these mutexes and instead fall
back to pthread mutex, it all works fine.

The fix is fairly simple:
1) ib_mutex_test_and_set needs a __sync_synchronize() before the call to
os_atomic_test_and_set_byte()
2) mutex_reset_lock_word needs a __sync_synchronize() before
os_atomic_test_and_set_byte() call
3) and  __sync_synchronize() is needed after the read in
mutex_get_lock_word().

The barriers aren't required on x86.

MariaDB, Percona Server and Percona XtraBackup are probably also
affected, although I haven't tried these.

Thread
InnoDB mutex atomics implementation incorrect on POWER and ARMStewart Smith2 May 2014
  • Re: InnoDB mutex atomics implementation incorrect on POWER and ARMMorgan Tocker5 May 2014
    • Re: InnoDB mutex atomics implementation incorrect on POWER and ARMStewart Smith6 May 2014
  • Re: InnoDB mutex atomics implementation incorrect on POWER and ARMMorgan Tocker5 May 2014