List:Internals« Previous MessageNext Message »
From:Mats Kindahl Date:February 3 2013 3:36am
Subject:Re: reducing fsyncs during handlerton->prepare and handlerton->commit
in 5.6
View as plain text  
On 01/31/2013 10:41 AM, Kristian Nielsen wrote:
> Mats Kindahl <mats.kindahl@stripped> writes:
>
>> In MySQL 5.6, there are no new APIs that you *have* to comply with.
> But MySQL 5.6 serialises calls to the commit handlerton method, the next one
> cannot start before the previous one completes. So if you did fsync() with
> group commit before in commit, your group commit will no longer work in 5.6
> and you will get a serious performance regression if you do not honour the
> HA_IGNORE_DURABILITY flag. I consider that breaking the storage engine API, as
> you see Mats and I disagree a bit on that point :-)

Yes, we do. :)

>
> Anyway, it should be easy to do for you. MySQL 5.6 sets HA_IGNORE_DURABILITY,
> this has similar semantics to when MariaDB 5.3+ calls the commit_ordered()
> method. So you can probably use the same code for both with a small amount of
> #ifdef.
>
> Note that for MySQL 5.6 you need to implement also the flush_logs() method to
> fsync() all prior commits durably to disk (if you did not already implement
> it). What happens is basically that in MySQL crash recovery looks only at the
> last binlog file written. So it calls flush_logs() before creating a new
> binlog, and storage engine must ensure that all commits become durable at that
> point. Otherwise commits may be lost if a crash happens just after binlog
> rotation.

Yup.

>
> This is actually the _only_ reason that fsync() was ever needed in commit, to
> ensure that it is done when binlog is rotated. So it is rather silly that we
> have done it for _every_ commit for so long. Anyway, it will be fixed now.

Not really, there's another issue with not fsync:ing every transaction
in that they can be sent to the slaves before they are committed on the
master.

This is not necessarily bad, but it could by some be considered unexpected.

/Matz

>
>  - Kristian.
>

-- 
Senior Principal Software Developer
Oracle, MySQL Department

Thread
reducing fsyncs during handlerton->prepare and handlerton->commit in 5.6Zardosht Kasheff30 Jan
  • Re: reducing fsyncs during handlerton->prepare and handlerton->commit in 5.6Kristian Nielsen31 Jan
  • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Mats Kindahl31 Jan
    • Re: reducing fsyncs during handlerton->prepare and handlerton->commit in 5.6Kristian Nielsen31 Jan
      • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Zardosht Kasheff31 Jan
        • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Mats Kindahl31 Jan
          • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Zardosht Kasheff31 Jan
            • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Mats Kindahl1 Feb
              • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Zardosht Kasheff1 Feb
                • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Mats Kindahl1 Feb
      • Re: reducing fsyncs during handlerton->prepare and handlerton->commitin 5.6Mats Kindahl3 Feb