MySQL Lists are EOL. Please join:

List:Replication« Previous MessageNext Message »
From:周振兴 Date:September 15 2011 2:20am
Subject:Re: A feature patch for semi-sync
View as plain text  
Hi Mark

Thank you very much for your great advise. And Feel free to give more advice
or "criticize". :-)

2011/9/15 MARK CALLAGHAN <mdcallag@stripped>

> 2011/9/13 周振兴 <orczhou@stripped>:
> > Hi all
> >    Thanks, Mark. Here are more details and my test data.
> > 2011/9/14 MARK CALLAGHAN <mdcallag@stripped>
> >>
> >> I think you should describe this in more detail for others to be
> >> interested. What you have done is very interesting, but also not
> >> trivial for us to understand.
> >
> > In MySQL-5.5 the semi-sync code is something like this:
> > original semi-sync
> >     binlog_prepare (do nothing)
> >     innodb_prepare
> >     ...
> >     binlog_commit
> >     innobase_commit
> > Now rpl_semi_sync_master_wait_before_commit=1
> >     binlog_prepare (do nothing)
> >     innodb_prepare
> >     ...
> >     binlog_commit
> >     innobase_commit
> Without looking at 5.5 code "WAITING FOR THE SLAVE" is done while that
> connection holds prepare_commit_mutex so all commits are blocked until
> the wait ends. From looking at your performance number I assume you
> have tested this on a system with a real (5 milliseconds or greater)
> fsync latency. Assuming that the binlog fsync takes 10ms and the
> average wait time for a slave to ack is 1ms, then your change won't do
> much to limit throughput. But on a system with HW RAID or SSD/flash
> then fsync latency is much lower and your change will reduce
> performance by much more. The overhead from this will be also much
> larger on versions of MySQL that use group commit for InnoDB+binlog
> (which is in MariaDB, Percona and the Facebook patch).
Once turn rpl_semi_sync_master_wait_before_commit ON, the network will make
every transaction slower 0.5ms(this rely on the network). The overhead of
0.5ms is big or not, as you said, if the rest works of transaction( retrieve
page, fsync the page etc. ) take *less* time , the overhead is *bigger*. For
example, if a cpu bound test with FusionIO/Virident card, 0.5ms may be a big

Turning Semi-sync on, it mean data is more safety, with less performance.
Since the network is good and the hardware is fabulous, even
turning Semi-sync /  rpl_semi_sync_master_wait_before_commit ON , still
MySQL can finish thousands transaction per second.

Still i'm trying to finger out how this affect Group Commit.

> I am not trying to criticize your work. I think is great work and hope

Actually thanks a lot for your advice. Feel free to give more advice or
"criticize". :-)

> you present this at one of the community conferences. But can you make
> the overhead of this low for systems with a fast fsync?  When fsync is
> fast and the workload has few concurrent transactions there isn't much
> that can be done to avoid the overhead. But for workloads with a lot
> of concurrency and on a MySQL variant that supports group commit it
> might be possible to minimize the overhead from this.

For checking rpl_semi_sync_master_wait_before_commit always working, a test
case has been finished and will be commit on launchpad later(using MySQL
Debug Sync Facility).

> --
> Mark Callaghan
> mdcallag@stripped

周振兴  159-9004-0105

A feature patch for semi-sync周振兴14 Sep
  • Re: A feature patch for semi-syncMARK CALLAGHAN14 Sep
    • Re: A feature patch for semi-sync周振兴14 Sep
      • Re: A feature patch for semi-syncMARK CALLAGHAN15 Sep
        • Re: A feature patch for semi-sync周振兴15 Sep
        • Re: A feature patch for semi-syncKristian Nielsen15 Sep
          • Re: A feature patch for semi-sync周振兴23 Sep
            • Re: A feature patch for semi-syncMARK CALLAGHAN23 Sep
              • Re: A feature patch for semi-sync周振兴23 Sep
  • Re: A feature patch for semi-sync周振兴20 Sep
    • Re: A feature patch for semi-syncMats Kindahl20 Sep
      • Re: A feature patch for semi-sync周振兴21 Sep
        • Re: A feature patch for semi-syncMats Kindahl21 Sep