List:Replication« Previous MessageNext Message »
From:周振兴 Date:September 23 2011 2:11pm
Subject:Re: A feature patch for semi-sync
View as plain text  
2011/9/23 MARK CALLAGHAN <mdcallag@stripped>

> I think the next challenge is to get this to work with group commit.
> But maybe that is not your challenge -- if your workload doesn't need
> it.
>

Everyone need group commit :) . But it's real complicated ,i am just diving
into the details.


> 2011/9/23 周振兴 <orczhou@stripped>:
> > Hi MARK CALLAGHAN
> > After a few reading about MySQL group commit, It seems that only
> >   ***"The 5.3 branch for MariaDB"*** (as you say) and
> >   ***"Percona Server 5.1.57 with the Group Commit Fix applied"***
> > support the group commit with :innodb_flush_log_at_trx_commit=1,
> > sync_binlog=1.
> > The test data listed in my last mail is
> > under innodb_flush_log_at_trx_commit=1, sync_binlog=0.
> > New test data about my "Enhanced Semi-sync" with Percona-MySQL5.5.15 :
> > thread         1  4     16    32
> > sync_binlog=1  182.3  329.4 416.1 415.7
> > So under innodb_flush_log_at_trx_commit=1, sync_binlog=1, There are no
> Group
> > Commit.
> > if only  innodb_flush_log_at_trx_commit=1 group commit works fine.
> >
> > On Thu, Sep 15, 2011 at 3:46 PM, Kristian Nielsen <
> knielsen@stripped>
> > wrote:
> >>
> >> [Thanks, Mark, for pointing me to this thread]
> >>
> >> MARK&nbsp;CALLAGHAN writes:
> >>
> >> > 2011/9/13 周振兴 <orczhou@stripped>:
> >>
> >> >> In MySQL-5.5 the semi-sync code is something like this:
> >> >> original semi-sync
> >> >>     binlog_prepare (do nothing)
> >> >>     innodb_prepare
> >> >>     ...
> >> >>     binlog_commit
> >> >>     innobase_commit
> >> >>     WAITING FOR THE SLAVE
> >> >> Now rpl_semi_sync_master_wait_before_commit=1
> >> >>     binlog_prepare (do nothing)
> >> >>     innodb_prepare
> >> >>     ...
> >> >>     binlog_commit
> >> >>     WAITING FOR THE SLAVE
> >> >>     innobase_commit
> >> >
> >>
> >> > performance by much more. The overhead from this will be also much
> >> > larger on versions of MySQL that use group commit for InnoDB+binlog
> >> > (which is in MariaDB, Percona and the Facebook patch).
> >>
> >> > you present this at one of the community conferences. But can you make
> >> > the overhead of this low for systems with a fast fsync?  When fsync is
> >> > fast and the workload has few concurrent transactions there isn't much
> >> > that can be done to avoid the overhead. But for workloads with a lot
> >> > of concurrency and on a MySQL variant that supports group commit it
> >> > might be possible to minimize the overhead from this.
> >>
> >> For MariaDB 5.3/5.5, which supports binlog group commit, I think it
> should
> >> be
> >> simple to minimize the overhead in this way.
> >>
> >> What we have there is basically a linked list of all the transactions
> that
> >> are
> >> waiting to commit in parallel. All of these transactions are
> >> written/fsynced
> >> to the binlog together. After this they are made visible (ie. committed
> >> inside
> >> innodb) together.
> >>
> >> So if rpl_semi_sync_master_wait_before_commit=1, then after syncing the
> >> binlog
> >> we just need to wait for the slave to acknowledge reception of the
> _last_
> >> transaction before we proceed to the next step of making them
> >> visible/committed in innodb:
> >>
> >>    foreach (transaction)
> >>      write to binlog
> >>    sync binlog
> >>    WAITING FOR THE SLAVE
> >>    foreach (transaction)
> >>      innobase_commit (first part)
> >>    foreach (transaction)
> >>      innobase_commit (second part)
> >>
> >> The overhead of waiting for 1 or 10 transactions to be acknowledged from
> >> the
> >> slave is about the same (for small transactions). So if many
> transactions
> >> commit in parallel, the overall overhead is minimized.
> >>
> >> Of course, there are some additional details to understand, but I think
> >> overall it should be simple.
> >>
> >> I already merged this MariaDB group commit with MySQL 5.5, however the
> >> rest of
> >> the merge is still being polished, so it will be a few weeks before it
> >> becomes
> >> ready. If you are interested, let me know and I will then point you to
> the
> >> relevant trees, files, etc. when we have them ready.
> >>
> >>  - Kristian.
> >
> >
> >
> > --
> > 此致
> >     敬礼
> > -----------------------------------------------------
> > 周振兴  159-9004-0105 Taobao.com
> > http://orczhou.com
> >
>
>
>
> --
> Mark Callaghan
> mdcallag@stripped
>



-- 
此致
    敬礼
-----------------------------------------------------
周振兴  159-9004-0105 Taobao.com
http://orczhou.com

Thread
A feature patch for semi-sync周振兴14 Sep
  • Re: A feature patch for semi-syncMARK CALLAGHAN14 Sep
    • Re: A feature patch for semi-sync周振兴14 Sep
      • Re: A feature patch for semi-syncMARK CALLAGHAN15 Sep
        • Re: A feature patch for semi-sync周振兴15 Sep
        • Re: A feature patch for semi-syncKristian Nielsen15 Sep
          • Re: A feature patch for semi-sync周振兴23 Sep
            • Re: A feature patch for semi-syncMARK CALLAGHAN23 Sep
              • Re: A feature patch for semi-sync周振兴23 Sep
  • Re: A feature patch for semi-sync周振兴20 Sep
    • Re: A feature patch for semi-syncMats Kindahl20 Sep
      • Re: A feature patch for semi-sync周振兴21 Sep
        • Re: A feature patch for semi-syncMats Kindahl21 Sep