List:Replication« Previous MessageNext Message »
From:周振兴 Date:September 23 2011 10:42am
Subject:Re: A feature patch for semi-sync
View as plain text  
Hi MARK CALLAGHAN

After a few reading about MySQL group commit, It seems that only
  ***"The 5.3 branch for
MariaDB<http://www.facebook.com/note.php?note_id=10150211546215933>"***
(as you say) and
  ***"Percona Server 5.1.57 with the Group Commit Fix
applied<http://www.mysqlperformanceblog.com/2011/07/13/testing-the-group-commit-fix/>
"***
support the group commit with :innodb_flush_log_at_trx_commit=1,
sync_binlog=1.

The test data listed in my last mail is under innodb_flush_log_at_trx_commit=1,
sync_binlog=0.

New test data about my "Enhanced
Semi-sync<http://code.google.com/p/enhanced-semi-sync-replication/>"
with Percona-MySQL5.5.15 :

thread         1  4     16    32
sync_binlog=1  182.3  329.4 416.1 415.7

So under innodb_flush_log_at_trx_commit=1, sync_binlog=1, There are no Group
Commit.

if only  innodb_flush_log_at_trx_commit=1 group commit works
fine<http://www.innodb.com/wp/products/innodb_plugin/plugin-performance/innodb-plugin-1-0-4-group-commit-test-sysbench/>
.


On Thu, Sep 15, 2011 at 3:46 PM, Kristian Nielsen
<knielsen@stripped>wrote:

> [Thanks, Mark, for pointing me to this thread]
>
> MARK&nbsp;CALLAGHAN writes:
>
> > 2011/9/13 周振兴 <orczhou@stripped>:
>
> >> In MySQL-5.5 the semi-sync code is something like this:
> >> original semi-sync
> >>     binlog_prepare (do nothing)
> >>     innodb_prepare
> >>     ...
> >>     binlog_commit
> >>     innobase_commit
> >>     WAITING FOR THE SLAVE
> >> Now rpl_semi_sync_master_wait_before_commit=1
> >>     binlog_prepare (do nothing)
> >>     innodb_prepare
> >>     ...
> >>     binlog_commit
> >>     WAITING FOR THE SLAVE
> >>     innobase_commit
> >
>
> > performance by much more. The overhead from this will be also much
> > larger on versions of MySQL that use group commit for InnoDB+binlog
> > (which is in MariaDB, Percona and the Facebook patch).
>
> > you present this at one of the community conferences. But can you make
> > the overhead of this low for systems with a fast fsync?  When fsync is
> > fast and the workload has few concurrent transactions there isn't much
> > that can be done to avoid the overhead. But for workloads with a lot
> > of concurrency and on a MySQL variant that supports group commit it
> > might be possible to minimize the overhead from this.
>
> For MariaDB 5.3/5.5, which supports binlog group commit, I think it should
> be
> simple to minimize the overhead in this way.
>
> What we have there is basically a linked list of all the transactions that
> are
> waiting to commit in parallel. All of these transactions are
> written/fsynced
> to the binlog together. After this they are made visible (ie. committed
> inside
> innodb) together.
>
> So if rpl_semi_sync_master_wait_before_commit=1, then after syncing the
> binlog
> we just need to wait for the slave to acknowledge reception of the _last_
> transaction before we proceed to the next step of making them
> visible/committed in innodb:
>
>    foreach (transaction)
>      write to binlog
>    sync binlog
>    WAITING FOR THE SLAVE
>    foreach (transaction)
>      innobase_commit (first part)
>    foreach (transaction)
>      innobase_commit (second part)
>
> The overhead of waiting for 1 or 10 transactions to be acknowledged from
> the
> slave is about the same (for small transactions). So if many transactions
> commit in parallel, the overall overhead is minimized.
>
> Of course, there are some additional details to understand, but I think
> overall it should be simple.
>
> I already merged this MariaDB group commit with MySQL 5.5, however the rest
> of
> the merge is still being polished, so it will be a few weeks before it
> becomes
> ready. If you are interested, let me know and I will then point you to the
> relevant trees, files, etc. when we have them ready.
>
>  - Kristian.
>



-- 
此致
    敬礼
-----------------------------------------------------
周振兴  159-9004-0105 Taobao.com
http://orczhou.com

Thread
A feature patch for semi-sync周振兴14 Sep
  • Re: A feature patch for semi-syncMARK CALLAGHAN14 Sep
    • Re: A feature patch for semi-sync周振兴14 Sep
      • Re: A feature patch for semi-syncMARK CALLAGHAN15 Sep
        • Re: A feature patch for semi-sync周振兴15 Sep
        • Re: A feature patch for semi-syncKristian Nielsen15 Sep
          • Re: A feature patch for semi-sync周振兴23 Sep
            • Re: A feature patch for semi-syncMARK CALLAGHAN23 Sep
              • Re: A feature patch for semi-sync周振兴23 Sep
  • Re: A feature patch for semi-sync周振兴20 Sep
    • Re: A feature patch for semi-syncMats Kindahl20 Sep
      • Re: A feature patch for semi-sync周振兴21 Sep
        • Re: A feature patch for semi-syncMats Kindahl21 Sep