List:Replication« Previous MessageNext Message »
From:MARK CALLAGHAN Date:September 23 2011 1:53pm
Subject:Re: A feature patch for semi-sync
View as plain text  
I think the next challenge is to get this to work with group commit.
But maybe that is not your challenge -- if your workload doesn't need
it.

2011/9/23 周振兴 <orczhou@stripped>:
> Hi MARK CALLAGHAN
> After a few reading about MySQL group commit, It seems that only
>   ***"The 5.3 branch for MariaDB"*** (as you say) and
>   ***"Percona Server 5.1.57 with the Group Commit Fix applied"***
> support the group commit with :innodb_flush_log_at_trx_commit=1,
> sync_binlog=1.
> The test data listed in my last mail is
> under innodb_flush_log_at_trx_commit=1, sync_binlog=0.
> New test data about my "Enhanced Semi-sync" with Percona-MySQL5.5.15 :
> thread         1  4     16    32
> sync_binlog=1  182.3  329.4 416.1 415.7
> So under innodb_flush_log_at_trx_commit=1, sync_binlog=1, There are no Group
> Commit.
> if only  innodb_flush_log_at_trx_commit=1 group commit works fine.
>
> On Thu, Sep 15, 2011 at 3:46 PM, Kristian Nielsen <knielsen@stripped>
> wrote:
>>
>> [Thanks, Mark, for pointing me to this thread]
>>
>> MARK&nbsp;CALLAGHAN writes:
>>
>> > 2011/9/13 周振兴 <orczhou@stripped>:
>>
>> >> In MySQL-5.5 the semi-sync code is something like this:
>> >> original semi-sync
>> >>     binlog_prepare (do nothing)
>> >>     innodb_prepare
>> >>     ...
>> >>     binlog_commit
>> >>     innobase_commit
>> >>     WAITING FOR THE SLAVE
>> >> Now rpl_semi_sync_master_wait_before_commit=1
>> >>     binlog_prepare (do nothing)
>> >>     innodb_prepare
>> >>     ...
>> >>     binlog_commit
>> >>     WAITING FOR THE SLAVE
>> >>     innobase_commit
>> >
>>
>> > performance by much more. The overhead from this will be also much
>> > larger on versions of MySQL that use group commit for InnoDB+binlog
>> > (which is in MariaDB, Percona and the Facebook patch).
>>
>> > you present this at one of the community conferences. But can you make
>> > the overhead of this low for systems with a fast fsync?  When fsync is
>> > fast and the workload has few concurrent transactions there isn't much
>> > that can be done to avoid the overhead. But for workloads with a lot
>> > of concurrency and on a MySQL variant that supports group commit it
>> > might be possible to minimize the overhead from this.
>>
>> For MariaDB 5.3/5.5, which supports binlog group commit, I think it should
>> be
>> simple to minimize the overhead in this way.
>>
>> What we have there is basically a linked list of all the transactions that
>> are
>> waiting to commit in parallel. All of these transactions are
>> written/fsynced
>> to the binlog together. After this they are made visible (ie. committed
>> inside
>> innodb) together.
>>
>> So if rpl_semi_sync_master_wait_before_commit=1, then after syncing the
>> binlog
>> we just need to wait for the slave to acknowledge reception of the _last_
>> transaction before we proceed to the next step of making them
>> visible/committed in innodb:
>>
>>    foreach (transaction)
>>      write to binlog
>>    sync binlog
>>    WAITING FOR THE SLAVE
>>    foreach (transaction)
>>      innobase_commit (first part)
>>    foreach (transaction)
>>      innobase_commit (second part)
>>
>> The overhead of waiting for 1 or 10 transactions to be acknowledged from
>> the
>> slave is about the same (for small transactions). So if many transactions
>> commit in parallel, the overall overhead is minimized.
>>
>> Of course, there are some additional details to understand, but I think
>> overall it should be simple.
>>
>> I already merged this MariaDB group commit with MySQL 5.5, however the
>> rest of
>> the merge is still being polished, so it will be a few weeks before it
>> becomes
>> ready. If you are interested, let me know and I will then point you to the
>> relevant trees, files, etc. when we have them ready.
>>
>>  - Kristian.
>
>
>
> --
> 此致
>     敬礼
> -----------------------------------------------------
> 周振兴  159-9004-0105 Taobao.com
> http://orczhou.com
>



-- 
Mark Callaghan
mdcallag@stripped
Thread
A feature patch for semi-sync周振兴14 Sep
  • Re: A feature patch for semi-syncMARK CALLAGHAN14 Sep
    • Re: A feature patch for semi-sync周振兴14 Sep
      • Re: A feature patch for semi-syncMARK CALLAGHAN15 Sep
        • Re: A feature patch for semi-sync周振兴15 Sep
        • Re: A feature patch for semi-syncKristian Nielsen15 Sep
          • Re: A feature patch for semi-sync周振兴23 Sep
            • Re: A feature patch for semi-syncMARK CALLAGHAN23 Sep
              • Re: A feature patch for semi-sync周振兴23 Sep
  • Re: A feature patch for semi-sync周振兴20 Sep
    • Re: A feature patch for semi-syncMats Kindahl20 Sep
      • Re: A feature patch for semi-sync周振兴21 Sep
        • Re: A feature patch for semi-syncMats Kindahl21 Sep