List:Internals« Previous MessageNext Message »
From:Yuan Wang Date:November 25 2009 8:56am
Subject:Re: Primary key is miss in row based replication.
View as plain text  
Hi Mats

Sorry but I can't catch your meaning. Calling bitmap_set_all() in
binlog_log_row() has nothing to do with read_set, and the engine read
just the columns needed in read_set. So calling bitmap_set_all()  will
write all columns in before_record. However, those columns that are
not denoted in read_set is already missing in before_record, for the
engine just doesn't read them. So you will write out some NULLs or
garbage.

Is that right?

On Wed, Nov 25, 2009 at 6:45 AM, Mats Kindahl <mats@stripped> wrote:
> Hi Yuan!
>
> Sorry for the late reply, I'm trying to juggle too many things currently.
>
> It is good that you managed to solve your problem, but there is still something
> that seems strange that I would like to make you aware of.
>
> Yuan Wang wrote:
>> Hi Mats
>>
>> I have checked the code of 5.1 (current head version of 5.1 branch in
>> launchpad), and it don't seem to fill missing attributes automatically
>> when RBR is used.
>>
>> The patch (http://lists.mysql.com/commits/40151) of bug 33055
>> (http://bugs.mysql.com/bug.php?id=33055) does the following change to
>> st_table::mark_columns_needed_for_delete().
>>
>> -  if (file->ha_table_flags() & HA_PRIMARY_KEY_REQUIRED_FOR_DELETE)
>> +  if (file->ha_table_flags() & HA_PRIMARY_KEY_REQUIRED_FOR_DELETE
> ||
>> +      mysql_bin_log.is_open() && in_use &&
>> +      in_use->current_stmt_binlog_row_based)
>>
>> This patch has been pushed into 6.0. However,
>> st_table::mar_columns_needed_for_update don't have this change.
>
> That is correct, and the reason is because there were a patch pushed to 6.0 to
> make Falcon work correctly that made row-based replication only replicate the
> columns that are explicitly marked, but no other.
>
> However, for 5.1, the full row is written to the binary log regardless of what
> columns that are marked in the write_set.
>
> You can see this in the binlog_log_row function, where the columns are set in
> the following code (I marked the line):
>
>    /*
>      If there are no table maps written to the binary log, this is
>      the first row handled in this statement. In that case, we need
>      to write table maps for all locked tables to the binary log.
>    */
>    if (likely(!(error= bitmap_init(&cols,
>                        
>            use_bitbuf ? bitbuf : NULL,
>                        
>            (n_fields + 7) & ~7UL,
>                        
>            FALSE))))
>    {
>>>>   bitmap_set_all(&cols);
>      if (likely(!(error= write_locked_table_maps(thd))))
>        error= (*log_func)(thd, table,
> table->file->has_transactions(),
>                        
>   &cols, table->s->fields,
>                        
>   before_record, after_record);
>
>> It seems that if my storage engine mark the
>> HA_PRIMARY_KEY_REQUIRED_FOR_DELETE flag in table_flags, then the
>> server will add primary key to binlog automatically, which is just
>> what we need.
>
> Yes, that is correct, but I am still surprised that it affects replication, for
> the reason given above.
>
> Just my few cents,
> Mats Kindahl
>
>>
>> On Fri, Nov 20, 2009 at 4:04 PM, Mats Kindahl <mats@stripped> wrote:
>>> Hi Yuan,
>>>
>>> In 5.1, the full row is sent over to the slave, so I am a little surprised
> that
>>> there is anything missing at all.
>>>
>>> Do you have an example I can look at?
>>>
>>> Best wishes,
>>> Mats Kindahl
>>>
>>> Yuan Wang wrote:
>>>> We are developed a storage engine using row based replication. During
>>>> the test we found that primary key values will not be included in the
>>>> binlog event. So the slave is impossible to identify which row to
>>>> update.
>>>>
>>>> Is this a bug or is there a flag to tell MySQL to include primary key?
>>>> We are based on 5.1.33 currently.
>>>>
>>> --
>>> Mats Kindahl
>>> Senior Software Engineer
>>> Database Technology Group
>>> Sun Microsystems
>>>
>
> --
> Mats Kindahl
> Senior Software Engineer
> Database Technology Group
> Sun Microsystems
>
Thread
Primary key is miss in row based replication.Yuan Wang20 Nov
  • Re: Primary key is miss in row based replication.Venu Kalyan20 Nov
  • Re: Primary key is miss in row based replication.Venu Kalyan20 Nov
  • Re: Primary key is miss in row based replication.Mats Kindahl20 Nov
    • Re: Primary key is miss in row based replication.Yuan Wang21 Nov
      • Re: Primary key is miss in row based replication.Mats Kindahl24 Nov
        • Re: Primary key is miss in row based replication.Yuan Wang25 Nov
          • Re: Primary key is miss in row based replication.Kristian Nielsen25 Nov
          • Re: Primary key is miss in row based replication.Mats Kindahl25 Nov
            • Re: Primary key is miss in row based replication.Yuan Wang26 Nov
              • Re: Primary key is miss in row based replication.Kristian Nielsen26 Nov
                • Re: Primary key is miss in row based replication.Mats Kindahl26 Nov
              • Re: Primary key is miss in row based replication.Mats Kindahl26 Nov
Re: Primary key is miss in row based replication.Yuan Wang20 Nov
  • Re: Primary key is miss in row based replication.Mats Kindahl20 Nov
    • Re: Primary key is miss in row based replication.Yuan Wang20 Nov
  • Re: Primary key is miss in row based replication.Mats Kindahl20 Nov
    • Re: Primary key is miss in row based replication.Yuan Wang20 Nov
      • Re: Primary key is miss in row based replication.Mats Kindahl20 Nov