List:Commits« Previous MessageNext Message »
From:cbell Date:July 11 2007 2:21am
Subject:bk commit into 5.1 tree (cbell:1.2538)
View as plain text  
Below is the list of changes that have just been committed into a local
5.1 repository of cbell. When cbell does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html

ChangeSet@stripped, 2007-07-10 22:20:32-04:00, cbell@mysql_cab_desk. +13 -0
  WL#3228 : RBR using different table defs on slave/master
  
  This patch adds the ability to store extra field metadata in the table
  map event. This data can include pack_length() or field_lenght() for
  fields such as CHAR or VARCHAR enabling developers to add code that
  can check for compatibilty between master and slave columns. More 
  importantly, the extra field metadata can be used to store data from the
  master correctly should a VARCHAR field on the master be <= 255 bytes 
  while the same field on the slave is > 255 bytes. '
  
  The patch also includes the needed changes to unpack to ensure that data
  which is smaller on the master can be unpacked correctly on the slave.
  
  See worklog for additional details.

  mysql-test/r/rpl_colSize.result@stripped, 2007-07-10 22:20:20-04:00, cbell@mysql_cab_desk. +0 -0
    WL#3228 : prototype test

  mysql-test/r/rpl_colSize.result@stripped, 2007-07-10 22:20:20-04:00, cbell@mysql_cab_desk. +0 -0

  mysql-test/t/rpl_colSize.test@stripped, 2007-07-10 22:20:20-04:00, cbell@mysql_cab_desk. +78 -0
    WL#3228 : prototype test

  mysql-test/t/rpl_colSize.test@stripped, 2007-07-10 22:20:20-04:00, cbell@mysql_cab_desk. +0 -0

  sql/field.cc@stripped, 2007-07-10 22:20:15-04:00, cbell@mysql_cab_desk. +33 -14
    WL#3228 : RBR using different table defs on slave/master
    
    This patch includes updates to the unpack() methods for the variable
    length fields. A new parameter was added (from_length) that is the
    value stored in the field_metadata of the table map from the table_def
    class. If the value is non-zero and less than what the field on the 
    slave is then use the from_length else use the original value from the
    field on the slave.

  sql/field.h@stripped, 2007-07-10 22:20:15-04:00, cbell@mysql_cab_desk. +10 -8
    WL#3228 : RBR using different table defs on slave/master
    
    This patch includes updates to the unpack() methods for the variable
    length fields. A new parameter was added (from_length) that is the
    value stored in the field_metadata of the table map from the table_def
    class. 

  sql/filesort.cc@stripped, 2007-07-10 22:20:16-04:00, cbell@mysql_cab_desk. +1 -1
    WL#3228 : RBR using different table defs on slave/master
    
    This patch adds the field's field length to the list of parameters for
    the unpack() method to comply with changes to field class.

  sql/log_event.cc@stripped, 2007-07-10 22:20:16-04:00, cbell@mysql_cab_desk. +139 -7
    WL#3228 : RBR using different table defs on slave/master
    
    This patch adds methods to calculate the field metadata size, prepare
    the field metadata for writing to the binlog, and additions to the
    Table_map_log_event::write_body method to include the field metadata 
    in the table map that is written to the binlog.

  sql/log_event.h@stripped, 2007-07-10 22:20:17-04:00, cbell@mysql_cab_desk. +8 -2
    WL#3228 : RBR using different table defs on slave/master
    
    This patch adds method declarations and variables needed to support
    storing field metadata in the table map that is written to the binlog.

  sql/rpl_record.cc@stripped, 2007-07-10 22:20:17-04:00, cbell@mysql_cab_desk. +12 -3
    WL#3228 : RBR using different table defs on slave/master
    
    This patch modifies the unpack_row() method to unpack fields passing in
    the value from the table_def class. This value is the extra field
    metadata stored there from the master.

  sql/rpl_record_old.cc@stripped, 2007-07-10 22:20:18-04:00, cbell@mysql_cab_desk. +5 -1
    WL#3228 : RBR using different table defs on slave/master
    
    This patch modifies the unpack_row() method to unpack fields passing in
    the value from the table_def class. This value is the extra field
    metadata stored there from the master.

  sql/rpl_rli.h@stripped, 2007-07-10 22:20:18-04:00, cbell@mysql_cab_desk. +10 -0
    WL#3228 : RBR using different table defs on slave/master
    
    This patch adds a helper function to retrieve the table_def for a given
    table in the RPL_TABLE_LIST structure.

  sql/rpl_utility.cc@stripped, 2007-07-10 22:20:19-04:00, cbell@mysql_cab_desk. +68 -0
    WL#3228 : RBR using different table defs on slave/master
    
    This patch adds a helper method that retrieves the correct size 
    parameter for the field. This method is used to compare the size as
    sent by the master with that on the slave for all types of fields that
    can vary in size and storage requirements. 

  sql/rpl_utility.h@stripped, 2007-07-10 22:20:19-04:00, cbell@mysql_cab_desk. +67 -2
    WL#3228 : RBR using different table defs on slave/master
    
    This patch changes the table_def class constructor to pass in the raw
    data read from the table map and extract it into an array of dimension
    size (number of fields). It also adds a method to return the field 
    metadata for any field. The method returns the data stored in the table
    map or 0 if no data was stored for that field.

  storage/archive/ha_archive.cc@stripped, 2007-07-10 22:20:19-04:00, cbell@mysql_cab_desk. +2 -1
    WL#3228 : RBR using different table defs on slave/master
    
    This patch adds the field's field length to the list of parameters for
    the unpack() method to comply with changes to field class.

diff -Nrup a/mysql-test/t/rpl_colSize.test b/mysql-test/t/rpl_colSize.test
--- /dev/null	Wed Dec 31 16:00:00 196900
+++ b/mysql-test/t/rpl_colSize.test	2007-07-10 22:20:20 -04:00
@@ -0,0 +1,78 @@
+-- source include/have_binlog_format_row.inc
+-- source include/master-slave.inc
+
+--disable_warnings
+DROP TABLE IF EXISTS t1;
+--enable_warnings
+
+--echo **** Testing WL#3228 changes. ****
+
+--echo *** On Slave ***
+sync_slave_with_master;
+STOP SLAVE;
+RESET SLAVE;
+
+eval CREATE TABLE t1 (
+  a float     (47),
+  b double    (143,9),
+  c decimal   (65,30),
+  d numeric   (4,0),
+  e bit       (32),
+  f char      (21),
+  g varchar   (1300),
+  h binary    (33),
+  j varbinary (200),
+  k enum      ('0','1'),
+  l set       ('1','2','3','4','5','6','7','8','9','0','11','12','13','14','15','16','17','18','19','21','22','23','24','25','26','27','28','29')
+);
+
+--echo *** Create t1 on Master ***
+connection master;
+eval CREATE TABLE t1 (
+  a float     (44),
+  b double    (10,3),
+  c decimal   (10,2),
+  d numeric   (3,0),
+  e bit       (20),
+  f char      (10),
+  g varchar   (100),
+  h binary    (7),
+  j varbinary (20),
+  k enum      ('0'),
+  l set       ('1','2','3','4','5','6','7','8','9','0')
+);
+
+RESET MASTER;
+
+--echo *** Start Slave ***
+connection slave;
+START SLAVE;
+
+--echo *** Master Data Insert ***
+connection master;
+
+INSERT INTO t1 () VALUES (
+  17.567, 
+  2.123, 
+  10.20, 
+  125,
+  3,
+  'TEST',
+  'This is a test',
+  'binary data',
+  'more binary data',
+  '0',
+  '0');
+
+SELECT * FROM t1 ORDER BY a;
+
+--echo *** Select from slave ***
+sync_slave_with_master;
+SELECT * FROM t1 ORDER BY a;
+
+--echo *** Drop t1  ***
+connection master;
+DROP TABLE t1;
+sync_slave_with_master;
+
+# END 5.1 Test Case
diff -Nrup a/sql/field.cc b/sql/field.cc
--- a/sql/field.cc	2007-06-21 11:12:54 -04:00
+++ b/sql/field.cc	2007-07-10 22:20:15 -04:00
@@ -6291,10 +6291,16 @@ uchar *Field_string::pack(uchar *to, con
 }
 
 
-const uchar *Field_string::unpack(uchar *to, const uchar *from)
+/*
+  When unpacking data that may be smaller in the "from" than the "to," use
+  the from_length to unpack rather than the to's length.
+*/
+const uchar *Field_string::unpack(uchar *to, const uchar *from, uint from_length)
 {
-  uint length;
-  if (field_length > 255)
+  uint length= 0;
+  uint f_length= (from_length && (from_length < field_length)) ? 
+                from_length : field_length;
+  if (f_length > 255)
   {
     length= uint2korr(from);
     from+= 2;
@@ -6783,12 +6789,16 @@ uchar *Field_varstring::pack_key_from_ke
 
 /*
   unpack field packed with Field_varstring::pack()
-*/
 
-const uchar *Field_varstring::unpack(uchar *to, const uchar *from)
+  When unpacking data that may be smaller in the "from" than the "to," use
+  the from_length to unpack rather than the to's length_bytes.
+*/
+const uchar *Field_varstring::unpack(uchar *to, const uchar *from, uint from_length)
 {
   uint length;
-  if (length_bytes == 1)
+  uint l_bytes= (from_length && (from_length < field_length)) ? 
+                (from_length <= 255) ? 1 : 2 : length_bytes;
+  if (l_bytes == 1)
     length= (uint) (*to= *from++);
   else
   {
@@ -6797,7 +6807,7 @@ const uchar *Field_varstring::unpack(uch
     to[1]= *from++;
   }
   if (length)
-    memcpy(to+ length_bytes, from, length);
+    memcpy(to+ l_bytes, from, length);
   return from+length;
 }
 
@@ -7476,8 +7486,11 @@ uchar *Field_blob::pack(uchar *to, const
   return to+packlength+length;
 }
 
-
-const uchar *Field_blob::unpack(uchar *to, const uchar *from)
+/*
+  Note: from_length included to satisfy inheritance rules, but is not needed
+  for blob fields.
+*/
+const uchar *Field_blob::unpack(uchar *to, const uchar *from, uint from_lenght)
 {
   memcpy(to,from,packlength);
   uint32 length=get_length(from);
@@ -8524,9 +8537,15 @@ uchar *Field_bit::pack(uchar *to, const 
 }
 
 
-const uchar *Field_bit::unpack(uchar *to, const uchar *from)
+/*
+  When unpacking data that may be smaller in the "from" than the "to," use
+  the from_length to unpack rather than the to's length.
+*/
+const uchar *Field_bit::unpack(uchar *to, const uchar *from, uint from_length)
 {
-  if (bit_len > 0)
+  uint b_length= (from_length && (from_length < bytes_in_rec)) ? 
+                  from_length : bytes_in_rec;
+  if (b_length > 0)
   {
     /*
       set_rec_bits is a macro, don't put the post-increment in the
@@ -8535,11 +8554,11 @@ const uchar *Field_bit::unpack(uchar *to
       For the choice of the second argument, see the explanation for
       Field_bit::pack().
     */
-    set_rec_bits(*from, bit_ptr + (to - ptr), bit_ofs, bit_len);
+    set_rec_bits(*from, bit_ptr + (to - ptr), bit_ofs, b_length);
     from++;
   }
-  memcpy(to, from, bytes_in_rec);
-  return from + bytes_in_rec;
+  memcpy(to, from, b_length);
+  return from + b_length;
 }
 
 
diff -Nrup a/sql/field.h b/sql/field.h
--- a/sql/field.h	2007-06-15 13:36:14 -04:00
+++ b/sql/field.h	2007-07-10 22:20:15 -04:00
@@ -343,11 +343,13 @@ public:
     memcpy(to,from,length);
     return to+length;
   }
-  virtual const uchar *unpack(uchar* to, const uchar *from)
+  virtual const uchar *unpack(uchar* to, const uchar *from, uint from_length)
   {
     uint length=pack_length();
-    memcpy(to,from,length);
-    return from+length;
+    uint len= (from_length && (from_length < length)) ?
+              from_length : length;
+    memcpy(to,from,len);
+    return from+len;
   }
   virtual uchar *pack_key(uchar* to, const uchar *from, uint max_length)
   {
@@ -361,7 +363,7 @@ public:
   virtual const uchar *unpack_key(uchar* to, const uchar *from,
                                   uint max_length)
   {
-    return unpack(to,from);
+    return unpack(to,from, max_length);
   }
   virtual uint packed_col_length(const uchar *to, uint length)
   { return length;}
@@ -1162,7 +1164,7 @@ public:
   void sort_string(uchar *buff,uint length);
   void sql_type(String &str) const;
   uchar *pack(uchar *to, const uchar *from, uint max_length=~(uint) 0);
-  const uchar *unpack(uchar* to, const uchar *from);
+  const uchar *unpack(uchar* to, const uchar *from, uint from_length);
   int pack_cmp(const uchar *a,const uchar *b,uint key_length,
                my_bool insert_or_update);
   int pack_cmp(const uchar *b,uint key_length,my_bool insert_or_update);
@@ -1238,7 +1240,7 @@ public:
   uchar *pack_key(uchar *to, const uchar *from, uint max_length);
   uchar *pack_key_from_key_image(uchar* to, const uchar *from,
                                  uint max_length);
-  const uchar *unpack(uchar* to, const uchar *from);
+  const uchar *unpack(uchar* to, const uchar *from, uint from_length);
   const uchar *unpack_key(uchar* to, const uchar *from, uint max_length);
   int pack_cmp(const uchar *a, const uchar *b, uint key_length,
                my_bool insert_or_update);
@@ -1377,7 +1379,7 @@ public:
   uchar *pack_key(uchar *to, const uchar *from, uint max_length);
   uchar *pack_key_from_key_image(uchar* to, const uchar *from,
                                  uint max_length);
-  const uchar *unpack(uchar *to, const uchar *from);
+  const uchar *unpack(uchar *to, const uchar *from, uint from_length);
   const uchar *unpack_key(uchar* to, const uchar *from, uint max_length);
   int pack_cmp(const uchar *a, const uchar *b, uint key_length,
                my_bool insert_or_update);
@@ -1552,7 +1554,7 @@ public:
   uint32 pack_length_in_rec() const { return bytes_in_rec; }
   void sql_type(String &str) const;
   uchar *pack(uchar *to, const uchar *from, uint max_length=~(uint) 0);
-  const uchar *unpack(uchar* to, const uchar *from);
+  const uchar *unpack(uchar* to, const uchar *from, uint from_length);
   virtual void set_default();
 
   Field *new_key_field(MEM_ROOT *root, struct st_table *new_table,
diff -Nrup a/sql/filesort.cc b/sql/filesort.cc
--- a/sql/filesort.cc	2007-06-11 06:20:26 -04:00
+++ b/sql/filesort.cc	2007-07-10 22:20:16 -04:00
@@ -1572,7 +1572,7 @@ unpack_addon_fields(struct st_sort_addon
       continue;
     }
     field->set_notnull();
-    field->unpack(field->ptr, buff + addonf->offset);
+    field->unpack(field->ptr, buff + addonf->offset, field->field_length);
   }
 }
 
diff -Nrup a/sql/log_event.cc b/sql/log_event.cc
--- a/sql/log_event.cc	2007-06-28 09:07:53 -04:00
+++ b/sql/log_event.cc	2007-07-10 22:20:16 -04:00
@@ -6023,10 +6023,10 @@ int Rows_log_event::do_apply_event(RELAY
 #ifdef HAVE_QUERY_CACHE
     query_cache.invalidate_locked_for_write(rli->tables_to_lock);
 #endif
-    const_cast<RELAY_LOG_INFO*>(rli)->clear_tables_to_lock();
+//    const_cast<RELAY_LOG_INFO*>(rli)->clear_tables_to_lock();
   }
 
-  DBUG_ASSERT(rli->tables_to_lock == NULL && rli->tables_to_lock_count == 0);
+//  DBUG_ASSERT(rli->tables_to_lock == NULL && rli->tables_to_lock_count == 0);
 
   TABLE* table= const_cast<RELAY_LOG_INFO*>(rli)->m_table_map.get_table(m_table_id);
 
@@ -6122,6 +6122,10 @@ int Rows_log_event::do_apply_event(RELAY
     }
   }
 
+  if (rli->tables_to_lock)
+    const_cast<RELAY_LOG_INFO*>(rli)->clear_tables_to_lock();
+  DBUG_ASSERT(rli->tables_to_lock == NULL && rli->tables_to_lock_count == 0);
+
   if (error)
   {                     /* error has occured during the transaction */
     rli->report(ERROR_LEVEL, thd->net.last_errno,
@@ -6370,6 +6374,102 @@ void Rows_log_event::print_helper(FILE *
 /**************************************************************************
 	Table_map_log_event member functions and support functions
 **************************************************************************/
+/**
+  * Calculate field metadata size based on the real_type of the field.
+  *
+  * @returns int Size of field metadata.
+  */
+#if !defined(MYSQL_CLIENT)
+int Table_map_log_event::get_field_metadata_size()
+{
+  DBUG_ENTER("Table_map_log_event::get_field_metadata_size");
+  int size= 0;
+  for (unsigned int i= 0 ; i < m_table->s->fields ; i++)
+  {
+    switch (m_table->s->field[i]->real_type()) {
+    case MYSQL_TYPE_DOUBLE:
+    case MYSQL_TYPE_NEWDECIMAL:
+    case MYSQL_TYPE_FLOAT:
+    case MYSQL_TYPE_BIT:
+    {
+      size++;                         // Store one byte here.
+      break;
+    }
+    case MYSQL_TYPE_ENUM:
+    case MYSQL_TYPE_STRING:
+    case MYSQL_TYPE_VARCHAR:
+    case MYSQL_TYPE_SET:
+    {
+      size= size + sizeof(short int); // Store short int here.
+      break;
+    }
+    default:
+      break;
+    }
+  }
+  m_field_metadata_size= size;
+  DBUG_PRINT("info", ("Table_map_log_event: %d bytes in field metadata.",
+                       m_field_metadata_size));
+  DBUG_RETURN(m_field_metadata_size);
+}
+#endif /* !defined(MYSQL_CLIENT) */
+
+/**
+  * Save the field metadata based on the real_type of the field.
+  * The metadata saved depends on the type of the field. Some fields
+  * store a single byte for pack_length() while others store two bytes
+  * for field_length (max length).
+  *
+  * @returns int 0 = Ok.
+  */
+#if !defined(MYSQL_CLIENT)
+int Table_map_log_event::get_field_metadata()
+{
+  DBUG_ENTER("Table_map_log_event::get_field_metadata");
+  int index= 0;
+  for (unsigned int i= 0 ; i < m_table->s->fields ; i++)
+  {
+    switch (m_table->s->field[i]->real_type()) {
+    case MYSQL_TYPE_DOUBLE:
+    case MYSQL_TYPE_NEWDECIMAL:
+    case MYSQL_TYPE_FLOAT:
+    {
+      m_field_metadata[index]= (byte)m_table->s->field[i]->pack_length();
+      byte x= m_field_metadata[index];
+      index++;
+      break;
+    }
+    case MYSQL_TYPE_BIT:
+    {
+      m_field_metadata[index]= 
+        (byte)((Field_bit *)m_table->s->field[i])->bytes_in_rec;
+      byte x= m_field_metadata[index];
+      index++;
+      break;
+    }
+    case MYSQL_TYPE_ENUM:
+    case MYSQL_TYPE_STRING:
+    case MYSQL_TYPE_VARCHAR:
+    {
+      short int *x= (short int *)&m_field_metadata[index];
+      int2store(x, m_table->s->field[i]->field_length);
+      index= index + sizeof(short int);
+      break;
+    }
+    case MYSQL_TYPE_SET:
+    {
+      short int *x= (short int *)&m_field_metadata[index];
+      int2store(x, m_table->s->field[i]->pack_length());
+      index= index + sizeof(short int);
+      break;
+    }
+    default:
+      break;
+    }
+  }
+  DBUG_RETURN(0);
+}
+#endif /* !defined(MYSQL_CLIENT) */
 
 /*
   Constructor used to build an event for writing to the binary log.
@@ -6385,7 +6485,7 @@ Table_map_log_event::Table_map_log_event
     m_dblen(m_dbnam ? tbl->s->db.length : 0),
     m_tblnam(tbl->s->table_name.str),
     m_tbllen(tbl->s->table_name.length),
-    m_colcnt(tbl->s->fields), m_coltype(0),
+    m_colcnt(tbl->s->fields), m_field_metadata(0),
     m_table_id(tid),
     m_flags(flags)
 {
@@ -6406,6 +6506,8 @@ Table_map_log_event::Table_map_log_event
   m_data_size+= m_dblen + 2;	// Include length and terminating \0
   m_data_size+= m_tbllen + 2;	// Include length and terminating \0
   m_data_size+= 1 + m_colcnt;	// COLCNT and column types
+  m_field_metadata_size= get_field_metadata_size();
+  m_data_size+= m_field_metadata_size + 1;
 
   /* If malloc fails, catched in is_valid() */
   if ((m_memory= (uchar*) my_malloc(m_colcnt, MYF(MY_WME))))
@@ -6414,6 +6516,12 @@ Table_map_log_event::Table_map_log_event
     for (unsigned int i= 0 ; i < m_table->s->fields ; ++i)
       m_coltype[i]= m_table->field[i]->type();
   }
+
+  /*
+    Create an array for the field metadata and store it.
+  */
+  m_field_metadata= new byte[m_field_metadata_size];
+  get_field_metadata();
 }
 #endif /* !defined(MYSQL_CLIENT) */
 
@@ -6429,7 +6537,8 @@ Table_map_log_event::Table_map_log_event
 #ifndef MYSQL_CLIENT
   m_table(NULL),
 #endif
-  m_memory(NULL)
+  m_memory(NULL),
+  m_field_metadata(0), m_field_metadata_size(0)
 {
   DBUG_ENTER("Table_map_log_event::Table_map_log_event(const char*,uint,...)");
 
@@ -6503,12 +6612,19 @@ Table_map_log_event::Table_map_log_event
     memcpy(m_coltype, ptr_after_colcnt, m_colcnt);
   }
 
+  ptr_after_colcnt= ptr_after_colcnt + m_colcnt;
+  m_field_metadata_size= net_field_length(&ptr_after_colcnt);
+
+  m_field_metadata= new byte[m_field_metadata_size];
+  memcpy(m_field_metadata, ptr_after_colcnt, m_field_metadata_size);
+
   DBUG_VOID_RETURN;
 }
 #endif
 
 Table_map_log_event::~Table_map_log_event()
 {
+  delete[] m_field_metadata;
   my_free(m_memory, MYF(MY_ALLOW_ZERO_PTR));
 }
 
@@ -6639,7 +6755,15 @@ int Table_map_log_event::do_apply_event(
       inside st_relay_log_info::clear_tables_to_lock() by calling the
       table_def destructor explicitly.
     */
-    new (&table_list->m_tabledef) table_def(m_coltype, m_colcnt);
+    new (&table_list->m_tabledef) table_def(m_coltype, m_colcnt, 
+         m_field_metadata, m_field_metadata_size);
+
+    //printf("\nThis is what is in table_def:\n");
+    //for (uint x= 0; x < m_colcnt; x++)
+    //{
+    //  printf("col %d metadata = %d\n", x, table_list->m_tabledef.field_metadata(x));
+    //}
+
     table_list->m_tabledef_valid= TRUE;
 
     /*
@@ -6653,7 +6777,7 @@ int Table_map_log_event::do_apply_event(
   }
 
   DBUG_RETURN(error);
-
+ 
 err:
   my_free(memory, MYF(MY_WME));
   DBUG_RETURN(error);
@@ -6711,12 +6835,20 @@ bool Table_map_log_event::write_data_bod
   uchar *const cbuf_end= net_store_length(cbuf, (size_t) m_colcnt);
   DBUG_ASSERT(static_cast<size_t>(cbuf_end - cbuf) <= sizeof(cbuf));
 
+  /*
+    Store the size of the field metadata.
+  */
+  uchar mbuf[sizeof(m_field_metadata_size)];
+  uchar *const mbuf_end= net_store_length(mbuf, (size_t) m_field_metadata_size);
+
   return (my_b_safe_write(file, dbuf,      sizeof(dbuf)) ||
           my_b_safe_write(file, (const uchar*)m_dbnam,   m_dblen+1) ||
           my_b_safe_write(file, tbuf,      sizeof(tbuf)) ||
           my_b_safe_write(file, (const uchar*)m_tblnam,  m_tbllen+1) ||
           my_b_safe_write(file, cbuf, (size_t) (cbuf_end - cbuf)) ||
-          my_b_safe_write(file, m_coltype, m_colcnt));
+          my_b_safe_write(file, m_coltype, m_colcnt) ||
+          my_b_safe_write(file, mbuf, (size_t) (mbuf_end - mbuf)) ||
+          my_b_safe_write(file, m_field_metadata, m_field_metadata_size));
  }
 #endif
 
diff -Nrup a/sql/log_event.h b/sql/log_event.h
--- a/sql/log_event.h	2007-06-04 19:15:00 -04:00
+++ b/sql/log_event.h	2007-07-10 22:20:17 -04:00
@@ -2049,6 +2049,8 @@ public:
 
   virtual int get_data_size() { return m_data_size; } 
 #ifndef MYSQL_CLIENT
+  virtual int get_field_metadata_size();
+  virtual int get_field_metadata();
   virtual bool write_data_header(IO_CACHE *file);
   virtual bool write_data_body(IO_CACHE *file);
   virtual const char *get_db() { return m_dbnam; }
@@ -2079,12 +2081,16 @@ private:
   size_t         m_tbllen;
   ulong          m_colcnt;
   uchar         *m_coltype;
-
   uchar         *m_memory;
   ulong          m_table_id;
   flag_set       m_flags;
-
   size_t         m_data_size;
+
+  byte          *m_field_metadata;        // buffer for field metadata
+  /*
+    The size of field metadata buffer set by calling get_field_metadata_size()
+  */
+  ulong         m_field_metadata_size;   
 };
 
 
diff -Nrup a/sql/rpl_record.cc b/sql/rpl_record.cc
--- a/sql/rpl_record.cc	2007-06-11 16:15:28 -04:00
+++ b/sql/rpl_record.cc	2007-07-10 22:20:17 -04:00
@@ -17,6 +17,7 @@
 #include "rpl_rli.h"
 #include "rpl_record.h"
 #include "slave.h"                  // Need to pull in slave_print_msg
+#include "rpl_utility.h"
 
 /**
    Pack a record of data for a table into a format suitable for
@@ -124,7 +125,6 @@ pack_row(TABLE *table, MY_BITMAP const* 
 }
 #endif
 
-
 /**
    Unpack a row into @c table->record[0].
 
@@ -191,10 +191,10 @@ unpack_row(RELAY_LOG_INFO const *rli,
   unsigned int null_mask= 1U;
   // The "current" null bits
   unsigned int null_bits= *null_ptr++;
+  int i= 0;
   for (field_ptr= begin_ptr ; field_ptr < end_ptr ; ++field_ptr)
   {
     Field *const f= *field_ptr;
-
     /*
       No need to bother about columns that does not exist: they have
       gotten default values when being emptied above.
@@ -222,12 +222,21 @@ unpack_row(RELAY_LOG_INFO const *rli,
         /*
           We only unpack the field if it was non-null
         */
-        pack_ptr= f->unpack(f->ptr, pack_ptr);
+        pack_ptr= f->unpack(f->ptr, pack_ptr, 
+                            const_cast<RELAY_LOG_INFO*>(rli)->
+                            get_tabledef(table)->field_metadata(i));
       }
 
       bitmap_set_bit(rw_set, f->field_index);
       null_mask <<= 1;
+      i++;
     }
+  }
+  printf("\nThis is what I got for the record:\n");
+  for (field_ptr= begin_ptr ; field_ptr < end_ptr ; ++field_ptr)
+  {
+    Field *const f= *field_ptr;
+    printf("Field %s = %s\n", f->field_name, f->ptr);
   }
 
   /*
diff -Nrup a/sql/rpl_record_old.cc b/sql/rpl_record_old.cc
--- a/sql/rpl_record_old.cc	2007-06-11 16:15:28 -04:00
+++ b/sql/rpl_record_old.cc	2007-07-10 22:20:18 -04:00
@@ -111,6 +111,7 @@ unpack_row_old(RELAY_LOG_INFO *rli,
   Field **field_ptr;
   uchar const *ptr= row + master_null_bytes;
   Field **const end_ptr= begin_ptr + colcnt;
+  int i= 0;
   for (field_ptr= begin_ptr ; field_ptr < end_ptr ; ++field_ptr)
   {
     Field *const f= *field_ptr;
@@ -118,13 +119,16 @@ unpack_row_old(RELAY_LOG_INFO *rli,
     if (bitmap_is_set(cols, field_ptr -  begin_ptr))
     {
       f->move_field_offset(offset);
-      ptr= f->unpack(f->ptr, ptr);
+      ptr= f->unpack(f->ptr, ptr, 
+                     const_cast<RELAY_LOG_INFO*>(rli)->
+                     get_tabledef(table)->field_metadata(i));
       f->move_field_offset(-offset);
       /* Field...::unpack() cannot return 0 */
       DBUG_ASSERT(ptr != NULL);
     }
     else
       bitmap_clear_bit(rw_set, field_ptr - begin_ptr);
+    i++;
   }
 
   *row_end = ptr;
diff -Nrup a/sql/rpl_rli.h b/sql/rpl_rli.h
--- a/sql/rpl_rli.h	2007-06-11 16:15:29 -04:00
+++ b/sql/rpl_rli.h	2007-07-10 22:20:18 -04:00
@@ -18,6 +18,7 @@
 
 #include "rpl_tblmap.h"
 #include "rpl_reporting.h"
+#include "rpl_utility.h"
 
 struct RPL_TABLE_LIST;
 
@@ -300,6 +301,15 @@ typedef struct st_relay_log_info : publi
   RPL_TABLE_LIST *tables_to_lock;           /* RBR: Tables to lock  */
   uint tables_to_lock_count;        /* RBR: Count of tables to lock */
   table_mapping m_table_map;      /* RBR: Mapping table-id to table */
+
+  inline table_def *get_tabledef(TABLE *tbl)
+  {
+    table_def *td= 0;
+    for (TABLE_LIST *ptr= tables_to_lock; ptr && !td; ptr= ptr->next_global)
+      if (ptr->table == tbl)
+        td= &((RPL_TABLE_LIST *)ptr)->m_tabledef;
+    return (td);
+  }
 
   /*
     Last charset (6 bytes) seen by slave SQL thread is cached here; it helps
diff -Nrup a/sql/rpl_utility.cc b/sql/rpl_utility.cc
--- a/sql/rpl_utility.cc	2007-06-11 16:15:29 -04:00
+++ b/sql/rpl_utility.cc	2007-07-10 22:20:19 -04:00
@@ -100,6 +100,49 @@ field_length_from_packed(enum_field_type
   return length;
 }
 
+/* 
+  This method gets the field metadata as pertaining to how the field
+  is stored in the raw data from the master. This can be used to determine
+  if the sizes are compatible in either max size (length) or if the
+  packed size is compatible.
+*/
+uint get_field_metadata (uint32 type, int col, TABLE *table)
+{
+  uint field_size= 0;
+  switch (type) {
+  case MYSQL_TYPE_DOUBLE:
+  case MYSQL_TYPE_NEWDECIMAL:
+  case MYSQL_TYPE_FLOAT:
+  {
+    field_size= table->field[col]->pack_length();
+    break;
+  }
+  case MYSQL_TYPE_BIT:
+  {
+    field_size= ((Field_bit *)table->field[col])->bytes_in_rec;
+    break;
+  }
+  case MYSQL_TYPE_SET:
+  case MYSQL_TYPE_STRING:
+  //{
+  //  if (table->field[col]->real_type() == MYSQL_TYPE_SET)
+  //    field_size= table->field[col]->pack_length();
+  //  else
+  //    field_size= table->field[col]->field_length;
+  //  break;
+  //}
+  case MYSQL_TYPE_ENUM:
+  case MYSQL_TYPE_VARCHAR:
+  {
+    field_size= table->field[col]->field_length;
+    break;
+  }
+  default:
+    field_size= 0;
+  }
+  return (field_size);
+}
+
 /*********************************************************************
  *                   table_def member definitions                    *
  *********************************************************************/
@@ -112,6 +155,9 @@ int
 table_def::compatible_with(RELAY_LOG_INFO const *rli_arg, TABLE *table)
   const
 {
+  uint received_size= 0;
+  uint field_size= 0;
+
   /*
     We only check the initial columns for the tables.
   */
@@ -138,8 +184,14 @@ table_def::compatible_with(RELAY_LOG_INF
                 ER(ER_BINLOG_ROW_WRONG_TABLE_DEF), buf);
   }
 
+  ///*
+  //  We now check for column type and size compatibility.
+  //*/
   for (uint col= 0 ; col < cols_to_check ; ++col)
   {
+    /*
+      Checking types.
+    */
     if (table->field[col]->type() != type(col))
     {
       DBUG_ASSERT(col < size() && col < tsh->fields);
@@ -153,6 +205,22 @@ table_def::compatible_with(RELAY_LOG_INF
       rli->report(ERROR_LEVEL, ER_BINLOG_ROW_WRONG_TABLE_DEF,
                   ER(ER_BINLOG_ROW_WRONG_TABLE_DEF), buf);
     }
+    ///*
+    //  Checking column width/size.
+    //*/
+    //received_size= field_metadata(col);
+    //field_size= get_field_metadata(type(col), col, table);
+    //if (received_size && (received_size > field_size))
+    //{
+    //    error= 1;
+    //    char buf[256];
+    //    my_snprintf(buf, sizeof(buf), "Column %d size mismatch - "
+    //                "received size %d, %s.%s has size %d",
+    //                col, received_size, tsh->db.str, 
+    //               tsh->table_name.str, field_size);
+    //    rli->report(ERROR_LEVEL, ER_BINLOG_ROW_WRONG_TABLE_DEF,
+    //                ER(ER_BINLOG_ROW_WRONG_TABLE_DEF), buf);
+    //  }
   }
 
   return error;
diff -Nrup a/sql/rpl_utility.h b/sql/rpl_utility.h
--- a/sql/rpl_utility.h	2007-05-23 18:39:21 -04:00
+++ b/sql/rpl_utility.h	2007-07-10 22:20:19 -04:00
@@ -27,6 +27,7 @@ typedef st_relay_log_info RELAY_LOG_INFO
 
 uint32
 field_length_from_packed(enum_field_types field_type, uchar const *data);
+uint get_field_metadata (uint32 type, int col, TABLE *table);
 
 /**
   A table definition from the master.
@@ -58,18 +59,67 @@ public:
     @param types Array of types
     @param size  Number of elements in array 'types'
    */
-  table_def(field_type *types, ulong size)
-    : m_size(size), m_type(new unsigned char [size])
+  table_def(field_type *types, ulong size, byte *field_metadata, 
+      int metadata_size)
+    : m_size(size), m_type(new unsigned char [size]), m_field_metadata(0)
   {
     if (m_type)
       memcpy(m_type, types, size);
     else
       m_size= 0;
+    /*
+      Extract the data from the table map into the field metadata array
+      iff there is field metadata. The variable metadata_size will be
+      0 if we are replicating from a down-level server hence no field
+      metadata was written to the table map. This can also happen if 
+      there were no fields in the master that needed extra metadata.
+    */
+    if (m_size && metadata_size)
+    {
+      m_field_metadata= new short int[m_size];
+      int index= 0;
+      for (unsigned int i= 0; i < m_size; i++)
+      {
+        switch (m_type[i]) {
+        case MYSQL_TYPE_DOUBLE:
+        case MYSQL_TYPE_NEWDECIMAL:
+        case MYSQL_TYPE_FLOAT:
+        case MYSQL_TYPE_BIT:
+        {
+          /*
+            These types store a single byte.
+          */
+          byte x= field_metadata[index];
+          m_field_metadata[i]= x;
+          index++;
+          break;
+        }
+        case MYSQL_TYPE_SET:
+        case MYSQL_TYPE_ENUM:
+        case MYSQL_TYPE_STRING:
+        case MYSQL_TYPE_VARCHAR:
+        {
+          /*
+            These types store two bytes.
+          */
+          short int *x= (short int *)&field_metadata[index];
+          m_field_metadata[i]= sint2korr(x);
+          index= index + sizeof(short int);
+          break;
+        }
+        default:
+          m_field_metadata[i]= 0;
+          break;
+        }
+      }
+    }
   }
 
   ~table_def() {
     if (m_type)
       delete [] m_type;
+    if (m_field_metadata)
+      delete [] m_field_metadata;
 #ifndef DBUG_OFF
     m_type= 0;
     m_size= 0;
@@ -99,6 +149,20 @@ public:
     return m_type[index];
   }
 
+  /*
+    This function allows callers to get the extra field data from the
+    table map for a given field. If there is no metadata for that field
+    or there is no extra metadata at all, the function returns 0.
+  */
+  uint field_metadata(uint index) const
+  {
+    DBUG_ASSERT(index < m_size);
+    if (m_field_metadata)
+      return m_field_metadata[index];
+    else
+      return 0;
+  }
+
   /**
     Decide if the table definition is compatible with a table.
 
@@ -121,6 +185,7 @@ public:
 private:
   ulong m_size;           // Number of elements in the types array
   field_type *m_type;                     // Array of type descriptors
+  short int *m_field_metadata;
 };
 
 /**
diff -Nrup a/storage/archive/ha_archive.cc b/storage/archive/ha_archive.cc
--- a/storage/archive/ha_archive.cc	2007-05-10 05:59:31 -04:00
+++ b/storage/archive/ha_archive.cc	2007-07-10 22:20:19 -04:00
@@ -1106,7 +1106,8 @@ int ha_archive::unpack_row(azio_stream *
   {
     if (!((*field)->is_null()))
     {
-      ptr= (*field)->unpack(record + (*field)->offset(table->record[0]), ptr);
+      ptr= (*field)->unpack(record + (*field)->offset(table->record[0]), ptr,
+                            (*field)->field_length);
     }
   }
   DBUG_RETURN(0);

Thread
bk commit into 5.1 tree (cbell:1.2538)cbell11 Jul