bk commit into 5.1 tree (holyfoot:1.2647) BUG#36793

[holyfoot]

Below is the list of changes that have just been committed into a local
5.1 repository of holyfoot.  When holyfoot does a push these changes
will be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html

ChangeSet@stripped, 2008-05-22 14:11:35+05:00, holyfoot@stripped +2 -0
  Bug #36793 rpl_innodb_bug28430 fails on Solaris.
  
  internal InnoDB to integer type conversion doesn't work
  on BIGENDIAN machine if the length of a field is 3 in
  the row_search_autoinc_column() function. It returns the
  the value*256 result.
  
  if the number in the field is 10 for example, the
  mach_read_int_type() returns the array of {0, 0, 10, 0}
  Then after ((uint32) array) conversion we get 256 * 10 = 2560
  as a result.
  
  Fixed by handling the lenght3 case separately.

  mysql-test/suite/rpl/t/disabled.def@stripped, 2008-05-22 14:11:00+05:00, holyfoot@stripped +0 -1
    Bug #36793 rpl_innodb_bug28430 fails on Solaris.
    
    test enabled

  storage/innobase/row/row0sel.c@stripped, 2008-05-22 14:11:00+05:00, holyfoot@stripped +8 -5
    Bug #36793 rpl_innodb_bug28430 fails on Solaris.
    
    Handle the field of length 3 case separately.

diff -Nrup a/mysql-test/suite/rpl/t/disabled.def b/mysql-test/suite/rpl/t/disabled.def
--- a/mysql-test/suite/rpl/t/disabled.def	2008-05-19 11:37:42 +05:00
+++ b/mysql-test/suite/rpl/t/disabled.def	2008-05-22 14:11:00 +05:00
@@ -11,4 +11,3 @@
 ##############################################################################
 
 rpl_redirect               : Failure is sporadic and and the test is superfluous (mats)
-rpl_innodb_bug28430        : Failure on Solaris Bug #36793
diff -Nrup a/storage/innobase/row/row0sel.c b/storage/innobase/row/row0sel.c
--- a/storage/innobase/row/row0sel.c	2008-05-14 13:45:29 +05:00
+++ b/storage/innobase/row/row0sel.c	2008-05-22 14:11:00 +05:00
@@ -4582,6 +4582,13 @@ row_search_autoinc_read_column(
 	ut_a(len != UNIV_SQL_NULL);
 	ut_a(len <= sizeof value);
 
+        if (len == 3)
+        {
+          value= (((ib_uint32_t) (data[0] ^ 128)) << 16) + (((ib_uint32_t) data[1]) << 8) +
+                   (ib_uint32_t) data[2];
+          goto break_point;
+        }
+
 	mach_read_int_type(dest, data, len, unsigned_type);
 
 	/* The assumption here is that the AUTOINC value can't be negative
@@ -4595,11 +4602,6 @@ row_search_autoinc_read_column(
 		value = *(ib_uint32_t*) dest;
 		break;
 
-	case 3:
-		value = *(ib_uint32_t*) dest;
-		value &= 0xFFFFFF;
-		break;
-
 	case 2:
 		value = *(uint16 *) dest;
 		break;
@@ -4611,6 +4613,7 @@ row_search_autoinc_read_column(
 	default:
 		ut_error;
 	}
+break_point:
 
 	if (UNIV_LIKELY_NULL(heap)) {
 		mem_heap_free(heap);