bk commit into 5.0 tree (evgen:1.2023) BUG#16752

[eugene]

Below is the list of changes that have just been committed into a local
5.0 repository of evgen. When evgen does a push these changes will
be propagated to the main repository and, within 24 hours after the
push, to the public repository.
For information on how to access the public repository
see http://dev.mysql.com/doc/mysql/en/installing-source-tree.html

ChangeSet
  1.2023 06/02/08 15:12:48 evgen@stripped +1 -0
  Fixed bug#16752 Binary table files created in mysqld v4.1 caused buffer overrun 
    and possibly server crash in mysqld v5.0.
  
  Reported MyISAM table was created in mysqld 4.1 and contains varchar field.
  When binary files of that table was moved to 5.0, mysqld treats that varchar 
  field as a string field. 
  In order to make grouping server calculates group buffer, and because
  that field is string server assumes it has fixed length and doesn't add
  space for length, but later that field is converted to varchar field. 
  Due to this, when field values were actually copied, additional space for
  length bytes is taken and buffer overrun occurs, which may lead to server crash.
  
  The calc_group_buffer() function now reserves additional space for length
  bytes for VAR_STRING fields, like for VARCHAR fields.

  sql/sql_select.cc
    1.390 06/02/08 15:10:08 evgen@stripped +4 -3
    Fixed bug#16752 Binary table files created in mysqld v4.1 caused buffer overrun and
possibly server crash in mysqld v5.0.
    The calc_group_buffer() function now reserves additional space for length
    bytes for VAR_STRING fields, like for VARCHAR fields.

# This is a BitKeeper patch.  What follows are the unified diffs for the
# set of deltas contained in the patch.  The rest of the patch, the part
# that BitKeeper cares about, is below these diffs.
# User:	evgen
# Host:	moonbone.local
# Root:	/work/16752-bug-5.0-mysql

--- 1.389/sql/sql_select.cc	2006-01-28 08:20:09 +03:00
+++ 1.390/sql/sql_select.cc	2006-02-08 15:10:08 +03:00
@@ -12717,11 +12717,12 @@
     Field *field= group_item->get_tmp_table_field();
     if (field)
     {
-      if (field->type() == FIELD_TYPE_BLOB)
+      enum_field_types type;
+      if ((type= field->type()) == FIELD_TYPE_BLOB)
 	key_length+=MAX_BLOB_WIDTH;		// Can't be used as a key
-      else if (field->type() == MYSQL_TYPE_VARCHAR)
+      else if (type == MYSQL_TYPE_VARCHAR || type == MYSQL_TYPE_VAR_STRING)
         key_length+= field->field_length + HA_KEY_BLOB_LENGTH;
-      else if (field->type() == FIELD_TYPE_BIT)
+      else if (type == FIELD_TYPE_BIT)
       {
         /* Bit is usually stored as a longlong key for group fields */
         key_length+= 8;                         // Big enough